Data Engineer / Databricks Developer (Onsite Interview)

Job Title: Data Engineer / Databricks DeveloperLocation: New York (Onsite – 3 days/week)Employment Type: Full-timeNote: Only local candidates will be considered. In-person interview is mandatory.Job OverviewWe are seeking an experienced Data Engineer / Databricks Developer with a strong background in the Investment Banking domain to join our team in New York. The ideal candidate will be responsible for designing, developing, and optimizing scalable data solutions using Azure-based technologies, with a focus on Databricks and modern data architecture.Key ResponsibilitiesDesign, develop, and implement scalable data pipelines using Azure Databricks and Azure Data Factory.Build and maintain robust ETL/ELT workflows using Apache Spark and PySpark DataFrames.Develop high-performance data transformation processes using Spark and SQL.Optimize data pipelines for efficient ingestion, processing, and transformation of large datasets.Implement and maintain data governance frameworks, including data access, security, compliance, and lifecycle management.Ensure data quality, consistency, and lineage across enterprise data platforms.Design and support modern Lakehouse architecture and distributed data processing systems.Apply Master Data Management (MDM) principles to maintain data integrity and standardization.Collaborate with cross-functional teams to deliver scalable, reliable, enterprise-grade data solutions.Required Skills & QualificationsStrong hands-on experience with Azure DatabricksProficiency in Apache Spark / PySparkExperience with Azure Data FactoryAdvanced knowledge of SQLExpertise in building data pipelines and ETL/ELT processesExperience with data governance, data lineage, and data quality frameworksStrong understanding of Lakehouse architectureExperience working in Investment Banking or Financial Services domain (Mandatory)Preferred QualificationsExperience with large-scale distributed data systemsFamiliarity with cloud-based data platforms and enterprise data architectureStrong problem-solving and performance optimization skills