Databricks Developer

Data Engineer/Databricks DeveloperNew York - 3 days a week onsite workOnsite Interview Required - local candidates onlyContract Opportunity**Must have Investment Banking industry experienceResponsibilities:Design and implement scalable data pipelines using Azure Databricks and Azure Data Factory.Develop and maintain robust ETL/ELT workflows using Apache Spark and PySpark DataFrames.Build and optimize data pipelines for efficient data ingestion, transformation, and processing of large datasets.Utilize data governance tools to manage data access, security, compliance, and data lifecycle.Implement data quality frameworks and maintain data lineage across enterprise data platforms.Design and support modern data architecture using Lakehouse and distributed data processing principles.Develop high-performance Spark and SQL-based data transformation.Apply Master Data Management (MDM) concepts to ensure data consistency, integrity, and standardization.Work within the Azure Databricks ecosystem to deliver scalable, reliable, and enterprise-grade data engineering solutions.Required Skills:Azure DatabricksApache Spark / PySparkAzureData FactorySQLData Pipelines and ETL/ELTData Governance and Data LineageLakehouse Architecture