Principal Data Engineer for Databricks

Principal Data Engineer for DatabricksLocation: Warren, NJ. (2-3 days in-office)Key ResponsibilitiesLead the architecture, design, and delivery of scalable enterprise data platforms using Azure Databricks, Delta Lake, and lakehouse and optimize reliable batch and real-time data pipelines for analytics, reporting, and AI workloads.Establish reusable engineering frameworks, coding standards, CI/CD practices, and performance and cost-optimization data governance, security, lineage, and quality controls, including Unity Catalog capabilities.Provide technical leadership, mentor engineering teams, and collaborate with business, architecture, analytics, and data science stakeholders.Key SkillsAzure Databricks and Lakehouse: Databricks platform, Delta Lake, Unity Catalog, and enterprise lakehouse architecture.Data Engineering: Apache Spark, PySpark or Scala, advanced SQL, ETL/ELT design, and distributed processing.Azure Data Services: Azure Data Factory, ADLS Gen2, Synapse Analytics, Key Vault, Monitor, and Log Analytics.Architecture and Governance: Data modeling, security, data quality, lineage, scalability, and performance Leadership: Azure DevOps, CI/CD, Agile delivery, stakeholder management, mentoring, and technical roadmap development.