Azure DataBricks Lead

At Staffworxs, we don’t just connect talent — we power transformation. Headquartered in Frisco, TX, with teams in Bengaluru and Hyderabad, we combine global reach with deep expertise. Our Digital & Data Analytics practice drives growth and innovation for some of the world’s top brands, who continue to retain us as their trusted partner. If you’re ready to make an impact, you’re in the right place.Job Details:Job Title: Azure DataBricks Lead/ Principal Data Engineer Experience: 8–12 YearsLocation: Dearborn, Michigan, United States (Weekly 2 day onsite)Duration: 6+MonthsJob SummaryWe are looking for an experienced DataBricks Lead and Architecture with deep expertise in Azure and Databricks to design and deliver scalable data platforms in a manufacturing environment. The ideal candidate combines strong hands-on engineering skills with architectural vision, governance practices, and cloud-native data solutions on Azure.Key ResponsibilitiesDesign and implement scalable, high-performance data architectures using Databricks on AzureLead the development of end-to-end data pipelines and lakehouse solutionsDefine and enforce data governance, security, and access control strategiesCollaborate with business stakeholders, data engineers, and analysts to deliver reliable data solutionsDrive best practices in data engineering, performance optimization, and cost efficiencyOversee CI/CD pipelines and DevOps processes for data platformsProvide technical leadership, mentoring, and guidance to engineering teamsCore Technical SkillsDatabricks & SparkAdvanced expertise in Apache Spark using PySpark/Scala for large-scale data processingStrong performance tuning skills (Spark UI, partitioning, joins, caching, shuffle optimization)Deep knowledge of the Databricks platform: notebooks, workflows/jobs, cluster configuration, and Databricks SQLHands-on experience implementing Medallion Architecture (Bronze/Silver/Gold layers)Delta Lake & LakehouseStrong experience with Delta Lake including ACID transactions, schema enforcement/evolution, MERGE operations, and time travelAbility to design optimized Delta tables for scalable lakehouse solutionsData Governance & SecurityExpertise in Unity Catalog including catalogs, schemas, permissions, lineage, and secure data sharingAzure EcosystemStrong experience with Microsoft Azure services including:Azure Data Lake StorageAzure Data Factory / Synapse PipelinesEvent Hubs / IoT integrationsNetworking, IAM, Key Vault, and monitoringData Modeling & SQLAdvanced SQL skills for transformations, aggregations, and analytics-ready datasetsStrong understanding of dimensional modeling (star/snowflake schemas)DevOps & CI/CDExperience with Git-based workflows and CI/CD pipelinesFamiliarity with Azure DevOps or GitHub Actions for deploying Databricks assetsRequired Skills & Qualifications8–12 years of experience in data engineering and architectureProven expertise in Databricks and Azure-based data platformsStrong problem-solving and analytical skillsExcellent communication and stakeholder management abilitiesExperience in manufacturing domain is highly preferredPreferred CertificationsMicrosoft Certified: Azure Solutions Architect ExpertMicrosoft Certified: Azure Data Engineer AssociateDatabricks Certified Data Engineer (Associate/Professional)Databricks Hadoop Migration Architect (or equivalent)Staffworxs is an Equal Opportunity Employer. We celebrate diversity and are committed to creating an inclusive workplace for all employees, regardless of race, color, religion, gender, sexual orientation, national origin, age, disability, or veteran status.