Azure Databricks Data Engineer
Job Title: Senior Azure Databricks Data EngineerExperience: 9-15 Years of IT ExperienceLocation: Warren, MI (Hybrid)Job SummaryWe are seeking a skilled Databricks Engineer with minimum of 5+ years of hands-on in Databricks who can design, develop, and optimize scalable data platforms and analytics solutions using the Databricks Lakehouse Platform. The ideal candidate will have expertise in data engineering, ETL/ELT development, cloud technologies preferably Azure, and big data processing to support enterprise analytics and AI initiatives.Key ResponsibilitiesDesign, develop, and maintain data pipelines using Databricks, Apache Spark, and cloud-native services.Build and optimize ETL/ELT workflows for large-scale structured and unstructured data.Develop data models and implement data quality, validation, and governance frameworks.Integrate data from multiple sources into a unified Lakehouse architecture.Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency.Implement security controls, access management, and data governance using Unity Catalog.Collaborate with business, analytics, and AI/ML teams to deliver trusted data products.Monitor, troubleshoot, and resolve data pipeline issues.Support CI/CD, DevOps, and infrastructure automation practices.Maintain technical documentation and best practices.Required Technical SkillsCore TechnologiesDatabricks Lakehouse PlatformApache Spark / PySparkDelta LakeSQLPythonData EngineeringETL / ELT DevelopmentData ModelingData WarehousingData Quality & ValidationStreaming & Real-Time ProcessingGovernance & SecurityUnity CatalogData LineageRow-Level SecurityAccess Control & ComplianceData Governance FrameworksCloud & DevOpsAzure / AWS / GCPTerraformGitHub Actions / Azure DevOpsCI/CD PipelinesAnalytics & AISemantic LayersData ProductsBI PlatformsMachine Learning SupportGenerative AI & RAG ArchitecturesRequired QualificationsBachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field.9-15 years of experience in Data Engineering and Data Warehousing.Minimum 5+ years of hands-on experience with Azure Data Engineering technologies.Minimum 4+ years of hands-on experience with Azure Databricks and Spark ecosystem.Strong understanding of data lake, lakehouse, and cloud-native architecture patterns.Experience in handling large-scale structured and unstructured datasets.Strong analytical, problem-solving, and troubleshooting skills.Preferred QualificationsMicrosoft Certified: Azure Data Engineer Associate (DP-203).Databricks Certified Data Engineer Associate/Professional.Experience with Snowflake, Power BI, or Microsoft Fabric.Experience in real-time streaming solutions using Kafka/Event Hubs.Exposure to Data Governance and Master Data Management initiatives.Soft SkillsStrong stakeholder management and communication skills.Ability to lead technical initiatives and drive architecture discussions.Experience working in Agile/Scrum environments.Excellent documentation and presentation skills.Strong mentoring and team leadership abilities.Nice to HaveMicrosoft FabricPower BIAzure Event HubsKafkaMachine Learning data pipelinesData Governance tools such as Purview