Lead Data Architect (Databricks)

Lead Data Architect (Databricks) Lead Data Architect (Databricks) Company Overview We are a rapidly leading healthcare technology company specializing in payment coordination. We partner with health plans/providers to identify, recover, and prevent payment errors. We are a fast-moving, collaborative environment that embraces AI and the latest technologies. Position Overview We are seeking a Lead Data Architect with deep experience on Databricks and Azure to design, build, and govern an enterprise-scale data platform. The role will lead architecture and delivery of modern analytics and data engineering solutions, drive data model and ingestion strategy, ensure performance, security and cost efficiency, and partner with engineering, product and business stakeholders to deliver high-impact data products. Strong SQL skills and hands-on experience with Databricks and Azure are required; healthcare domain experience and familiarity with Data Factory/ADF, Python and platform rationalization are highly desirable. Key Responsibilities Lead end-to-end design and delivery of a Databricks-based data platform on Azure, including architecture, patterns, and best practices. Define and maintain logical and physical data models, schema design, and data contracts to support analytics, ML and reporting use cases. Architect and oversee ingestion pipelines and ETL/ELT workflows (batch and streaming), ensuring robustness, scalability and data quality. Collaborate with engineering, product and business teams to translate requirements into reusable data products and APIs. Establish and enforce data governance, security, access controls and compliance (including healthcare/HIPAA where applicable). Optimize performance and cost across Databricks, storage (ADLS) and SQL workloads; tune Spark jobs, queries and cluster configuration. Drive platform rationalization and delivery: evaluate tooling, reduce technical debt, and standardize patterns across projects. Mentor and lead engineers and data professionals; run architecture reviews, code reviews and support onboarding. Implement CI/CD, automated testing, monitoring, observability and operational runbooks for data processes. Document architecture, operational procedures, SLAs and data lineage; communicate designs and trade-offs to stakeholders. Qualifications Proven experience as a Data Architect or Lead Data Engineer with demonstrable architecture ownership in enterprise environments. Strong hands-on experience with Databricks (Spark), Delta Lake and Databricks best practices. Deep Azure platform knowledge (ADLS, Azure Databricks, Azure SQL, Azure Data Factory/ADF) and cloud networking/security patterns. Expert SQL skills for data modeling, query optimization and analytics. Experience designing and implementing ingestion pipelines and ETL/ELT processes (batch and streaming). Proficiency in Python for data engineering and automation tasks. Experience with platform rationalization, delivery leadership, and reducing technical debt across data ecosystems. Familiarity with healthcare data standards, privacy and compliance (nice-to-have). Experience with Agent Bricks or similar orchestration/agent-based tooling is a plus. Excellent communication, stakeholder management and mentorship skills; ability to translate business needs into technical solutions. Bachelors degree in Computer Science, Engineering, Information Systems or equivalent practical experience; 5+ years in data architecture/engineering roles preferred. Benefits Medical Dental Vision 401k w/ match Remote PTO - For this position, you must be currently authorized to work in the United States without the need for sponsorship for a non-immigrant visa. This job was first posted by CyberCoders on 09/25/2026 and applications will be accepted on an ongoing basis until the position is filled or closed.Everforth CyberCoders is proud to be an Equal Opportunity Employer All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, age, sexual orientation, gender identity or expression, national origin, ancestry, citizenship, genetic information, registered domestic partner status, marital status, status as a crime victim, disability, protected veteran status, or any other characteristic protected by law. Our hiring process includes AI screening for keywords and minimum qualifications, and a virtual recruiter as part of the application process. A human recruiter reviews all results. Click here for details on our virtual recruiter . Everforth CyberCoders will consider qualified applicants with criminal histories in a manner consistent with the requirements of applicable state and local law, including but not limited to the Los Angeles County Fair Chance Ordinance, the San Francisco Fair Chance Ordinance, and the California Fair Chance Act. Everforth CyberCoders is committed to working with and providing reasonable accommodation to individuals with physical and mental disabilities. Individuals needing special assistance or an accommodation while seeking employment can contact a member of our Human Resources team at Benefits@CyberCoders.com to make arrangements. Salary $180000 - $200000 USD per year