Lead Databricks Data Engineer

Job TitleWe are CirrusLabs. Our vision is to become the world's most sought-after niche digital transformation company that helps customers realize value through innovation. Our mission is to co-create success with our customers, partners, and community. Our goal is to enable employees to dream, grow, and make things happen. We are committed to excellence. We are a dependable partner organization that delivers on commitments. We strive to maintain integrity with our employees and customers. Every action we take is driven by value. The core of who we are is through our well-knit teams and employees. You are the core of a values-driven organization. You have an entrepreneurial spirit. You enjoy working as a part of well-knit teams. You value the team over the individual. You welcome diversity at work and within the greater community. You aren't afraid to take risks. You appreciate a growth path with your leadership team that journeys how you can grow inside and outside of the organization. You thrive upon continuing education programs that your company sponsors to strengthen your skills and for you to become a thought leader ahead of the industry curve. You are excited about creating change because your skills can help the greater good of every customer, industry, and community. We are hiring a talented J Lead Databricks Data Engineer.Location: Atlanta, GA (Onsite)Certification preferred: Databricks Certified ProfessionalJob DescriptionDesigns, builds, and manages scalable data platforms, leading teams to create robust, cloud-based data pipelines (ETL/ELT) using tools like Python, PySpark, SQL, Databricks (Lakehouse, Delta Lake, Unity Catalog), and cloud services (AWS/Azure/GCP). Key responsibilities include technical leadership, mentoring junior engineers, ensuring data quality/governance, optimizing performance (Spark optimizations, Medallion Architecture), stakeholder communication, and driving DataOps practices for data-driven insights.Key ResponsibilitiesTechnical Leadership: Own architecture, set standards, guide technical direction, and lead complex projects.Pipeline Development: Design, build, and optimize scalable data pipelines (ingestion, transformation, storage) on Databricks.Mentorship: Coach and develop data engineers, conduct code reviews, and promote best practices.Architecture: Implement Data Lakehouse (Bronze/Silver/Gold) and Medallion Architecture patterns.Performance Tuning: Identify and resolve bottlenecks, optimize Spark configurations, and manage cluster optimization.Collaboration: Work with architects, product managers, and business stakeholders to translate requirements into solutions.Data Governance: Ensure data quality, integrity, security, and compliance.DataOps/CI/CD: Implement modern engineering practices for automated, reliable data flows.Required Skills & ExperienceDatabricks Expertise: Deep knowledge of the platform (Lakehouse, Delta Lake, Unity Catalog, SQL).Programming: Python, PySpark, Scala, SQL.Cloud Platforms: AWS, Azure, or GCP.Big Data Technologies: Spark, Data Warehousing, ETL/ELT.Leadership: Proven experience leading technical initiatives and teams.Data Modeling: Strong understanding of data modeling concepts.Desirable SkillsApache Airflow, Kafka, CI/CD tools (Jenkins), orchestration.Experience in specific domains (e.g., Healthcare).DataOps, MDM, data governance.