Lead Data Platform Engineer
Job Title: Lead Data Platform EngineerLocation: Irvine, CA (Onsite)Employment Type: W2 OnlyJob SummaryWe are seeking an experienced Lead Data Platform Engineer to design, build, and manage enterprise-scale cloud-native data platforms. The ideal candidate will have extensive experience with Databricks, AWS, Terraform, DevOps, and Platform Engineering, along with a strong background in building secure, scalable, and highly available data platforms. This role requires close collaboration with development, infrastructure, security, and operations teams to deliver reliable and efficient data platform solutions.Key ResponsibilitiesDesign, build, and manage enterprise-scale cloud-native data platforms.Lead Databricks platform administration, engineering, and operational activities.Design and implement Lakehouse Architecture using Delta Lake.Manage Databricks Workflows, Unity Catalog, security configurations, and access controls.Optimize platform performance, scalability, availability, and cost efficiency.Implement governance controls across development, testing, and production environments.Design and deploy AWS-based infrastructure using services such as S3, Glue, Lambda, ECS, EKS, CloudWatch, IAM, Secrets Manager, and VPC.Develop Infrastructure as Code (IaC) solutions using Terraform.Create reusable Terraform modules for infrastructure provisioning.Design, build, and maintain CI/CD pipelines for automated application and infrastructure deployments.Improve deployment automation and platform reliability using DevOps best practices.Utilize GitHub Actions, GitLab CI/CD, Azure DevOps, Jenkins, Docker, and Kubernetes.Implement monitoring, observability, logging, and incident management processes.Perform root cause analysis and implement preventive measures to improve platform stability.Collaborate with engineering, infrastructure, security, and operations teams to deliver secure, scalable, and highly available platform solutions.Ensure compliance with enterprise security, governance, and operational standards.Provide technical leadership, best practices, and mentoring to engineering teams.Required QualificationsBachelor's degree in Computer Science, Information Technology, Engineering, or a related field.10+ years of experience in Platform Engineering, Cloud Engineering, Data Engineering, or a related field.Hands-on experience designing, implementing, and supporting enterprise-scale cloud-native data platforms.Strong expertise in Databricks platform administration and Lakehouse Architecture.Extensive experience with AWS cloud infrastructure and Infrastructure as Code (Terraform).Strong understanding of DevOps, Cloud Security, Infrastructure Automation, and Platform Operations.Experience with containerization technologies such as Docker and Kubernetes.Strong scripting/programming experience using Python.Excellent analytical, troubleshooting, communication, and collaboration skills.Required SkillsData PlatformDatabricks AdministrationLakehouse ArchitectureDelta LakeUnity CatalogdbtCloud & InfrastructureAWS Cloud ServicesTerraformInfrastructure as Code (IaC)DevOps & CI/CDGitHub ActionsGitLab CI/CDAzure DevOpsJenkinsDockerKubernetesCI/CD PipelinesDevOpsProgrammingPythonPlatform OperationsMonitoring & ObservabilitySecurity & Access ManagementPlatform EngineeringCloud EngineeringPreferred QualificationsExperience building highly available enterprise data platforms.Experience with cloud governance, security, and compliance.Strong understanding of platform reliability engineering (SRE) principles.Experience supporting large-scale data analytics and AI/ML platforms.Ability to lead platform modernization and infrastructure automation initiatives