JOBSEARCHER

Lead Data Engineer

Lead Data Engineer / Architect Candidate Preference: This team operates on a onsite schedule out of our Diamond Bar, CA facility. To ensure seamless in-person collaboration and team alignment, we are seeking local or nearby Greater Los Angeles/Inland Empire-based candidates.
Relevant Experience required:
  • 12+ years Responsibilities
  • Design, build, and maintain scalable ETL/ELT pipelines using Azure Data Factory, Azure Synapse, and Azure Databricks.
  • Architect and manage secure enterprise cloud storage, including ADLS Gen2, Delta Lake, and relational/NoSQL databases.
  • Ingest and process both high-volume batch data and real-time streaming workloads.
  • Optimize database queries, PySpark jobs, and data pipeline performance to minimize execution time and cloud costs.
  • Collaborate with BI and analytics teams to model dimensional data marts (Star/Snowflake schemas) for reporting.
  • Implement data governance, automated data quality validation, and end-to-end lineage tracking (e.g., via Microsoft Purview).Secure data pipelines and storage using Azure Role-Based Access Control (RBAC), Key Vaults, and encryption standards.
  • Automate infrastructure deployments and code releases using Azure DevOps, GitHub Actions, and CI/CD best practices.