JOBSEARCHER

Lead Data Engineer

Lead Data Engineer / Architect (Azure)
Relevant Experience required:
  • 12+ years Responsibilities
  • Design, build, and maintain scalable ETL/ELT pipelines using Azure Data Factory, Azure Synapse, and Azure Databricks.
  • Architect and manage secure enterprise cloud storage, including ADLS Gen2, Delta Lake, and relational/NoSQL databases.
  • Ingest and process both high-volume batch data and real-time streaming workloads.
  • Optimize database queries, PySpark jobs, and data pipeline performance to minimize execution time and cloud costs.
  • Collaborate with BI and analytics teams to model dimensional data marts (Star/Snowflake schemas) for reporting.
  • Implement data governance, automated data quality validation, and end-to-end lineage tracking (e.g., via Microsoft Purview).Secure data pipelines and storage using Azure Role-Based Access Control (RBAC), Key Vaults, and encryption standards.
  • Automate infrastructure deployments and code releases using Azure DevOps, GitHub Actions, and CI/CD best practices.