{"schemaVersion":"jobsearcher.job.v1","id":"fc043e509d630077ae63faad","url":"https://jobsearcher.com/jobs/fc043e509d630077ae63faad","canonicalUrl":"https://jobsearcher.com/jobs/fc043e509d630077ae63faad","title":"Lead Data Platform enginner","description":"Job Summary\n\nWe are seeking an experienced *Lead Data Platform Engineer* to lead the design, development, and optimization of an enterprise-scale data platform. The ideal candidate will drive technical strategy, establish engineering best practices, and collaborate with cross-functional teams to deliver scalable, secure, and high-performing data solutions across the *Hadoop ecosystem and Azure, including Databricks*.\n\nKey Responsibilities\n\n* Lead the architecture, development, and enhancement of large-scale data platforms supporting analytics, machine learning, and operational workloads.\n* Design and optimize big data solutions using Hadoop ecosystem technologies including *HDFS, Hive, Spark, YARN, and related components*.\n* Develop and manage data pipelines and transformations using *Azure Data Lake Storage, Azure Data Factory, Azure Synapse, and Azure Databricks*.\n* Implement and enforce robust *data governance, security, and data quality frameworks* across all data layers.\n* Partner with data engineering, analytics, product, and infrastructure teams to translate business requirements into scalable technical solutions.\n* Drive *performance tuning, capacity planning, and cost optimization* across on-premises and cloud-based data platforms.\n* Mentor and technically guide data engineers while establishing engineering standards, reusable patterns, and best practices.\n* Oversee *CI/CD, deployment, and monitoring* for data workflows.\n* Evaluate emerging technologies and contribute to long-term *data platform modernization and technology strategy*.\n\nRequired Qualifications\n\n* *8+ years* of experience in data engineering or data platform roles, including *3+ years in a technical lead or architect capacity*.\n* Strong hands-on experience with the *Hadoop ecosystem*, including HDFS, Hive, Spark, Oozie, Ranger, Airflow, and related technologies.\n* Deep expertise in *Azure Data Services*, including:\n* Azure Data Lake Storage\n* Azure Data Factory\n* Azure Synapse\n* Azure Functions\n* Azure Key Vault\n* Advanced hands-on experience with *Databricks*, including:\n* Apache Spark optimization\n* Delta Lake\n* Unity Catalog\n* Strong proficiency in *Python and SQL* and distributed data processing frameworks.\n* Experience with *DevOps, CI/CD pipelines, and Infrastructure as Code*, such as Terraform or ARM.\n* Strong understanding of *data modeling, storage formats, and data governance*, including Parquet, ORC, and Delta.\n* Proven ability to lead technical teams, communicate effectively, and influence architecture decisions.\n\nPreferred Qualifications\n\n* Experience migrating *on-premises Hadoop workloads to cloud platforms*, preferably Azure Databricks.\n* Knowledge of *real-time data processing technologies*, including:\n* Kafka\n* Azure Event Hubs\n* Spark Streaming\n\nPay: $55.00 - $65.00 per hour\n\nWork Location: In person","company":"Cloudsecurityweb","rawCompany":"cloudsecurityweb","city":"Ontario","state":"CA","isRemote":false,"isActive":false,"createdAt":"2026-09-26T08:59:56.836Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Lead Data Platform enginner","description":"Job Summary\n\nWe are seeking an experienced *Lead Data Platform Engineer* to lead the design, development, and optimization of an enterprise-scale data platform. The ideal candidate will drive technical strategy, establish engineering best practices, and collaborate with cross-functional teams to deliver scalable, secure, and high-performing data solutions across the *Hadoop ecosystem and Azure, including Databricks*.\n\nKey Responsibilities\n\n* Lead the architecture, development, and enhancement of large-scale data platforms supporting analytics, machine learning, and operational workloads.\n* Design and optimize big data solutions using Hadoop ecosystem technologies including *HDFS, Hive, Spark, YARN, and related components*.\n* Develop and manage data pipelines and transformations using *Azure Data Lake Storage, Azure Data Factory, Azure Synapse, and Azure Databricks*.\n* Implement and enforce robust *data governance, security, and data quality frameworks* across all data layers.\n* Partner with data engineering, analytics, product, and infrastructure teams to translate business requirements into scalable technical solutions.\n* Drive *performance tuning, capacity planning, and cost optimization* across on-premises and cloud-based data platforms.\n* Mentor and technically guide data engineers while establishing engineering standards, reusable patterns, and best practices.\n* Oversee *CI/CD, deployment, and monitoring* for data workflows.\n* Evaluate emerging technologies and contribute to long-term *data platform modernization and technology strategy*.\n\nRequired Qualifications\n\n* *8+ years* of experience in data engineering or data platform roles, including *3+ years in a technical lead or architect capacity*.\n* Strong hands-on experience with the *Hadoop ecosystem*, including HDFS, Hive, Spark, Oozie, Ranger, Airflow, and related technologies.\n* Deep expertise in *Azure Data Services*, including:\n* Azure Data Lake Storage\n* Azure Data Factory\n* Azure Synapse\n* Azure Functions\n* Azure Key Vault\n* Advanced hands-on experience with *Databricks*, including:\n* Apache Spark optimization\n* Delta Lake\n* Unity Catalog\n* Strong proficiency in *Python and SQL* and distributed data processing frameworks.\n* Experience with *DevOps, CI/CD pipelines, and Infrastructure as Code*, such as Terraform or ARM.\n* Strong understanding of *data modeling, storage formats, and data governance*, including Parquet, ORC, and Delta.\n* Proven ability to lead technical teams, communicate effectively, and influence architecture decisions.\n\nPreferred Qualifications\n\n* Experience migrating *on-premises Hadoop workloads to cloud platforms*, preferably Azure Databricks.\n* Knowledge of *real-time data processing technologies*, including:\n* Kafka\n* Azure Event Hubs\n* Spark Streaming\n\nPay: $55.00 - $65.00 per hour\n\nWork Location: In person","datePosted":"2026-09-26T08:59:56.836Z","dateModified":"2026-09-26T08:59:56.836Z","hiringOrganization":{"@type":"Organization","name":"Cloudsecurityweb","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Ontario","addressRegion":"CA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"fc043e509d630077ae63faad"},"url":"https://jobsearcher.com/jobs/fc043e509d630077ae63faad"}}