ETRM Databricks Data Engineer

Job Title: ETRM Databricks Data EngineerLocation: Houston, TexasJob Description:Job Summary: ETRM Databricks Data Engineer to design, develop, and maintain scalable data solutions that support business intelligence, analytics, and data-driven decision-making with understanding of Data Domain & Data Assets in ETRM Domain. The ideal candidate will have strong expertise in Databricks, Apache Spark, Azure Cloud platforms, and modern data engineering practices.Key Responsibilities:Design, develop, and optimize data pipelines using Databricks & ADF.Must possess good understanding of Trading Domain knowledge.Build and maintain scalable ETL/ELT processes for structured and unstructured data.Integrate data from multiple sources into enterprise data platforms.Develop and manage data lakes, lakehouses, and data warehouses.Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.Ensure data quality, integrity, security, and governance across data platforms.Monitor and troubleshoot data processing jobs and optimize performance.Implement CI/CD practices and automate data engineering workflows.Create and maintain technical documentation, data models, and process flows.Support cloud-based data solutions and migration initiatives.Required Qualifications:• Hands-on experience with Databricks and Azure.• Strong proficiency in Python, SQL, and PySpark.• Experience with cloud platforms such as Azure• Knowledge of data warehousing, data modeling, and ETL/ELT concepts.• Experience with version control tools such as Git.• Strong analytical, problem-solving, and communication skills.• Trading Domain experience related for Crude, Refined Products, Natural gas is a must.Preferred Qualifications:• Experience with Azure Data Factory, Azure Synapse Analytics, Delta Lake, or similar technologies.• Knowledge of streaming data processing and real-time analytics.• Familiarity with DevOps practices and CI/CD pipelines.• Databricks, Azure, or related cloud certifications.