Data Engineer

Job DescriptionWe are seeking a Data Engineer with strong experience with data wrangling and building enterprise data pipelines including ingestion, transformation and management.Responsibilities:Design, develop, and maintain end-to-end data pipelines using data ingestion tools and Azure DatabricksImplement scalable data ingestion frameworks (batch and incremental) from diverse data sources (APIs, databases, files, cloud storage)Build and optimize data transformation logic using Spark (Databricks), SQL, and PythonDevelop and maintain data models supporting analytics, reporting, and machine learning use casesCollaborate with analytics teams to enable Power BI datasets and reporting solutionsEnsure data quality, reliability, and consistency through validation frameworks and monitoringOptimize performance of pipelines, storage, and compute resources in AzureImplement data governance practices, including lineage, cataloging, and documentationSupport CI/CD deployment pipelines for data engineering workflowsMinimum Qualifications5+ years of experience in Data Engineering rolesStrong hands-on experience with data ingestion / ETL / ELT tools like Azure Data Factory and data modellingProficient in using Azure Databricks (PySpark/Spark SQL) and medallion architectureExperience with other SQL databases, relational databases and NoSQL databasesUnderstanding of data lake and data warehouse architectures (e.g., Azure Data Lake, One Lake, Delta Tables) Experience working with large-scale structured and semi-structured datasetsAbility to work with cross-functional teams and translate business requirements into technical solutions