Azure Data Engineer

Role Descriptions:Design and build batch and streaming data pipelines using Azure, Databricks, and Kafka.Develop ETL/ELT solutions using Spark, SQL and Python/Pyspark, or ScalaImplement and support Data Lake / Lakehouse architecturesIntegrate data from legacy and modern platforms. Preferred postgres DB migration to Azure Cloud.Expertise in handling Data migration using API’s.Apply data quality checks and support data governance basics (metadata, lineage, access controls)Optimize Spark jobs for performance, reliability, and costCollaborate with cross-functional teams, Product owners, Admin teams, BSAs, Scrum & Project managersRequired Skills & Experience:8+ years of experience in data engineering in azure using Azure Databricks.Strong experience with Azure Data Lake, Databricks, ADF, Postgres.Strong Data Engineering experience in Big Data (Cloudera)Hands-on expertise in SQL, Python and PySpark or ScalaProven experience with batch & real-time ingestionExperience with Kafka and big data processing Added good to have experience with:Informatica PowerCenterInformatica IDMC or equivalent cloud data integration toolsSQL Server, Oracle, Azure SQLCosmos DBUnix/LinuxMaestro (or similar enterprise schedulers)Data tools such as Precisely or equivalent Added AdvantageInsurance domain knowledge (Policy, Claims, Billing, Underwriting, Actuarial, Regulatory data)Experience with Azure DevOps (AzDO) and GitHub-based CI/CDPractical usage of Copilot for data engineering and development productivityExposure to Generative AI / GenAI use cases in data platforms and analytics