Senior Data Engineer

Overview Senior Data Engineer at LABUR client, you will design and operate scalable data ingestion and transformation pipelines using Databricks, Azure Data Factory, and PySpark. You’ll drive data lake construction, governance, and AI-enabled optimization while collaborating with business users to translate needs into production-ready solutions. This role emphasizes autonomy, strong communication, and hands-on leadership in migrating and modernizing data platforms. You’ll contribute to a regulated, life sciences-focused environment with a strong emphasis on data quality and discoverability. ResponsibilitiesDesign, develop, and maintain scalable ingestion and transformation pipelines using Azure Data Factory, Databricks, and PySparkMonitor daily scheduled jobs, troubleshoot failures, and resolve data issues promptlyEnforce data governance, data quality standards, and coding best practicesCollaborate with business and technical stakeholders to translate requirements into scalable solutionsIdentify and implement opportunities to optimize pipelines and data lake infrastructure leveraging AIImplement and manage data catalog solutions using Unity Catalog for reliable data productsLead hands-on integration work in Informatica IDMC, including migrations from PowerCenter to IDMCLead data analysis and solution design efforts aligning cross-functional stakeholders around a shared visionEmbed governance principles into every stage of the data delivery lifecycle Key requirements8+ years of hands-on data engineering, analytics engineering, or related rolesStrong proficiency with Databricks (Delta Lake, PySpark, Unity Catalog)Experience with Informatica; IDMC implementation strongly preferredProficiency in SQL and PythonSolid experience with Azure Data Factory for orchestration and schedulingExperience in life sciences or other highly regulated industries preferredAbility to apply AI to solve business problems and optimize data lifecycleExcellent communication skills and self-directed work styleexcellent communicationself-directed / autonomous work styleability to explain technical issues to business stakeholdersDatabricksDelta LakePySpark