Data Engineer

Senior Databricks Data Engineer (Azure)Designed, developed, and optimized scalable data pipelines on the Databricks Lakehouse platform using Azure, Python, PySpark, Spark SQL, and Delta Lake to support enterprise analytics and reporting.Modernized cloud-based data ecosystems by implementing distributed data engineering solutions, data modeling, security best practices, CI/CD automation, and cloud-native architecture.Built and maintained robust ETL/ELT pipelines using Databricks notebooks, Apache Airflow, Databricks Workflows, and Delta Live Tables to automate data ingestion and processing.Developed scalable ingestion frameworks for APIs, relational databases, files, streaming sources, and MDM systems, including REST API integrations and workflow automation.Implemented Unity Catalog to enforce data governance, role-based access control (RBAC), data lineage, metadata management, and secure data sharing across enterprise platforms.Designed and optimized Delta Lake data models to improve data quality, performance, scalability, and reliability for downstream business intelligence, analytics, and data science workloads.Wrote high-performance SQL/T-SQL queries, stored procedures, and curated datasets while supporting reporting, dashboarding, and enterprise data warehousing initiatives.Automated deployments, testing, and infrastructure configuration through DevOps and CI/CD pipelines to improve development efficiency and platform reliability.Worked extensively with financial and regulated data, ensuring compliance with governance standards while delivering secure, enterprise-grade data solutions.Collaborated with cross-functional teams to deliver end-to-end data engineering initiatives, leveraging Azure, Databricks, and Snowflake to build modern, scalable cloud data platforms.