Databricks & Data Engineering
Role Descriptions: Design| develop| and maintain scalable| high-performance data engineering solutions on Azure Databricks.Build and support batch and real-time data pipelines using Apache Spark and Structured Streaming.Design and implement Lakehouse and Medallion Architecture (Bronze| Silver| Gold) using Delta Lake.Develop reusable ETL/ELT frameworks| standardized data processing components| and enterprise data ingestion patterns.Ingest and process large-scale data from diverse enterprise sources| including databases| APIs| message queues| and file-based systems. Develop complex data transformation logic using PySpark| Python| and SQL.Create trusted| governed| and consumption-ready datasets for reporting| analytics| and downstream applications.Implement data quality| validation| reconciliation| audit| and governance controls across data pipelines.Manage and optimize Databricks Workflows| Delta Live Tables (DLT)| Unity Catalog| and Databricks SQL.Collaborate with business| analytics| architecture| and delivery teams in an Agile environment to deliver high-quality data products.RequiredTechnical Skills: Databricks & Data EngineeringStrong hands-on experience with Databricks| Lakehouse Architecture| and Delta LakeExpertise in PySpark| Python| and advanced SQLExperience with Apache Spark| Structured Streaming| Delta Live Tables (DLT)| Databricks Workflows| and Unity CatalogExperience designing and implementing Medallion Architecture and modern Data Lake/Data Warehouse solutionsStrong background in data ingestion| ETL/ELT| data transformation| optimization| and deployment practicesExperience with event-driven and streaming technologies such as Kafka or Azure Event HubCloud PlatformsMicrosoft Azure (preferred)AWS or GCP experience is also valuableApplication IntegrationWorking knowledge of React and/or Angular for front-end integration and data-centric application developmentDomain Experience (Preferred)Experience in Finance| Insurance| Regulatory Reporting| or US GAAP reportingUnderstanding of financial data models| reporting processes| and governance requirements is highly desirable Essential Skills: Design| develop| and maintain scalable| high-performance data engineering solutions on Azure Databricks.Build and support batch and real-time data pipelines using Apache Spark and Structured Streaming.Design and implement Lakehouse and Medallion Architecture (Bronze| Silver| Gold) using Delta Lake.Develop reusable ETL/ELT frameworks| standardized data processing components| and enterprise data ingestion patterns.Ingest and process large-scale data from diverse enterprise sources| including databases| APIs| message queues| and file-based systems. Develop complex data transformation logic using PySpark| Python| and SQL.Create trusted| governed| and consumption-ready datasets for reporting| analytics| and downstream applications.Implement data quality| validation| reconciliation| audit| and governance controls across data pipelines.Manage and optimize Databricks Workflows| Delta Live Tables (DLT)| Unity Catalog| and Databricks SQL.Collaborate with business| analytics| architecture| and delivery teams in an Agile environment to deliver high-quality data products.RequiredTechnical Skills: Databricks & Data EngineeringStrong hands-on experience with Databricks| Lakehouse Architecture| and Delta LakeExpertise in PySpark| Python| and advanced SQLExperience with Apache Spark| Structured Streaming| Delta Live Tables (DLT)| Databricks Workflows| and Unity CatalogExperience designing and implementing Medallion Architecture and modern Data Lake/Data Warehouse solutionsStrong background in data ingestion| ETL/ELT| data transformation| optimization| and deployment practicesExperience with event-driven and streaming technologies such as Kafka or Azure Event HubCloud PlatformsMicrosoft Azure (preferred)AWS or GCP experience is also valuableApplication IntegrationWorking knowledge of React and/or Angular for front-end integration and data-centric applResponsibilitiesDesign, develop, and maintain scalable, high-performance data engineering solutions on Azure Databricks.Build and support batch and real-time data pipelines using Apache Spark and Structured Streaming.Design and implement Lakehouse and Medallion Architecture (Bronze, Silver, Gold) using Delta Lake.Develop reusable ETL/ELT frameworks, standardized data processing components, and enterprise data ingestion patterns.Ingest and process large-scale data from diverse enterprise sources, including databases, APIs, message queues, and file-based systems.Develop complex data transformation logic using PySpark, Python, and SQL.Create trusted, governed, and consumption-ready datasets for reporting, analytics, and downstream applications.Implement data quality, validation, reconciliation, audit, and governance controls across data pipelines.Manage and optimize Databricks Workflows, Delta Live Tables (DLT), Unity Catalog, and Databricks SQL.Collaborate with business, analytics, architecture, and delivery teams in an Agile environment to deliver high-quality data products. Required Technical Skills: Databricks & Data EngineeringStrong hands-on experience with Databricks, Lakehouse Architecture, and Delta LakeExpertise in PySpark, Python, and advanced SQLExperience with Apache Spark, Structured Streaming, Delta Live Tables (DLT), Databricks Workflows, and Unity CatalogExperience designing and implementing Medallion Architecture and modern Data Lake/Data Warehouse solutionsStrong background in data ingestion, ETL/ELT, data transformation, optimization, and deployment practicesExperience with event-driven and streaming technologies such as Kafka or Azure Event Hub Cloud PlatformsMicrosoft Azure (preferred)WS or GCP experience is also valuable Application IntegrationWorking knowledge of React and/or Angular for front-end integration and data-centric application development Domain Experience (Preferred)Experience in Finance, Insurance, Regulatory Reporting, or US GAAP reportingUnderstanding of financial data models, reporting processes, and governance requirements is highly desirable