Senior Microsoft Fabric Data Engineer
Job Title: Sr Microsoft Fabric Data Engineer
Location: Dallas TX (Can be Remote)
ROLE SUMMARY:
Design and build enterprise-scale Microsoft Fabric data platforms with a strong focus on metadata-driven ingestion and orchestration frameworks. Lead the development of reusable, configuration-driven data integration solutions capable of dynamically executing Microsoft Fabric resources including Data Pipelines, Notebooks, Dataflows Gen2, Lakehouses, Warehouses, and Semantic Models. Establish and govern Fabric Medallion Architecture (Bronze/Silver/Gold), implement scalable framework standards, and support migration of enterprise Tableau dashboards from on-prem SQL Server (Epic Clarity ~90%) to governed Microsoft Fabric data products.
KEY RESPONSIBILITIES:
- Design and implement metadata-driven ingestion frameworks in Microsoft Fabric to onboard and manage multiple source systems through centralized configurations.
- Build dynamic orchestration frameworks capable of executing Fabric resources including Data Pipelines, Spark Notebooks, Dataflows Gen2, Lakehouses, Warehouses, and Semantic Models based on metadata and business rules.
- Develop reusable and parameter-driven Fabric Data Factory pipelines to support scalable ingestion, transformation, dependency management, auditing, and monitoring.
- Implement framework capabilities for full loads, incremental loads, CDC processing, restartability, error handling, and automated recovery mechanisms.
- Design and maintain execution control tables, metadata repositories, and dependency frameworks to support enterprise-scale orchestration and workload management.
- Stand up the Fabric Medallion Architecture using a hybrid model: Bronze and Silver in the Lakehouse (Delta), Gold in the Fabric Warehouse (T-SQL).
- Build Bronze and Silver layers in the Lakehouse using Spark/PySpark Notebooks and Data Factory Pipelines (source-aligned Bronze; standardized, reusable Silver).
- Build the Gold layer in the Fabric Warehouse as enterprise dimensional models (facts, dimensions, aggregations, KPIs) using T-SQL, loading from Silver via Data Factory pipelines, Spark Notebooks, or Dataflow Gen2.
- Translate parsed Tableau embedded/custom SQL into reusable Silver business logic and Gold fact/dimension data products.
- Build Gold dimensional models and Fabric Semantic Models / Direct Lake datasets that replace embedded SQL and serve as enterprise Tableau published data sources.
- Act as the human-in-the-loop reviewer on automated Bronze/Silver/Gold recommendations by evaluating reusable business concepts, existing Fabric assets, and fact vs dimension vs report-specific mart classifications.
- Implement governance and security including Microsoft Purview cataloging, lineage tracking, sensitivity labels, RLS/OLS, and PHI/HIPAA compliance controls.
- Establish and enforce enterprise standards for metadata management, orchestration, naming conventions, medallion architecture, and governance processes.
- Implement CI/CD using Azure DevOps, Fabric Git Integration, and Deployment Pipelines.
- Lead data reconciliation and validation activities against source systems, partnering with business owners on automated comparisons and sign-off processes.
- Collaborate with Tableau teams to repoint existing dashboards and published data sources to Fabric-based Lakehouse, Warehouse, and Semantic Model assets.
REQUIRED QUALIFICATIONS:
- 12+ years of Data Engineering and Data Warehousing experience, including lead-level ownership of enterprise data platform architecture.
- 3+ years of hands-on Microsoft Fabric experience (Lakehouse, Warehouse, Data Factory, Spark Notebooks, Dataflows Gen2, Direct Lake, Semantic Models) or equivalent Azure Synapse/ADF experience with recent Fabric delivery.
- Extensive experience building metadata-driven ingestion and orchestration frameworks for enterprise data platforms.
- Strong experience developing dynamic, configuration-driven ETL/ELT frameworks capable of orchestrating Fabric workloads at scale.
- Proven experience implementing dependency-based job orchestration, execution control frameworks, and operational monitoring solutions.
- Expert T-SQL skills with proven Fabric Warehouse (or Azure SQL DW / Dedicated SQL Pool) delivery of enterprise dimensional Gold models.
- Strong Spark/PySpark expertise for Lakehouse Bronze/Silver transformations and proven Medallion Architecture implementation experience.
- Strong Python development experience for framework automation, metadata processing, orchestration, and platform engineering.
- Strong dimensional modeling (Kimball) expertise for Warehouse-based fact and dimension design.
- Hands-on experience with Microsoft Purview, lineage management, sensitivity labels, and row-level/object-level security implementation.
- Experience implementing CI/CD for data platforms using Azure DevOps, Git-based deployment, and Fabric deployment pipelines.
- Proven track record migrating or modernizing large BI and data warehouse ecosystems onto modern cloud platforms.
- Experience with Agentic AI, MCP-driven automation, and intelligent orchestration concepts. TECH STACK
- Microsoft Fabric (Lakehouse, Warehouse, Data Factory, Notebooks/Spark, Dataflows Gen2, Direct Lake, Semantic Models)
- OneLake
- Fabric Data Factory
- Metadata-Driven Frameworks
- Dynamic Orchestration Frameworks
- T-SQL
- PySpark
- Python
- Delta Lake
- Azure DevOps
- Git Integration / Deployment Pipelines
- Power BI / Tableau
- SQL Server / Epic ClarityPREFERRED QUALIFICATIONS
- Microsoft Certified: Fabric Data Engineer Associate (DP-700), Fabric Analytics Engineer Associate, and/or Azure Data Engineer Associate (DP-203).
- Demonstrated experience designing enterprise metadata-driven ingestion and orchestration frameworks in Microsoft Fabric.
- Experience building centralized execution engines, dependency-management frameworks, and operational monitoring solutions for Fabric workloads.
- Epic Clarity / Caboodle experience.
- Experience integrating Fabric Warehouse (SQL Endpoint, XMLA Endpoint, Direct Lake) as published data sources for Tableau consumption.
- Familiarity with SQL parsing, normalization, and automation techniques using SQLGlot and Python.
- Experience leveraging AI-assisted development, Agentic AI, MCP Servers, and automation frameworks for accelerating Fabric platform delivery and operations.
- Experience supporting large-scale healthcare, HIPAA, or regulated data environments. Share your resume to shravya.nikhila@gmail.com and sunil@risamsoft.com