Data Engineer

Data Engineer – Microsoft FabricDevelops scripts, processes, and tools to ingest, transform, and deliver data within a Microsoft Fabric environment, enabling robust analytics and reporting across the organization.Key ResponsibilitiesLakehouse Architecture & Platform EngineeringDesign, build, and maintain scalable lakehouse architecture using Microsoft Fabric and OneLakeEnsure high availability, reliability, and performance of data platformsData Pipelines & IngestionBuild and maintain end-to-end data pipelines covering ingestion, transformation, and servingDevelop scalable ingestion frameworks supporting both batch and real-time sources (APIs, databases, event streams)Integrate enterprise systems, including Jira, ERP, CRM, flat files, and streaming sourcesData Processing & TransformationDevelop and execute large-scale Spark jobs for both batch and streaming workloadsAuthor and maintain notebooks (PySpark, SQL) for data transformation and analysisImplement data cleansing, enrichment, and transformation logicData Quality & IntegrationBuild ingestion logic to handle structured, semi-structured, and unstructured dataImplement data validation and quality control processesData EnablementDeliver analytics-ready datasets for Power BI and other downstream consumption needsSkill RequirementsMicrosoft Fabric (Lakehouse, OneLake, Pipelines, Notebooks)Apache Spark (PySpark, Spark SQL)SQLPythonDelta Lake / Delta TablesData Pipeline Design (ETL / ELT)• • API & Streaming Data Integration