Data Engineer
ARCHIVED
We can't find an active application page for this role right now. It may reopen or be listed elsewhere. Use Next Steps to search for an active apply link and similar live jobs.
Data Engineer – Microsoft Fabric Develops scripts, processes, and tools to ingest, transform, and deliver data within a Microsoft Fabric environment, enabling robust analytics and reporting across the organization.
Key Responsibilities:
- Lakehouse Architecture & Platform Engineering
- Design, build, and maintain scalable lakehouse architecture using Microsoft Fabric and OneLakeEnsure high availability, reliability, and performance of data platforms
- Data Pipelines & Ingestion
- Build and maintain end-to-end data pipelines covering ingestion, transformation, and serving
- Develop scalable ingestion frameworks supporting both batch and real-time sources (APIs, databases, event streams)
- Integrate enterprise systems, including Jira, ERP, CRM, flat files, and streaming sources
- Data Processing & Transformation
- Develop and execute large-scale Spark jobs for both batch and streaming workloads
- Author and maintain notebooks (PySpark, SQL) for data transformation and analysis
- Implement data cleansing, enrichment, and transformation logic
- Data Quality & Integration
- Build ingestion logic to handle structured, semi-structured, and unstructured data
- Implement data validation and quality control processes
- Data Enablement
- Deliver analytics-ready datasets for Power BI and other downstream consumption needs
- Skill Requirements
- Microsoft Fabric (Lakehouse, OneLake, Pipelines, Notebooks)
- Apache Spark (PySpark, Spark SQL)SQLPythonDelta Lake / Delta TablesData Pipeline Design (ETL / ELT) • • API & Streaming Data Integration