Senior Data Engineer

Position Summary:The Senior Data Engineer is a foundational hire on PTL's data team, responsible for architecting and building the data platform that will power analytics, operational reporting, and decision-making across our downstream fuel distribution business. This is a hands-on senior individual contributor role with a dual mandate: building the modern data foundation on Microsoft Fabric while simultaneously delivering against live business asks from operations, commercial, finance, and supply chain.Reporting to the Data Lead, this engineer will own the discovery-to-delivery lifecycle: analyzing source systems across ERP, dispatch, telematics, and trading platforms; partnering with business stakeholders to translate fuel logistics requirements into technical specifications; and engineering scalable, secure, and performant Fabric solutions. The right candidate is someone who can design the Medallion architecture in the morning, write the PySpark notebook in the afternoon, and explain the results to a dispatch manager or a CFO by end of day. Over time, this role is positioned to grow into a lead position with direct reports as the data engineering function scales.Responsibilities:Data Platform Architecture and BuildFoundational build " Architect and build the Microsoft Fabric lakehouse foundation " OneLake structure, workspace design, Medallion layering (Bronze/Silver/Gold), and orchestration patterns that will scale with PTL's data footprint.Pipeline engineering " Design and deploy automated ETL/ELT workflows using Fabric Data Factory, Dataflows Gen2, and PySpark notebooks; establish reusable ingestion patterns and orchestration frameworks.Data modeling " Design star schemas, Bus Matrix architectures, and dimensional models that support operational reporting, commercial analytics, fuel margin analysis, dispatch performance, and financial reconciliation.Engineering practices " Establish coding standards, Git-based source control, CI/CD patterns, deployment pipelines, and environment management (dev/test/prod) for the data platform.Source System IntegrationSource system discovery " Conduct deep-dive analysis of PTL source systems including Sage/DM2, PDI, SkyBitz telematics, dispatch and TMS platforms, CRM, ETRM, and Master Data Management (MDM) systems.Heterogeneous ingestion " Design ingestion patterns for varied data types: transactional ERP data, high-volume telematics and geospatial feeds, rack pricing data, EDI flows, flat files, and third-party APIs.Data lineage " Map end-to-end data lineage across systems; document data frequency, quality constraints, and business rules inherent in downstream fuel operations.Master data partnership " Work with the VP of Business and Process Integration and functional owners to establish master data governance for customers, vendors, products, locations, and pricing.Delivery Against Business NeedsRequirements translation " Partner with logistics, dispatch, commercial, supply, and finance stakeholders to understand business questions, define KPIs, and translate them into data models and delivered outputs.Operational analytics " Deliver against live business asks in parallel with foundational build work " fuel margin analysis, load-level profitability, dispatch efficiency, rack-to-customer flows, driver and asset utilization, regulatory reporting.Semantic layer " Partner with Power BI developers and analytics consumers to build semantic models, enforce consistent metric definitions, and deliver trustworthy outputs to the business.Pragmatic trade-offs " Balance speed and rigor " deliver tactical wins without compromising the long-term architecture; pay down technical debt as the platform matures.Data Quality, Governance, and ReliabilityQuality frameworks " Implement rigorous validation frameworks, data quality checks, and reconciliation routines so the business can act on trusted data.Pipeline observability " Build proactive monitoring, alerting, and observability across pipelines; identify and resolve upstream data issues before they break downstream reporting.Security and governance " Partner with IT and security on access controls, data classification, PII handling, and compliance with relevant regulatory requirements.Stakeholder Collaboration and CommunicationDesign leadership " Lead design reviews with technical teammates and non-technical business leaders; bridge the gap between engineering detail and business outcomes.Business fluency " Build credibility with operations, dispatch, and commercial leaders by learning the business, asking sharp questions, and proposing solutions rather than surfacing only problems.Knowledge transfer " Document architectures, data models, and pipelines so the platform remains maintainable as the team grows.Minimum Requirements:5+ years of hands-on data engineering experience, with demonstrated delivery on the Microsoft Azure data stack (Fabric, Synapse, Databricks, ADLS, Data Factory) or comparable modern lakehouse platforms.Expert-level Python (including PySpark) and SQL; comfort writing custom scripts for data parsing, transformation, and API consumption.Proven experience designing dimensional models and data warehouse architectures " star schemas, Bus Matrix, slowly changing dimensions, conformed dimensions " at enterprise scale.Demonstrated experience in source-to-target mapping, gap analysis, and integrating data from heterogeneous systems (ERP, operational, telemetry, third-party APIs, flat files).Track record of building data platforms that served real business users, not just proof-of-concept work.Strong engineering fundamentals " source control, testing, CI/CD, code review, environment management.Domain Experience (Preferred)Direct experience with downstream fuel, refined products logistics, midstream, or energy distribution data " and ideally with the platforms that generate it: Sage/DM2, PDI, SkyBitz, or comparable ERP, dispatch, and telematics systems.Familiarity with fuel logistics concepts " rack pricing, BOLs, loads, dispatch and routing, inventory management, driver settlement, fuel margin, and regulatory reporting.Exposure to ETRM (Energy Trading and Risk Management), CRM, and MDM platforms and the data patterns they produce.Experience with geospatial data and telematics feeds for transportation or logistics analytics.Preferred Technical SkillsProduction experience with Microsoft Fabric " OneLake, Lakehouses, Warehouses, Data Factory, and Notebooks.Power BI semantic model development, DAX, and tabular modeling.Real-time and streaming data patterns (Eventstream, Event Hubs, Kafka).Data quality and testing frameworks (Great Expectations, dbt tests, or comparable).Infrastructure-as-code and DevOps tooling (Azure DevOps, GitHub Actions, Terraform).Soft SkillsThe translator ability " Able to explain complex technical concepts to business managers, and to translate business questions into precise technical requirements. This is non-negotiable for this role.Solution ownership " Identifies bottlenecks, bad data, and broken assumptions upstream before they break the pipeline or reach the business.Pragmatism " Comfortable operating with ambiguity, shifting priorities, and the reality that foundational build work happens while the business keeps asking for answers.Communication " Direct, concise communicator who writes clear documentation and runs productive design reviews.Education:Bachelor's degree in Computer Science, Data Engineering, Information Systems, Mathematics, Engineering, or a related quantitative field; equivalent professional experience will be considered.