Principal Software Engineer Data Platform Engineering

Principal Software Engineer Data Platform Engineering08/19/2025Full TimeActiveJob Description:We are enabling the transformation of how energy and water are managed. As part of this mission, we are investing in cloud-native data platform technologies that power intelligent analytics at scale — particularly for large IoT data workloads. We are looking for a Principal Software Engineer - Data Platform Engineering with a strong background in data engineering to help design, build, and scale our next-generation data platform and services.This role will focus on developing performant, scalable, and secure data analytics and data pipelines using Python, Apache Spark, Databricks, and Azure-native technologies. The ideal candidate is passionate about working with large-scale datasets, understands Spark query execution plans, and has experience deploying infrastructure using Terraform and container technologies like Docker.Duties & Responsibilities:Design and implement scalable data pipelines for high-volume IoT telemetry data using PySpark, Spark SQL, and DatabricksAnalyze and optimize Spark jobs using execution plans, caching strategies, and memory tuningDevelop Python-based micro-services and reusable libraries to support data processing workflowsWork in a DevOps-driven Agile/Scrum environment to deliver high-quality code in fast iterationsCollaborate with product managers, data scientists, and software engineers to define system requirements and data integration needsBuild and manage infrastructure as code using Terraform to deploy workloads on AzureCreate CI/CD pipelines for deploying and testing data workflows and servicesPackage and deploy applications in Docker containers and manage runtime environmentsApply strong testing practices, including unit tests, integration tests, and test automationParticipate in architectural discussions and technical design reviewsEnsure secure, reliable, and cost-effective operation of data workloads on cloud platforms Required Skills & Experience:Minimum of 10 years of software engineering experience, with at least 3 years focused on data platform or data engineering rolesProficient in Python, with strong knowledge of PySpark and Apache SparkHands-on experience with Databricks and Spark SQLStrong understanding of Spark performance tuning and query execution plansExperience writing infrastructure-as-code using TerraformWorking knowledge of Docker and containerized deploymentExperience with CI/CD pipelines, version control (e.g., Git), and automated testingExperience with Agile/Scrum methodologiesBachelor's degree in Computer Science, Engineering, or related field — or equivalent experience Preferred Skills & Experience:Experience with large-scale IoT or telemetry data processingFamiliarity with Kafka, Event Hubs, or other streaming platformsFamiliarity with cost optimization in cloud-based data processingExperience with monitoring and observability tools like Prometheus, Grafana, or DatadogEmail ID * This field is required Please enter valid emailId.Cell phone * This field is required Please enter valid cell phone.First Name * This field is required Please enter valid first name.Last Name * This field is required Please enter valid last name.J-18808-Ljbffr