JOBSEARCHER

Data Engineer (Python) - Remote

Robert HalfRaleigh, NCRemoteL6 LeadNovember 1st, 2026
We're partnering with a highly respected organization that is making significant investments in its enterprise data and AI capabilities. This is an opportunity for a senior-level engineer to help shape the future of a modern Azure-based data platform while working with cutting-edge technologies including Databricks, Unity Catalog, AI-ready data architectures, governance tooling, and cloud-native engineering practices. This role is ideal for someone who enjoys building platforms from the ground up, solving complex enterprise data challenges, and driving best practices across data engineering, platform engineering, security, and governance. As a Senior Data Platform Engineer, you'll be responsible for the design, operation, optimization, and evolution of a modern enterprise data platform supporting analytics, business intelligence, AI, and machine learning initiatives. Designing and managing scalable Azure and Databricks-based data platforms Building and optimizing lakehouse architectures leveraging Azure Data Lake, Delta Lake, Spark, and PySparkCreating robust data ingestion, integration, ETL/ELT, and orchestration pipelines Managing platform reliability, scalability, observability, and cost optimization Implementing governance, lineage, metadata management, and security controls Developing AI-ready data foundations to support advanced analytics and machine learning initiatives Automating infrastructure deployments and platform management through Infrastructure-as-Code and CI/CD practices Providing technical leadership, mentoring engineers, and influencing platform strategy 10+ years of experience in Data Engineering, Data Platform Engineering, Cloud Engineering, or related disciplines Microsoft Azure DatabricksPythonSQLAzure Data Factory Azure Data Lake Storage Experience building and supporting enterprise-scale cloud data platformsAI-focused data engineering Data readiness for Machine Learning and Generative AI initiatives Data governance, metadata, lineage, and cataloging Performance tuning, scalability, and cost optimization Experience leading projects and providing technical direction to engineering teams We're looking for engineers who have worked in mature, modern technology environments rather than traditional legal organizations. Software product companies Companies operating large-scale cloud platforms and modern engineering practices Experience designing and launching enterprise data platforms from the ground upExpertise in Databricks architecture, optimization, and governance Strong Azure cloud engineering knowledge Proven success supporting AI and advanced analytics initiatives through scalable data architecture A platform engineering mindset focused on reliability, automation, observability, and security