JOBSEARCHER

Databricks Data Engineer

SotalentDenver, COL6 LeadNovember 4th, 2026
Databricks Data Engineer Location: Remote, US Industry: Business Consulting and Services Work Setting: Remote Are you passionate about building scalable cloud data platforms, engineering high-performance data pipelines, and enabling advanced analytics and AI solutions through modern data engineering practices? We are seeking a Databricks Data Engineer to design, develop, and support enterprise-scale data solutions that drive data modernization, analytics, reporting, and AI initiatives. This role combines hands-on data engineering, cloud platform expertise, performance optimization, and cross-functional collaboration to deliver reliable, secure, and scalable data products.
Position Summary: The Databricks Data Engineer is responsible for building and maintaining cloud-native data pipelines and processing frameworks that support ingestion, transformation, governance, and delivery of data across the organization. Working closely with architects, data scientists, analysts, DevOps teams, and business stakeholders, you will enable data-driven decision-making through modern analytics platforms and well-engineered data ecosystems. The ideal candidate brings strong expertise in Databricks, Spark, Python, SQL, Delta Lake, and cloud data engineering, along with a passion for solving complex data challenges and optimizing large-scale processing environments.
Key Responsibilities
Data Pipeline Development: Design, build, and maintain scalable data pipelines using modern cloud-native technologies. Develop robust ETL/ELT workflows for data ingestion, transformation, and delivery. Support both batch and real-time data processing requirements. Ensure data pipelines are reliable, maintainable, and performant. Build reusable data engineering frameworks that support multiple business use cases.
Data Processing & Transformation: Develop and optimize data transformation solutions using Python, PySpark, and SQL.Process large-scale structured and unstructured datasets efficiently. Implement data cleansing, enrichment, aggregation, and validation processes. Improve processing performance through optimization and tuning techniques. Support large-scale analytics and reporting workloads.
Cloud Data Platform Engineering: Work with cloud-based data platforms and distributed processing technologies. Develop scalable solutions supporting enterprise analytics workloads. Manage data storage, processing, and access across cloud environments. Optimize platform utilization, performance, and cost efficiency. Support platform modernization and cloud migration initiatives.
Data Architecture & Modeling: Develop and maintain scalable data models and analytical datasets. Support data warehousing and lakehouse architectures. Design solutions that enable efficient reporting, analytics, and machine learning. Implement standards for data organization and lifecycle management. Support enterprise data architecture initiatives.
Data Quality & Reliability: Implement data validation and quality assurance processes. Develop monitoring and alerting capabilities for pipeline health. Identify and resolve data inconsistencies and processing errors. Perform root cause analysis and corrective actions. Ensure delivery of trusted and accurate data products.
Troubleshooting & Performance Optimization:
  • Investigate and resolve pipeline failures and workflow bottlenecks.
  • Optimize processing workloads for scalability and efficiency.
  • Improve job execution times and resource utilization.
  • Analyze system performance metrics and recommend improvements.
  • Support operational stability across production environments.
  • Collaboration & Stakeholder Engagement
  • Partner with:
  • Data Scientists
  • Solution Architects
  • Data Analysts
  • DevOps Engineers
  • Business Stakeholders
  • Translate business requirements into technical data solutions.
  • Communicate engineering designs and implementation approaches clearly.
  • Contribute to project planning, estimation, and delivery activities.
  • Support analytics and AI initiatives through data platform enablement.
Governance, Security & Compliance:
  • Implement data governance and access control standards.
  • Support data lineage, metadata management, and auditability requirements.
  • Ensure compliance with organizational security and regulatory standards.
  • Maintain secure handling of enterprise data assets.
  • Support governance frameworks for cloud data platforms.
  • Documentation & Knowledge Sharing
  • Create and maintain:
  • Data Flow Documentation
  • Architecture Diagrams
  • Pipeline Specifications
  • Data Models
  • Technical Runbooks
  • Document engineering decisions and operational procedures.
  • Promote reusable engineering practices and standards.
  • Support team knowledge sharing and mentoring activities.
Required Qualifications: EducationBachelor's degree in:
  • Computer Science
  • EngineeringInformation Systems
  • MathematicsStatisticsRelated technical discipline
  • Experience 3-8 years of experience in:
  • Data Engineering
  • Data Platform Development
  • Cloud Data Solutions
  • Analytics Engineering
  • Experience building enterprise-scale data pipelines and processing frameworks.
  • Strong problem-solving and analytical skills.
  • Experience working in collaborative delivery environments.