Databricks Engineer
ARCHIVED
We can't find an active application page for this role right now. It may reopen or be listed elsewhere. Use Next Steps to search for an active apply link and similar live jobs.
Type: Contract-to-Hire (flexible based on candidate preference)
Job Summary:
We are seeking a highly skilled Databricks Engineer to support the design, development, and optimization of large-scale data processing pipelines and machine learning workflows using the Databricks Lakehouse platform. This role requires expertise in Spark, Delta Lake, and a strong understanding of cloud-native data architectures (AWS).
Key Responsibilities:
Design and implement scalable ETL/ELT pipelines using PySpark and Databricks notebooks.
Optimize performance across Delta Lake tables, jobs, and clusters.
Integrate Databricks with external tools and services (e.g., Azure Data Factory, AWS Glue, Kafka, Snowflake).
Collaborate with data engineers, scientists, and architects to build reliable, high-performance solutions.
Develop CI/CD pipelines for Databricks workflows using tools like GitHub Actions, Azure DevOps, or Jenkins.
Apply best practices for security, cost governance, and platform observability.
Required Qualifications:
4+ years of experience in data engineering or analytics roles.
2+ years of hands-on experience with Databricks and Apache Spark (preferably in a production setting).
Strong Python skills (PySpark) and SQL expertise.
Familiarity with Delta Lake , Unity Catalog, and data governance practices.
Solid understanding of modern data architectures: Lakehouse, medallion architecture, etc.
Nice to Have:
Experience with MLflow or Databricks Machine Learning.
Certifications in Databricks, Azure Data Engineer, or AWS Big Data.
Exposure to tools like Airflow, dbt, or Terraform.
#J-18808-Ljbffr