Databricks Engineer

We are seeking a Senior Databricks Engineer with 8+ years of experience to design, build, and optimize enterprise-grade data solutions on Databricks. In this role, you will work with broad latitude and independence, translating complex business objectives into scalable, cost-effective data architecture. You will bridge the gap between technical requirements and business needs—conducting feasibility studies, analyzing user workflows, and delivering high-performance lakehouse analytics.Key ResponsibilitiesPipeline & Lakehouse Architecture: Design, build, and maintain scalable ETL/ELT ingestion pipelines utilizing Lakeflow Declarative Pipelines (DLT), PySpark/Scala, and the Medallion Architecture (Bronze, Silver, Gold).Business Analysis & Data Modeling: Collaborate with stakeholders to capture requirements, perform cost/benefit analyses, and build robust dimensional models (Star/Snowflake schemas).Analytics & Visualization: Develop dynamic native Databricks SQL dashboards, Databricks Apps, and custom analytical solutions to surface actionable insights.Governance & Operations: Implement data quality, security, and governance frameworks. Optimize workload performance, monitor compute efficiency, and manage pipeline orchestration via Lakeflow Jobs or Airflow.Candidate QualificationsRequired Experience (8+ Years)Databricks & Spark: Hands-on experience architecting and tuning ETL/ELT data pipelines on Databricks using Apache Spark (PySpark or Scala).Lakehouse Stack: Deep knowledge of Delta Lake, Medallion architecture, and Lakeflow Declarative Pipelines (DLT).Data Modeling & SQL: Strong proficiency in SQL and dimensional data modeling (Star/Snowflake schemas).Analytics & Governance: Experience developing native Databricks dashboards/apps and establishing data security, quality, and governance practices.Orchestration: Experience scheduling and managing offline jobs using Lakeflow Jobs (Databricks Workflows) or similar tools (e.g., Airflow).Communication: Exceptional verbal and written communication skills with a proven ability to translate complex data concepts for business leadership.Preferred QualificationsActive Databricks Certification (Data Engineer Associate or Professional).Experience with CI/CD practices for data pipelines (Git-based workflows, DevOps).Prior experience in public sector or state government environments.