Lead Data Engineer – Databricks
🚀 We're Hiring: Lead Data Engineer – Databricks📍 Location: USA – RemoteWe are looking for an experienced Lead Data Engineer with strong hands-on expertise in Databricks, Advanced SQL, and AWS to join our team!
🔹 Key Skills & Requirements:
5+ years of hands-on experience with Databricks.
Strong expertise in Medallion Architecture (Bronze, Silver & Gold).Advanced SQL, PySpark, and performance tuning.
Pipeline optimization and DBU cost optimization across All-Purpose and Job Clusters.
Data ingestion from multiple source systems.
Parsing and processing JSON/XML data.
Ingesting data from APIs into Delta Tables.
Hands-on experience with Databricks Auto Loader.
Publishing data to AWS services such as S3, SQS, and SNS.Experience working with relational and non-relational databases.
Strong proficiency in Unity Catalog.
🔹 Good to Have:
Declarative Pipelines.
DBUtils commands.
Knowledge of broader AWS services.🎯 Looking for candidates with strong hands-on Databricks experience and expertise in building, optimizing, and managing scalable data engineering solutions.