Databricks Architect
Job SummaryWe are seeking an experienced Databricks Architect to design, architect, and implement scalable, cloud-based data platforms using Databricks. The ideal candidate will have strong expertise in modern data engineering, Lakehouse architecture, cloud platforms, and big data technologies. This role involves collaborating with business stakeholders, data engineers, and solution architects to deliver high-performance analytics and data solutions.Key ResponsibilitiesDesign and architect enterprise-scale data platforms using Databricks Lakehouse.Lead the design and implementation of scalable data pipelines using Databricks, Apache Spark, and Delta Lake.Develop and optimize ETL/ELT processes for structured and semi-structured data.Design data models, ingestion frameworks, and data transformation pipelines.Implement Medallion Architecture (Bronze, Silver, Gold) and Delta Lake best practices.Optimize Databricks workloads for performance, scalability, and cost efficiency.Build reusable frameworks, notebooks, and deployment pipelines.Collaborate with data engineering, analytics, and business teams to understand data requirements.Establish data governance, security, and access controls within Databricks.Integrate Databricks with cloud services such as Azure, AWS, or GCP.Design CI/CD pipelines for Databricks deployments using Azure DevOps, GitHub Actions, or Jenkins.Provide technical leadership, architecture guidance, and mentoring to engineering teams.Troubleshoot complex production issues and recommend architecture improvements.Document solution architecture, technical designs, and implementation standards.Required Qualifications8+ years of experience in Data Engineering or Data Architecture.4+ years of hands-on experience with Databricks.Strong expertise in Apache Spark (PySpark/Scala/Spark SQL).Experience with Delta Lake, Unity Catalog, and Databricks Workflows.Strong SQL and data modeling skills.Experience designing enterprise data lakes and Lakehouse architectures.Experience with cloud platforms (Azure, AWS, or GCP).Strong understanding of ETL/ELT frameworks and distributed data processing.Experience with Git and CI/CD pipelines.Knowledge of data governance, security, and performance tuning.Excellent communication and stakeholder management skills.Preferred QualificationsExperience with Azure Data Factory (ADF), Azure Synapse, or Microsoft Fabric.Experience with AWS Glue, EMR, or Redshift (for AWS environments).Experience with Apache Kafka, Event Hubs, or streaming technologies.Experience with Terraform or Infrastructure as Code.Databricks Certified Professional or Databricks Certified Data Engineer certification.Experience with Unity Catalog, MLflow, and Databricks Asset Bundles.Knowledge of Power BI, Tableau, or other BI tools.Required SkillsDatabricksApache SparkPySparkSpark SQLDelta LakeUnity CatalogDatabricks WorkflowsSQLPythonAzure / AWS / GCPData LakeLakehouseETL / ELTCI/CDGitPerformance TuningData GovernanceData Modeling