Databricks Engineer

Databricks EngineerRemoteContract roleDesign, develop, and maintain ETL/ELT pipelines using Databricks and PySparkBuild scalable data processing workflows for structured and semi-structured dataImplement Delta Lake and Medallion Architecture (Bronze, Silver, Gold layers)Optimize Spark jobs for performance, scalability, and cost efficiencyDevelop batch and streaming data pipelines using Structured StreamingWork with cloud storage services such as AWS S3, Azure ADLS, or GCP StorageCollaborate with data analysts, data scientists, and stakeholders to deliver business solutionsMonitor and troubleshoot data pipelines and cluster performanceImplement CI/CD and version control practices using Git and DevOps toolsEnsure data quality, governance, security, and compliance standardsCreate technical documentation and support production deployments