Databricks Data Engineer

Company Description:Cogniforge Global is a leading consulting firm specializing in modern data platforms, including Databricks, Snowflake, Microsoft Azure, and AWS. We empower businesses to optimize their data assets by delivering tailored solutions for data integration, machine learning optimization, and data governance. Our team of experts focuses on cutting-edge technologies to enable seamless cloud-native services, ensuring maximum ROI and business success. With a commitment to innovation, we help organizations become truly data-driven, leveraging advanced data and AI capabilities.Role Description:This is a contract-based, on-site role for a Databricks Data Engineer. The primary responsibilities of this role involve designing, building, and maintaining scalable data pipelines and workflows in a cloud-based environment, particularly leveraging Databricks. The Data Engineer will collaborate with cross-functional teams to develop and optimize data models, oversee Extract, Transform, and Load (ETL) processes, and create data infrastructure that supports analytics and reporting. The role also involves ensuring data quality, governance, and security within the organization.Qualifications:Design, develop, and maintain scalable data pipelines using Databricks and Apache SparkBuild and optimize ETL/ELT workflows for structured and unstructured dataImplement data models following best practices (e.g., medallion architecture – Bronze/Silver/Gold)Work with large-scale datasets in distributed environmentsCollaborate with data analysts, data scientists, and business stakeholders to deliver high-quality data solutionsEnsure data quality, governance, and security standards are metOptimize performance of data pipelines and queriesIntegrate data from multiple sources such as APIs, databases, and streaming platformsSupport CI/CD processes and deployment of data workflowsMonitor and troubleshoot data pipeline issuesExperience with streaming frameworks (e.g., Spark Streaming, Kafka)Knowledge of data governance and cataloging (e.g., Unity Catalog)Familiarity with CI/CD tools and DevOps practicesExposure to BI tools such as Tableau or Power BIUnderstanding of performance tuning and cost optimization in DatabricksMust be Databricks certified Data Engineer Professional