Data Engineer

Job DescriptionJob Summary: (Data Engineer) SkillsetHands-on experience with GCP (BigQuery, Composer, Dataproc, Dataflow, Pub/Sub)Working knowledge of Databricks for data processing and pipelinesStrong programming skills in SQL and PySparkExperience with Oracle and SQL Server databasesUnderstanding of data modelling (dimensional / medallion basics)Experience in batch pipelines and exposure to streamingFamiliarity with data ingestion, ETL/ELT processesBasic understanding of data quality and validationExposure to Airflow/Composer and schedulingWorking knowledge of Git and CI/CDDetailed ResponsibilitiesDevelop and maintain batch and streaming data pipelinesBuild SQL and PySpark transformations for large datasetsIngest data from multiple sources into BigQuery/cloud platformsImplement data transformation logic as per requirementsWork with teams to consume curated datasetsDevelop workflows using Cloud Composer (Airflow)Perform data validation and reconciliationOptimise queries and pipelines for performanceSupport migration of legacy ETL jobs to cloudTroubleshoot pipeline and performance issuesParticipate in testing, UAT, and deploymentDocument pipelines, transformations, and sourcesCollaborate with analytics and reporting teamsSoft SkillsStrong analytical and problem-solving skillsExcellent communication and stakeholder managementLeadership and mentoring capabilitiesAttention to detail and quality-focused mindsetCollaboration and teamworkTime management and prioritizationAbility to work in Agile and fast-paced environmentsQualificationsBachelor'sRange Of Year Experience-Min Year1Range Of Year Experience-Max Year4