Senior Data Reliability Engineer (CI/CD & Data Platform)

Data Reliability Engineer (CI/CD & Data Platform)We are looking for a Data Reliability Engineer to ensure the reliability, scalability, and operational excellence of our data platform. The role focuses on building resilient data pipelines, implementing CI/CD for data engineering workflows, automating deployments, monitoring production systems, and maintaining high standards of data quality and availability.Key ResponsibilitiesDesign, build, and maintain reliable batch and streaming data pipelines.Develop and manage CI/CD pipelines for data applications, ETL workflows, and infrastructure.Automate build, test, deployment, and rollback processes for data solutions.Monitor data pipelines, platforms, and infrastructure using observability tools.Troubleshoot production incidents and perform root cause analysis (RCA).Implement data quality validation, testing, and monitoring.Manage Infrastructure as Code (IaC) using Terraform or with Data Engineering, DevOps, Platform Engineering, QA, and Analytics teams.Optimize Spark jobs, SQL workloads, and cloud data services for performance and reliability.Ensure high availability, disaster recovery, backup, and security compliance.Define and monitor SLAs, SLOs, and reliability metrics.Support release management, version control, and environment promotion Technical SkillsProgrammingPythonSQLShell ScriptingJava or Scala (preferred)Data EngineeringApache SparkApache AirflowdbtKafkaSnowflake, Databricks, Redshift, BigQuery, or SynapseCI/CDJenkinsGitHub ActionsGitLab CI/CDAzure DevOps PipelinesBitbucket PipelinesVersion ControlGitGitHubGitLabBitbucketContainers & OrchestrationDockerKubernetesCloud PlatformsAWSAzureGoogle Cloud Platform (GCP)Infrastructure as CodeTerraformCloudFormationAzure Bicep (preferred)Monitoring & ObservabilityGrafanaPrometheusDatadogSplunkELK StackCloudWatch / Azure Monitor / Google Cloud MonitoringTestingUnit TestingIntegration TestingData Quality TestingGreat ExpectationsSodaPyTestCI/CD ResponsibilitiesBuild automated pipelines for data applications and ETL deployments.Integrate automated testing into deployment pipelines.Implement deployment strategies such as blue-green and rolling deployments where schema migrations and database deployment processes.Enforce code quality through static code analysis and pull request secure secret management and credential release pipelines across development, QA, staging, and production environments.Monitor deployment success rates and automate rollback mechanisms.