{"schemaVersion":"jobsearcher.job.v1","id":"19a3934b540a5eed5326570b","url":"https://jobsearcher.com/jobs/19a3934b540a5eed5326570b","canonicalUrl":"https://jobsearcher.com/jobs/19a3934b540a5eed5326570b","title":"Sr Big Data / OpenShift API Developer","description":"Sr Big Data / OpenShift API DeveloperLocation: Charlotte, NC (Hybrid – 3 days office in a week)Key ResponsibilitiesPlatform ManagementDeploy, configure, maintain, and support containerized Apache Spark applications on OpenShift and Google Cloud.Manage Spark workloads running on Kubernetes and OpenShift clusters.Configure pods, deployments, services, routes, secrets, config maps, resource quotas, and namespaces.Define resource requests and limits for Spark drivers, executors, and supporting services.Support cluster capacity planning, scaling, patching, upgrades, and environment management.Maintain Helm charts and deployment templates for repeatable application releases.Data Pipeline DevelopmentDesign and implement large-scale batch and streaming data processing workflows using Apache Spark and Apache Airflow.Develop robust, reusable, and production-ready PySpark, Scala, or Java Airflow DAGs for scheduling, orchestration, dependency management, retries, and alerting.Implement data ingestion, transformation, validation, enrichment, and publishing processes.Support both real-time and batch data processing use cases.Apply data engineering best practices for reliability, scalability, data quality, and maintainability.API and Microservices DevelopmentDesign, develop, test, and maintain RESTful APIs and microservices supporting data and analytics API contracts, request/response schemas, authentication mechanisms, error handling, and service-level requirements.Integrate APIs with Spark applications, data platforms, databases, messaging systems, and cloud services.Develop automated unit, integration, and API tests.Apply microservices architecture principles, including loose coupling, scalability, resiliency, and observability.Performance OptimizationTune Spark jobs for performance, scalability, and efficient resource partitioning, joins, caching, serialization, shuffle operations, file formats, and executor OpenShift and Kubernetes resource-management capabilities to improve application performance.Configure horizontal and vertical scaling, autoscaling, and workload scheduling where Spark execution plans, application logs, cluster metrics, and resource-consumption and resolve bottlenecks across applications, infrastructure, networking, and storage layers.IntegrationIntegrate Apache Spark with data sources and platforms such as:Apache KafkaAmazon S3Google Cloud StorageAzure Blob StorageHDFSRelational and NoSQL databasesData lakes and data warehousesDevelop reliable ingestion and publishing patterns for batch and streaming schema management, checkpointing, offset handling, retry logic, and failure recovery.Support secure connectivity between applications, cloud platforms, messaging systems, and /CD ImplementationBuild and maintain CI/CD pipelines for deploying Spark applications, APIs, and supporting services to OpenShift.Use tools such as GitHub Actions, SonarQube, Harness, Docker, Helm, and Git.Automate build, code-quality analysis, security scanning, testing, packaging, and deployment processes.Implement environment-specific configurations and promotion strategies across development, QA, staging, and production.Manage container images, registries, versioning, release tagging, and rollback procedures.Establish deployment standards and support automated release management.Monitoring and TroubleshootingMonitor OpenShift cluster health, Spark job performance, application availability, and resource OpenShift monitoring capabilities and tools such as Prometheus, Grafana, and centralized logging dashboards, alerts, health checks, and operational metrics.Troubleshoot complex issues involving Spark applications, APIs, containers, Kubernetes resources, networking, storage, and root-cause analysis and implement proactive in incident, problem, and change-management activities.Provide production support and participate in on-call rotations when required.Security and ComplianceImplement security controls for applications, data pipelines, containers, and OpenShift environments.Configure and manage Role-Based Access Control (RBAC), service accounts, secrets, and namespace permissions.Ensure encryption of data in transit secure coding, container security, vulnerability management, and least-privilege principles.Support compliance with enterprise security standards, regulatory requirements, and internal governance policies.Protect sensitive data during ingestion, processing, storage, and transmission.CollaborationWork with product owners, data engineers, application developers, architects, DevOps teams, security teams, and operations teams to define requirements.Participate in architecture reviews, solution design, estimation, sprint planning, and technical technical documentation, deployment guides, operational runbooks, and support procedures.Support production releases and provide knowledge transfer to engineering and operations teams.Mentor junior developers and promote engineering best practices.Required QualificationsBachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.7+ years of experience in software development, data engineering, big data, or distributed systems.4+ years of hands-on experience with Apache Spark and large-scale data processing.3+ years of experience deploying and managing containerized applications on OpenShift and/or Kubernetes.Strong experience developing Spark applications using Python/PySpark, Scala, or Java.Hands-on experience with Apache Airflow and workflow designing and developing REST APIs and microservices.Strong knowledge of Docker and Kubernetes concepts, including:PodsDeploymentsServicesConfigMapsSecretsNamespacesIngress or RoutesResource requests and limitsContainer imagesExperience with cloud platforms such as AWS, Google Cloud Platform, or Microsoft Azure.Experience working with data storage and processing technologies such as:Amazon S3Google Cloud StorageAzure Blob StorageHDFSData lakesRelational databasesNoSQL databasesStrong programming skills in Python, Scala, or Java.Experience with shell scripting is preferred.","company":"Conflux Systems","rawCompany":"conflux systems","city":"Charlotte","state":"NC","isRemote":false,"isActive":false,"createdAt":"2026-09-11T07:12:10.546Z","occupations":[{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"}],"industries":[{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Sr Big Data / OpenShift API Developer","description":"Sr Big Data / OpenShift API DeveloperLocation: Charlotte, NC (Hybrid – 3 days office in a week)Key ResponsibilitiesPlatform ManagementDeploy, configure, maintain, and support containerized Apache Spark applications on OpenShift and Google Cloud.Manage Spark workloads running on Kubernetes and OpenShift clusters.Configure pods, deployments, services, routes, secrets, config maps, resource quotas, and namespaces.Define resource requests and limits for Spark drivers, executors, and supporting services.Support cluster capacity planning, scaling, patching, upgrades, and environment management.Maintain Helm charts and deployment templates for repeatable application releases.Data Pipeline DevelopmentDesign and implement large-scale batch and streaming data processing workflows using Apache Spark and Apache Airflow.Develop robust, reusable, and production-ready PySpark, Scala, or Java Airflow DAGs for scheduling, orchestration, dependency management, retries, and alerting.Implement data ingestion, transformation, validation, enrichment, and publishing processes.Support both real-time and batch data processing use cases.Apply data engineering best practices for reliability, scalability, data quality, and maintainability.API and Microservices DevelopmentDesign, develop, test, and maintain RESTful APIs and microservices supporting data and analytics API contracts, request/response schemas, authentication mechanisms, error handling, and service-level requirements.Integrate APIs with Spark applications, data platforms, databases, messaging systems, and cloud services.Develop automated unit, integration, and API tests.Apply microservices architecture principles, including loose coupling, scalability, resiliency, and observability.Performance OptimizationTune Spark jobs for performance, scalability, and efficient resource partitioning, joins, caching, serialization, shuffle operations, file formats, and executor OpenShift and Kubernetes resource-management capabilities to improve application performance.Configure horizontal and vertical scaling, autoscaling, and workload scheduling where Spark execution plans, application logs, cluster metrics, and resource-consumption and resolve bottlenecks across applications, infrastructure, networking, and storage layers.IntegrationIntegrate Apache Spark with data sources and platforms such as:Apache KafkaAmazon S3Google Cloud StorageAzure Blob StorageHDFSRelational and NoSQL databasesData lakes and data warehousesDevelop reliable ingestion and publishing patterns for batch and streaming schema management, checkpointing, offset handling, retry logic, and failure recovery.Support secure connectivity between applications, cloud platforms, messaging systems, and /CD ImplementationBuild and maintain CI/CD pipelines for deploying Spark applications, APIs, and supporting services to OpenShift.Use tools such as GitHub Actions, SonarQube, Harness, Docker, Helm, and Git.Automate build, code-quality analysis, security scanning, testing, packaging, and deployment processes.Implement environment-specific configurations and promotion strategies across development, QA, staging, and production.Manage container images, registries, versioning, release tagging, and rollback procedures.Establish deployment standards and support automated release management.Monitoring and TroubleshootingMonitor OpenShift cluster health, Spark job performance, application availability, and resource OpenShift monitoring capabilities and tools such as Prometheus, Grafana, and centralized logging dashboards, alerts, health checks, and operational metrics.Troubleshoot complex issues involving Spark applications, APIs, containers, Kubernetes resources, networking, storage, and root-cause analysis and implement proactive in incident, problem, and change-management activities.Provide production support and participate in on-call rotations when required.Security and ComplianceImplement security controls for applications, data pipelines, containers, and OpenShift environments.Configure and manage Role-Based Access Control (RBAC), service accounts, secrets, and namespace permissions.Ensure encryption of data in transit secure coding, container security, vulnerability management, and least-privilege principles.Support compliance with enterprise security standards, regulatory requirements, and internal governance policies.Protect sensitive data during ingestion, processing, storage, and transmission.CollaborationWork with product owners, data engineers, application developers, architects, DevOps teams, security teams, and operations teams to define requirements.Participate in architecture reviews, solution design, estimation, sprint planning, and technical technical documentation, deployment guides, operational runbooks, and support procedures.Support production releases and provide knowledge transfer to engineering and operations teams.Mentor junior developers and promote engineering best practices.Required QualificationsBachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.7+ years of experience in software development, data engineering, big data, or distributed systems.4+ years of hands-on experience with Apache Spark and large-scale data processing.3+ years of experience deploying and managing containerized applications on OpenShift and/or Kubernetes.Strong experience developing Spark applications using Python/PySpark, Scala, or Java.Hands-on experience with Apache Airflow and workflow designing and developing REST APIs and microservices.Strong knowledge of Docker and Kubernetes concepts, including:PodsDeploymentsServicesConfigMapsSecretsNamespacesIngress or RoutesResource requests and limitsContainer imagesExperience with cloud platforms such as AWS, Google Cloud Platform, or Microsoft Azure.Experience working with data storage and processing technologies such as:Amazon S3Google Cloud StorageAzure Blob StorageHDFSData lakesRelational databasesNoSQL databasesStrong programming skills in Python, Scala, or Java.Experience with shell scripting is preferred.","datePosted":"2026-09-11T07:12:10.546Z","dateModified":"2026-09-11T07:12:10.546Z","hiringOrganization":{"@type":"Organization","name":"Conflux Systems","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Charlotte","addressRegion":"NC","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"19a3934b540a5eed5326570b"},"url":"https://jobsearcher.com/jobs/19a3934b540a5eed5326570b"}}