{"schemaVersion":"jobsearcher.job.v1","id":"f1d7034ed072fbe2d1eb1254","url":"https://jobsearcher.com/jobs/f1d7034ed072fbe2d1eb1254","canonicalUrl":"https://jobsearcher.com/jobs/f1d7034ed072fbe2d1eb1254","title":"Apache Spark / Big Data Administrator","description":"Role: Apache Spark / Big Data AdministratorLocation: Berkley Heights, NJWork Type: 5 days a week at client officeJob Description10+ years solid experience with deploying, configuring, monitoring, and troubleshooting Apache Spark clusters and associated big data frameworks (such as Hadoop, Kafka, or cloud-based data platforms) to ensure high availability and performance.Focuses on cluster health, performance tuning, and production data pipelines.Technical Experience: Deep hands-on experience with Apache Spark architecture (Driver, Workers, Executors, RDDs/DataFrames) and cluster managers (YARN, Kubernetes, or Mesos).OS & Scripting: Strong proficiency in Linux/Unix administration and scripting languages (Bash, Python, or Scala).Ecosystem Knowledge: Familiarity with big data and streaming tools like Hadoop HDFS, Kafka, Hive, or monitoring suites (Splunk, Prometheus, Grafana).Debugging Skills: Strong analytical capabilities to inspect logs, thread dumps, and metrics under heavy production loadsExcellent coordination with stakeholders and communication skillsMust have experience with below responsibilitiesCluster Management: Install, configure, and maintain Apache Spark clusters and node infrastructure in production and staging environments.Performance & Monitoring: Monitor cluster health, resource utilization (CPU, memory), and execution logs using tools like Spark UI, Splunk, or Elastic. Identify bottlenecks and optimize job execution times.Troubleshooting: Diagnose and resolve out-of-memory errors, executor failures, shuffle issues, and node crashes in distributed workloads.Pipeline & Integration Support: Support data engineers and developers in managing Spark applications, streaming jobs, and integrations with message brokers like Kafka.Security & Access Control: Implement security principles, role-based access control, authentication, and data encryption across the cluster.Backup & Disaster Recovery: Execute backup, recovery, and failover plans for Spark applications, configurations, and critical data stores.Patching & Upgrades: Apply software patches, driver updates, and version upgrades with minimal disruption to ongoing production jobs","company":"Mphasis","rawCompany":"mphasis","city":"Denver","state":"CO","isRemote":false,"isActive":false,"createdAt":"2026-11-12T11:37:31.796Z","occupations":[{"code":"15-1244.00","title":"Network and Computer Systems Administrators","slug":"network-and-computer-systems-administrators"},{"code":"15-1242.00","title":"Database Administrators","slug":"database-administrators"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"518210","title":"Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services","slug":"computing-infrastructure-providers-data-processing-web-hosting-and-related-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Apache Spark / Big Data Administrator","description":"Role: Apache Spark / Big Data AdministratorLocation: Berkley Heights, NJWork Type: 5 days a week at client officeJob Description10+ years solid experience with deploying, configuring, monitoring, and troubleshooting Apache Spark clusters and associated big data frameworks (such as Hadoop, Kafka, or cloud-based data platforms) to ensure high availability and performance.Focuses on cluster health, performance tuning, and production data pipelines.Technical Experience: Deep hands-on experience with Apache Spark architecture (Driver, Workers, Executors, RDDs/DataFrames) and cluster managers (YARN, Kubernetes, or Mesos).OS & Scripting: Strong proficiency in Linux/Unix administration and scripting languages (Bash, Python, or Scala).Ecosystem Knowledge: Familiarity with big data and streaming tools like Hadoop HDFS, Kafka, Hive, or monitoring suites (Splunk, Prometheus, Grafana).Debugging Skills: Strong analytical capabilities to inspect logs, thread dumps, and metrics under heavy production loadsExcellent coordination with stakeholders and communication skillsMust have experience with below responsibilitiesCluster Management: Install, configure, and maintain Apache Spark clusters and node infrastructure in production and staging environments.Performance & Monitoring: Monitor cluster health, resource utilization (CPU, memory), and execution logs using tools like Spark UI, Splunk, or Elastic. Identify bottlenecks and optimize job execution times.Troubleshooting: Diagnose and resolve out-of-memory errors, executor failures, shuffle issues, and node crashes in distributed workloads.Pipeline & Integration Support: Support data engineers and developers in managing Spark applications, streaming jobs, and integrations with message brokers like Kafka.Security & Access Control: Implement security principles, role-based access control, authentication, and data encryption across the cluster.Backup & Disaster Recovery: Execute backup, recovery, and failover plans for Spark applications, configurations, and critical data stores.Patching & Upgrades: Apply software patches, driver updates, and version upgrades with minimal disruption to ongoing production jobs","datePosted":"2026-11-12T11:37:31.796Z","dateModified":"2026-11-12T11:37:31.796Z","hiringOrganization":{"@type":"Organization","name":"Mphasis","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Denver","addressRegion":"CO","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"f1d7034ed072fbe2d1eb1254"},"url":"https://jobsearcher.com/jobs/f1d7034ed072fbe2d1eb1254"}}