Lead, Big Data Administrator
About Our Client:
The organization operates within the data and identity sector, addressing the challenge of unlocking the full potential of data to power connected marketing. It supports brands in delivering personalized experiences at scale and building lasting customer relationships. The company has a global presence spanning over 30 countries across EMEA, the Americas, and APAC, employing more than 8,000 people.
About the Opportunity:
The LEAD, BIG DATA ADMINISTRATOR is responsible for deploying and managing large-scale distributed systems to ensure their performance, availability, and scalability. This role is crucial for maintaining and optimizing Hadoop and Kafka ecosystems, supporting data pipeline orchestration, and enhancing operational efficiency through automation. The position contributes directly to enabling data-driven solutions that support business requirements and analytics capabilities.
Responsibilities:
- Deploy and manage large-scale distributed systems
- Oversee installation, configuration, and maintenance of Hadoop components (HDFS, Yarn, Hive, Spark) and Kafka clusters
- Implement high-availability and disaster recovery strategies
- Conduct performance tuning and resource allocation for system efficiency
- Manage data pipeline integration between Kafka and processing frameworks
- Ensure data security via encryption, access control, and auditing
- Troubleshoot cluster health, data loss, and Kafka offset issues
- Automate repetitive tasks using tools like Ansible and shell scripting
- Collaborate with development, data engineering, and analytics teams
- Stay updated on emerging big data and streaming technologies for system improvements
Requirements:
- Bachelor’s degree in Computer Applications or related field, plus eight years of relevant progressive experience
- Proficiency with Hadoop components (HDFS, Yarn, Hive, Spark) and Kafka clusters
- Experience using ZooKeeper for coordination in Hadoop and Kafka
- Knowledge of Kerberos, LDAP integration, SSL/TLS encryption, and audit tools such as Ranger or Atlas
- Ability to use Prometheus and Grafana for monitoring and alerting
- Skill in diagnosing and resolving cluster connectivity and performance issues
- Expertise in fine-tuning cluster configurations
- Proficiency in Bash and Python scripting for automation
- Pay Range and Compensation Package:
The pay range and compensation package for this role will be determined based on the candidate's experience, skills, and other relevant factors.
Benefits & Perks:
- Flexible time off, paid holidays, and paid sick time
- Parental and new child leave, childcare and elder care assistance, adoption assistance
- Comprehensive health coverage 401(k) retirement plan
- Tuition assistance
- Commuter benefits
Equal Opportunity Statement: Our client is an equal opportunity employer. They celebrate diversity and are committed to creating an inclusive environment for all employees. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, or national origin.
Note:
RemoteHunter is a recruitment partner of this role. Please note that all employment decisions, including candidate assessment, interviews, hiring, compensation, and employment terms, are made exclusively by the hiring employer.