Senior Hadoop Developer (AB Initio, Spark, Python)

Job SummaryWe are seeking a skilled Senior Hadoop Developer with strong experience in AB Initio, Hadoop ecosystem, Spark, Python, and SQL. The ideal candidate will be responsible for designing, developing, and maintaining scalable data processing systems and ETL pipelines that support enterprise-level data platforms.This role requires strong technical expertise in big data technologies, hands-on development experience, and the ability to work collaboratively in a fast-paced Agile environment.Key ResponsibilitiesDesign, develop, and maintain scalable data pipelines and distributed data processing systems.Develop and optimize ETL workflows using AB Initio and Hadoop ecosystem tools.Build and maintain data processing applications using Apache Spark and Python.Write efficient SQL queries and stored procedures for data extraction and transformation.Develop APIs and integrate data across multiple platforms and systems.Participate in code reviews and ensure adherence to coding standards and best practices.Perform unit testing, debugging, and performance tuning of applications.Collaborate with business analysts, data engineers, and project managers to define requirements.Document technical specifications, system designs, and integration points.Ensure compliance with IT security, data governance, and control standards.Support production deployments, troubleshooting, and incident resolution.Required Skills & Qualifications8–10+ years of experience in Data Engineering or Big Data development.Strong hands-on experience with AB Initio.Expertise in Hadoop ecosystem (HDFS, Hive, YARN, MapReduce concepts).Strong experience with Apache Spark (batch processing, optimization techniques).Proficiency in Python for data processing and automation.Strong knowledge of SQL and stored procedures.Experience with version control tools (Git/SVN).Solid understanding of SDLC, Agile methodologies, and release processes.Experience in unit testing, debugging, and performance tuning.Strong analytical, problem-solving, and communication skills.Preferred SkillsExperience with Snowflake or other cloud data platforms.Experience with CI/CD pipelines and DevOps practices.Knowledge of data warehousing concepts and data modeling.Experience with API development and system integration.