Java/ Big Data so Java Spark databricks

Key Responsibilities: Deliver large-scale cloud-native data platforms primarily on AWS using REST APIs, microservices, and event-driven applications. Work hands-on across Java, Python, Spark, PySpark, TypeScript, JavaScript, Angular, AWS services, event-driven architectures, and SQL/NoSQL databases. Design and develop scalable distributed systems and high-performance data processing solutions using Apache Spark and PySpark. Develop and maintain REST APIs, microservices, and Kubernetes-based containerized applications. Drive performance optimization, scalability, reliability, security, governance, and cost efficiency across data platforms. Design and implement solutions involving Data Warehousing, Data Lakes, and Delta Lake architectures. Develop cloud-native data platforms using AWS services such as S3, Lambda, API Gateway, and EventBridge. Implement messaging and event-driven solutions using Kafka, SNS, and SQS.Work with relational and NoSQL databases including PostgreSQL, SQL Server, Aurora, DynamoDB, MongoDB, and Redis. Implement Infrastructure as Code using Terraform and Ansible. Support CI/CD and DevOps practices using Jenkins, GitHub/GitLab, Bitbucket, GoCD, and automated deployment pipelines. Implement unit, integration, and regression testing strategies while following engineering best practices. Own the end-to-end software development lifecycle, including requirements gathering, solution design, development, deployment, observability, and documentation. Collaborate with global engineering, product management, architecture, and business stakeholders to align technical solutions with business objectives. Diagnose, troubleshoot, and resolve complex technical problems using strong critical thinking and analytical skills. Apply awareness of Generative AI technologies, including LLMs, RAG architectures, and Agentic AI systems, where applicable.

Required Qualifications: B.E., B.Tech, M.Tech, MCA, or equivalent degree in Computer Science, Information Technology, or a related field.5-8 years of strong software engineering experience building scalable applications and distributed systems. Strong hands-on expertise in Java and Python. Strong experience with Apache Spark and PySpark for distributed data processing. Experience with Data Warehousing, Data Lakes, Delta Lake architecture, and modern Big Data ecosystem designs. Proven experience designing and building scalable REST APIs, microservices, Kubernetes-based applications, and distributed systems. Strong understanding of object-oriented design patterns and functional programming. Experience with AWS services including S3, Lambda, API Gateway, and EventBridge. Strong experience with Kafka, SNS, and SQS and event-driven architectures. Experience with relational and NoSQL databases such as PostgreSQL, SQL Server, Aurora, DynamoDB, MongoDB, and Redis. Hands-on experience with Terraform and Ansible for Infrastructure as Code. Understanding of CI/CD and DevOps practices and automated deployment pipelines. Experience implementing unit, integration, and regression testing strategies. Strong critical thinking, analytical, troubleshooting, and problem-solving skills. Awareness of Generative AI technologies, including LLMs, RAG architectures, and Agentic AI systems.

Preferred Qualifications: Working knowledge of PySpark with Databricks. Experience working with Azure and/or Google Cloud Platform (GCP).Experience building data platforms in privacy-safe, Customer Data Platform, or Marketing Technology environments. Experience with Angular, TypeScript, and JavaScript in UX-driven applications.