Data / ML Platform Engineer (Python)

Overview:Data / ML Platform Engineer (Python, LLM Pipelines, Batch Processing)Job SummaryWe are looking for aData / ML Platform Engineerto design and managePython-based batch pipelinesfor evaluating machine learning and LLM models.The role involves building scalable pipelines that process large datasets, interact with LLM APIs, and store structured outputs for analytics and benchmarking.Key ResponsibilitiesML / Data Pipeline DevelopmentBuild and maintainPython batch pipelinesfor:Model evaluation Dataset processingDesignresumable and fault-tolerant workflows LLM IntegrationIntegrate with:GPT / Claude / other LLM APIsExecute large-scale evaluation workflows Data EngineeringLoad and process data from:Cloud storage (S3, GCS, Azure Blob)Transform and structure data for analytics Database & StorageStore results in:MongoDB (structured outputs)Ensure efficient data retrieval and storage Performance & ScalabilityOptimize:Pipeline execution Cost and latencyHandle high-volume batch jobs Cloud & DevOpsDeploy pipelines on:AWS / GCP / AzureUse:CI/CD pipelines Version control (Git)Monitoring & ReliabilityImplement:Logging and monitoring Error handling & retriesEnsure pipeline stability •Required SkillsCore SkillsStrongPython development Experience with:Batch pipelines / ETL Data processingAI / MLHands-on with:LLM APIs (GPT, Claude) Prompt engineeringExperience with:RAG pipelines (preferred)Data & StorageMongoDB / NoSQL databases Data structuring and transformation CloudAWS / GCP / Azure Cloud storage services DevOpsGit / GitHub CI/CD pipelines Experience Required5-8 years total experience 2+ years inML / data pipelines / LLM-based systems