Data Platform Engineer - AI
We are looking for a hands-on Technical Lead – Data Platform & Services Engineering with strong expertise in Azure Data Engineering, backend services, APIs, analytics, and cloud architecture. The candidate will design, build, modernize, and operate enterprise-scale data platforms and services on Microsoft Azure.
Key Responsibilities: Data Platform & Engineering:
- Design and build scalable Azure data platforms, ETL/ELT pipelines, and analytical solutions.
- Develop large-scale data processing using Azure Databricks, Spark/PySpark, Azure Synapse, and Azure Data Factory.
- Design Lakehouse/Medallion architectures using ADLS Gen2, Delta Lake, and Bronze/Silver/Gold layers
- Backend Services & Integration:
- Design and develop REST APIs and backend services using C#, .NET, and ASP.NET Core.
- Build data and integration services using Azure SQL, Cosmos DB, Kusto, Azure Functions, Event Hubs, and Service Bus.
- Implement microservices and event-driven processing patterns with secure authentication and authorization. Modernization & Reliability
- Modernize legacy data pipelines, services, APIs, storage platforms, and application components.
- Support platform and data migrations while ensuring functional and data parity.
- Implement logging, telemetry, dashboards, and proactive alerting using Azure Monitor and Application Insights
- Cloud, DevOps & Technical Leadership
- Build and maintain CI/CD pipelines using Azure DevOps and Git.
- Implement secure cloud practices using Azure Key Vault, identity, authentication, and access controls.
- Support Docker and Kubernetes/AKS-based workloads where applicable.
- Mandatory Skills:
- Data Engineering
- Azure Databricks, Spark/PySparkAzure Synapse Analytics, Azure Data FactoryADLS Gen2, Delta LakeSQL Server / Azure SQLCosmos DB, Kusto / Azure Data ExplorerETL/ELT, Data Warehousing & Data Modeling
- Lakehouse / Medallion Architecture
- Data Quality, Reconciliation & Governance
- Backend & ServicesC#, .NET / ASP.NET CoreREST APIs & Microservices
- Azure Functions
- Event Hubs & Service BusEntity Framework Core
- Authentication & Authorization
- Cloud & DevOpsMicrosoft AzureAzure DevOps, Git & CI/CDAzure Monitor & Application Insights
- Azure Key VaultDocker / Kubernetes
- Good to HaveReact or Angular, TypeScriptGraphQLPower BIInfrastructure as CodeSRE / Production Support experienceAI/ML or AI-enabled analytics exposure