{"schemaVersion":"jobsearcher.job.v1","id":"3410ff12d42ffa9e831ba707","url":"https://jobsearcher.com/jobs/3410ff12d42ffa9e831ba707","canonicalUrl":"https://jobsearcher.com/jobs/3410ff12d42ffa9e831ba707","title":"Sr. Data Engineer","description":"Designs and automates deployment of our distributed system for ingesting and transforming data from various types of sources (relational, event-based, unstructured).\r\nOwn end-to-end delivery of AI and ML solutions from problem definition to production deployment\r\nBuild and maintain data pipelines using PySpark and Spark in Azure Databricks\r\nDesigns and implements framework to continuously monitor and troubleshoot data quality and data integrity issues.\r\nImplements data governance processes and methods for managing metadata, access, retention to data for internal and external users.\r\nDesigns and provide guidance on building reliable, efficient, scalable and quality data pipelines with monitoring and alert mechanisms that combine a variety of sources using ETL/ELT tools or scripting languages.\r\nPerform feature engineering and data preparation for machine learning models\r\nDeploy ML models and AI agents for real business use cases\r\nDesign and implement agentic AI workflows including multi-step reasoning and tool usage\r\nTrack experiments, manage models, and support deployment using MLflow\r\nDefine and execute model evaluation frameworks including both ML and AI agent performance\r\nDesigns and implements physical data models to define the database structure. Optimizing database performance through efficient indexing and table relationships.\r\nParticipates in optimizing, testing, and troubleshooting of data pipelines.\r\nDesigns, develops and operates large scale data storage and processing solutions using different distributed and cloud-based platforms for storing data (e.g. Azure , AWS, Spark , Pyspark, Scala , advanced SQL , Hadoop others).\r\nUses innovative and modern tools, techniques and architectures to partially or completely automate the most-common, repeatable and tedious data preparation and integration tasks to minimize manual and error-prone processes and improve productivity. Assists with renovating the data management infrastructure to drive automation in data integration and management.\r\nEnsures the timeliness and success of critical analytics initiatives by using agile development technologies such as DevOps, Scrum, Kanban\r\nCoaches and develops less experienced team members.\r\nDefine and execute model evaluation frameworks including both ML and AI agent performance\r\nWork independently on ambiguous business problems and convert them into scalable solutions\r\nCollaborate with other data engineers, analysts, solution architects and business teams to deliver solutions\r\nGuide and support team members on data engineering, ML, and AI best practices\r\nWrite clean, production-ready, and well-documented code\r\nRequirementsSystem Requirements Engineering - Uses appropriate methods and tools to translate stakeholder needs into verifiable requirements to which designs are developed; establishes acceptance criteria for the system of interest through analysis, allocation and negotiation; tracks the status of requirements throughout the system lifecycle; assesses the impact of changes to system requirements on project scope, schedule, and resources; creates and maintains information linkages to related artifacts.\r\nCollaborates - Building partnerships and working collaboratively with others to meet shared objectives.\r\nCommunicates effectively - Developing and delivering multi-mode communications that convey a clear understanding of the unique needs of different audiences.\r\nCustomer focus - Building strong customer relationships and delivering customer-centric solutions.\r\nDecision quality - Making good and timely decisions that keep the organization moving forward.\r\nData Extraction - Performs data extract-transform-load (ETL) activities from variety of sources and transforms them for consumption by various downstream applications and users using appropriate tools and technologies.\r\nProgramming - Creates, writes and tests computer code, test scripts, and build scripts using algorithmic analysis and design, industry standards and tools, version control, and build and test automation to meet business, technical, security, governance and compliance requirements.\r\nQuality Assurance Metrics - Applies the science of measurement to assess whether a solution meets its intended outcomes using the IT Operating Model (ITOM), including the SDLC standards, tools, metrics and key performance indicators, to deliver a quality product.\r\nSolution Documentation - Documents information and solution based on knowledge gained as part of product development activities; communicates to stakeholders with the goal of enabling improved productivity and effective knowledge transfer to others who were not originally part of the initial learning.\r\nSolution Validation Testing - Validates a configuration item change or solution using the Function's defined best practices, including the Systems Development Life Cycle (SDLC) standards, tools and metrics, to ensure that it works as designed and meets customer requirements.\r\nData Quality - Identifies, understands and corrects flaws in data that supports effective information governance across operational business processes and decision making.\r\nProblem Solving - Solves problems and may mentor others on effective problem solving by using a systematic analysis process by leveraging industry standard methodologies to create problem traceability and protect the customer; determines the assignable cause; implements robust, data-based solutions; identifies the systemic root causes and ensures actions to prevent problem reoccurrence are implemented.\r\nValues differences - Recognizing the value that different perspectives and cultures bring to an organization.\r\nCollege, university, or equivalent degree in relevant technical discipline, or relevant equivalent experience required.\r\nAt least 5 years of experience in data engineering with a strong background on Azure Databricks and Scala/Python.\r\nExperience in handling unstructured data processing and transformation with programming knowledge.\r\nHands on experience in building data pipelines using Scala/Python\r\nBig data technologies such as Apache Spark, Structured Streaming, Advanced SQL, Databricks, Delta Lake, Azure/AWS\r\nStrong analytical and problem-solving skills with the ability to troubleshoot spark applications and resolve data pipeline issues.\r\nFamiliarity with version control systems like Git, CICD pipelines.\r\nExperience with Azure Databricks and MLflow\r\nGood understanding of ML workflows, model development, and evaluation\r\nKnowledge of MLOps fundamentals such as CI/CD, versioning, and monitoring\r\nAbility to build end-to-end data and ML solutions\r\nExposure to production ML or AI systems\r\nUnderstanding of data engineering and data modeling basics\r\nAbility to work independently on loosely defined problems\r\nStrong problem-solving and communication skills\r\nMentoring experience is a plus\r\nPreferredExperience with AI agents, LLMs, or agentic AI systems\r\nCompensation and BenefitsAlong with competitive pay, as a full-time KPIT employee, you are eligible for the following benefits:\r\nGeo Blue PPO and HSA plan.\r\nMetLife – Dental and Vision plan.\r\nHealthcare and Dependent care flexible spending account(FSA).\r\n401k with employer match.\r\nCompany-paid Basic Life and Long-term disability insurance.\r\nVoluntary benefits include Critical Illness, Hospital indemnity, accident insurance, theft, and legal service.\r\nEmployee Assistance Program.\r\nPaid Holidays.\r\nEmployee discounts and perks.\r\nRequirementESSENTIAL SKILLS /COMPETENCIESSystem Requirements Engineering\r\nRequirements Management\r\nRequirements Traceability\r\nRequirements Analysis\r\nAcceptance Criteria Definition\r\nChange Impact Analysis\r\nStakeholder Management\r\nCommunication Skills\r\nCustomer Focus\r\nETL Development\r\nData Extraction Transformation and Loading (ETL)\r\nScala\r\nPython\r\nPREFFERED SKILLS /COMPETENCIESAI agents\r\nLLMs\r\nAgentic AI systems\r\nVideo\r\nWe are Automobelievers!\r\nAcross mobility & technology domains - Upskill yourself in an environment designed for constant learning\r\nKPIT is the Best Place to Grow\r\nWork on cutting edge technology programs of leading OEMS and Tier 1s in automotive and mobility\r\nWork across Autonomous, Connected, Electrification, AUTOSAR, Cybersecurity, OTA technology driving mobility transformation\r\nRobust competency development framework, Individual development plans and Mentors to bring focus on YOU\r\nHighest numbers of promotions backed by solid performance\r\nCollaboration with global university for technology & management master's program#J-18808-Ljbffr","company":"Kpit Technologies","rawCompany":"kpit technologies","city":"Columbus","state":"OH","isRemote":false,"isActive":false,"createdAt":"2026-09-02T01:10:50.668Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-2051.00","title":"Data Scientists","slug":"data-scientists"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Sr. Data Engineer","description":"Designs and automates deployment of our distributed system for ingesting and transforming data from various types of sources (relational, event-based, unstructured).\r\nOwn end-to-end delivery of AI and ML solutions from problem definition to production deployment\r\nBuild and maintain data pipelines using PySpark and Spark in Azure Databricks\r\nDesigns and implements framework to continuously monitor and troubleshoot data quality and data integrity issues.\r\nImplements data governance processes and methods for managing metadata, access, retention to data for internal and external users.\r\nDesigns and provide guidance on building reliable, efficient, scalable and quality data pipelines with monitoring and alert mechanisms that combine a variety of sources using ETL/ELT tools or scripting languages.\r\nPerform feature engineering and data preparation for machine learning models\r\nDeploy ML models and AI agents for real business use cases\r\nDesign and implement agentic AI workflows including multi-step reasoning and tool usage\r\nTrack experiments, manage models, and support deployment using MLflow\r\nDefine and execute model evaluation frameworks including both ML and AI agent performance\r\nDesigns and implements physical data models to define the database structure. Optimizing database performance through efficient indexing and table relationships.\r\nParticipates in optimizing, testing, and troubleshooting of data pipelines.\r\nDesigns, develops and operates large scale data storage and processing solutions using different distributed and cloud-based platforms for storing data (e.g. Azure , AWS, Spark , Pyspark, Scala , advanced SQL , Hadoop others).\r\nUses innovative and modern tools, techniques and architectures to partially or completely automate the most-common, repeatable and tedious data preparation and integration tasks to minimize manual and error-prone processes and improve productivity. Assists with renovating the data management infrastructure to drive automation in data integration and management.\r\nEnsures the timeliness and success of critical analytics initiatives by using agile development technologies such as DevOps, Scrum, Kanban\r\nCoaches and develops less experienced team members.\r\nDefine and execute model evaluation frameworks including both ML and AI agent performance\r\nWork independently on ambiguous business problems and convert them into scalable solutions\r\nCollaborate with other data engineers, analysts, solution architects and business teams to deliver solutions\r\nGuide and support team members on data engineering, ML, and AI best practices\r\nWrite clean, production-ready, and well-documented code\r\nRequirementsSystem Requirements Engineering - Uses appropriate methods and tools to translate stakeholder needs into verifiable requirements to which designs are developed; establishes acceptance criteria for the system of interest through analysis, allocation and negotiation; tracks the status of requirements throughout the system lifecycle; assesses the impact of changes to system requirements on project scope, schedule, and resources; creates and maintains information linkages to related artifacts.\r\nCollaborates - Building partnerships and working collaboratively with others to meet shared objectives.\r\nCommunicates effectively - Developing and delivering multi-mode communications that convey a clear understanding of the unique needs of different audiences.\r\nCustomer focus - Building strong customer relationships and delivering customer-centric solutions.\r\nDecision quality - Making good and timely decisions that keep the organization moving forward.\r\nData Extraction - Performs data extract-transform-load (ETL) activities from variety of sources and transforms them for consumption by various downstream applications and users using appropriate tools and technologies.\r\nProgramming - Creates, writes and tests computer code, test scripts, and build scripts using algorithmic analysis and design, industry standards and tools, version control, and build and test automation to meet business, technical, security, governance and compliance requirements.\r\nQuality Assurance Metrics - Applies the science of measurement to assess whether a solution meets its intended outcomes using the IT Operating Model (ITOM), including the SDLC standards, tools, metrics and key performance indicators, to deliver a quality product.\r\nSolution Documentation - Documents information and solution based on knowledge gained as part of product development activities; communicates to stakeholders with the goal of enabling improved productivity and effective knowledge transfer to others who were not originally part of the initial learning.\r\nSolution Validation Testing - Validates a configuration item change or solution using the Function's defined best practices, including the Systems Development Life Cycle (SDLC) standards, tools and metrics, to ensure that it works as designed and meets customer requirements.\r\nData Quality - Identifies, understands and corrects flaws in data that supports effective information governance across operational business processes and decision making.\r\nProblem Solving - Solves problems and may mentor others on effective problem solving by using a systematic analysis process by leveraging industry standard methodologies to create problem traceability and protect the customer; determines the assignable cause; implements robust, data-based solutions; identifies the systemic root causes and ensures actions to prevent problem reoccurrence are implemented.\r\nValues differences - Recognizing the value that different perspectives and cultures bring to an organization.\r\nCollege, university, or equivalent degree in relevant technical discipline, or relevant equivalent experience required.\r\nAt least 5 years of experience in data engineering with a strong background on Azure Databricks and Scala/Python.\r\nExperience in handling unstructured data processing and transformation with programming knowledge.\r\nHands on experience in building data pipelines using Scala/Python\r\nBig data technologies such as Apache Spark, Structured Streaming, Advanced SQL, Databricks, Delta Lake, Azure/AWS\r\nStrong analytical and problem-solving skills with the ability to troubleshoot spark applications and resolve data pipeline issues.\r\nFamiliarity with version control systems like Git, CICD pipelines.\r\nExperience with Azure Databricks and MLflow\r\nGood understanding of ML workflows, model development, and evaluation\r\nKnowledge of MLOps fundamentals such as CI/CD, versioning, and monitoring\r\nAbility to build end-to-end data and ML solutions\r\nExposure to production ML or AI systems\r\nUnderstanding of data engineering and data modeling basics\r\nAbility to work independently on loosely defined problems\r\nStrong problem-solving and communication skills\r\nMentoring experience is a plus\r\nPreferredExperience with AI agents, LLMs, or agentic AI systems\r\nCompensation and BenefitsAlong with competitive pay, as a full-time KPIT employee, you are eligible for the following benefits:\r\nGeo Blue PPO and HSA plan.\r\nMetLife – Dental and Vision plan.\r\nHealthcare and Dependent care flexible spending account(FSA).\r\n401k with employer match.\r\nCompany-paid Basic Life and Long-term disability insurance.\r\nVoluntary benefits include Critical Illness, Hospital indemnity, accident insurance, theft, and legal service.\r\nEmployee Assistance Program.\r\nPaid Holidays.\r\nEmployee discounts and perks.\r\nRequirementESSENTIAL SKILLS /COMPETENCIESSystem Requirements Engineering\r\nRequirements Management\r\nRequirements Traceability\r\nRequirements Analysis\r\nAcceptance Criteria Definition\r\nChange Impact Analysis\r\nStakeholder Management\r\nCommunication Skills\r\nCustomer Focus\r\nETL Development\r\nData Extraction Transformation and Loading (ETL)\r\nScala\r\nPython\r\nPREFFERED SKILLS /COMPETENCIESAI agents\r\nLLMs\r\nAgentic AI systems\r\nVideo\r\nWe are Automobelievers!\r\nAcross mobility & technology domains - Upskill yourself in an environment designed for constant learning\r\nKPIT is the Best Place to Grow\r\nWork on cutting edge technology programs of leading OEMS and Tier 1s in automotive and mobility\r\nWork across Autonomous, Connected, Electrification, AUTOSAR, Cybersecurity, OTA technology driving mobility transformation\r\nRobust competency development framework, Individual development plans and Mentors to bring focus on YOU\r\nHighest numbers of promotions backed by solid performance\r\nCollaboration with global university for technology & management master's program#J-18808-Ljbffr","datePosted":"2026-09-02T01:10:50.668Z","dateModified":"2026-09-02T01:10:50.668Z","hiringOrganization":{"@type":"Organization","name":"Kpit Technologies","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Columbus","addressRegion":"OH","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"3410ff12d42ffa9e831ba707"},"url":"https://jobsearcher.com/jobs/3410ff12d42ffa9e831ba707"}}