{"schemaVersion":"jobsearcher.job.v1","id":"1ebab07c7d42d4c06055dfac","url":"https://jobsearcher.com/jobs/1ebab07c7d42d4c06055dfac","canonicalUrl":"https://jobsearcher.com/jobs/1ebab07c7d42d4c06055dfac","title":"Databricks Data Engineer","description":"Location: Tarrytown, NY (4 days per week onsite)Duration: 1 year ongoing contractOverviewWe are looking for a Senior Data Engineer with 7+ years of experience to support the newly launched VI Train Command Center. VI Train Command Center is a new internal platform designed to create one shared, current view of antibody campaigns, connecting data and work across teams to improve visibility into priorities, capacity, readiness, and progress.This person will own a data engineering stream end-to-end, from designing the solution and building data pipelines through processing and preparing datasets for use within Databricks applications. This is a highly independent role for someone who can not only execute technically, but also think ahead, identify potential gaps, and build scalable solutions that support future functionality.Must-Haves7+ years of Data Engineering experienceStrong hands-on Databricks experienceAdvanced Python and SQL skillsExperience designing and building end-to-end data pipelines and datasetsCloud experience with AWS strongly preferred; Azure or GCP experience is also applicableAbility to independently design data solutions, including data mapping, transformations, expected outputs, and overall pipeline architectureExcellent communication skills with the ability to present technical designs, explain decisions, and collaborate with the broader teamNice-to-HaveFront-end development experience with React, JavaScript, or similar technologiesExperience supporting data for user-facing applicationsExperience designing solutions with scalability, extensibility, and different user/access needs in mindDay-to-DayThis person will take full ownership of a stream within the VI Train Command Center. The role is approximately 80% backend/data engineering, focused on designing and building end-to-end pipelines, processing and preparing data, creating datasets, and making that data available within Databricks applications.The engineer will be responsible for understanding the required inputs and outputs, developing the technical design, presenting that design to the team for alignment, and then independently implementing the solution.Since VI Train is a new and evolving platform, the team needs someone who thinks beyond the immediate requirement and anticipates future needs. This includes considering scalability, flexibility, and how functionality or access requirements may differ across users as the platform grows.While the initial focus will be VI Train, this person will not be limited to this project long-term and can take ownership of additional data engineering streams as needs evolve.","company":"Insight Global","rawCompany":"insight global","city":"Tarrytown","state":"NY","isRemote":false,"isActive":false,"createdAt":"2026-10-23T01:26:44.983Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-2051.00","title":"Data Scientists","slug":"data-scientists"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"518210","title":"Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services","slug":"computing-infrastructure-providers-data-processing-web-hosting-and-related-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Databricks Data Engineer","description":"Location: Tarrytown, NY (4 days per week onsite)Duration: 1 year ongoing contractOverviewWe are looking for a Senior Data Engineer with 7+ years of experience to support the newly launched VI Train Command Center. VI Train Command Center is a new internal platform designed to create one shared, current view of antibody campaigns, connecting data and work across teams to improve visibility into priorities, capacity, readiness, and progress.This person will own a data engineering stream end-to-end, from designing the solution and building data pipelines through processing and preparing datasets for use within Databricks applications. This is a highly independent role for someone who can not only execute technically, but also think ahead, identify potential gaps, and build scalable solutions that support future functionality.Must-Haves7+ years of Data Engineering experienceStrong hands-on Databricks experienceAdvanced Python and SQL skillsExperience designing and building end-to-end data pipelines and datasetsCloud experience with AWS strongly preferred; Azure or GCP experience is also applicableAbility to independently design data solutions, including data mapping, transformations, expected outputs, and overall pipeline architectureExcellent communication skills with the ability to present technical designs, explain decisions, and collaborate with the broader teamNice-to-HaveFront-end development experience with React, JavaScript, or similar technologiesExperience supporting data for user-facing applicationsExperience designing solutions with scalability, extensibility, and different user/access needs in mindDay-to-DayThis person will take full ownership of a stream within the VI Train Command Center. The role is approximately 80% backend/data engineering, focused on designing and building end-to-end pipelines, processing and preparing data, creating datasets, and making that data available within Databricks applications.The engineer will be responsible for understanding the required inputs and outputs, developing the technical design, presenting that design to the team for alignment, and then independently implementing the solution.Since VI Train is a new and evolving platform, the team needs someone who thinks beyond the immediate requirement and anticipates future needs. This includes considering scalability, flexibility, and how functionality or access requirements may differ across users as the platform grows.While the initial focus will be VI Train, this person will not be limited to this project long-term and can take ownership of additional data engineering streams as needs evolve.","datePosted":"2026-10-23T01:26:44.983Z","dateModified":"2026-10-23T01:26:44.983Z","hiringOrganization":{"@type":"Organization","name":"Insight Global","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Tarrytown","addressRegion":"NY","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"1ebab07c7d42d4c06055dfac"},"url":"https://jobsearcher.com/jobs/1ebab07c7d42d4c06055dfac"}}