{"schemaVersion":"jobsearcher.job.v1","id":"9fe698c067de943ecc3b458c","url":"https://jobsearcher.com/jobs/9fe698c067de943ecc3b458c","canonicalUrl":"https://jobsearcher.com/jobs/9fe698c067de943ecc3b458c","title":"Senior Data Engineer","description":"About the RoleWe are looking for an experienced Lead Data Engineer with deep expertise in Databricks, PySpark, and AWS. The ideal candidate will have a strong understanding of media measurement datasets (Nielsen) and a proven ability to lead multiple concurrent projects across onsite and offshore teams. This role requires a senior technical leader to define architecture, manage delivery, and mentor teams within the fast-paced New York media industry.Key Responsibilities- Technical Leadership: Architect and deliver enterprise-scale data platforms using Databricks, PySpark, Delta Lake, and AWS. Lead the development of media analytics pipelines, including the ingestion and modeling of Nielsen data.- Project & Delivery Management: Manage scope, timelines, and risks for multiple simultaneous projects. Collaborate with New York-based stakeholders and drive sprint delivery.- Onsite–Offshore Coordination: Act as the primary technical lead onsite, coordinating with offshore teams to ensure alignment, quality, and continuity.- Team Mentorship: Mentor junior and mid-level engineers through code reviews and architectural guidance.Required Skills- Databricks: Delta Lake, Workflows, Unity Catalog, Notebooks, and SQL.- Data Processing: Strong PySpark and distributed computing experience.- AWS: S3, EMR, Glue, Lambda, Step Functions, Kinesis, and IAM.- Domain Expertise: Proven experience handling Nielsen datasets (National/Local TV ratings, Digital Ad ratings, Panels).- Leadership: Experience leading large data engineering programs and managing senior stakeholders.Good to Have- Knowledge of Snowflake, Airflow, or Dagster.- Infrastructure as Code (Terraform / CloudFormation).- Exposure to ad-tech ecosystems (Comscore, Kantar, IAS, Moat) or ML feature pipelines.Soft Skills- Strong decision-making and ownership mindset.- Exceptional communication skills for interacting with senior business stakeholders.- Strategic, problem-solving, and delivery-focused approach.","company":"Sg Analytics","rawCompany":"sg analytics","city":"Brooklyn","state":"NY","isRemote":false,"isActive":false,"createdAt":"2026-05-14T05:57:23.192Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-2051.00","title":"Data Scientists","slug":"data-scientists"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"518210","title":"Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services","slug":"computing-infrastructure-providers-data-processing-web-hosting-and-related-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Senior Data Engineer","description":"About the RoleWe are looking for an experienced Lead Data Engineer with deep expertise in Databricks, PySpark, and AWS. The ideal candidate will have a strong understanding of media measurement datasets (Nielsen) and a proven ability to lead multiple concurrent projects across onsite and offshore teams. This role requires a senior technical leader to define architecture, manage delivery, and mentor teams within the fast-paced New York media industry.Key Responsibilities- Technical Leadership: Architect and deliver enterprise-scale data platforms using Databricks, PySpark, Delta Lake, and AWS. Lead the development of media analytics pipelines, including the ingestion and modeling of Nielsen data.- Project & Delivery Management: Manage scope, timelines, and risks for multiple simultaneous projects. Collaborate with New York-based stakeholders and drive sprint delivery.- Onsite–Offshore Coordination: Act as the primary technical lead onsite, coordinating with offshore teams to ensure alignment, quality, and continuity.- Team Mentorship: Mentor junior and mid-level engineers through code reviews and architectural guidance.Required Skills- Databricks: Delta Lake, Workflows, Unity Catalog, Notebooks, and SQL.- Data Processing: Strong PySpark and distributed computing experience.- AWS: S3, EMR, Glue, Lambda, Step Functions, Kinesis, and IAM.- Domain Expertise: Proven experience handling Nielsen datasets (National/Local TV ratings, Digital Ad ratings, Panels).- Leadership: Experience leading large data engineering programs and managing senior stakeholders.Good to Have- Knowledge of Snowflake, Airflow, or Dagster.- Infrastructure as Code (Terraform / CloudFormation).- Exposure to ad-tech ecosystems (Comscore, Kantar, IAS, Moat) or ML feature pipelines.Soft Skills- Strong decision-making and ownership mindset.- Exceptional communication skills for interacting with senior business stakeholders.- Strategic, problem-solving, and delivery-focused approach.","datePosted":"2026-05-14T05:57:23.192Z","dateModified":"2026-05-14T05:57:23.192Z","hiringOrganization":{"@type":"Organization","name":"Sg Analytics","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Brooklyn","addressRegion":"NY","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"9fe698c067de943ecc3b458c"},"url":"https://jobsearcher.com/jobs/9fe698c067de943ecc3b458c"}}