{"schemaVersion":"jobsearcher.job.v1","id":"b68091b04f7f7ca66b38721d","url":"https://jobsearcher.com/jobs/b68091b04f7f7ca66b38721d","canonicalUrl":"https://jobsearcher.com/jobs/b68091b04f7f7ca66b38721d","title":"Data Engineer","description":"Job DescriptionOverviewWe are seeking a highly experienced Data Engineer with 5+ years of experience. This role is critical to hitting product rollout deadlines, as the team's work is a hard, direct dependency for other product feature rollouts. The ideal candidate will be a hands-on developer with deep expertise in the AWS data stack, focusing primarily on data engineering and pipeline development.Key ResponsibilitiesDevelop and Implement Data Pipelines: Design, build, and maintain robust data pipelines primarily using AWS Glue and PySpark.Data Sourcing and Transformation: Source data from various systems, including Redshift and Aurora, performing necessary streaming transformations and heavy data cleaning.Data Delivery: Push resulting, cleaned datasets into S3 buckets.External Integration: Manage the secure transfer of resulting files via SFTP to an external 3rd party company's server, adhering to non-negotiable external integration deadlines.Collaboration: Work closely with the team to consult on the best and most efficient solutions for achieving required data outputs, given the constraints of the AWS Glue/PySpark environment. Required Qualifications and Skills WS Data Stack: Heavy expertise in the AWS ecosystem, specifically AWS Glue.PySpark Expertise: Hands-on experience working with PySpark on complex application implementations is required.Database Knowledge: Heavy knowledge of both relational (e.g., Redshift, Aurora) and non-SQL databases, and how to leverage them within the AWS Glue/PySpark environment.Experience Level: Looking for experienced engineers with .Data Engineering Fundamentals: Strong general knowledge of how to efficiently get, transform, and push out data. Job Responsibilities Key Responsibilities Develop and Implement Data Pipelines: Design, build, and maintain robust data pipelines primarily using AWS Glue and PySpark.Data Sourcing and Transformation: Source data from various systems, including Redshift and Aurora, performing necessary streaming transformations and heavy data cleaning.Data Delivery: Push resulting, cleaned datasets into S3 buckets.External Integration: Manage the secure transfer of resulting files via SFTP to an external 3rd party company's server, adhering to non-negotiable external integration deadlines.Collaboration: Work closely with the team to consult on the best and most efficient solutions for achieving required data outputs, given the constraints of the AWS Glue/PySpark environment.","company":"Intersources","rawCompany":"intersources","city":"Malvern","state":"PA","isRemote":false,"isActive":false,"createdAt":"2026-06-26T03:30:33.764Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Data Engineer","description":"Job DescriptionOverviewWe are seeking a highly experienced Data Engineer with 5+ years of experience. This role is critical to hitting product rollout deadlines, as the team's work is a hard, direct dependency for other product feature rollouts. The ideal candidate will be a hands-on developer with deep expertise in the AWS data stack, focusing primarily on data engineering and pipeline development.Key ResponsibilitiesDevelop and Implement Data Pipelines: Design, build, and maintain robust data pipelines primarily using AWS Glue and PySpark.Data Sourcing and Transformation: Source data from various systems, including Redshift and Aurora, performing necessary streaming transformations and heavy data cleaning.Data Delivery: Push resulting, cleaned datasets into S3 buckets.External Integration: Manage the secure transfer of resulting files via SFTP to an external 3rd party company's server, adhering to non-negotiable external integration deadlines.Collaboration: Work closely with the team to consult on the best and most efficient solutions for achieving required data outputs, given the constraints of the AWS Glue/PySpark environment. Required Qualifications and Skills WS Data Stack: Heavy expertise in the AWS ecosystem, specifically AWS Glue.PySpark Expertise: Hands-on experience working with PySpark on complex application implementations is required.Database Knowledge: Heavy knowledge of both relational (e.g., Redshift, Aurora) and non-SQL databases, and how to leverage them within the AWS Glue/PySpark environment.Experience Level: Looking for experienced engineers with .Data Engineering Fundamentals: Strong general knowledge of how to efficiently get, transform, and push out data. Job Responsibilities Key Responsibilities Develop and Implement Data Pipelines: Design, build, and maintain robust data pipelines primarily using AWS Glue and PySpark.Data Sourcing and Transformation: Source data from various systems, including Redshift and Aurora, performing necessary streaming transformations and heavy data cleaning.Data Delivery: Push resulting, cleaned datasets into S3 buckets.External Integration: Manage the secure transfer of resulting files via SFTP to an external 3rd party company's server, adhering to non-negotiable external integration deadlines.Collaboration: Work closely with the team to consult on the best and most efficient solutions for achieving required data outputs, given the constraints of the AWS Glue/PySpark environment.","datePosted":"2026-06-26T03:30:33.764Z","dateModified":"2026-06-26T03:30:33.764Z","hiringOrganization":{"@type":"Organization","name":"Intersources","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Malvern","addressRegion":"PA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"b68091b04f7f7ca66b38721d"},"url":"https://jobsearcher.com/jobs/b68091b04f7f7ca66b38721d"}}