{"schemaVersion":"jobsearcher.job.v1","id":"bcdbf58f66d2c0a8ab9410e2","url":"https://jobsearcher.com/jobs/bcdbf58f66d2c0a8ab9410e2","canonicalUrl":"https://jobsearcher.com/jobs/bcdbf58f66d2c0a8ab9410e2","title":"Data Engineer ETL","description":"We are seeking a highly motivated Data Engineer (ETL) to design, develop, and maintain scalable data engineering solutions supporting enterprise data aggregation and integration initiatives. This role focuses on building high-performance ETL pipelines, developing Apache Spark applications, processing large-scale datasets, and leveraging cloud-native technologies to deliver reliable and efficient data solutions. The ideal candidate will have strong expertise in Spark, Scala, Java, cloud platforms, and big data technologies while collaborating with cross-functional teams in an Agile environment.\r\nKey Responsibilities Design, develop, and maintain scalable ETL pipelines for enterprise data integration and movement.\r\nDevelop, test, deploy, and support ETL batch jobs using Apache Spark and Amazon EMR.\r\nBuild and maintain Spark services for large-scale data processing.\r\nSchedule, monitor, troubleshoot, and optimize Spark batch processing jobs.\r\nDevelop Java clients for consuming REST and SOAP APIs.\r\nDevelop and maintain Scala-based Spark batch applications.\r\nProcess and transform large datasets using Apache Spark.\r\nStore and manage processed data in NoSQL databases such as Cassandra or Yugabyte.\r\nDesign and implement cloud-based data warehouse and data lake solutions using Snowflake.\r\nWork with large-scale data storage formats including Parquet and HDF5.\r\nCollaborate with cross-functional teams to design innovative data solutions that address business requirements.\r\nParticipate in code reviews, pair programming, and technical discussions to ensure code quality and best practices.\r\nSupport the full Software Development Life Cycle (SDLC), including analysis, design, development, testing, deployment, and maintenance.\r\nTroubleshoot production issues and optimize data processing performance.\r\nIdentify opportunities for innovation and continuous improvement in data engineering practices.\r\nCollaborate across multiple Agile teams to deliver scalable and reliable data solutions.\r\nApply a customer-first mindset when designing and delivering enterprise data solutions.\r\nRequired Qualifications Bachelor's degree in Computer Science, Information Systems, or a related field.\r\n10+ years of experience developing Spark services for large-scale data movement.\r\nExperience scheduling, monitoring, debugging, and optimizing ETL Spark batch processes.\r\nHands-on experience developing Java clients for REST and SOAP API integrations.\r\nExperience developing Scala Spark batch applications.\r\nExperience developing, deploying, and maintaining Apache Spark and Amazon EMR ETL jobs.\r\nExperience processing big data using Apache Spark.\r\nExperience with NoSQL databases such as Cassandra or Yugabyte.\r\nExperience with cloud-based data warehouse and data lake platforms such as Snowflake.\r\nExperience managing large datasets using storage formats such as Parquet and HDF5.\r\nStrong understanding of software development lifecycle (SDLC) methodologies.\r\nStrong analytical, troubleshooting, and problem-solving skills.\r\nExperience working in Agile software development environments.\r\nExcellent collaboration and communication skills.\r\nPreferred Qualifications Experience designing enterprise-scale data integration and aggregation solutions.\r\nExperience supporting external-facing data platforms and APIs.\r\nExperience working with financial services or FinTech data platforms.\r\nPassion for continuous learning and adopting modern data engineering best practices.\r\nJ-18808-Ljbffr","company":"Compunnel","rawCompany":"compunnel","city":"Durham","state":"NC","isRemote":false,"isActive":false,"createdAt":"2026-07-16T02:02:25.677Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Data Engineer ETL","description":"We are seeking a highly motivated Data Engineer (ETL) to design, develop, and maintain scalable data engineering solutions supporting enterprise data aggregation and integration initiatives. This role focuses on building high-performance ETL pipelines, developing Apache Spark applications, processing large-scale datasets, and leveraging cloud-native technologies to deliver reliable and efficient data solutions. The ideal candidate will have strong expertise in Spark, Scala, Java, cloud platforms, and big data technologies while collaborating with cross-functional teams in an Agile environment.\r\nKey Responsibilities Design, develop, and maintain scalable ETL pipelines for enterprise data integration and movement.\r\nDevelop, test, deploy, and support ETL batch jobs using Apache Spark and Amazon EMR.\r\nBuild and maintain Spark services for large-scale data processing.\r\nSchedule, monitor, troubleshoot, and optimize Spark batch processing jobs.\r\nDevelop Java clients for consuming REST and SOAP APIs.\r\nDevelop and maintain Scala-based Spark batch applications.\r\nProcess and transform large datasets using Apache Spark.\r\nStore and manage processed data in NoSQL databases such as Cassandra or Yugabyte.\r\nDesign and implement cloud-based data warehouse and data lake solutions using Snowflake.\r\nWork with large-scale data storage formats including Parquet and HDF5.\r\nCollaborate with cross-functional teams to design innovative data solutions that address business requirements.\r\nParticipate in code reviews, pair programming, and technical discussions to ensure code quality and best practices.\r\nSupport the full Software Development Life Cycle (SDLC), including analysis, design, development, testing, deployment, and maintenance.\r\nTroubleshoot production issues and optimize data processing performance.\r\nIdentify opportunities for innovation and continuous improvement in data engineering practices.\r\nCollaborate across multiple Agile teams to deliver scalable and reliable data solutions.\r\nApply a customer-first mindset when designing and delivering enterprise data solutions.\r\nRequired Qualifications Bachelor's degree in Computer Science, Information Systems, or a related field.\r\n10+ years of experience developing Spark services for large-scale data movement.\r\nExperience scheduling, monitoring, debugging, and optimizing ETL Spark batch processes.\r\nHands-on experience developing Java clients for REST and SOAP API integrations.\r\nExperience developing Scala Spark batch applications.\r\nExperience developing, deploying, and maintaining Apache Spark and Amazon EMR ETL jobs.\r\nExperience processing big data using Apache Spark.\r\nExperience with NoSQL databases such as Cassandra or Yugabyte.\r\nExperience with cloud-based data warehouse and data lake platforms such as Snowflake.\r\nExperience managing large datasets using storage formats such as Parquet and HDF5.\r\nStrong understanding of software development lifecycle (SDLC) methodologies.\r\nStrong analytical, troubleshooting, and problem-solving skills.\r\nExperience working in Agile software development environments.\r\nExcellent collaboration and communication skills.\r\nPreferred Qualifications Experience designing enterprise-scale data integration and aggregation solutions.\r\nExperience supporting external-facing data platforms and APIs.\r\nExperience working with financial services or FinTech data platforms.\r\nPassion for continuous learning and adopting modern data engineering best practices.\r\nJ-18808-Ljbffr","datePosted":"2026-07-16T02:02:25.677Z","dateModified":"2026-07-16T02:02:25.677Z","hiringOrganization":{"@type":"Organization","name":"Compunnel","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Durham","addressRegion":"NC","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"bcdbf58f66d2c0a8ab9410e2"},"url":"https://jobsearcher.com/jobs/bcdbf58f66d2c0a8ab9410e2"}}