{"schemaVersion":"jobsearcher.job.v1","id":"a38071b9f76fc9fe73b4849a","url":"https://jobsearcher.com/jobs/a38071b9f76fc9fe73b4849a","canonicalUrl":"https://jobsearcher.com/jobs/a38071b9f76fc9fe73b4849a","title":"Ab Initio Data Engineer","description":"Job Description: We are seeking a highly skilled resource to design and implement high-performance, event-driven data pipelines, ensuring low-latency data processing and high availability system for the large credit card processing system. The ideal candidate will work with the Ab Initio ecosystem (GDE, EME, Conduct>It) to build stateful services that ingest, filter, and transform data from sources like Kafka or message queues, pushing updates to dashboards or downstream databases in near-real-time.ResponsibilitiesDesign, develop, and maintain complex ETL graphs using Ab Initio GDE (Graphical Development Environment)Architect end-to-end data integration solutions spanning multiple source/target systems (RDBMS, mainframe, flat files, message queues)Optimize graph performance - parallelism, partitioning (broadcast, hash, range), layout tuning, checkpointingLead development using Ab Initio components: Co>Operating System, EME (Enterprise Meta>Environment), Conduct>It, Data Profiler, Continuous FlowsOwn metadata management and data lineage within EMEDesign and implement CDC (Change Data Capture) and near-real-time data pipelinesTroubleshoot production job failures, perform root-cause analysis, and implement fixes with minimal downtimeDefine coding standards, review peer-developed graphs, and enforce best practicesMentor junior/mid-level Ab Initio developersCollaborate with data architects, DBAs, and business analysts to translate requirements into technical designRequired QualificationsBachelor's degree in Computer Science, Engineering, or related field7+ years of ETL development experience, with 5+ years specifically in Ab InitioStrong hands-on expertise with Ab Initio GDE, Co>Op, EME, and Conduct>ItDeep understanding of parallel processing concepts and Ab Initio partitioning/multi-file system (MFS)Strong SQL skills and experience with major RDBMS (DB2)Experience with Unix/Linux shell scriptingSolid understanding of data warehousing concepts (dimensional modeling, SCDs, star/snowflake schemas)Experience with KafkaStrong debugging and performance-tuning skills for large-volume data processingPreferred QualificationsExperience with mainframe data integration (COBOL copybooks, VSAM, flat files)Exposure to Kubernetes, OpenShiftFamiliarity with Hadoop/Spark ecosystems and hybrid ETL architecturesKnowledge of API-based or streaming integration (Kafka)Prior experience as a technical lead or in a client-facing architecture roleSoft Skills / LeadershipStrong analytical and problem-solving skills for complex data issuesAbility to lead design discussions and drive technical consensusEffective mentorship and knowledge-sharing across the teamClear communication with both technical and business stakeholdersBehavioral Skills:Good Communication skills5 days' Work from Office at Berkley Heights, NJTeam PlayerAbility to work in a changing environmentStrong problem solving and analytical skillsAbility to work independently or within a teamManage day-to-day challenges and communicate developmental risks with the technical team","company":"Diamondpick","rawCompany":"diamondpick","city":"Berkeley Heights","state":"NJ","isRemote":false,"isActive":false,"createdAt":"2026-08-06T23:36:56.479Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Ab Initio Data Engineer","description":"Job Description: We are seeking a highly skilled resource to design and implement high-performance, event-driven data pipelines, ensuring low-latency data processing and high availability system for the large credit card processing system. The ideal candidate will work with the Ab Initio ecosystem (GDE, EME, Conduct>It) to build stateful services that ingest, filter, and transform data from sources like Kafka or message queues, pushing updates to dashboards or downstream databases in near-real-time.ResponsibilitiesDesign, develop, and maintain complex ETL graphs using Ab Initio GDE (Graphical Development Environment)Architect end-to-end data integration solutions spanning multiple source/target systems (RDBMS, mainframe, flat files, message queues)Optimize graph performance - parallelism, partitioning (broadcast, hash, range), layout tuning, checkpointingLead development using Ab Initio components: Co>Operating System, EME (Enterprise Meta>Environment), Conduct>It, Data Profiler, Continuous FlowsOwn metadata management and data lineage within EMEDesign and implement CDC (Change Data Capture) and near-real-time data pipelinesTroubleshoot production job failures, perform root-cause analysis, and implement fixes with minimal downtimeDefine coding standards, review peer-developed graphs, and enforce best practicesMentor junior/mid-level Ab Initio developersCollaborate with data architects, DBAs, and business analysts to translate requirements into technical designRequired QualificationsBachelor's degree in Computer Science, Engineering, or related field7+ years of ETL development experience, with 5+ years specifically in Ab InitioStrong hands-on expertise with Ab Initio GDE, Co>Op, EME, and Conduct>ItDeep understanding of parallel processing concepts and Ab Initio partitioning/multi-file system (MFS)Strong SQL skills and experience with major RDBMS (DB2)Experience with Unix/Linux shell scriptingSolid understanding of data warehousing concepts (dimensional modeling, SCDs, star/snowflake schemas)Experience with KafkaStrong debugging and performance-tuning skills for large-volume data processingPreferred QualificationsExperience with mainframe data integration (COBOL copybooks, VSAM, flat files)Exposure to Kubernetes, OpenShiftFamiliarity with Hadoop/Spark ecosystems and hybrid ETL architecturesKnowledge of API-based or streaming integration (Kafka)Prior experience as a technical lead or in a client-facing architecture roleSoft Skills / LeadershipStrong analytical and problem-solving skills for complex data issuesAbility to lead design discussions and drive technical consensusEffective mentorship and knowledge-sharing across the teamClear communication with both technical and business stakeholdersBehavioral Skills:Good Communication skills5 days' Work from Office at Berkley Heights, NJTeam PlayerAbility to work in a changing environmentStrong problem solving and analytical skillsAbility to work independently or within a teamManage day-to-day challenges and communicate developmental risks with the technical team","datePosted":"2026-08-06T23:36:56.479Z","dateModified":"2026-08-06T23:36:56.479Z","hiringOrganization":{"@type":"Organization","name":"Diamondpick","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Berkeley Heights","addressRegion":"NJ","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"a38071b9f76fc9fe73b4849a"},"url":"https://jobsearcher.com/jobs/a38071b9f76fc9fe73b4849a"}}