{"schemaVersion":"jobsearcher.job.v1","id":"5ab22cbf33b48c7925c04cf3","url":"https://jobsearcher.com/jobs/5ab22cbf33b48c7925c04cf3","canonicalUrl":"https://jobsearcher.com/jobs/5ab22cbf33b48c7925c04cf3","title":"Data Engineer","description":"HomeAboutAdaptive AIIndustriesConsulting Jobs PortalContact UsHomeAboutAdaptive AIIndustriesConsulting Jobs PortalContact UsJob DescriptionJob OverviewJob ID:J55442Job Title:Data EngineerLocation:Wilmington, DEDuration:14 Months + ExtensionDomain Exposure:Aerospace/ AirlinesClient:To Be Discussed LaterEmployment Type:W-2, 1099, C2CWork Authorization:US Citizen, Green Card, H-1B, OPT-EAD, CPT, OtherView All JobsSimilar Jobs Data Product Solution Engin... - Boston, MA Technical Project Manager –... - Santa Clara, CA Data Scientist - Houston, TX GIS Cloud Data Engineer - Spring, TX Azure Data Engineer - Raleigh, NCJob Title: Data EngineerLocation: Wilmington , DEDuration: 14 Months + ExtensionBill Rate: $79/hour (C2C)Job Type: W-2, 1099, C2CClient: To Be Discussed LaterWork Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, OtherJob DescriptionA hands-on Senior Data Engineer to build and run end-to-end data pipelines on AWS DMS and Databricks using PySpark, Python, and SQL. The position owns pipelines from source-database replication through transformation to curated, analytics-ready data, working directly with the customer. Immediate delivery on AWS is required with no ramp-up time from Azure or another cloud.Key ResponsibilitiesDesign, build, and maintain end-to-end data pipelines, from source ingestion through transformation to curated output tables.Configure and run AWS DMS replication tasks for full loads and ongoing Change Data Capture (CDC) from relational sources such as Oracle, SQL Server, and MariaDB.Land DMS output in Amazon S3 and process it in Databricks with PySpark, using Delta Lake and a bronze/silver/gold (medallion) design.Write complex, efficient SQL for transformations, data validation, reconciliation, and troubleshooting.Handle CDC logic: upserts and merges, deletes, late-arriving data, schema changes, and deduplication.Tune Spark jobs and DMS tasks for performance, cost, and reliability; diagnose replication lag and job failures.Build in data quality checks, logging, alerting, and restart/recovery so pipelines run without hand-holding.Orchestrate workloads with Databricks Workflows (or similar) and keep code in source control with peer review.Meet regularly with customer stakeholders: gather requirements, explain design choices and trade-offs, give status updates, and walk through issues and fixes.Write clear technical documentation for pipelines, data flows, and runbooks.Required Qualifications (Must Have)All experience below must be hands-on and on AWS (experience only on Azure without AWS does not meet the bar):AWS DMS: Personally set up, run, and troubleshot DMS replication (full load + CDC) in a production setting, including endpoints, replication instances, task settings, and table mappings.Databricks: Strong hands-on development on Databricks on AWS: notebooks and jobs, Delta Lake, MERGE/upsert patterns, cluster configuration, and performance tuning.PySpark and Python: Writes production-grade PySpark transformations and Python code (modular, tested, version-controlled).End-to-end pipelines: Has built complex pipelines from source to curated data, not only the reporting or visualization layer.SQL: Strong SQL, including joins across large tables, window functions, CTEs, query tuning, and data reconciliation.Communication: Can explain work clearly to technical and non-technical audiences, and comfortable leading customer conversations.Experience: Typically 6+ years in data engineering, with at least 3 years currently on AWS data services and Databricks.Nice To HaveAWS Glue: Jobs, crawlers, Data CatalogPower BI: Enough to understand how downstream reports consume the dataOther AWS services: S3, IAM, Lambda, Step Functions, CloudWatch, RedshiftDatabricks features: Unity Catalog, Delta Live Tables, or Auto LoaderCI/CD for data code: Git, Databricks Asset Bundles, TerraformCertifications: Databricks or AWS certifications (e.g., Databricks Data Engineer Professional, AWS Data Engineer Associate)Sector Experience: Experience on federal or public-sector projectsWho This Role Is Not ForDevOps or platform engineers whose DMS/Databricks work is mostly infrastructure, provisioning, or CI/CD, not building pipelines.Reporting or BI developers whose main work is dashboards, Power BI, or report-layer SQL.Azure-first engineers who would need to learn AWS on the job.Glue-only engineers whose AWS pipeline experience is mainly AWS Glue, without strong DMS and Databricks.Apply NowEqual Opportunity EmployerWe are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.Home|About|Adaptive AI|Industries|Consulting |Contact Us© 2004-2026, Machine Learning Technologies LLC, or its affiliates","company":"Machine Learning Technologies","rawCompany":"machine learning technologies","city":"Wilmington","state":"DE","isRemote":false,"isActive":false,"createdAt":"2026-11-18T09:33:21.875Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"},{"code":"15-2051.00","title":"Data Scientists","slug":"data-scientists"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"518210","title":"Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services","slug":"computing-infrastructure-providers-data-processing-web-hosting-and-related-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Data Engineer","description":"HomeAboutAdaptive AIIndustriesConsulting Jobs PortalContact UsHomeAboutAdaptive AIIndustriesConsulting Jobs PortalContact UsJob DescriptionJob OverviewJob ID:J55442Job Title:Data EngineerLocation:Wilmington, DEDuration:14 Months + ExtensionDomain Exposure:Aerospace/ AirlinesClient:To Be Discussed LaterEmployment Type:W-2, 1099, C2CWork Authorization:US Citizen, Green Card, H-1B, OPT-EAD, CPT, OtherView All JobsSimilar Jobs Data Product Solution Engin... - Boston, MA Technical Project Manager –... - Santa Clara, CA Data Scientist - Houston, TX GIS Cloud Data Engineer - Spring, TX Azure Data Engineer - Raleigh, NCJob Title: Data EngineerLocation: Wilmington , DEDuration: 14 Months + ExtensionBill Rate: $79/hour (C2C)Job Type: W-2, 1099, C2CClient: To Be Discussed LaterWork Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, OtherJob DescriptionA hands-on Senior Data Engineer to build and run end-to-end data pipelines on AWS DMS and Databricks using PySpark, Python, and SQL. The position owns pipelines from source-database replication through transformation to curated, analytics-ready data, working directly with the customer. Immediate delivery on AWS is required with no ramp-up time from Azure or another cloud.Key ResponsibilitiesDesign, build, and maintain end-to-end data pipelines, from source ingestion through transformation to curated output tables.Configure and run AWS DMS replication tasks for full loads and ongoing Change Data Capture (CDC) from relational sources such as Oracle, SQL Server, and MariaDB.Land DMS output in Amazon S3 and process it in Databricks with PySpark, using Delta Lake and a bronze/silver/gold (medallion) design.Write complex, efficient SQL for transformations, data validation, reconciliation, and troubleshooting.Handle CDC logic: upserts and merges, deletes, late-arriving data, schema changes, and deduplication.Tune Spark jobs and DMS tasks for performance, cost, and reliability; diagnose replication lag and job failures.Build in data quality checks, logging, alerting, and restart/recovery so pipelines run without hand-holding.Orchestrate workloads with Databricks Workflows (or similar) and keep code in source control with peer review.Meet regularly with customer stakeholders: gather requirements, explain design choices and trade-offs, give status updates, and walk through issues and fixes.Write clear technical documentation for pipelines, data flows, and runbooks.Required Qualifications (Must Have)All experience below must be hands-on and on AWS (experience only on Azure without AWS does not meet the bar):AWS DMS: Personally set up, run, and troubleshot DMS replication (full load + CDC) in a production setting, including endpoints, replication instances, task settings, and table mappings.Databricks: Strong hands-on development on Databricks on AWS: notebooks and jobs, Delta Lake, MERGE/upsert patterns, cluster configuration, and performance tuning.PySpark and Python: Writes production-grade PySpark transformations and Python code (modular, tested, version-controlled).End-to-end pipelines: Has built complex pipelines from source to curated data, not only the reporting or visualization layer.SQL: Strong SQL, including joins across large tables, window functions, CTEs, query tuning, and data reconciliation.Communication: Can explain work clearly to technical and non-technical audiences, and comfortable leading customer conversations.Experience: Typically 6+ years in data engineering, with at least 3 years currently on AWS data services and Databricks.Nice To HaveAWS Glue: Jobs, crawlers, Data CatalogPower BI: Enough to understand how downstream reports consume the dataOther AWS services: S3, IAM, Lambda, Step Functions, CloudWatch, RedshiftDatabricks features: Unity Catalog, Delta Live Tables, or Auto LoaderCI/CD for data code: Git, Databricks Asset Bundles, TerraformCertifications: Databricks or AWS certifications (e.g., Databricks Data Engineer Professional, AWS Data Engineer Associate)Sector Experience: Experience on federal or public-sector projectsWho This Role Is Not ForDevOps or platform engineers whose DMS/Databricks work is mostly infrastructure, provisioning, or CI/CD, not building pipelines.Reporting or BI developers whose main work is dashboards, Power BI, or report-layer SQL.Azure-first engineers who would need to learn AWS on the job.Glue-only engineers whose AWS pipeline experience is mainly AWS Glue, without strong DMS and Databricks.Apply NowEqual Opportunity EmployerWe are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.Home|About|Adaptive AI|Industries|Consulting |Contact Us© 2004-2026, Machine Learning Technologies LLC, or its affiliates","datePosted":"2026-11-18T09:33:21.875Z","dateModified":"2026-11-18T09:33:21.875Z","hiringOrganization":{"@type":"Organization","name":"Machine Learning Technologies","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Wilmington","addressRegion":"DE","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"5ab22cbf33b48c7925c04cf3"},"url":"https://jobsearcher.com/jobs/5ab22cbf33b48c7925c04cf3"}}