{"schemaVersion":"jobsearcher.job.v1","id":"93fec8d206e2e46b9fdf7beb","url":"https://jobsearcher.com/jobs/93fec8d206e2e46b9fdf7beb","canonicalUrl":"https://jobsearcher.com/jobs/93fec8d206e2e46b9fdf7beb","title":"Data Engineer - AWS/Databricks","description":"Overview\nIn this Data Engineer role you will design and deliver AWS cloud-scale data platforms for federal clients, enabling scalable data ingestion and analytics. You will work with Spark, Delta Lake, and Databricks to modernize enterprise data assets and support governance and quality. You’ll collaborate with data architects to build cloud-native solutions and optimize performance across large datasets. This is a hands-on role with a clear impact on mission-critical data enablement and modernization initiatives.\n\nCompensation / Benefitstraining and certifications allowance up to $3,000 annuallydegree-seeking program support up to $3,000competitive compensationcomprehensive benefitswork-life balanceinclusive, diverse culture\nResponsibilitiesBuild and maintain PySpark-based data pipelines in Databricks for ingestion, transformation, and enrichment of structured and semi-structured dataDesign and implement Delta Lake tables with ACID, partition pruning, schema enforcement, and performance optimizationDevelop ETL/ELT workflows integrating multiple sources into a centralized data warehouseLeverage Spark SQL/DataFrame APIs for business rules, joins, and aggregations aligned to warehouse modelingCollaborate with data architects to implement cloud-native data solutions on AWS (S3, Glue, RDS, IAM)Optimize pipeline performance via partitioning, caching, broadcast joins, and adaptive tuningDeploy and version data assets with Git-integrated workflows and CI/CD tools (GitLab or Jenkins)Monitor pipelines and cluster usage using Databricks tools and AWS CloudWatch; address bottlenecks and cost-performance tradeoffsConduct discovery and mapping of legacy sources to design end-to-end data flowsImplement governance includes metadata tagging, data quality validation, audit logging, and lineage trackingSupport ad hoc data access, develop reusable data assets, and maintain shared notebooks for cross-team analytics\nKey requirements2+ years of data engineering and Agile analytics experience2+ years of experience retrieving, parsing, and processing structured and unstructured data1–2 years building scalable ETL/ELT workflows for reporting and analytics1+ year experience building enterprise data solutions in the cloud, with AWS and Databricks preferredExperience with data quality, validation frameworks, and storage optimizationBachelor degree (BA/BS)collaborationproblem-solvingcommunicationSparkDelta LakeDatabricks","company":"Acuity","rawCompany":"acuity","city":"McLean","state":"VA","isRemote":false,"isActive":true,"createdAt":"2026-09-15T04:56:47.500Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"518210","title":"Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services","slug":"computing-infrastructure-providers-data-processing-web-hosting-and-related-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Data Engineer - AWS/Databricks","description":"Overview\nIn this Data Engineer role you will design and deliver AWS cloud-scale data platforms for federal clients, enabling scalable data ingestion and analytics. You will work with Spark, Delta Lake, and Databricks to modernize enterprise data assets and support governance and quality. You’ll collaborate with data architects to build cloud-native solutions and optimize performance across large datasets. This is a hands-on role with a clear impact on mission-critical data enablement and modernization initiatives.\n\nCompensation / Benefitstraining and certifications allowance up to $3,000 annuallydegree-seeking program support up to $3,000competitive compensationcomprehensive benefitswork-life balanceinclusive, diverse culture\nResponsibilitiesBuild and maintain PySpark-based data pipelines in Databricks for ingestion, transformation, and enrichment of structured and semi-structured dataDesign and implement Delta Lake tables with ACID, partition pruning, schema enforcement, and performance optimizationDevelop ETL/ELT workflows integrating multiple sources into a centralized data warehouseLeverage Spark SQL/DataFrame APIs for business rules, joins, and aggregations aligned to warehouse modelingCollaborate with data architects to implement cloud-native data solutions on AWS (S3, Glue, RDS, IAM)Optimize pipeline performance via partitioning, caching, broadcast joins, and adaptive tuningDeploy and version data assets with Git-integrated workflows and CI/CD tools (GitLab or Jenkins)Monitor pipelines and cluster usage using Databricks tools and AWS CloudWatch; address bottlenecks and cost-performance tradeoffsConduct discovery and mapping of legacy sources to design end-to-end data flowsImplement governance includes metadata tagging, data quality validation, audit logging, and lineage trackingSupport ad hoc data access, develop reusable data assets, and maintain shared notebooks for cross-team analytics\nKey requirements2+ years of data engineering and Agile analytics experience2+ years of experience retrieving, parsing, and processing structured and unstructured data1–2 years building scalable ETL/ELT workflows for reporting and analytics1+ year experience building enterprise data solutions in the cloud, with AWS and Databricks preferredExperience with data quality, validation frameworks, and storage optimizationBachelor degree (BA/BS)collaborationproblem-solvingcommunicationSparkDelta LakeDatabricks","datePosted":"2026-09-15T04:56:47.500Z","dateModified":"2026-09-15T04:56:47.500Z","hiringOrganization":{"@type":"Organization","name":"Acuity","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"McLean","addressRegion":"VA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"93fec8d206e2e46b9fdf7beb"},"url":"https://jobsearcher.com/jobs/93fec8d206e2e46b9fdf7beb"}}