{"schemaVersion":"jobsearcher.job.v1","id":"f8bbb2da8a429d6ca05080ac","url":"https://jobsearcher.com/jobs/f8bbb2da8a429d6ca05080ac","canonicalUrl":"https://jobsearcher.com/jobs/f8bbb2da8a429d6ca05080ac","title":"Big Data Engineer","description":"Title: Big Data EngineerLocation: Rockville, MD or McLean, VA (Hybrid)Contract: 6+ Months ContractOnly Local candidates who can take Assessment and only who are in DC/VA/MD who can got for F2F interviewOverviewMust haves: spark, Hadoop, scala, hivescripting is a must- python or perlmust be expert level in Complex SQL- window functioning, complex multiple joins, cloud experience is mandatory-S3, glue, emr, athenaAI- How to use AI for prompt engineeringGithubCopiliotChjatgoptQNeed someone who is well versed in agile, test automations, CICD practicesFinancial Experience Is PreferredROLE FIT5+ years building enterprise-scale data solutions using Spark, Hadoop, Hive, and ScalaStrong scripting skills (Python or Perl) and expert-level complex SQL (window functions, multi-joins)AWS cloud experience required (S3, EMR, Glue, Athena)Experience with Agile delivery, CI/CD pipelines, automated testing, and GitHub workflowsFinancial services or regulated industry experience preferredOBJECTIVESDesign and maintain scalable, reliable big data pipelinesOptimize Spark/Hadoop workloads for performance, scalability, and cost efficiencyImplement automated testing and data quality validationEnable analytics and data science teams with high-quality, accessible datasetsLeverage AI-assisted tools (Copilot, ChatGPT, Q Developer) to improve development productivityPROBLEM-SOLVINGDiagnose and resolve Spark performance bottlenecks and data pipeline failuresOptimize complex SQL transformations and large-scale joinsTroubleshoot data quality, latency, and reliability issues in productionImprove AWS workload efficiency through tuning and resource optimizationAutomate repetitive engineering tasks using AI-assisted development toolsExperience ValidationDelivered end-to-end pipelines using Spark and Hadoop ecosystem toolsOptimized SQL and pipeline performance with measurable improvementsDeployed and supported AWS data workloads (EMR, Glue, Athena, S3)Implemented CI/CD and automated testing for data pipelinesUsed AI coding assistants and GitHub workflows in team-based developmentSkills To ProbeSpark internals, performance tuning, and partitioning strategiesAdvanced SQL techniques and Hive/Trino optimizationAWS data architecture and cost/performance tuning practicesPrompt engineering and safe use of AI coding assistantsCollaboration and communication in fast-paced, cross-functional environments'","company":"Comtec Information Systems It","rawCompany":"comtec information systems it","city":"McLean","state":"VA","isRemote":false,"isActive":false,"createdAt":"2026-09-17T08:18:48.509Z","occupations":[{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-2051.00","title":"Data Scientists","slug":"data-scientists"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Big Data Engineer","description":"Title: Big Data EngineerLocation: Rockville, MD or McLean, VA (Hybrid)Contract: 6+ Months ContractOnly Local candidates who can take Assessment and only who are in DC/VA/MD who can got for F2F interviewOverviewMust haves: spark, Hadoop, scala, hivescripting is a must- python or perlmust be expert level in Complex SQL- window functioning, complex multiple joins, cloud experience is mandatory-S3, glue, emr, athenaAI- How to use AI for prompt engineeringGithubCopiliotChjatgoptQNeed someone who is well versed in agile, test automations, CICD practicesFinancial Experience Is PreferredROLE FIT5+ years building enterprise-scale data solutions using Spark, Hadoop, Hive, and ScalaStrong scripting skills (Python or Perl) and expert-level complex SQL (window functions, multi-joins)AWS cloud experience required (S3, EMR, Glue, Athena)Experience with Agile delivery, CI/CD pipelines, automated testing, and GitHub workflowsFinancial services or regulated industry experience preferredOBJECTIVESDesign and maintain scalable, reliable big data pipelinesOptimize Spark/Hadoop workloads for performance, scalability, and cost efficiencyImplement automated testing and data quality validationEnable analytics and data science teams with high-quality, accessible datasetsLeverage AI-assisted tools (Copilot, ChatGPT, Q Developer) to improve development productivityPROBLEM-SOLVINGDiagnose and resolve Spark performance bottlenecks and data pipeline failuresOptimize complex SQL transformations and large-scale joinsTroubleshoot data quality, latency, and reliability issues in productionImprove AWS workload efficiency through tuning and resource optimizationAutomate repetitive engineering tasks using AI-assisted development toolsExperience ValidationDelivered end-to-end pipelines using Spark and Hadoop ecosystem toolsOptimized SQL and pipeline performance with measurable improvementsDeployed and supported AWS data workloads (EMR, Glue, Athena, S3)Implemented CI/CD and automated testing for data pipelinesUsed AI coding assistants and GitHub workflows in team-based developmentSkills To ProbeSpark internals, performance tuning, and partitioning strategiesAdvanced SQL techniques and Hive/Trino optimizationAWS data architecture and cost/performance tuning practicesPrompt engineering and safe use of AI coding assistantsCollaboration and communication in fast-paced, cross-functional environments'","datePosted":"2026-09-17T08:18:48.509Z","dateModified":"2026-09-17T08:18:48.509Z","hiringOrganization":{"@type":"Organization","name":"Comtec Information Systems It","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"McLean","addressRegion":"VA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"f8bbb2da8a429d6ca05080ac"},"url":"https://jobsearcher.com/jobs/f8bbb2da8a429d6ca05080ac"}}