{"schemaVersion":"jobsearcher.job.v1","id":"66fc5c99eaaf2e40b594c122","url":"https://jobsearcher.com/jobs/66fc5c99eaaf2e40b594c122","canonicalUrl":"https://jobsearcher.com/jobs/66fc5c99eaaf2e40b594c122","title":"Data Engineer","description":"About DataVueDataVue provides consumer credit data, audience solutions, and predictive analytics to lenders and other businesses. Our data environment spans more than one billion records, with individual source files exceeding 200 million records and some datasets containing more than 4,000 attributes.The roleWe are looking for a hands-on Data Engineer. You will build and operate workflows that ingest, cleanse, match, profile, and integrate large datasets, making them reliable and usable for customer delivery, analytics, and modeling.Major feeds refresh approximately every four to six weeks. Between refreshes, you will improve automation, onboard sources, maintain data dictionaries, investigate quality issues, and support production data requests. Database responsibilities focus on tables, processing logic, and performance optimization; infrastructure administration is handled separately.What You Will DoData pipelines and automationBuild and maintain Python and Spark ETL/ELT workflows in AWS. Manage source mappings, schema changes, full and incremental loads, reconciliation, and recovery from failed processingAutomate scheduling, dependencies, logging, alerts, retries, and safe reruns. Optimize runtime, memory use, and processing costs; maintain version-controlled code, tests, and runbooksData cleansing and matchingIntegrate CASS address processing and other cleansing steps. Develop match keys, crosswalks, and record-linkage logic connecting consumers, households, addresses, and properties across sourcesHandle duplicates, conflicting values, missing identifiers, and ambiguous matches. Monitor match quality and coverage, prevent unintended record multiplication, and maintain consistent identities across refreshesAttribute profiling and data qualityBuild configurable profiling across thousands of attributes. Measure meaningful population rates excluding blanks and placeholder codes, validity, distinct values, distributions, ranges, percentiles, and concentration as appropriateMonitor drift between deliveries and assess coverage within relevant populations. Establish quality checks and exception reports, investigate discrepancies, and validate refreshed data before production releaseMaintain data dictionaries, source lineage, definitions, and quality findings. Help technical, modeling, sales, and operations teams understand attribute usability and limitationsDatabase development and data operationsDesign and maintain staging, integrated, and downstream tables. Optimize SQL, large joins, schemas, indexing, partitioning, and data distribution as appropriate to the platformSupport customer orders, audience extractions, suppressions, match-and-append requests, and modeling datasets as priorities require. Apply established controls for sensitive consumer dataHistorical data and derived attributesContribute to historical data capabilities using Apache Iceberg or comparable technology, with potential petabyte-scale requirements. Support historical extracts for modeling and potential data productsPreserve source versions, effective and ingestion dates, and transformation lineage. Build reproducible point-in-time datasets that reflect when information became availableImplement and validate time-series attributes, including changes, trends, rolling measures, and event counts, using definitions developed with business and ML teams. Account for missing periods, corrections, and changing source or matching rulesRequirementsRequired experienceAdvanced Python engineering: production-grade, reusable code for large-scale processing and automation, with strong debugging, memory management, testing, and error-handling skillsAdvanced SQL and database development: complex joins, window functions, execution-plan analysis, data modeling, and performance optimization across large tablesHands-on Spark/PySpark: building and optimizing distributed jobs, including partitioning, large joins, shuffles, data skew, and job troubleshootingProduction AWS experience: S3, processing services such as Glue or EMR, and secure data access patterns (IAM, encryption, least-privilege)Demonstrated processing experience with hundreds of millions of records, efficient storage formats such as Parquet, and practical methods for avoiding unnecessary scans and recomputationExperience orchestrating multi-step data workflows with a production scheduler such as Airflow, Dagster, Step Functions, or Glue workflows, including dependency management, retries, and alertingCross-source matching, deduplication, attribute profiling, data validation, and investigation of quality issuesOwnership of reliable production workflows, including Git, code review, automated testing, controlled releases, monitoring, and recoveryAbility to interpret complex data dictionaries, translate business requirements into accurate processing logic, and communicate clearly in a remote teamBachelor's degreeAuthorized to work in the United States without current or future visa sponsorshipPreferred experienceSingleStore or comparable distributed SQL databasesApache Iceberg and historical data lake or lakehouse developmentpoint-in-time modeling datasets and time-series feature pipelinesConsumer credit, property, mortgage, or direct marketing dataCASS processing and postal data hygiene","company":"Datavue","rawCompany":"datavue","city":"Denver","state":"CO","isRemote":false,"isActive":false,"createdAt":"2026-10-31T07:59:03.126Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-2051.00","title":"Data Scientists","slug":"data-scientists"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"}],"industries":[{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"518210","title":"Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services","slug":"computing-infrastructure-providers-data-processing-web-hosting-and-related-services"},{"code":"561450","title":"Credit Bureaus","slug":"credit-bureaus"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Data Engineer","description":"About DataVueDataVue provides consumer credit data, audience solutions, and predictive analytics to lenders and other businesses. Our data environment spans more than one billion records, with individual source files exceeding 200 million records and some datasets containing more than 4,000 attributes.The roleWe are looking for a hands-on Data Engineer. You will build and operate workflows that ingest, cleanse, match, profile, and integrate large datasets, making them reliable and usable for customer delivery, analytics, and modeling.Major feeds refresh approximately every four to six weeks. Between refreshes, you will improve automation, onboard sources, maintain data dictionaries, investigate quality issues, and support production data requests. Database responsibilities focus on tables, processing logic, and performance optimization; infrastructure administration is handled separately.What You Will DoData pipelines and automationBuild and maintain Python and Spark ETL/ELT workflows in AWS. Manage source mappings, schema changes, full and incremental loads, reconciliation, and recovery from failed processingAutomate scheduling, dependencies, logging, alerts, retries, and safe reruns. Optimize runtime, memory use, and processing costs; maintain version-controlled code, tests, and runbooksData cleansing and matchingIntegrate CASS address processing and other cleansing steps. Develop match keys, crosswalks, and record-linkage logic connecting consumers, households, addresses, and properties across sourcesHandle duplicates, conflicting values, missing identifiers, and ambiguous matches. Monitor match quality and coverage, prevent unintended record multiplication, and maintain consistent identities across refreshesAttribute profiling and data qualityBuild configurable profiling across thousands of attributes. Measure meaningful population rates excluding blanks and placeholder codes, validity, distinct values, distributions, ranges, percentiles, and concentration as appropriateMonitor drift between deliveries and assess coverage within relevant populations. Establish quality checks and exception reports, investigate discrepancies, and validate refreshed data before production releaseMaintain data dictionaries, source lineage, definitions, and quality findings. Help technical, modeling, sales, and operations teams understand attribute usability and limitationsDatabase development and data operationsDesign and maintain staging, integrated, and downstream tables. Optimize SQL, large joins, schemas, indexing, partitioning, and data distribution as appropriate to the platformSupport customer orders, audience extractions, suppressions, match-and-append requests, and modeling datasets as priorities require. Apply established controls for sensitive consumer dataHistorical data and derived attributesContribute to historical data capabilities using Apache Iceberg or comparable technology, with potential petabyte-scale requirements. Support historical extracts for modeling and potential data productsPreserve source versions, effective and ingestion dates, and transformation lineage. Build reproducible point-in-time datasets that reflect when information became availableImplement and validate time-series attributes, including changes, trends, rolling measures, and event counts, using definitions developed with business and ML teams. Account for missing periods, corrections, and changing source or matching rulesRequirementsRequired experienceAdvanced Python engineering: production-grade, reusable code for large-scale processing and automation, with strong debugging, memory management, testing, and error-handling skillsAdvanced SQL and database development: complex joins, window functions, execution-plan analysis, data modeling, and performance optimization across large tablesHands-on Spark/PySpark: building and optimizing distributed jobs, including partitioning, large joins, shuffles, data skew, and job troubleshootingProduction AWS experience: S3, processing services such as Glue or EMR, and secure data access patterns (IAM, encryption, least-privilege)Demonstrated processing experience with hundreds of millions of records, efficient storage formats such as Parquet, and practical methods for avoiding unnecessary scans and recomputationExperience orchestrating multi-step data workflows with a production scheduler such as Airflow, Dagster, Step Functions, or Glue workflows, including dependency management, retries, and alertingCross-source matching, deduplication, attribute profiling, data validation, and investigation of quality issuesOwnership of reliable production workflows, including Git, code review, automated testing, controlled releases, monitoring, and recoveryAbility to interpret complex data dictionaries, translate business requirements into accurate processing logic, and communicate clearly in a remote teamBachelor's degreeAuthorized to work in the United States without current or future visa sponsorshipPreferred experienceSingleStore or comparable distributed SQL databasesApache Iceberg and historical data lake or lakehouse developmentpoint-in-time modeling datasets and time-series feature pipelinesConsumer credit, property, mortgage, or direct marketing dataCASS processing and postal data hygiene","datePosted":"2026-10-31T07:59:03.126Z","dateModified":"2026-10-31T07:59:03.126Z","hiringOrganization":{"@type":"Organization","name":"Datavue","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Denver","addressRegion":"CO","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"66fc5c99eaaf2e40b594c122"},"url":"https://jobsearcher.com/jobs/66fc5c99eaaf2e40b594c122"}}