JOBSEARCHER

DATA QUALITY ENGINEER

BuzzclanPlano, TXL6 LeadOctober 8th, 2026
Job Title: Senior Data QA / Test Engineer
Location: Plano, TX 75024
Visa: Independent Visa Only
Duration: Contractual / Fulltime Role (W2)
Note: We are looking for a Senior Data QA / Test Engineer with strong experience in data/ETL testing, SQL Server, Databricks, Azure Data Factory, PySpark, and data migration testing. The ideal candidate will own the testing strategy for migrating a SQL Server data platform to Databricks on Azure, validate data parity between legacy and modern platforms, establish quality gates, and ensure end-to-end validation of ADF pipelines and Delta Lake workloads.
Key Skills:
  • (50%)Strong experience with Data QA / ETL Testing / Data Validation
  • Strong SQL experience including T-SQL, Spark SQL, joins, window functions, aggregates, and data reconciliation
  • Hands-on experience with Databricks, Delta Lake, notebooks, Jobs/Workflows, and Unity Catalog
  • Hands-on experience with Azure Data Factory (ADF) including pipelines, triggers, datasets, monitoring, and debugging
  • Strong experience with PySpark for test automation and reconciliation frameworks
  • Experience with source-to-target reconciliation, row counts, checksums/hashes, aggregates, and row-level data comparisons
  • Experience testing data transformations, migrations, schema changes, incremental loads, and business logicExperience integrating automated tests into CI/CD using Azure DevOps or GitHub Actions
  • Experience (30%)5+ years of experience in Data / ETL Testing
  • Experience testing at least one data platform migration
  • Strong experience validating SQL Server data platforms and modern cloud data platforms
  • Experience converting SQL business logic, stored procedures, and views into test cases
  • Experience validating PySpark / Spark SQL outputs against legacy SQL results
  • Experience defining quality gates, entry/exit criteria, test metrics, and release readiness
  • Experience testing ADF pipelines, dependencies, parameters, triggers, retries, and incremental/watermark logicExperience with Delta Lake MERGE/upsert, SCD Type 1/2, schema evolution, deduplication, and late-arriving data
  • Experience with UAT, regression testing, defect management, and production validation
  • Other Skills (20%)Strong analytical, troubleshooting, and problem-solving skillsExperience reporting defects, reconciliation results, test metrics, and go/no-go readiness
  • Strong communication and collaboration with Data Engineers, Data Quality Engineers, Developers, and Project Leadership
  • Experience working in Agile/Scrum environments
  • Strong documentation and test strategy development skillsExperience with pytest or similar test automation frameworks
  • Knowledge of Terraform, Databricks Asset Bundles, and Azure DevOpsKnowledge of Databricks Labs DQX, Great Expectations, Soda, or Lakeflow/DLT expectations is an asset
  • Experience with Spark performance testing including partitioning, file sizing, and cluster sizing is an asset
  • Knowledge of SSIS, SSRS, or Power BI report validation is an asset
  • Must HaveData / ETL TestingSQL / T-SQL / Spark SQLDatabricks / Delta LakeAzure Data Factory
  • PySparkData Migration Testing
  • Source-to-Target Reconciliation
  • Data Validation / Quality Testing
  • Test Automation / pytestCI/CD / Azure DevOps or GitHub Actions
  • Quality Gates / Test Strategy
Responsibilities:
  • Define and execute the migration test strategy and test plan covering schema, data, transformation logic, performance, and regression testing
  • Build automated source-to-target reconciliation for migrated tables
  • Validate row counts, column-level checksums/hashes, aggregates, and row-level differences
  • Convert legacy SQL business logic, stored procedures, and views into test cases
  • Validate equivalent PySpark/Spark SQL outputs against legacy SQL results
  • Define and enforce quality gates across code merge, Bronze/Silver/Gold promotion, UAT sign-off, and production cutover
  • Validate Azure Data Factory pipelines including parameters, triggers, dependencies, mappings, failure/retry paths, incremental loads, and watermark logic
  • Test Delta Lake behavior including MERGE/upsert, SCD Type 1/2, schema evolution, deduplication, and late-arriving data
  • Integrate automated tests into CI/CD pipelines using Azure DevOps, GitHub Actions, or equivalent
  • Partner with the Data Quality Engineer to embed DQX/data-quality checks into pipelines
  • Report defects, reconciliation results, test metrics, and go/no-go readiness to project leadership
  • Support UAT, regression testing, production validation, defect resolution, and migration cutover activities
  • Ensure zero critical data defects are introduced into production following each migration wave
  • If interested, please share your resume at:📧 Afzal.Khan@buzzclan.com