QA Data Engineer

Job Title: QA Data EngineerLocation: Remote (United States)Key Responsibilities:As a QA Data Engineer, you will work in an agile environment with a development team to:Design, develop, and maintain comprehensive test plans and test cases for scalable data pipelines.Implement automated testing solutions using Python and PySpark on a cloud-native Lakehouse data platform.Write efficient SQL queries to validate data extraction, transformation, and loading processes.Collaborate with product management and analysts to understand data requirements and ensure quality assurance.Optimize and troubleshoot data pipelines for performance and reliability.Ensure data quality and integrity through comprehensive testing and validation processes.Follow DevOps principles and use CI/CD to deploy and operate automated testing frameworks.Required Skills:Proficiency in Python and PySpark.Strong experience with SQL and database management.Knowledge of software development patterns and best practices in quality assurance.Experience with ETL/ELT processes and data pipeline testing.Proficiency in using version control, automated testing, and deployments with git-based tools like GitHub and GitHub Actions.Qualifications:Intense intellectual curiosity and an ability to view old problems with a fresh perspective.Bachelor’s degree in Computer Science, Engineering, or a related field.8+ years of experience in quality assurance or a related role.Understanding of testing methodologies for data pipelines, including unit, integration, and end-to-end testing.Knowledge of data governance and security best practices.Familiarity with data warehousing concepts and tools.Experience with cloud platforms (e.g., Azure, AWS, GCP), with Azure preferred.Knowledge of big data technologies (e.g., Microsoft Fabric, Azure Synapse, Lakehouse, Databricks).Familiarity with advanced data orchestration tools and frameworks like dbt or Airflow.Experience in the healthcare industry is a plus.