AWS Data Engineer – Qualtrics Integration
AWS Data Engineer - Qualtrics IntegrationRole Type: ContractLocation: Torrance, CA / RemoteDomain: Enterprise Survey Platform / QualtricsPrimary Stack: AWS | Python | SQL | Qualtrics APIThe AWS Data Engineer - Qualtrics Integration is responsible for designing, building, and maintaining scalable, automated data pipelines that support Qualtrics survey ingestion, transformation, and downstream reporting.This role focuses on serverless AWS data engineering, integrating Qualtrics APIs with AWS services to process structured survey data, dealer hierarchies, and reporting files. The engineer will ensure data accuracy, automation, monitoring, and performance across end-to-end workflows.Key ResponsibilitiesAWS Cloud & Data EngineeringDesign and maintain ETL pipelines using AWS Glue (PySpark)Develop AWS Lambda functions (Python / Node.js) for serverless data processingManage AWS S3 for Qualtrics input/output storage and optimized data accessOrchestrate workflows using AWS Step Functions and MWAA (Airflow)Query structured datasets using AWS AthenaMonitor pipelines using CloudWatch logs and metricsData Processing & ETLTransform and aggregate Qualtrics datasets (CSV, JSON, XML)Merge multiple source files (e.g., dealer master + employee files) into unified hierarchiesAutomate ingestion, transformation, validation, and report generationImplement reusable, scalable ETL frameworksAPI & IntegrationsIntegrate with Qualtrics APIs to extract raw survey data and response filesImplement event-driven processing (S3 triggers → Lambda)Connect AWS pipelines with CRM, ERP, BI tools, or downstream platformsData Quality & ReportingPerform data validation and quality checks prior to Qualtrics ingestionGenerate formatted output files aligned to business-defined templatesSupport ad-hoc analysis using SQL and AthenaProgramming & AutomationDevelop robust Python scripts for Glue, Lambda, and automation tasksWrite optimized SQL queries for structured data accessUse Bash/Shell scripting for file movement and preparationDevOps & Security (Nice to Have)Configure IAM roles and permissions securelyImplement Infrastructure as Code (Terraform / CloudFormation)Support CI/CD pipelines for data workflowsRequired Skills & ExperienceStrong experience in AWS Data Engineering & Serverless ArchitectureHands-on expertise with AWS Glue, Lambda, S3, Athena, Step FunctionsExperience with MWAA (Airflow) for orchestrationStrong Python and SQL skillsExperience integrating Qualtrics API or structured survey dataAbility to troubleshoot pipeline failures and performance issuesPreferred / Nice-to-Have SkillsExperience with Terraform or CloudFormationCI/CD for data pipelinesExperience supporting BI tools (Power BI, Tableau, etc.)Knowledge of data governance and security best practicesIdeal Candidate ProfileAWS-focused Data Engineer with strong automation mindsetComfortable working with survey / VoC / structured dataStrong debugging, monitoring, and optimization skillsAble to work independently in enterprise environmentsRecruiter Submission Template - Full Name:Degree Major with University & Completion Year:Total Years of Data Engineering Experience:Total Years of AWS Data Engineering Experience:Total Years of Experience with Serverless Architecture:Experience with AWS Glue (PySpark ETL)? (Yes/No - please elaborate):Experience with AWS Lambda (Python/Node.js)? (Yes/No - please elaborate):Experience with AWS S3 data storage & optimization?Experience with AWS Athena? (Yes/No):Experience implementing data validation & quality checks?Experience working with Qualtrics API? (Yes/No - please specify):Experience with event-driven processing (S3 → Lambda)?Experience integrating AWS with CRM / ERP / BI tools?Python Experience (years & libraries used):SQL Experience (years & databases/tools):Experience with IAM & role-based access?Brief description of a recent AWS data pipeline you built (tools + outcome):Motivation / Reason for Interest in Qualtrics Data Engineering Role:Contact Number:Email ID:LinkedIn Profile URL:Full Address (Street, City, State, Zip):Notice Period (in weeks):Current Work Authorization Status:Expected Hourly Rate:W2 / C2C (If C2C, corporation name):Are you comfortable working primarily Remote with Limited Onsite Flexibility?