Data Engineer
Data Engineer
Location: Remote (USA)
Employment Type: Contract
Work Authorization: US Citizen (USC) or Green Card (GC) Holders Only
Experience Level: 3–5 YearsRole Summary
We are seeking a detail-oriented and skilled Data Engineer with 3–5 years of hands-on experience to design, build, and maintain scalable data pipelines and cloud-based analytics solutions. In this role, you will play a crucial part in architecting seamless ETL/ELT workflows using Snowflake, Azure Data Factory, and Azure Databricks. The ideal candidate brings strong technical expertise in data modeling, pipeline orchestration, and query optimization, with a strict commitment to data quality, security, and system reliability.
Key Responsibilities:
Pipeline Engineering: Design, build, test, deploy, and maintain batch and near-real-time ETL/ELT data pipelines for structured and semi-structured datasets.
Orchestration & Workflow Management: Build and orchestrate data ingestion and transformation workflows using Azure Data Factory and Azure Databricks.
Data Warehousing & Modeling: Develop scalable data models, transformations, stored procedures, and performance-optimized workloads within Snowflake.
Data Processing: Utilize PySpark, Python, and SQL to cleanse, integrate, validate, and transform data across various enterprise and cloud data sources.
Reliability & Monitoring: Implement robust logging, monitoring, alerting, retry, and error-handling mechanisms to ensure high pipeline availability and uptime.
Governance & Security: Apply data-quality frameworks, reconciliation controls, lineage tracking, documentation standards, and strict role-based access security.
Cross-Functional Collaboration: Partner with solution architects, data analysts, application teams, and business stakeholders to translate complex requirements into scalable data solutions.
DevOps & Maintenance: Participate in code reviews, CI/CD deployment pipelines, production support, root-cause troubleshooting, and continuous query/cost optimization.
Required Qualifications
Education:
Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related technical field.
Experience:
3–5 years of hands-on experience in data engineering, data integration, or cloud data platform development.
Core ETL/ELT: Strong background in designing end-to-end data pipelines and deep familiarity with dimensional modeling and data warehousing concepts.
Snowflake Proficiency: Solid hands-on experience with Snowflake, including data loading, complex transformations, query tuning, and virtual warehouse optimization.
Azure Data Factory (ADF): Practical experience setting up linked services, datasets, pipelines, triggers, and orchestration.
Azure Databricks & PySpark: Hands-on experience with Azure Databricks, Apache Spark/PySpark, notebook workflows, cluster management, and Delta Lake concepts.
Programming & Querying: Advanced proficiency in SQL and solid working experience with Python for integrating data from relational databases, flat files, REST APIs, and cloud storage.
Version Control & Methodology: Familiarity with Git-based version control, CI/CD practices, Agile/Scrum delivery, and cloud security best practices.
Soft Skills: Excellent problem-solving, analytical, troubleshooting, written documentation, and verbal communication skills.
Preferred Skills & Certifications
Technologies:
Azure Data Lake Storage Gen2 (ADLS Gen2), Delta Lake, Snowpipe, and dbt (data build tool).
Tools:
Azure DevOps, Git, and REST API integration. Concepts: Data governance, data lineage, and metadata management.
Certifications:
Active certifications in Microsoft Azure (e.g., Azure Data Engineer Associate) or Snowflake (e.g., SnowPro Core) are a strong plus.
How to Apply:
If you meet the requirements above and are looking to join a dynamic, data-driven environment, please submit your updated resume outlining your relevant experience.