JOBSEARCHER

Data Architect with Databricks -(Only W2)

Below is the Job Summary Job Role/Title: Data Architect with Databricks Contract-12-month contract Location: Remote Work Authorization: GC, GC EAD / H4 EAD / L2S / J2 EAD / TN / E3, USC (Any Visa Except H1B and OPT)
Job Summary: We are seeking a highly skilled Data Engineer with hands on experience in Databricks, Snowflake, and modern cloud data pipelines. The ideal candidate has deep expertise in PySpark, SQL, Delta Lake, Snowflake ELT, and distributed data processing. This role focuses on building scalable data pipelines, optimizing performance, ensuring data quality, and supporting analytics, AI/ML, and enterprise reporting workloads
Key Responsibilities
Databricks Engineering: Build and optimize ETL/ELT pipelines using PySpark, Spark SQL, Databricks Workflows, and Delta Lake. Develop Bronze/Silver/Gold Medallion architecture pipelines. Implement Delta Live Tables (DLT) for automated ingestion and transformation. Manage and optimize Databricks clusters, jobs, notebooks, repos, and workflows. Perform Spark performance tuning (partitioning, caching, AQE, broadcast joins).Implement Unity Catalog governance (catalogs, schemas, tables, permissions).Integrate Databricks with AWS S3 / Azure Data Lake / Kafka / APIs.
Snowflake Engineering: Design and develop Snowflake ELT pipelines using Snowflake SQL, Streams, Tasks, and Snow pipe. Build warehouse models, fact/dimension tables, and curated datasets. Optimize Snowflake performance (clustering, micro-partitioning, query tuning).Implement RBAC, masking policies, row-level security, and governance. Integrate Snowflake with Fivetran, DBT, ADF, Glue, Kafka, or custom ingestion frameworks.
Data Pipeline & Integration:
  • Build scalable ingestion frameworks for structured, semi structured, and unstructured data.
  • Integrate data from databases, APIs, cloud storage, streaming sources, and enterprise systems.
  • Implement data quality checks, validation rules, reconciliation, and lineage.
  • Support ML/AI workloads, feature engineering, and model-ready datasets.
  • Cloud & DevOpsWork with AWS, Azure, or GCP cloud-native services.
  • Implement CI/CD using GitHub Actions, Azure DevOps, GitLab, Jenkins.
  • Containerize workloads using Docker and orchestrate with Kubernetes (nice to have).Monitor pipelines using CloudWatch, Azure Monitor, Databricks metrics, Prometheus/Grafana.
  • Required SkillsCore Technical Skills
  • Databricks (PySpark, Spark SQL, Delta Lake, Workflows, DLT)
  • Snowflake (SQL, Streams, Tasks, Snowpipe, RBAC)
Python Advanced SQLCloud platforms: AWS / Azure / GCPData modelling (Star schema, dimensional modelling)ETL/ELT pipeline development Data quality, governance, lineageCI/CD pipelinesAPI integration & REST services Nice-to-Have SkillsDBTKafka / Spark Structured StreamingMLflow / Feature Store TerraformAirflow / ADF / Glue / DataflowRAG/LLM data preparation (bonus) Healthcare, finance, or regulated industry experience