{"schemaVersion":"jobsearcher.job.v1","id":"d76f807c9a8dcfcb6b94e7bb","url":"https://jobsearcher.com/jobs/d76f807c9a8dcfcb6b94e7bb","canonicalUrl":"https://jobsearcher.com/jobs/d76f807c9a8dcfcb6b94e7bb","title":"Data Engineer","description":"Location: Remote (United States)Employment Type: contractAbout the RoleAs a Data Engineer, you’ll build and operate the pipelines, tables, and services that power our hybrid data platform across on-premises and AWS. You’ll implement lakehouse patterns, productionize batch and streaming workloads, and partner with security, platform, analytics, and application teams to deliver governed, high-performance data products at scale.What You’ll DoBuild data pipelines: Develop reliable ELT/ETL jobs in Spark/SQL/dbt/Airflow/Glue to ingest from on-prem and cloud sources into S3-backed lakes and warehouses.Implement lakehouse tables: Create and maintain Iceberg (or Delta/Hudi) tables using the appropriate catalog (AWS Glue, Hive Metastore, Polaris/REST) with ACID, time travel, and schema evolution.Operate compute engines: Run and tune Spark (Databricks/EMR), Trino/Starburst, Dremio, and Snowflake workloads; leverage pushdown and query acceleration where applicable.Model data for analytics: Deliver dimensional models, semantic layers, and domain-oriented data products; document contracts and SLAs with consumers.Governance & security: Apply data cataloging, lineage, PII classification, Lake Formation permissions, IAM roles, and row/column-level security; contribute to audit readiness.Performance & cost tuning: Optimize partitioning, clustering/Z-order, predicate pushdown, file sizing/compaction, caching, and workload isolation; monitor and right-size clusters.Migrations: Execute migration workstreams from legacy/on-prem EDW (Informatica/SSIS/SAP BW, SQL Server/Oracle, Hadoop) to lakehouse patterns (Spark/dbt/ELT), including dual-run cutovers and reconciliation.Streaming & CDC: Build real-time and near-real-time pipelines using Kafka/MSK/Kinesis and Spark Structured Streaming; implement CDC with Debezium, DMS, or Fivetran.Quality & observability: Add unit/integration tests, expectations/rules, data contracts, lineage, alerting, and SLO dashboards; participate in on-call rotations.Platform guardrails: Contribute to standards for naming, zones, schemas, S3 layout, encryption, backup/DR, and multi-region replication; write clear runbooks and docs.DevOps for data: Use Terraform/CloudFormation and CI/CD (GitHub Actions/GitLab/Azure DevOps) to version, test, and deploy data assets.Required Experience5+ years in data engineering (or equivalent), delivering production pipelines and tables on at least two large-scale platforms.Hands-on with AWS data services: S3, Glue/EMR, Lake Formation, IAM, and at least one warehouse (Snowflake or Redshift).Deep experience with Apache Spark and at least one of: Databricks, EMR Spark, Snowflake Snowpark, Dremio, or Starburst/Trino.Production experience with open table formats: Apache Iceberg (preferred), Delta Lake, or Apache Hudi; strong grasp of metadata/manifest, compaction, and schema evolution.Comfortable with on-prem stacks: Hadoop/Hive, Spark on Kubernetes, SQL Server/SSIS and/or Oracle/Exadata; Netezza/Teradata a plus.Proven data modeling (3NF, dimensional, Data Vault), ELT/ETL design, and SQL performance tuning.Security/governance: RBAC/ABAC, row/column-level security, masking/tokenization, KMS/key management.IaC & CI/CD for data workloads.Excellent communicator who can collaborate with platform engineers, analysts, and stakeholders to meet SLAs and roadmap goals.Nice to Havedbt for ELT, Airflow orchestration, Great Expectations/Deequ for quality, OpenLineage/Marquez for lineage.Streaming experience with Kafka/MSK, Kinesis, or Flink.Catalogs/semantic: AWS Glue Data Catalog, Unity Catalog, Amundsen/DataHub, Atlan/Collibra.BI/serving: DuckDB, Athena, QuickSight, Tableau/Power BI/Looker.Compliance: SOC 2, HIPAA/PHI, GDPR, PCI; SSO/OIDC with Okta.Multi-tenant platforms or federated governance exposure.Location & Work StyleRemote with core hours in PST/CST;","company":"Pashtek Salesforce Partner Data Ai","rawCompany":"pashtek salesforce partner data ai","city":"Millbrae","state":"CA","isRemote":false,"isActive":false,"createdAt":"2026-10-08T03:27:36.531Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Data Engineer","description":"Location: Remote (United States)Employment Type: contractAbout the RoleAs a Data Engineer, you’ll build and operate the pipelines, tables, and services that power our hybrid data platform across on-premises and AWS. You’ll implement lakehouse patterns, productionize batch and streaming workloads, and partner with security, platform, analytics, and application teams to deliver governed, high-performance data products at scale.What You’ll DoBuild data pipelines: Develop reliable ELT/ETL jobs in Spark/SQL/dbt/Airflow/Glue to ingest from on-prem and cloud sources into S3-backed lakes and warehouses.Implement lakehouse tables: Create and maintain Iceberg (or Delta/Hudi) tables using the appropriate catalog (AWS Glue, Hive Metastore, Polaris/REST) with ACID, time travel, and schema evolution.Operate compute engines: Run and tune Spark (Databricks/EMR), Trino/Starburst, Dremio, and Snowflake workloads; leverage pushdown and query acceleration where applicable.Model data for analytics: Deliver dimensional models, semantic layers, and domain-oriented data products; document contracts and SLAs with consumers.Governance & security: Apply data cataloging, lineage, PII classification, Lake Formation permissions, IAM roles, and row/column-level security; contribute to audit readiness.Performance & cost tuning: Optimize partitioning, clustering/Z-order, predicate pushdown, file sizing/compaction, caching, and workload isolation; monitor and right-size clusters.Migrations: Execute migration workstreams from legacy/on-prem EDW (Informatica/SSIS/SAP BW, SQL Server/Oracle, Hadoop) to lakehouse patterns (Spark/dbt/ELT), including dual-run cutovers and reconciliation.Streaming & CDC: Build real-time and near-real-time pipelines using Kafka/MSK/Kinesis and Spark Structured Streaming; implement CDC with Debezium, DMS, or Fivetran.Quality & observability: Add unit/integration tests, expectations/rules, data contracts, lineage, alerting, and SLO dashboards; participate in on-call rotations.Platform guardrails: Contribute to standards for naming, zones, schemas, S3 layout, encryption, backup/DR, and multi-region replication; write clear runbooks and docs.DevOps for data: Use Terraform/CloudFormation and CI/CD (GitHub Actions/GitLab/Azure DevOps) to version, test, and deploy data assets.Required Experience5+ years in data engineering (or equivalent), delivering production pipelines and tables on at least two large-scale platforms.Hands-on with AWS data services: S3, Glue/EMR, Lake Formation, IAM, and at least one warehouse (Snowflake or Redshift).Deep experience with Apache Spark and at least one of: Databricks, EMR Spark, Snowflake Snowpark, Dremio, or Starburst/Trino.Production experience with open table formats: Apache Iceberg (preferred), Delta Lake, or Apache Hudi; strong grasp of metadata/manifest, compaction, and schema evolution.Comfortable with on-prem stacks: Hadoop/Hive, Spark on Kubernetes, SQL Server/SSIS and/or Oracle/Exadata; Netezza/Teradata a plus.Proven data modeling (3NF, dimensional, Data Vault), ELT/ETL design, and SQL performance tuning.Security/governance: RBAC/ABAC, row/column-level security, masking/tokenization, KMS/key management.IaC & CI/CD for data workloads.Excellent communicator who can collaborate with platform engineers, analysts, and stakeholders to meet SLAs and roadmap goals.Nice to Havedbt for ELT, Airflow orchestration, Great Expectations/Deequ for quality, OpenLineage/Marquez for lineage.Streaming experience with Kafka/MSK, Kinesis, or Flink.Catalogs/semantic: AWS Glue Data Catalog, Unity Catalog, Amundsen/DataHub, Atlan/Collibra.BI/serving: DuckDB, Athena, QuickSight, Tableau/Power BI/Looker.Compliance: SOC 2, HIPAA/PHI, GDPR, PCI; SSO/OIDC with Okta.Multi-tenant platforms or federated governance exposure.Location & Work StyleRemote with core hours in PST/CST;","datePosted":"2026-10-08T03:27:36.531Z","dateModified":"2026-10-08T03:27:36.531Z","hiringOrganization":{"@type":"Organization","name":"Pashtek Salesforce Partner Data Ai","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Millbrae","addressRegion":"CA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"d76f807c9a8dcfcb6b94e7bb"},"url":"https://jobsearcher.com/jobs/d76f807c9a8dcfcb6b94e7bb"}}