JOBSEARCHER

Senior Platform Data Engineer

Senior Platform Data Engineer We are seeking a Senior Platform Data Engineer. This is a production engineering role focused on building and operating a centralized, enterprise-grade data platform that institutionalizes proprietary knowledge and delivers data as a product — not a side effect. You will design, build, and operate models, pipelines, integrations, and infrastructure across the full medallion architecture (Bronze → Silver → Gold). This role requires true end-to-end ownership: design, build, test, deploy, monitor, and continuously improve.
What You’ll Do:
  • Build and maintain ingestion pipelines from raw Bronze landing through curated Gold domain models
  • Develop transformation models using dbt with testing, Slim CI, and exposure tracking
  • Author and manage production-grade Airflow DAGs with SLA monitoring and backfill strategies
  • Design and implement streaming and event-driven pipelines (Kafka, Glue Streaming, Delta Live Tables)
  • Collaborate with data scientists, platform engineers, and product teams to ship reliable data products
  • Implement data quality frameworks and enforce governance standards
  • Containerize workloads and deploy to Kubernetes (EKS)
  • Contribute to infrastructure as code (Terraform provisioning for S3, IAM, Glue, etc.)Document lineage, metadata, ownership, and sensitivity classification
  • This is not a “pipeline-only” role — it is platform ownership at scale.
  • Core Technical SkillsPythonProduction-level standardsSQLAdvanced analytical queries, window functions, optimization
  • Experience with Redshift, Snowflake, or Databricks SQLdbtGold-layer modeling and testing
  • Slim CI, exposures, meta configurations
  • OrchestrationApache Airflow (DAG authoring, task dependencies, SLAs, backfills)
  • Prefect or Dagster is a plus
  • Streaming & Event Pipelines
  • Kafka topic consumption
  • Bronze landing jobs
  • Cloud Data Platforms (hands-on with at least one)
  • Databricks (Delta Lake, Unity Catalog, Auto Loader, DLT)AWS (S3, Glue, Lake Formation, Redshift)
  • Data Quality
  • Great Expectations, dbt tests, or equivalent
  • Docker & Kubernetes
  • Containerized jobs
  • Kubernetes Jobs and CronJobsEKS deployments
  • Infrastructure as Code
  • Terraform familiarity
  • Data Modeling
  • Dimensional modeling
  • Slowly Changing Dimensions
  • Medallion architecture design
  • Data Platform-Specific Experience
  • SaaS ingestion via Fivetran, Airbyte, or custom connectors (REST, SFTP, JDBC)
  • Change Data Capture (Debezium or connector-based CDC modes)
  • Handling semi-structured and unstructured data (PDF, Excel, JSON)
  • Designing Bronze schemas for fidelity and downstream flexibility
  • Vector embedding pipelines (chunking strategies, embeddings, pgvector/OpenSearch)
  • Data lineage tooling (OpenLineage, DataHub, Unity Catalog)
  • Governance standards: PII handling, sensitivity classification, column masking, partition-level access
  • Ways of Working
  • Agile sprint participation (planning, standups, retros)
  • Treat pipelines as production software (tests, runbooks, SLAs, alerts)
  • Migration-safe schema changes (expand/contract pattern)
  • Proactive data quality ownership
  • Rigorous peer code reviews
  • Close collaboration with data science and platform engineering teams
Nice to Have:
  • Migration of legacy data assets (Excel, Access, shared drives)ML feature engineering and point-in-time correct pipelines
  • Open-source data tooling contributions
  • Qualifications5–8+ years of experience developing data products and platforms
  • Strong background in Agile software development
  • Demonstrated success in prior engineering roles
  • Strong analytical and problem-solving capabilities
What We’re Looking For:
  • Clear communicator who can document complex data flows
  • Results-driven and impact-focused
  • Proactive and self-directed
  • Comfortable interfacing across technical and business stakeholders
  • Entrepreneurial mindset with a willingness to innovate
If you’re a production-minded data engineer who thrives on building scalable platforms and owning pipelines end-to-end, we’d love to connect.