{"schemaVersion":"jobsearcher.job.v1","id":"d55b36a7e8311d2fdf6099d8","url":"https://jobsearcher.com/jobs/d55b36a7e8311d2fdf6099d8","canonicalUrl":"https://jobsearcher.com/jobs/d55b36a7e8311d2fdf6099d8","title":"Data Engineering Lead","description":"Remote (Australasia Time Zone)\n\nHands-on leadership role | Title and compensation are flexible based on experience\n\nAbout the role\n\nTrillion is looking for a hands-on data engineering leader to own our ad-tech analytics and reporting platform and manage the engineers who build it. You will be accountable for the full path from event generation and Kafka-based transport through ClickHouse storage, aggregation, and reporting in Metabase.\n\nThis is a player-coach role with direct reports. You will set technical direction, coach the team, and own delivery, while still writing production code, reviewing schemas and queries, and working through difficult data problems alongside the engineers. The role may become less hands-on as the team grows, but it will remain technical.\n\nOne of the first priorities is to keep the current legacy pipeline stable while designing and building its replacement. That work will require a practical migration plan, parallel processing, careful reconciliation, and the ability to improve the platform without disrupting business-critical reporting.\n\nWorking hours: Our team spans the United States and Australia, so this is not a fully asynchronous role. Depending on where you are based, your schedule will need regular, agreed overlap with both US and Australian working hours. We will discuss the specific hours during the interview process.\n\nWhat you’ll own\n\nData platform and hands-on engineering\n\nSet the technical direction for the ad-tech data platform, with Kafka and ClickHouse at its core.\n\nDesign, build, and operate high-volume ETL/ELT pipelines that ingest ad, traffic, and application events from real-time streams and legacy sources.\n\nMaintain the existing pipeline and reporting outputs while leading a phased replacement of the legacy architecture.\n\nBuild and maintain a detailed canonical event model as the source of truth, together with efficient roll-up tables for analytics, Metabase, and application use.\n\nDevelop minute-level and near-real-time aggregation pipelines, including deterministic reprocessing and backfills for late, duplicate, or corrected data.\n\nWrite and review production code for pipelines, Kafka consumers and producers, rollups, schema migrations, reconciliation, and operational tooling.\n\nDefine standards for ClickHouse schema design, partitioning, sharding, replication, retention, and query performance.\n\nReview application logging and instrumentation, and introduce new or parallel event publishing when existing data cannot support accurate reporting or attribution.\n\nEstablish practical standards for testing, monitoring, documentation, data quality, and service-level expectations.\n\nPeople leadership and delivery\n\nDirectly manage the current data engineers through regular one-to-ones, coaching, clear goals, actionable feedback, and performance support.\n\nSet priorities, assign work thoughtfully, remove blockers, and create an engineering rhythm that makes delivery predictable.\n\nGrow the team over time by helping define roles, interview candidates, and onboard new hires.\n\nTurn business and reporting needs into clear technical designs, milestones, and ownership.\n\nOwn delivery commitments, operational stability, and the quality of the team’s data products.\n\nLead investigations into discrepancies that affect revenue, optimization, customer reporting, or trust in the data.\n\nCross-functional partnership\n\nWork closely with Product, Engineering, Analytics, and company leadership to define monetization and reporting requirements.\n\nIdentify gaps between the current platform and future reporting needs, then propose phased solutions that balance speed, risk, and long-term maintainability.\n\nPartner on metric definitions and data models exposed through Metabase so reporting remains consistent, explainable, and fast.\n\nCommunicate trade-offs, risks, and progress clearly to both technical and non-technical stakeholders.\n\nWhat we’re looking for\n\n6+ years building and operating production data systems, including high-volume event data and near-real-time pipelines.\n\nExperience directly managing data engineers while remaining hands-on in production systems.\n\nStrong production experience with ClickHouse or a comparable columnar OLAP database; direct ClickHouse experience is strongly preferred.\n\nSubstantial hands-on experience with Kafka, including event design, producers and consumers, delivery semantics, replay, and operational troubleshooting.\n\nAdvanced SQL skills and a strong understanding of analytical and time-series query patterns.\n\nWorking knowledge of the ad-tech ecosystem and its data flows. You should be comfortable with concepts such as requests, impressions, clicks, conversions, revenue attribution, CPM, RPM, CTR, and common causes of reporting discrepancies.\n\nA solid grasp of data modeling and the partitioning, sharding, replication, and performance trade-offs involved in distributed systems.\n\nExperience shipping pipelines with CI/CD, orchestration, observability, and safe backfill or reprocessing workflows.\n\nThe ability to read application code well enough to improve event schemas, logging, and instrumentation.\n\nStrong debugging judgment and clear written and verbal communication.\n\nHelpful experience\n\nOperating and tuning ClickHouse clusters at scale.\n\nModernizing or replacing a legacy analytics and reporting platform while keeping existing outputs reliable.\n\nUsing stream-processing frameworks such as Flink or Spark Structured Streaming.\n\nWorking with data observability and data quality frameworks.\n\nBuilding data platforms for high-scale monetization, advertising, or traffic-driven products.\n\nApplying data science or machine learning to optimization, experimentation, forecasting, or monetization performance.\n\nWhat success looks like in the first 3–6 months\n\nYou understand the existing architecture, have earned the team’s trust, and have taken clear ownership of priorities and delivery.\n\nYou have audited the current pipelines, event logs, ClickHouse environment, and legacy reporting systems, with documented risks around correctness, latency, and scale.\n\nThe legacy pipeline has clear operational ownership, monitoring, and a plan for addressing its most immediate reliability risks.\n\nYou have completed a gap analysis against current and planned ad-tech reporting needs, including attribution accuracy and latency.\n\nThere is an agreed, phased roadmap for replacing the legacy platform without interrupting critical reporting.\n\nNew or parallel event publishing is in place for at least one important data flow where the existing instrumentation is insufficient.\n\nAt least one end-to-end pipeline has been delivered from raw Kafka events to ClickHouse tables and Metabase-ready reporting, with monitoring, documentation, and reprocessing support.\n\nWhat we offer\n\nFlexible remote work, with optional in-office days.\n\nHealth coverage and a 401(k) match.\n\nA collaborative environment with room to shape the platform, the team, and how data engineering is practiced at Trillion.\n\nA lead-level scope with direct reports. The final title and salary will be agreed based on the successful candidate’s experience and the scope they are best positioned to own.\n\n.grecaptcha-badge { visibility: visible; } .custom-control-label::before {left: -1.40rem;} .custom-control-label::after {left: -1.40rem;} /* sc-60523: salary is now a numeric text field for all locations (dropdown removed) */ #inputWagesExp {display: inline-block;} .form-group {align-items: center;} .form-group label {font-size: 24px; line-height: 1.5;} label {margin-bottom: 0;} .form-control{font-size: 20px;}","company":"Trillion","rawCompany":"trillion","city":"Remote","state":"OR","isRemote":false,"isActive":false,"createdAt":"2026-08-09T14:55:04.729Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"}],"industries":[{"code":"513210","title":"Software Publishers","slug":"software-publishers"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Data Engineering Lead","description":"Remote (Australasia Time Zone)\n\nHands-on leadership role | Title and compensation are flexible based on experience\n\nAbout the role\n\nTrillion is looking for a hands-on data engineering leader to own our ad-tech analytics and reporting platform and manage the engineers who build it. You will be accountable for the full path from event generation and Kafka-based transport through ClickHouse storage, aggregation, and reporting in Metabase.\n\nThis is a player-coach role with direct reports. You will set technical direction, coach the team, and own delivery, while still writing production code, reviewing schemas and queries, and working through difficult data problems alongside the engineers. The role may become less hands-on as the team grows, but it will remain technical.\n\nOne of the first priorities is to keep the current legacy pipeline stable while designing and building its replacement. That work will require a practical migration plan, parallel processing, careful reconciliation, and the ability to improve the platform without disrupting business-critical reporting.\n\nWorking hours: Our team spans the United States and Australia, so this is not a fully asynchronous role. Depending on where you are based, your schedule will need regular, agreed overlap with both US and Australian working hours. We will discuss the specific hours during the interview process.\n\nWhat you’ll own\n\nData platform and hands-on engineering\n\nSet the technical direction for the ad-tech data platform, with Kafka and ClickHouse at its core.\n\nDesign, build, and operate high-volume ETL/ELT pipelines that ingest ad, traffic, and application events from real-time streams and legacy sources.\n\nMaintain the existing pipeline and reporting outputs while leading a phased replacement of the legacy architecture.\n\nBuild and maintain a detailed canonical event model as the source of truth, together with efficient roll-up tables for analytics, Metabase, and application use.\n\nDevelop minute-level and near-real-time aggregation pipelines, including deterministic reprocessing and backfills for late, duplicate, or corrected data.\n\nWrite and review production code for pipelines, Kafka consumers and producers, rollups, schema migrations, reconciliation, and operational tooling.\n\nDefine standards for ClickHouse schema design, partitioning, sharding, replication, retention, and query performance.\n\nReview application logging and instrumentation, and introduce new or parallel event publishing when existing data cannot support accurate reporting or attribution.\n\nEstablish practical standards for testing, monitoring, documentation, data quality, and service-level expectations.\n\nPeople leadership and delivery\n\nDirectly manage the current data engineers through regular one-to-ones, coaching, clear goals, actionable feedback, and performance support.\n\nSet priorities, assign work thoughtfully, remove blockers, and create an engineering rhythm that makes delivery predictable.\n\nGrow the team over time by helping define roles, interview candidates, and onboard new hires.\n\nTurn business and reporting needs into clear technical designs, milestones, and ownership.\n\nOwn delivery commitments, operational stability, and the quality of the team’s data products.\n\nLead investigations into discrepancies that affect revenue, optimization, customer reporting, or trust in the data.\n\nCross-functional partnership\n\nWork closely with Product, Engineering, Analytics, and company leadership to define monetization and reporting requirements.\n\nIdentify gaps between the current platform and future reporting needs, then propose phased solutions that balance speed, risk, and long-term maintainability.\n\nPartner on metric definitions and data models exposed through Metabase so reporting remains consistent, explainable, and fast.\n\nCommunicate trade-offs, risks, and progress clearly to both technical and non-technical stakeholders.\n\nWhat we’re looking for\n\n6+ years building and operating production data systems, including high-volume event data and near-real-time pipelines.\n\nExperience directly managing data engineers while remaining hands-on in production systems.\n\nStrong production experience with ClickHouse or a comparable columnar OLAP database; direct ClickHouse experience is strongly preferred.\n\nSubstantial hands-on experience with Kafka, including event design, producers and consumers, delivery semantics, replay, and operational troubleshooting.\n\nAdvanced SQL skills and a strong understanding of analytical and time-series query patterns.\n\nWorking knowledge of the ad-tech ecosystem and its data flows. You should be comfortable with concepts such as requests, impressions, clicks, conversions, revenue attribution, CPM, RPM, CTR, and common causes of reporting discrepancies.\n\nA solid grasp of data modeling and the partitioning, sharding, replication, and performance trade-offs involved in distributed systems.\n\nExperience shipping pipelines with CI/CD, orchestration, observability, and safe backfill or reprocessing workflows.\n\nThe ability to read application code well enough to improve event schemas, logging, and instrumentation.\n\nStrong debugging judgment and clear written and verbal communication.\n\nHelpful experience\n\nOperating and tuning ClickHouse clusters at scale.\n\nModernizing or replacing a legacy analytics and reporting platform while keeping existing outputs reliable.\n\nUsing stream-processing frameworks such as Flink or Spark Structured Streaming.\n\nWorking with data observability and data quality frameworks.\n\nBuilding data platforms for high-scale monetization, advertising, or traffic-driven products.\n\nApplying data science or machine learning to optimization, experimentation, forecasting, or monetization performance.\n\nWhat success looks like in the first 3–6 months\n\nYou understand the existing architecture, have earned the team’s trust, and have taken clear ownership of priorities and delivery.\n\nYou have audited the current pipelines, event logs, ClickHouse environment, and legacy reporting systems, with documented risks around correctness, latency, and scale.\n\nThe legacy pipeline has clear operational ownership, monitoring, and a plan for addressing its most immediate reliability risks.\n\nYou have completed a gap analysis against current and planned ad-tech reporting needs, including attribution accuracy and latency.\n\nThere is an agreed, phased roadmap for replacing the legacy platform without interrupting critical reporting.\n\nNew or parallel event publishing is in place for at least one important data flow where the existing instrumentation is insufficient.\n\nAt least one end-to-end pipeline has been delivered from raw Kafka events to ClickHouse tables and Metabase-ready reporting, with monitoring, documentation, and reprocessing support.\n\nWhat we offer\n\nFlexible remote work, with optional in-office days.\n\nHealth coverage and a 401(k) match.\n\nA collaborative environment with room to shape the platform, the team, and how data engineering is practiced at Trillion.\n\nA lead-level scope with direct reports. The final title and salary will be agreed based on the successful candidate’s experience and the scope they are best positioned to own.\n\n.grecaptcha-badge { visibility: visible; } .custom-control-label::before {left: -1.40rem;} .custom-control-label::after {left: -1.40rem;} /* sc-60523: salary is now a numeric text field for all locations (dropdown removed) */ #inputWagesExp {display: inline-block;} .form-group {align-items: center;} .form-group label {font-size: 24px; line-height: 1.5;} label {margin-bottom: 0;} .form-control{font-size: 20px;}","datePosted":"2026-08-09T14:55:04.729Z","dateModified":"2026-08-09T14:55:04.729Z","hiringOrganization":{"@type":"Organization","name":"Trillion","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Remote","addressRegion":"OR","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"d55b36a7e8311d2fdf6099d8"},"url":"https://jobsearcher.com/jobs/d55b36a7e8311d2fdf6099d8"}}