Senior Software Engineer, Data Pipelines

Senior Software Engineer, Data Pipelines About PerimeterPerimeter builds for resilience in an era of accelerating biological risk. Our end-to-end biosecurity infrastructure platform spans detection, characterization, forecasting, and response — powered by frontier AI and deployed across airports, hospitals, wastewater systems, and critical infrastructure worldwide. Perimeter's infrastructure maps the biological landscape, transforming biological data into biointelligence, enabling decisive response as soon as a threat emerges.https://www.perimeter.bio/About The RoleOn our BUILD team, you are a software engineer focused on building and operating critical biosecurity data systems. You design reliable data pipelines and models, productionize analytics, and ensure data quality across programs spanning PCR, sequencing, wastewater, biosurveillance, and large-scale environmental monitoring.This role requires strong software engineering fundamentals—including system design, testing, and code quality—applied to data infrastructure challenges. You will work primarily on backend data systems, designing data warehouses, building ETL/ELT pipelines, and managing data architecture. The role combines platform engineering (e.g., orchestration with Airflow, observability, infrastructure-as-code) with analytics engineering (SQL modeling, testing, documentation) to deliver reliable data products that support threat detection, pathogen attribution, and operational decision-making.ResponsibilitiesData Platform Architecture & EngineeringPlan, architect, test, and deploy data warehouses, data marts, and ETL/ELT pipelines primarily within AWS and Snowflake environmentsBuild scalable data pipelines capable of handling structured, unstructured, and high-throughput biological data from diverse sourcesDevelop data models using dbt with rigorous testing, documentation, and stakeholder-aligned semantics to ensure analytics-ready datasetsData Quality & GovernanceEnsure data integrity, consistency, and accessibility across internal and external biosecurity data productsDevelop, document, and enforce coding and data modeling standards to improve code quality, maintainability, and system performanceServe as the in-house data expert, making recommendations on data architecture, pipeline improvements, and best practices; define and adapt data engineering processes to deliver reliable answers to critical biosecurity questionsAPI & Integration DevelopmentBuild high-performance APIs and microservices in Python that enable seamless integration between the biosecurity data platform and user-facing applicationsDesign backend services that support real-time and batch data access for biosecurity operationsCreate data products that empower public health officials, analysts, and partners with actionable biosecurity intelligenceAI & Data DemocratizationDemocratize access to complex biosecurity datasets using AI and LLMs, making data more discoverable and usable for stakeholdersApply AI-assisted development tools to accelerate code generation, data modeling, and pipeline development while maintaining high quality standardsCloud Infrastructure & PerformanceBuild robust, production-ready data workflows using AWS, Kubernetes, Docker, Airflow, and infrastructure-as-code (Terraform/CloudFormation)Diagnose system bottlenecks, optimize for cost and speed, and ensure the reliability and fault tolerance of mission-critical data pipelinesImplement observability, monitoring, and alerting to maintain high availability for biosecurity operationsTechnical Leadership & CollaborationLead data projects from scoping through execution, including design, documentation, and stakeholder communicationCollaborate with technical leads, product managers, scientists, and data analysts to build robust data products and analytics capabilitiesMinimum Qualifications7+ years of professional experience in data or software engineering, with a focus on building production-grade data products and scalable architecturesExpert proficiency with SQL for complex transformations, performance tuning, and query optimizationStrong Python skills for data engineering workflows, including pipeline development, ETL/ELT processes, and data processing; experience with backend frameworks (FastAPI, Flask) for API development; focus on writing modular, testable, and reusable codeProven experience with dbt for data modeling and transformation, including testing frameworks and documentation practicesHands-on experience with cloud data warehouses (Snowflake, BigQuery, or Redshift), including performance tuning, security hardening, and managing complex schemasExperience with workflow orchestration tools (Airflow, Dagster, or equivalent) for production data pipelines, including DAG development, scheduling, monitoring, and troubleshootingSolid grounding in software engineering fundamentals: system design, version control (Git), CI/CD pipelines, containerization (Docker), and infrastructure-as-code (Terraform, CloudFormation)Hands-on experience managing AWS resources, including S3, IAM roles/policies, API integrations, and security configurationsStrong ability to analyze large datasets, identify data quality issues, debug pipeline failures, and propose scalable solutionsExcellent communication skills and ability to work cross-functionally with scientists, analysts, and product teams to turn ambiguous requirements into maintainable data productsPreferred Capabilities & ExperienceDomain familiarity with biological data (PCR, sequencing, wastewater surveillance, TAT metrics) and experience working with lab, bioinformatics, NGS, or epidemiology teamsProduction ownership of Snowflake environments including RBAC, secure authentication patterns, and cost/performance optimizationExperience with observability and monitoring stacks (Grafana, Datadog, or similar) and data quality monitoring (anomaly detection, volume/velocity checks, schema drift detection)Familiarity with container orchestration platforms (Kubernetes) for managing production workloadsExperience with data ingestion frameworks (Airbyte, Fivetran) or building custom ingestion solutions for external partner data deliveryFamiliarity with data cataloging, governance practices, and reference data management to prevent silent data driftExperience designing datasets for visualization tools (Tableau, Looker, Metabase) with strong understanding of dashboard consumption patterns; familiarity with JavaScript for custom visualizations or front-end dashboard developmentComfort with AI-assisted development tools (GitHub Copilot, Cursor) to accelerate code generation while maintaining quality standardsStartup or fast-paced environment experience with evolving priorities and rapid iterationScientific or data-intensive domain experience (life sciences, healthcare, materials science)You Should Apply If You...Are passionate about working on a mission that matters & has real global impactAre a self-starter who thrives in dynamic, fast-moving environments and gets energized by ambiguity rather than slowed down by itWant to be part of building something from the ground up — we're a small, scrappy team, and we're looking for people who are comfortable operating without a full playbook but are equally excited about helping us build the processes and foundations that will carry us forward as we grow