{"schemaVersion":"jobsearcher.job.v1","id":"a6b6cdc4bedbb1405590113f","url":"https://jobsearcher.com/jobs/a6b6cdc4bedbb1405590113f","canonicalUrl":"https://jobsearcher.com/jobs/a6b6cdc4bedbb1405590113f","title":"Data/Full Stack Engineer, Data Storage & Ingestion Consultant","description":"About us\nAt Eon, we are at the forefront of large-scale neuroscientific data collection. Our mission is to enable the safe and scalable development of brain emulation technology to empower humanity over the next decade, beginning with the creation of a fully emulated digital twin of a mouse.\nRole\nWe’re a San Francisco team collecting very large microscopy datasets and we need an expert to design and implement our end-to-end data pipeline, from high-rate ingest to multi-petabyte storage and downstream processing. You’ll own the strategy (on-prem vs. S3 or hybrid), the bill of materials, and the deployment, and you’ll be on the floor wiring, racking, tuning, and validating performance.\nOur current instruments generate data at ~1+ GB/s sustained (higher during bursts) and the program will accumulate multiple petabyes total over time. You’ll help us choose and implement the right architecture considering reliability and cost controls.\nOutcomes (what success looks like)\nWithin 2 weeks: Implement an immediate data-handling strategy that reliably ingests our initial data streams.\nWithin 2 weeks: Deliver a documented medium-term data architecture covering storage, networking, ingest, and durability.\nWithin 1 month: Operationalize the medium-term pipeline in production (ingest buffer long-term store compute access).\nOngoing: Maintain 95% uptime for the end-to-end data-handling pipeline after setup.\n\nResponsibilities\nArchitect ingest & storage: Choose and implement an on-prem hardware and data pipeline design or a cloud/S3 alternative with explicit cost and performance tradeoffs at multi-petabyte scale.\nSet up a sustained-write ingest path 1 GB/s with adequate burst headroom (camera/frame-to-disk), including networking considerations, cooling, and throttling safeguards.\nOptimize footprint & cost: Incorporate on-the-fly compression/downsampling options and quantify CPU budget vs. write-speed tradeoffs; document when/where to compress to control $/PB.\nIntegrate with acquisition workflows ensuring image data and metadata are compatible with downstream stitching/flat-field correction pipelines.\nEnable downstream compute: Expose the data to segmentation/analysis stacks (local GPU nodes or cloud).\n\nSkills\n5+ years designing and deploying high-throughput storage or HPC pipelines ( 1 GB/s sustained ingest) in production.\nDeep hands-on with: NVMe RAID/striping, ZFS/MDRAID/erasure coding, PCIe topology, NUMA pinning, Linux performance tuning, and NIC offload features.\nProven delivery of multi-GB/s ingest systems and petabyte-scale storage in production (life-sciences, vision, HPC, or media).\nExperience building tiered storage systems (NVMe HDD/object) and validating real-world throughput under sustained load.\nPractical S3/object-storage know-how (AWS S3 and/or on-prem S3-compatible systems) with lifecycle, versioning, and cost controls.\nData integrity & reliability: snapshots, scrubs, replication, erasure coding, and backup/DR for PB-scale systems.\nNetworking: ****25/40/100 GbE (SFP+/SFP28), RDMA/ RoCE/iWARP familiarity; switch config and path tuning.\nAbility to spec and rack hardware: selecting chassis/backplanes, RAID/HBA cards, NICs, and cooling strategies to prevent NVMe throttling under sustained writes.\n\nIdeal skills:\nExperience with microscopy or scientific imaging ingest at frame-to-disk speeds, including Micro-Manager-based pipelines and raw-to-containerized format conversions.\nExperience with life science imaging data a plus.\n\nEngagement details\nContract (1099 or corp-to-corp); contract-to-hire if there’s a mutual fit.\nOn-site requirement: You must be physically present in San Francisco during build-out and initial operations; local field work (e.g., UCSF) as needed.\nCompensation: Contract, $100-300/hour\nTimeline: Immediate start","company":"Eon Systems","rawCompany":"eon systems","city":"Millbrae","state":"CA","isRemote":false,"isActive":false,"createdAt":"2026-08-05T11:28:11.767Z","occupations":[{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"518210","title":"Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services","slug":"computing-infrastructure-providers-data-processing-web-hosting-and-related-services"},{"code":"334112","title":"Computer Storage Device Manufacturing","slug":"computer-storage-device-manufacturing"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Data/Full Stack Engineer, Data Storage & Ingestion Consultant","description":"About us\nAt Eon, we are at the forefront of large-scale neuroscientific data collection. Our mission is to enable the safe and scalable development of brain emulation technology to empower humanity over the next decade, beginning with the creation of a fully emulated digital twin of a mouse.\nRole\nWe’re a San Francisco team collecting very large microscopy datasets and we need an expert to design and implement our end-to-end data pipeline, from high-rate ingest to multi-petabyte storage and downstream processing. You’ll own the strategy (on-prem vs. S3 or hybrid), the bill of materials, and the deployment, and you’ll be on the floor wiring, racking, tuning, and validating performance.\nOur current instruments generate data at ~1+ GB/s sustained (higher during bursts) and the program will accumulate multiple petabyes total over time. You’ll help us choose and implement the right architecture considering reliability and cost controls.\nOutcomes (what success looks like)\nWithin 2 weeks: Implement an immediate data-handling strategy that reliably ingests our initial data streams.\nWithin 2 weeks: Deliver a documented medium-term data architecture covering storage, networking, ingest, and durability.\nWithin 1 month: Operationalize the medium-term pipeline in production (ingest buffer long-term store compute access).\nOngoing: Maintain 95% uptime for the end-to-end data-handling pipeline after setup.\n\nResponsibilities\nArchitect ingest & storage: Choose and implement an on-prem hardware and data pipeline design or a cloud/S3 alternative with explicit cost and performance tradeoffs at multi-petabyte scale.\nSet up a sustained-write ingest path 1 GB/s with adequate burst headroom (camera/frame-to-disk), including networking considerations, cooling, and throttling safeguards.\nOptimize footprint & cost: Incorporate on-the-fly compression/downsampling options and quantify CPU budget vs. write-speed tradeoffs; document when/where to compress to control $/PB.\nIntegrate with acquisition workflows ensuring image data and metadata are compatible with downstream stitching/flat-field correction pipelines.\nEnable downstream compute: Expose the data to segmentation/analysis stacks (local GPU nodes or cloud).\n\nSkills\n5+ years designing and deploying high-throughput storage or HPC pipelines ( 1 GB/s sustained ingest) in production.\nDeep hands-on with: NVMe RAID/striping, ZFS/MDRAID/erasure coding, PCIe topology, NUMA pinning, Linux performance tuning, and NIC offload features.\nProven delivery of multi-GB/s ingest systems and petabyte-scale storage in production (life-sciences, vision, HPC, or media).\nExperience building tiered storage systems (NVMe HDD/object) and validating real-world throughput under sustained load.\nPractical S3/object-storage know-how (AWS S3 and/or on-prem S3-compatible systems) with lifecycle, versioning, and cost controls.\nData integrity & reliability: snapshots, scrubs, replication, erasure coding, and backup/DR for PB-scale systems.\nNetworking: ****25/40/100 GbE (SFP+/SFP28), RDMA/ RoCE/iWARP familiarity; switch config and path tuning.\nAbility to spec and rack hardware: selecting chassis/backplanes, RAID/HBA cards, NICs, and cooling strategies to prevent NVMe throttling under sustained writes.\n\nIdeal skills:\nExperience with microscopy or scientific imaging ingest at frame-to-disk speeds, including Micro-Manager-based pipelines and raw-to-containerized format conversions.\nExperience with life science imaging data a plus.\n\nEngagement details\nContract (1099 or corp-to-corp); contract-to-hire if there’s a mutual fit.\nOn-site requirement: You must be physically present in San Francisco during build-out and initial operations; local field work (e.g., UCSF) as needed.\nCompensation: Contract, $100-300/hour\nTimeline: Immediate start","datePosted":"2026-08-05T11:28:11.767Z","dateModified":"2026-08-05T11:28:11.767Z","hiringOrganization":{"@type":"Organization","name":"Eon Systems","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Millbrae","addressRegion":"CA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"a6b6cdc4bedbb1405590113f"},"url":"https://jobsearcher.com/jobs/a6b6cdc4bedbb1405590113f"}}