Founding Machine learning Engineer - Evaluation

Senior ML Engineer Medical Imaging Evaluation & AI ReliabilityAbout the Role:My client is building evaluation and evidence infrastructure for safety-critical AI systems, starting with diagnostic medical imaging.AI systems are increasingly used in settings where their outputs affect clinical decisions and patient outcomes. In medical imaging, benchmark accuracy alone is not enough. Hospitals, regulators, and clinical stakeholders need evidence that models will behave reliably across real-world deployment environments, populations, scanners, and workflows.This role sits at the intersection of:medical imaging AI,model robustness and evaluation,regulatory evidence generation,and real-world deployment behavior.The work is highly investigative and requires strong technical judgment, scientific reasoning, and the ability to operate effectively in ambiguous environments.The RoleThis is not a traditional “train models on benchmark datasets” ML role.You will work directly with medical imaging companies and healthcare stakeholders to investigate how AI systems behave in practice and what evidence is required for deployment, regulatory, and clinical decisions.You will:Design and execute evaluations for medical imaging AI systemsInvestigate model failure modes, robustness, and generalization gapsAnalyze behavior across populations, scanners, imaging protocols, and clinical settingsDetermine what evidence is sufficient for stakeholders making deployment or regulatory decisionsTranslate technical findings into actionable recommendations for customers and clinical stakeholdersBuild reusable evaluation pipelines, evidence schemas, and model assessment frameworksWork with messy, incomplete, and noisy real-world clinical dataHelp shape how evaluation investigations are conducted across the organizationThe important work is not simply running experiments. It is identifying what questions actually matter, what evidence is missing, and how to generate defensible conclusions under real-world constraints.Required Qualifications:Strong experience in machine learning for medical imaging (radiology, pathology, cardiology imaging, or related domains)Experience evaluating or validating real-world ML systems, not just training modelsDeep understanding of:model robustness,distribution shift,uncertainty,failure analysis,and real-world deployment behaviorStrong Python skills across the full investigation workflow:data analysis,experimentation,evaluation,and reportingExperience working with noisy or imperfect clinical datasetsAbility to communicate technical findings clearly to both technical and non-technical stakeholdersHigh tolerance for ambiguity and open-ended investigative workStrongly Preferred:Experience with FDA-regulated AI/ML systems or medical device submissions (510(k), De Novo, SaMD, etc.)Experience with medical imaging deployment evaluation or clinical validationExperience with interpretability, post-deployment monitoring, uncertainty estimation, or model auditingExperience designing reproducible evaluation frameworks or benchmarking systemsBackground in healthcare AI or other safety-critical ML domainsCustomer-facing or cross-functional technical leadership experiencePhD or equivalent research depth in ML, medical imaging, computer vision, or related areasIdeal Candidate ProfileCandidates who tend to succeed in this role often come from backgrounds such as:Medical imaging ML researchFDA or healthcare AI evaluationClinical AI validationAI robustness and reliability researchApplied ML investigation in safety-critical environmentsHealthcare-focused computer vision researchWhat Success Looks Like:The strongest people in this role become experts in how medical AI systems behave in the real world.They develop the judgment to answer questions such as:Where are the model’s true weaknesses?Which deployment conditions introduce risk?What concerns are real versus theoretical?What evidence is sufficient for a hospital or regulator to trust the system?What additional validation is required before deployment proceeds?