Software Calibration Engineer

Job Description Position Description:Own the formal Deliverable Acceptance process for TOP's external vendor engagement, reviewing each submitted Deliverable against the Acceptance Criteria defined in the vendor SOW and issuing written acceptance or a specific, actionable defect list · Verify vendor container image deliverables against the Software Bill of Materials (SBOM), confirming that all declared components are present and no unapproved components are included · Validate that vendor-delivered AI engine deployments comply with Ford's technical architecture requirements, including confirming that fine-tuned model weights are stored in Ford's Vertex AI Model Registry and not embedded in container images · Design, build, and maintain automated test suites for TOP platform backend services, APIs, and data pipelines · Develop AI engine output evaluation frameworks that test inference quality against defined accuracy benchmarks for each dealer service use case · Own the UAT (User Acceptance Testing) process for dealer-facing interfaces, coordinating with Ford Service stakeholders to recruit pilot users, design test scenarios, and capture structured feedback · Manage Jira project bug triage workflow: creating Bug records for confirmed Tier 2 issues, assigning priority in alignment with SLA tiers, tracking vendor acknowledgment and resolution timelines, and reporting SLA compliance metrics · Perform regression testing for each new container image version before Ford authorizes production deployment · Define test environments within Ford's GCP project space in collaboration with the GCP Cloud Engineer, ensuring test environments accurately reflect production configurations · Produce quality metrics reports for Ford program leadership covering defect rates, SLA compliance, and test coverage across all TOP platform componentsSkills Required:API, Cloud Infrastructure, Google Cloud Platform, User Acceptance Testing, Application Testing, Software Testing, Test Cases, Jira, Ad Hoc Reporting, Test Integration Testing 1. API - 3-5 years designing, executing, and validating API test cases using tools such as Postman or REST-assured. This includes verifying request/response contracts, authentication flows, error handling, and integration behavior across microservices within the Telemetry & Observability Platform. Cloud Infrastructure - 3-5 years of working knowledge of cloud-native infrastructure concepts including containerization, networking, IAM, and storage. Experience validating deployments and testing service behavior in cloud environments is expected. Google Cloud Platform - 2-5 years of hands-on experience working within GCP, including familiarity with services such as Pub/Sub, BigQuery, GCS, or Cloud Run as they relate to testing data pipelines and telemetry workloads. User Acceptance Testing - 2-5 years coordinating and executing UAT cycles with internal stakeholders and platform consumers, ensuring delivered features meet business and functional requirements before production release. Application Testing - 3-5 years developing and executing test plans covering functional, regression, and smoke testing across platform applications and services throughout the SDLC. Software Testing - 3-5 years of broad software testing experience including unit, integration, system, and end-to-end testing, with the ability to contribute to or maintain automated test suites. Test Cases - 2-5 years authoring clear, traceable, and reusable test cases tied to acceptance criteria, with documentation maintained in Jira or a comparable test management tool. Jira - Ad Hoc Reporting - 2-5 years creating ad hoc Jira queries, dashboards, and filters to surface test coverage, defect trends, and sprint quality metrics for engineering and leadership audiences. Test Integration Testing - 3-5 years planning and executing integration tests that validate end-to-end data and event flows across platform services. including ingestion, processing, and telemetry delivery pipelines.Skills Preferred:Artificial Intelligence & Expert Systems, Dynatrace, Quality Assurance Concepts and Standards, Quality Assurance/Control, Testing - Performance Artificial Intelligence & Expert Systems - 1-3 years of familiarity with AI-assisted testing approaches, including the use of LLM-based tools for test generation, anomaly detection, or intelligent defect triage within developer workflows. Dynatrace - 1-3 years using Dynatrace for observability-driven testing, including leveraging traces, dashboards, and alerting to validate system health and performance during test cycles. Quality Assurance Concepts and Standards - 2-5 years applying QA methodologies such as shift-left testing, risk-based testing, and test-driven development, with experience maintaining consistent quality standards across teams. Quality Assurance/Control - 2-5 years owning quality gates within a CI/CD pipeline, including defining acceptance thresholds, managing test environments, and working with engineering teams to resolve defects efficiently. 5. Testing - Performance - 2-4 years designing and running load, stress, or scalability tests against platform services, with the ability to interpret results and collaborate with engineers to resolve bottlenecks.Experience Required:4 or more years of professional software quality assurance experience, with demonstrated experience in cloud-native service testing · Experience performing formal vendor Deliverable acceptance reviews against documented acceptance criteria in a contractual delivery context · Proficiency in test automation frameworks for API and service testing, such as pytest, Postman, or equivalent · Experience testing containerized applications in GCP or equivalent cloud environments, including container deployment validation and service integration testing · Familiarity with Jira for bug management and workflow configuration; experience managing structured escalation workflows in Jira is strongly preferred · Experience designing and executing UAT programs with real end users, including test scenario design, facilitating sessions, and synthesizing structured feedback · Ability to write clear, specific defect reports that enable developers to reproduce and resolve issues without further clarification · Strong understanding of software acceptance criteria and the ability to evaluate whether a delivered artifact meets them objectively and defensiblyExperience Preferred:Experience evaluating LLM or AI model outputs for production quality, including prompt regression testing and AI output consistency validation · Familiarity with Dynatrace or equivalent APM (Application Performance Monitoring) tooling for test environment monitoring · Experience in automotive or manufacturing quality contexts where structured acceptance processes are the norm · ISTQB certification or equivalent formal QA training · Experience with performance and load testing for cloud-native APIs serving high-volume, real-time workloadsAdditional Information:HYBRID / 4 days per week in the office