{"schemaVersion":"jobsearcher.job.v1","id":"80eb2c3a811b852505940d0b","url":"https://jobsearcher.com/jobs/80eb2c3a811b852505940d0b","canonicalUrl":"https://jobsearcher.com/jobs/80eb2c3a811b852505940d0b","title":"Software Development Engineer in Test","description":"About SOFICO IntelligenceSOFICO Intelligence applies advanced AI and intelligence techniques to transform and extract value from the deep business controls embedded in back-office systems. We build agentic AI that reasons over the complex financial, leasing, fleet, and mobility-management data that runs the enterprise, surfacing fraud, billing errors, hidden risk, and value that traditional tools miss. We are backed by SOFICO, a global leader in ERP software for automotive finance, leasing, fleet, and mobility management, which gives us deep domain and back-office expertise to build on. Remote work is part of how we operate, and our team is made up of curious, self-directed people who like to build and ship.Our flagship AI platform, Velona, is a multi-agent system that turns years of business and operational data into financial intelligence. It finds fraud, ghost assets, billing errors, and risk on its own, and reports each finding as a specific asset, a dollar amount, and a recommended action. Rather than acting as a chatbot or a dashboard, it performs real analytical reasoning, runs investigations overnight without supervision, and answers in plain language.What The Job Is AboutThis role is responsible for making our autonomous AI trustworthy. As an SDET on the AI team, you own quality for non-deterministic, multi-agent systems, which raises questions most testing roles never face. How do you prove an AI reached the right answer for the right reason? How do you catch a hallucination before a customer sees it? How do you guarantee one tenant can never see another's data, that an agent never exceeds its guard rails, and that quality does not quietly drift when a model or prompt changes?You will help define how SOFICO Intelligence evaluates agentic AI, designing and building the evaluation harness that underpins our AI governance and our confidence to ship. This is real engineering against a production AI system at scale, not script-running against a static application. You write code to test code, and code to assess the quality of the AI itself. It is one of the more interesting testing problems in software today, and you would own it.What you'll do (about 85% of the role):Design and build SOFICO Intelligence's evaluation harness for agentic AI: golden datasets, automated scoring, faithfulness and grounding checks, and regression detection across prompt, model, and agent changesVerify grounding and faithfulness, confirming every insight and every number traces back to real data, and build the automated checks that stop hallucinations from reaching productionTest multi-agent orchestration end to end: agentic tool-calling loops, supervisor-specialist pipelines, autonomous background jobs, and the synthesis of findings into trustworthy reportsRed-team the system for prompt injection, jailbreaks, and tool misuse, and rigorously prove multi-tenant data isolation so one organization can never reach another's dataValidate that AI guard rails hold under pressure: iteration limits, agent-recursion depth, and token/cost ceilings behave correctly at the edgesPressure-test performance and scale: real-time streaming, high-concurrency conversations, latency, and cost under realistic and adversarial loadBuild automated tests for the Go microservices, data/SQL tools, and APIs that power the platform, and wire them into CI/CD for continuous evaluationPartner with AI and data engineers to define quality bars and acceptance criteria, and help evolve the supervisor review-and-retry loop that keeps the system honestThe remaining :15%, breadth across the platform:Strengthen automated testing across the broader SOFICO Intelligence platform and, where it intersects, the wider SOFICO engineering environment, bringing modern test practices to established systems and toolingHelp raise quality engineering standards consistently across the combined groupWhy this role mattersHard, important work. Evaluating autonomous, reasoning AI is one of the harder open problems in software today, and you would help define how we approach itReal ownership. You won't inherit a mature test suite to maintain. You'll build the evaluation foundation from the ground up and see it become core to how we shipReal impact. Your work decides whether customers can trust an AI with their money. The stakes and the visibility are both highStrong technology. Modern Go services, LLM orchestration, a multi-year proprietary data asset, and a team that has shipped at scale for yearsYour Skills And InterestsYou are comfortable with the ambiguity of testing systems that don't give the same answer twice. You take pride in shipping software people can trust, and you want to see the products you work on succeed in the real world. You do well in collaborative problem-solving environments, prefer coding and collaborating over sitting in meetings, and can prioritize your own work while working closely with a team. You give and take constructive criticism well, and you bring a structured, analytical approach to building scalable evaluation systems.What Else Is ExpectedMandatory skills:Proven experience spanning both software development and quality assurance / quality engineeringStrong coding skills, able to program confidently in at least one language (we work primarily in Go and Python)Hands-on experience with open-source automated testing tools and frameworksA solid understanding of RESTful APIs and the ability to perform HTTP requests from the command line or a tool like PostmanComfortable working from the command line and reasoning about distributed, concurrent systemsHands-on experience testing, evaluating, or building with LLM-based or agentic systems, not just interest. You have worked directly with model APIs, tool-calling, or multi-agent workflows and understand how they failDemonstrated ability to evaluate non-deterministic AI output against a measurable quality bar. For example, building or running eval harnesses, scoring grounding and faithfulness, detecting quality regressions, or red-teaming for prompt injection and jailbreaksWorking knowledge of LLM concepts that drive quality: prompting, context windows and token limits, hallucination and grounding, and the sources of non-determinism in model outputA sense of pride and a strong desire to ensure trustworthy, bug-free software ships every dayFluent in English, as the role involves interaction with international teamsBonus points:Depth in AI evaluation methodology, such as designing golden datasets, LLM-as-judge scoring, supervisor/critic patterns, or automated regression detection for agentic pipelinesExperience with Go, data warehouses / SQL analytics (e.g. Databricks), event-driven systems, or real-time streamingFamiliarity with enterprise back-office or ERP environments, including Java back-ends and relational databases (Oracle, MSSQL, Postgres), is a plus for the cross-platform workExposure to the automotive finance, leasing, fleet, or mobility domainWhat You'll Work WithAI & data platform: a production multi-agent AI system built on modern Go microservices, LLM orchestration, and a multi-year proprietary fleet-data assetEvaluation & testing: AI evaluation harnesses, golden datasets, Playwright, REST Assured, PyTest, JMeter, and similar open-source frameworksLanguages: Go and Python primarily, with JavaScript across the stackData & infrastructure: PostgreSQL, Redis, RabbitMQ, Databricks, Docker, and KubernetesAPIs: RESTful and real-time WebSocket / streaming interfaces, exercised via Postman, REST Assured, and the command lineOrganizational tools: Jira, TestRail, and Git, with tests integrated into CI/CD pipelinesBenefits & How We WorkWe have operated as a remote-first team for years and have a long track record of running productive, professional remote teams. We offer competitive benefits, including health care (medical, dental, and vision) for US employees and a flexible PTO policy, and we support your growth through conferences, advanced training, and certifications.","company":"Sofico","rawCompany":"sofico","city":"Dallas","state":"TX","isRemote":false,"isActive":true,"createdAt":"2026-10-28T09:37:38.526Z","occupations":[{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-1253.00","title":"Software Quality Assurance Analysts and Testers","slug":"software-quality-assurance-analysts-and-testers"},{"code":"15-1251.00","title":"Computer Programmers","slug":"computer-programmers"}],"industries":[{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"},{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Software Development Engineer in Test","description":"About SOFICO IntelligenceSOFICO Intelligence applies advanced AI and intelligence techniques to transform and extract value from the deep business controls embedded in back-office systems. We build agentic AI that reasons over the complex financial, leasing, fleet, and mobility-management data that runs the enterprise, surfacing fraud, billing errors, hidden risk, and value that traditional tools miss. We are backed by SOFICO, a global leader in ERP software for automotive finance, leasing, fleet, and mobility management, which gives us deep domain and back-office expertise to build on. Remote work is part of how we operate, and our team is made up of curious, self-directed people who like to build and ship.Our flagship AI platform, Velona, is a multi-agent system that turns years of business and operational data into financial intelligence. It finds fraud, ghost assets, billing errors, and risk on its own, and reports each finding as a specific asset, a dollar amount, and a recommended action. Rather than acting as a chatbot or a dashboard, it performs real analytical reasoning, runs investigations overnight without supervision, and answers in plain language.What The Job Is AboutThis role is responsible for making our autonomous AI trustworthy. As an SDET on the AI team, you own quality for non-deterministic, multi-agent systems, which raises questions most testing roles never face. How do you prove an AI reached the right answer for the right reason? How do you catch a hallucination before a customer sees it? How do you guarantee one tenant can never see another's data, that an agent never exceeds its guard rails, and that quality does not quietly drift when a model or prompt changes?You will help define how SOFICO Intelligence evaluates agentic AI, designing and building the evaluation harness that underpins our AI governance and our confidence to ship. This is real engineering against a production AI system at scale, not script-running against a static application. You write code to test code, and code to assess the quality of the AI itself. It is one of the more interesting testing problems in software today, and you would own it.What you'll do (about 85% of the role):Design and build SOFICO Intelligence's evaluation harness for agentic AI: golden datasets, automated scoring, faithfulness and grounding checks, and regression detection across prompt, model, and agent changesVerify grounding and faithfulness, confirming every insight and every number traces back to real data, and build the automated checks that stop hallucinations from reaching productionTest multi-agent orchestration end to end: agentic tool-calling loops, supervisor-specialist pipelines, autonomous background jobs, and the synthesis of findings into trustworthy reportsRed-team the system for prompt injection, jailbreaks, and tool misuse, and rigorously prove multi-tenant data isolation so one organization can never reach another's dataValidate that AI guard rails hold under pressure: iteration limits, agent-recursion depth, and token/cost ceilings behave correctly at the edgesPressure-test performance and scale: real-time streaming, high-concurrency conversations, latency, and cost under realistic and adversarial loadBuild automated tests for the Go microservices, data/SQL tools, and APIs that power the platform, and wire them into CI/CD for continuous evaluationPartner with AI and data engineers to define quality bars and acceptance criteria, and help evolve the supervisor review-and-retry loop that keeps the system honestThe remaining :15%, breadth across the platform:Strengthen automated testing across the broader SOFICO Intelligence platform and, where it intersects, the wider SOFICO engineering environment, bringing modern test practices to established systems and toolingHelp raise quality engineering standards consistently across the combined groupWhy this role mattersHard, important work. Evaluating autonomous, reasoning AI is one of the harder open problems in software today, and you would help define how we approach itReal ownership. You won't inherit a mature test suite to maintain. You'll build the evaluation foundation from the ground up and see it become core to how we shipReal impact. Your work decides whether customers can trust an AI with their money. The stakes and the visibility are both highStrong technology. Modern Go services, LLM orchestration, a multi-year proprietary data asset, and a team that has shipped at scale for yearsYour Skills And InterestsYou are comfortable with the ambiguity of testing systems that don't give the same answer twice. You take pride in shipping software people can trust, and you want to see the products you work on succeed in the real world. You do well in collaborative problem-solving environments, prefer coding and collaborating over sitting in meetings, and can prioritize your own work while working closely with a team. You give and take constructive criticism well, and you bring a structured, analytical approach to building scalable evaluation systems.What Else Is ExpectedMandatory skills:Proven experience spanning both software development and quality assurance / quality engineeringStrong coding skills, able to program confidently in at least one language (we work primarily in Go and Python)Hands-on experience with open-source automated testing tools and frameworksA solid understanding of RESTful APIs and the ability to perform HTTP requests from the command line or a tool like PostmanComfortable working from the command line and reasoning about distributed, concurrent systemsHands-on experience testing, evaluating, or building with LLM-based or agentic systems, not just interest. You have worked directly with model APIs, tool-calling, or multi-agent workflows and understand how they failDemonstrated ability to evaluate non-deterministic AI output against a measurable quality bar. For example, building or running eval harnesses, scoring grounding and faithfulness, detecting quality regressions, or red-teaming for prompt injection and jailbreaksWorking knowledge of LLM concepts that drive quality: prompting, context windows and token limits, hallucination and grounding, and the sources of non-determinism in model outputA sense of pride and a strong desire to ensure trustworthy, bug-free software ships every dayFluent in English, as the role involves interaction with international teamsBonus points:Depth in AI evaluation methodology, such as designing golden datasets, LLM-as-judge scoring, supervisor/critic patterns, or automated regression detection for agentic pipelinesExperience with Go, data warehouses / SQL analytics (e.g. Databricks), event-driven systems, or real-time streamingFamiliarity with enterprise back-office or ERP environments, including Java back-ends and relational databases (Oracle, MSSQL, Postgres), is a plus for the cross-platform workExposure to the automotive finance, leasing, fleet, or mobility domainWhat You'll Work WithAI & data platform: a production multi-agent AI system built on modern Go microservices, LLM orchestration, and a multi-year proprietary fleet-data assetEvaluation & testing: AI evaluation harnesses, golden datasets, Playwright, REST Assured, PyTest, JMeter, and similar open-source frameworksLanguages: Go and Python primarily, with JavaScript across the stackData & infrastructure: PostgreSQL, Redis, RabbitMQ, Databricks, Docker, and KubernetesAPIs: RESTful and real-time WebSocket / streaming interfaces, exercised via Postman, REST Assured, and the command lineOrganizational tools: Jira, TestRail, and Git, with tests integrated into CI/CD pipelinesBenefits & How We WorkWe have operated as a remote-first team for years and have a long track record of running productive, professional remote teams. We offer competitive benefits, including health care (medical, dental, and vision) for US employees and a flexible PTO policy, and we support your growth through conferences, advanced training, and certifications.","datePosted":"2026-10-28T09:37:38.526Z","dateModified":"2026-10-28T09:37:38.526Z","hiringOrganization":{"@type":"Organization","name":"Sofico","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Dallas","addressRegion":"TX","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"80eb2c3a811b852505940d0b"},"url":"https://jobsearcher.com/jobs/80eb2c3a811b852505940d0b"}}