Commit to quality: AI-enhanced testing in open source

"Commit to quality: AI-enhanced testing in open source," presented by Shelley Lambert (IBM), explores how large-scale open source projects are applying AI to improve testing, triage, and release quality. This session was recorded at Open Community Experience 2026 (OCX26) in Brussels, Belgium, as part of the Open Community for AI. This session examines how AI techniques can be applied across the software testing lifecycle in large-scale open source projects, using the Adoptium and AQAvit initiatives as a case study. The testing context involves high-volume pipelines, with over 17 million tests executed per release cycle. At this scale, manual analysis is infeasible, requiring automated approaches for test execution, triage, and reporting. The approach focuses on augmenting key stages of the testing lifecycle. In triage, tools such as Commit Hunter identify likely fault-inducing changes using a combination of rule-based strategies and AI-assisted analysis. In pre-commit stages, Glitch Witcher applies defect prediction techniques to assess the likelihood of introducing errors based on historical code patterns. Performance testing is addressed through projects like Perf Savvy, which integrates benchmarking data and explores AI-driven detection of regressions using historical test results. Additionally, retrieval-augmented generation (RAG) techniques are used to improve access to project-specific knowledge, enabling more effective querying of test frameworks, documentation, and historical data. The session also explores opportunities for AI-assisted test generation and selection, using combinatorial analysis to reduce redundant test execution while maintaining coverage. Across these use cases, the focus remains on model-agnostic, data-driven approaches that leverage existing project artefacts rather than relying on proprietary tooling. Key topics covered - AI in software testing - large-scale test automation - Adoptium and AQAvit - commit analysis and fault detection - defect prediction models - performance regression detection - RAG for developer workflows - test generation and selection - combinatorial testing - CI/CD test analytics Why this matters At scale, software quality depends on the ability to prioritise, analyse, and act on large volumes of test data. AI-assisted workflows enable teams to improve triage, reduce redundant testing, and surface actionable insights, making testing pipelines more efficient and maintainable. About OCX26 Open Community Experience 2026 is the Eclipse Foundation’s flagship event, held in Brussels, Belgium. It brings together developers, architects, and industry leaders to explore open source technologies across domains including AI, automotive, tooling, and cloud systems, with a focus on practical implementation. Learn more at https://www.ocxconf.org/ Chapters 00:00 introduction and testing scale 00:49 Adoptium and AQAvit overview 03:46 testing lifecycle stages 05:28 AI-assisted testing opportunities 10:02 Perf Savvy and performance testing 15:10 Commit Hunter fault detection 17:17 Glitch Witcher defect prediction 21:42 RAG-based assistant (VITAI) 27:20 test generation and combinatorial testing 31:23 test results analytics and TRSS 33:52 challenges and future direction