A RAG system connected to Odoo through Chainlit, FastAPI and APIs passed 17–19 of 20 fixed questions across three evaluation frameworks, while three users gave a SUS score of 66.67.
Key findings
- Pass rates were 95% (19/20), 90% (18/20) and 85% (17/20). Mean expert score was 3.82, user satisfaction 4.33 and SUS 66.67, near the acceptable/good boundary; independent-use scored 2.33 and need-for-expert-support 2.67.
Why this matters globally
The work shows how grounded natural-language access may reduce ERP complexity for SMEs lacking specialists, while accuracy and authorization must take priority over convenience.
Thai researcher contribution
Khon Kaen University researchers designed and evaluated the prototype in a Thai SME, connecting RAG research with an operational business system.
Limitations to consider
The study involved one organization, 20 fixed questions, three users and five experts, with no baseline for time, errors or longitudinal use. Hallucination, prompt injection, permission leakage, privacy and cost require testing.