Thai University RankingsRESEARCH RADAR
Potential

OMNI's 10 Domains for Evaluating Generative AI in Health Care: An International Study With Prince of Songkla University Participation

Cornelisse and colleagues published a four-stage mixed methods study in the Journal of Medical Internet Research on October 27, 2026 (DOI: 10.2196/100531). They developed OMNI, a standardized, expert-led rubric evaluating generative AI output in ten domains: Accuracy, Safety, Recognizes Uncertainty, Completeness, Communicates Clearly, Empathetic, Context Aware, Referencing, Follows Instructions, and Avoids Coerciveness. Each applicable domain uses a four-point Likert rating supplemented by qualitative comments. In four health care scenarios, 15 clinicians assessed 250 input-output pairs, including 200 AI-generated outputs and 50 physician-tagged replies, generating 9,600 domain-level ratings and 4,010 qualitative comments. Recurring concerns included referencing, recognition of uncertainty, completeness, empathy, and accuracy; health-related images and videos were particularly challenging. Warit Ruanglertboon is a coauthor affiliated with Prince of Songkla University, as verified by the journal's author page. The accessible sources do not establish his individual responsibilities in the four study stages, so leadership or a specific technical role should not be attributed to him. These are pilot findings about an evaluation rubric, not clinical certification of AI models or evidence of adoption in Thai hospitals.

01

Key findings

  • OMNI comprises ten evaluation domains and was developed through four mixed-methods stages informed by 69 publications and 23 experts.
  • Fifteen clinical assessors evaluated 250 input-output pairs in four scenarios, recording 9,600 domain-level ratings and 4,010 comments.
  • Recurring weaknesses concerned referencing, recognition of uncertainty, completeness, empathy, and accuracy, with multimodal image/video outputs especially challenging.
  • Inter-rater agreement varied by setting (Gwet's AC2 0.49-0.80), so the rubric does not demonstrate perfect reliability.
  • Warit Ruanglertboon of Prince of Songkla University is a verified coauthor; his precise technical tasks in this project have not been established.
02

Why this matters globally

The main contribution is a common, expert-accessible language for examining AI output quality and safety without reducing performance to a single aggregate score. Ten domains and qualitative justifications can make weaknesses—especially poor referencing and uncertainty handling—visible to evaluators. In the pilot, mean pairwise agreement was 30.8%-53.4% on the original categories, increasing to 59.3%-79.6% with broader grouping; Gwet's AC2 ranged from 0.49 to 0.80. Thus, assessor calibration remains important. Potential use for procurement, governance, quality assurance, and model improvement is a plausible application discussed by the authors, not evidence that OMNI guarantees clinical safety or has already been validated in deployment.

03

Thai researcher contribution

The journal's author page identifies Warit Ruanglertboon as a coauthor with two affiliations in Hat Yai, Songkhla, Thailand: (16) Discipline of Pharmacology, Division of Health and Applied Sciences, Faculty of Science, Prince of Songkla University, and (17) Research Center in Mathematics and Statistics with Applications, Faculty of Science, Prince of Songkla University. The university's own Faculty of Science personnel page independently confirms his institutional affiliation. This establishes Thailand-affiliated participation in an international publication. It does not establish that the study was conducted in Thailand, led by the Thai coauthor, or validated on Thai clinical systems. Individual responsibilities in rubric design, data collection, or analysis should not be inferred without further authorship-contribution evidence.

04

Limitations to consider

The stage-1 literature synthesis was targeted rather than a formal systematic or scoping review and could have missed sources. Stage-2 consensus was operationalized as an absence of substantive objections, which need not imply genuine agreement. The pilot covered only four use cases, primarily publicly accessible foundation models and single-turn interactions, with one generation per prompt. All pilot assessors were Australian-trained clinicians, and ratings involved subjective judgment with variable agreement. Evaluation time and workload were not measured prospectively. OMNI assesses observable outputs, not hidden reasoning or internal safety mechanisms. Its generalizability to specialized medical systems, agentic workflows, other jurisdictions, and real-world patient outcomes remains unestablished. No study finding should be presented as proof of implementation or clinical safety in Thailand.

05

Verify the original sources

Journal of Medical Internet ResearchRead the original article↗

DOI: 10.2196/100531

KEEP EXPLORING

More Thai research to explore