Frequently Asked Questions

No. PrAImaan does not certify, approve, or regulate AI products for clinical use—that authority sits with bodies such as the Central Drugs Standard Control Organisation (CDSCO). PrAImaan generates independent, clinician-led evidence on how AI tools perform against Indian clinical scenarios. This evidence can inform regulatory decisions, institutional adoption choices, and further model development, but participating in a ValiDATAthon does not constitute regulatory clearance for any AI product, and a strong result does not imply endorsement by PrAImaan, ICMR-NIRDH, or Ashoka University.

Models are selected by the PrAImaan research team based on relevance to the clinical domain being evaluated at a given session, and may include a mix of open-source and commercial systems. Institutions do not select specific vendors to test, and vendors do not pay to be included or excluded. This is intentional: it keeps the evaluation independent of commercial influence, which is central to why the results carry evidentiary weight.

Findings are published in aggregate, and specific institutions are not identified in connection with specific model performance results without their consent. Institutions are, however, recognised publicly as ValiDATAthon partners, separate from any performance outcomes. If your institution has concerns about how results involving your clinicians' evaluations may be used or attributed, this can be addressed directly during LoA (Letter of Agreement) finalisation.

Before the session: a point of contact to coordinate logistics, a venue, and clinician recruitment from your institution (typically over 2–4 weeks of lead time). On the day: a half-day to full-day commitment from participating clinicians. After the session: no ongoing obligation is required, though institutions that wish to pursue further stages (e.g., proposing a project-specific clinical AI question) can expect a longer engagement, scoped separately.

Institutions retain rights to the evaluation data generated by their own clinicians and may use it for internal analysis or their own publications, subject to standard academic data-sharing agreements finalised as part of the LoA. Aggregated, anonymised data across institutions also feeds the shared national evidence base; this dual structure (institution-owned, plus contribution to a shared pool) is set out explicitly in the partnership agreement, not left ambiguous.

No. A ValiDATAthon is an evaluation exercise, not a procurement process. Participation and hosting carry no obligation to purchase, license, or deploy any AI system, regardless of how it performs.

Participation does not require disclosing or altering existing vendor relationships. If a currently-deployed tool happens to be among those evaluated, PrAImaan's role remains limited to independent evaluation; it does not intervene in or comment on existing commercial arrangements.

No prior AI experience is expected or required. The pre-session knowledge component (covering how these models are built and where they tend to fail) is designed to bring clinicians without a technical background to a working level of understanding before evaluation begins. Clinical judgment, not technical expertise, is what the evaluation depends on.

While each ValiDATAthon uses a structured, validated rubric to ensure consistency and inter-rater reliability across sessions, clinicians who contribute at scale—or who join PrAImaan's tiered consortium—have the opportunity to shape which clinical scenarios and evaluation dimensions are prioritised in future rounds. This is part of what distinguishes PrAImaan from a one-off vendor benchmark: the criteria themselves evolve based on frontline clinical input.

Each scenario is typically reviewed by multiple clinicians. Disagreement across reviewers isn't treated as a problem to be smoothed over; it's itself a data point. Divergence in scoring is recorded and factored into how confidently a given result can be reported, rather than being resolved by majority vote alone.

PrAImaan is building out its evaluation scenarios specialty by specialty, grounded in clinical guidelines and real epidemiological data specific to each domain, rather than attempting broad coverage across all of medicine simultaneously. This means the scenarios clinicians evaluate are built to reflect the specific clinical decision-points and comorbidity patterns relevant to Indian patients in that specialty, not generic case templates.

This is an active area of methodological development. Early-stage evaluation prioritises clinical and demographic representativeness (comorbidity patterns, presentation patterns typical of Indian patient populations); incorporating facility-tier and regional variation (e.g., primary/rural versus tertiary/urban settings) is part of the platform's intended evolution, reflecting the reality that "safe for Indian healthcare" isn't a single uniform bar.