AI tools are rapidly entering India’s health system, often without regulation or transparent validation. Models built by companies, researchers, and open communities are already influencing clinical decisions. In a country as diverse as India, accuracy alone is not enough — models must be tested for contextual relevance and equity. Without a coordinated national framework, unvalidated AI risks misinforming care, deepening disparities, and compromising patient safety.