Eros GenAI researchers win Google DeepMind × Kaggle Grand Prize

Eros Innovation said two Eros GenAI researchers were named among four Grand Prize winners in a Google DeepMind × Kaggle hackathon focused on progress toward AGI. Their GAUGE benchmark tests whether AI models can recognize uncertainty and choose to abstain or escalate when answers may be unreliable.

Why it matters: - GAUGE targets a growing gap in AI safety: models can sound confident even when they are uncertain. - In high-stakes settings such as government, education, healthcare and enterprise systems, knowing when to abstain can matter as much as getting the answer right. - The benchmark is designed to measure whether a model’s self-assessment leads to responsible action.

What happened: - Eros Innovation said Eros GenAI researchers Arjun Thilak R. and Ramkumar M. V. were named among the four Grand Prize winners in the Google DeepMind × Kaggle “Measuring Progress Toward AGI: Cognitive Abilities” Hackathon. - Their winning benchmark is called GAUGE: Measuring the Gap Between What LLMs Know and What They Do About It. - GAUGE uses a multi-turn evaluation protocol to test whether large language models recognize uncertainty and respond by abstaining when an answer may be unreliable.

The details: - GAUGE evaluates three dimensions: how accurately a model assesses its own confidence, whether it changes behavior when uncertain, and whether confidence and decision-making are meaningfully connected. - The Eros GenAI researchers tested eight leading models across four model families. - The evaluation used a three-stage metacognitive protocol across mathematics, logic and factual knowledge. - Ridhima Lulla, Co-Founder and Co-President of Eros Innovation, said the recognition marks a milestone for Eros GenAI and reflects research focused on trust, oversight and responsible action. - Eros Innovation plans to explore integrating the GAUGE methodology into its wider model-evaluation and AI assurance framework. - The company also plans to extend the research into Cultural Intelligence, rights-aware AI, education, citizen services and other sovereign AI applications. - Arjun Thilak R. said a model that knows it is uncertain but still acts with confidence creates a serious risk for human operators. - Ramkumar M. V. said GAUGE goes beyond single-answer testing by examining how a model assesses uncertainty over multiple turns and whether that self-assessment changes behavior.

Between the lines: - The win gives Eros Innovation a research credibility boost in a crowded field where model benchmarks are increasingly used to define safety and performance. - The focus on abstention and escalation signals a broader industry shift away from raw accuracy alone and toward operational trust. - Eros is tying the result to its larger pitch around sovereign AI, cultural intelligence and human oversight built into system design.

What's next: - Eros Innovation intends to apply GAUGE in assessments of global, open-weight and Eros-developed models. - The company says the framework will support evaluation across reliability, uncertainty, safety and human-escalation behavior. - Eros Innovation will continue building its broader Sovereign Cultural Intelligence platform, including model orchestration, Cultural AI, rights and identity infrastructure and country-specific intelligence systems.

The bottom line: - The award gives Eros GenAI a visible win in frontier AI benchmarking while reinforcing a simple message: in real-world AI, knowing when not to answer may be as important as being right.