Sports-magazine editors helped JioStar develop tests for cricket-statistics questions in JioHotstar’s AI search, according to chief architect Vijay Seshadri in an OpenAI interview published on November 11, 2026. For businesses adding conversational search, the account shows where subject expertise fits into checking customer-facing answers.
This is not a new launch. JioStar announced the partnership on February 19, and told investors on October 30 that conversational discovery was available in the app. The November interview describes an approach to testing, not independently audited accuracy.
A recommendation and a statistic need different tests
JioStar’s original announcement described voice and text discovery across languages, covering entertainment recommendations and sports information. Those tasks need different standards: several films might suit a viewer’s preferences, but a cricket statistic must match the requested player, match and record.
For a buyer, the practical implication is to approve each kind of answer separately. Recommendations need relevance checks; factual lookups need trusted reference records. An ambiguous request needs a useful follow-up question. Combining them into one overall satisfaction score could hide a weak category behind a strong one.
OpenAI’s evaluation guidance recommends tests tailored to the actual task and human feedback to calibrate automated scoring. A small business can put that responsibility with staff who already resolve customer questions: have them define acceptable answers, missing-information cases and requests the assistant should not handle.
The distinction also matters outside consumer search. In Kuken’s engineering-drawing search case, finding a similar drawing is only an intermediate step; engineers still need to establish that it is the right record for the equipment.
Test the reviewer, not just the answer
Seshadri says AI expanded the initial examples and another model checked answers. The reviewer needed its own evaluation; the account supplies neither a public test set nor a measured error rate.
There is a research basis for treating automated review as a separate problem. A 2025 study of 15 AI judges found that the order in which candidate answers appeared could influence judgments, with effects varying across judges and tasks. That study does not establish a fault in JioHotstar’s system.
For businesses commissioning a reviewer, a useful acceptance check would include specialist-labelled examples of both correct and incorrect answers. Count two errors separately: wrong answers it approves, and right answers it rejects. The first can mislead customers; the second can create needless refusals or staff work. Agreement between two models is not a substitute for a checked reference answer.
Where a reviewer chooses between two candidate answers, the study suggests another practical check: reverse their order and investigate changes in the verdict. A stable choice still needs checking against the facts. Keep acceptance examples separate from the examples used to tune the system, as OpenAI’s guidance on held-out evaluation sets illustrates.
After launch, count completed tasks
Seshadri says he would invest earlier in monitoring answer quality after launch. Fast replies alone do not show that customers received useful answers.
JioStar reported 75% quarter-on-quarter growth in engagement with conversational discovery in October. The transcript does not define that measure or give its starting value. Its separate figure of 440 million monthly active app users is not an assistant-user count. Neither number establishes answer accuracy or financial return.
For a pilot, a more useful comparison would track whether customers complete the intended task with conversational search versus the existing route. Include the time taken, unresolved requests and any staff assistance. For multilingual services, break out results by supported language and input mode rather than assuming one overall score represents everyone.
Budget for the whole task, too: retrieving information, generating and checking answers, handling retries and providing human review. The implementation account does not disclose enough costs to calculate savings for another business.
Before widening a pilot, ask the supplier and the staff responsible for answer quality to show which tasks passed, how the reviewer was checked, and who will act when quality drops. Feed authorised examples of new problems back into the tests. Expansion should follow evidence for each task, not the confidence of the conversation.