NVIDIA's Indic data recipe is the part worth copying

Most of the 2.8 trillion tokens in NVIDIA and AI4Bharat's new Indic corpus began as English-language documents. Nemotron-pretraining-Indic-v1 takes web text, math, code and Q&A documents from Nemotron-CC-v2 and translates them into the 22 languages listed in India's Constitution. Gemma-4-31B-it does the translation and then scores its own output against the English source on a rubric modeled on MQM, a standard scheme for classifying translation errors. Outputs under 75 out of 100 get repaired using that feedback and rescored. The authors say this loop recovered 438.4 billion tokens.

The pipeline, which the authors say will be released alongside the prompts, quality checks, and record source IDs, will probably be the most useful piece for others looking to adapt their work. Provenance is what lets you audit synthetic data instead of taking it on faith. The piece I'd want to test is the grading. A model judging its own translations shares its own blind spots. The team backs it up with sampled human review and deliberately injected errors to catch omissions and leftover English. That is reasonable, but a sample is not 2.8 trillion tokens, and the 75 is a rubric score, not accuracy, as the authors themselves point out.

Translated English also carries English contextual knowledge, and the authors say plainly that natural Indic text is still needed for local context. For European teams working on Basque, Maltese or Irish, the open recipe is the interesting export. Whether self-graded translation holds up for those languages is something that needs to be measured. However, it is a promising bit of research for teams looking to build models that support languages that are under-represented digitally.