Did Nemotron 3.5 Double NVIDIA’s Score? The Open-Weight Gap Remains

🔍 Read the full analysis: Did Nemotron 3.5 Double NVIDIA’s Score? The Open-Weight Gap Remains on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get hardware and tech essentials delivered free with Prime

• Fast, free delivery on millions of items
• Prime Video, Amazon Music and more included
• Member-only deals all year

Start your free Prime trial →Free trial for eligible customers · Cancel anytime

As an affiliate, we earn on qualifying purchases.Hide

NVIDIA released Nemotron 3.5 Super VL and Nemotron 3.5 Bolt 74B-A9B on October 15, with same-day Artificial Analysis evaluations showing Super VL scored 26 on its Intelligence Index, up from 13 for its predecessor. The result marks a substantial improvement for NVIDIA, but the highest-scoring open-weight models listed by the evaluator still come from China and the United Arab Emirates.

Super VL is a 120-billion-parameter mixture-of-experts model with 12 billion parameters active per token. It accepts text and images, returns text, supports a one-million-token context window and is released under OpenMDW-1.1, which allows commercial use. Artificial Analysis gives it an unrounded Intelligence Index score of 26.5. The predecessor, Nemotron 3 Super, scored 13 in the comparison cited by the source report.

The evaluation also reports lower resource use per index task: Super VL generated about 49,000 output tokens, compared with roughly 114,000 for its predecessor. Its reported task cost fell from $1.64 to $0.25. The source says Super VL is distributed in NVIDIA’s NVFP4 four-bit format; Artificial Analysis found that version’s score was within 0.1 points of the full-precision BF16 weights, with no individual evaluation moving by more than 2.6 points.

The second release, Bolt 74B-A9B, scored 21, with 74 billion total and nine billion active parameters. Artificial Analysis ranks it fourth among models below 100 billion total parameters and first among US models in that size group. It has a 256,000-token context window, but no public API was listed in the source report, so a hosted API cost comparison was not available.

NVIDIA Gains Ground in Efficient Models

Super VL’s score and reported task cost suggest NVIDIA has improved the performance-cost trade-off of its open-weight offering. In the comparison supplied by the source, it also scores above Nemotron 3 Ultra, a 550-billion-parameter model with 55 billion active parameters, which scored 23 and cost $0.60 per index task. Those results may matter to developers choosing models for deployment, though actual performance and cost can vary with provider, hardware and workload.

The rankings also put the improvement in perspective. The source lists China’s MiMo-V2.6-Pro at 46 and a UAE model, K2 Horizon 375B A23B, at 31. Super VL is 18th among the open-weight models in the cited table and 11th when limited to permissively licensed models. It leads the permissively licensed US models identified in the report, but does not lead the global field.

For operators considering local deployment, the NVFP4 result could reduce memory requirements compared with 16-bit weights while preserving a similar aggregate score in this evaluation. That does not establish identical results for every task or hardware configuration. Bolt’s reported long output length also shows why a model’s per-token price alone would not settle its overall serving cost.

How the Rankings Compare

The source report describes both releases as reasoning models that accept text and images and produce text. Both use a mixture-of-experts design, activating only part of their networks for each token. Super VL is the larger-context flagship, while Bolt is the first model in a new NVIDIA family intended to run on less hardware.

Artificial Analysis’ Intelligence Index combines evaluation results into a score; it is not a direct measure of performance on every user’s tasks. The source reports that Super VL generated about 49,000 output tokens per index task, versus 28,000 for Nemotron 3 Ultra. Bolt generated about 82,000 tokens per task, nearly three times Ultra’s figure. Bolt exceeded Ultra on AutomationBench-AA and AA-Briefcase in the cited results, while trailing it on long-context reasoning and Humanity’s Last Exam.

A separate same-day release affects comparisons among US models. Meta’s Muse Spark Open scored 39, according to the source report, but uses the Muse Model License, which the report says has commercial restrictions. The report therefore distinguishes scores across all open-weight models from results among models whose licenses permit commercial use.

Limits of the Published Comparisons

The source report does not provide the full evaluation methodology or underlying results for every model, so the scores alone cannot show how each system would perform on a particular application. It also does not establish that the reported cost per index task will match a buyer’s costs, which can depend on provider, hardware, usage and pricing.

Super VL’s speed figures vary by serving provider: the report cites about 630 output tokens per second from a pre-release DeepInfra endpoint with 100,000-token prompts, while the median across six listed providers was 196 tokens per second. Bolt had no public API listed, leaving its hosted cost and provider-dependent speed unclear. The source material also ends before completing its description of the Muse Model License, so its precise restrictions cannot be detailed here beyond the report’s statement that commercial conditions apply.

API Access and Deployment Costs

Further provider listings and public API availability will make it possible to compare Bolt’s serving speed and price with other models. For Super VL, additional provider measurements can clarify how closely hosted performance matches the fastest reported endpoint. Developers assessing either model will need to check the applicable license, their hardware requirements and results on their own workloads before relying on the benchmark comparisons.

Key Questions

What score did Nemotron 3.5 Super VL receive?

Artificial Analysis gave it an Intelligence Index score of 26, or 26.5 unrounded, in the evaluation cited by the source report.

Did Super VL score twice as high as its predecessor?

In the cited comparison, Nemotron 3 Super scored 13 and Super VL scored 26. The report also says output tokens per index task fell from about 114,000 to about 49,000.

Does Super VL lead all open-weight models?

No. The source report places it 18th in the overall open-weight table and 11th among permissively licensed models. It describes Super VL as the leading US model among permissively licensed models.

How does Bolt compare with Super VL?

Bolt scored 21, compared with Super VL’s 26. Its total and active parameter counts are smaller, but the report says it produced about 82,000 output tokens per index task and had no public API listed.

Can the reported benchmark results predict performance for every use?

No. The Intelligence Index summarizes results across evaluations. Performance, speed and cost can vary with the task, provider and hardware, so the reported figures do not guarantee the same results in a particular deployment.

Source: ThorstenMeyerAI.com