We run frontier open-weight models on a desktop-class supercomputer. People assume that means we think local models have caught up to Claude, GPT, and Gemini. They haven’t, and pretending otherwise is the fastest way to make a bad architecture decision.

This is how we actually compare models, open and closed, local and cloud, and how that comparison plays out for sensitive domains like federal health.

The honest starting point: closed frontier still leads

As of June 2026, the most capable general models are closed and cloud-hosted. On the Artificial Analysis Intelligence Index, the leaders are Claude Opus 4.8 (~61), GPT-5.5 (~60), and Gemini 3.1 Pro (~57). No open-weight model you can self-host reaches that tier.

It gets harder for the on-prem story. Every open model that clearly beats a mid-size open MoE like Qwen3.6-35B (MiniMax-M3, Kimi K2.6, GLM-5.1, DeepSeek-V4-Pro) is now 1T+ total parameters. Even at 4-bit, that’s past the ~256 GB ceiling of a two-node commodity setup. The open frontier has outgrown the hardware most organizations can realistically put in a rack.

So if the question is “what’s the smartest model I can use,” the answer is a cloud frontier model. Full stop. (Benchmarks move weekly; treat the specific numbers as a June 2026 snapshot.)

Then why run open models at all?

Because “smartest” is rarely the binding constraint. For a large class of real workloads, the constraint is the data boundary: where the data is allowed to go, who can see it, and what can be done with it.

When that’s the constraint, the relevant questions change:

  • Can the model run inside our trust boundary (on-prem, air-gapped, or in a controlled enclave)?
  • Is there a no-training, no-retention guarantee on every byte that touches it?
  • Is the cost predictable at volume, independent of per-token cloud pricing?
  • Is latency under our control, not subject to a shared API?

A mid-size open MoE that we fully control can win on all four while losing on raw capability, and for a bounded task (summarize this record, classify this document, draft this section), “good enough and inside the boundary” beats “smarter and outside it.”

How we compare every option

We don’t rank models on a single number. Every candidate gets scored on four axes, together:

  1. Capability: public benchmarks (AA Index, domain evals) as a first filter, then task-specific evaluation on our actual workloads.
  2. Throughput on real hardware: not the vendor’s headline number. We measure single-stream tok/s on the exact silicon we’d deploy (the DGX Spark’s GB10 / SM121), because architecture-vs-kernel mismatches routinely cut throughput by 2–4×. (We’ve written about the SM121 traps at length.)
  3. Architecture fit: does the model’s structure run well on our kernels? Pure mixture-of-experts models run cleanly; GatedDeltaNet/Mamba hybrids fall into a kernel gap and crawl. The smartest model that runs at 9 tok/s is not a usable model.
  4. The open-vs-closed gap: for the specific task, how much capability are we actually giving up by staying inside the boundary? Sometimes it’s decisive. Often it’s negligible.

The output isn’t “the best model.” It’s “the right model for this constraint.”

What it means for federal health

Federal health is the clearest case where the data boundary, not the leaderboard, drives the decision. Protected health information (PHI), HIPAA obligations, data-sovereignty requirements, and air-gapped enclaves all push toward inference that stays inside a controlled environment.

For workloads like de-identification, record summarization, document classification, or coding and QA assist on sensitive corpora, a self-hosted open model offers:

  • PHI never leaves the boundary: no third-party API call, no retention question to answer.
  • Sovereignty: the model and the data sit in infrastructure you control and can audit.
  • Predictable cost: fixed hardware, not metered per-token billing that scales with usage.
  • Air-gap capable: it runs with no outbound network at all.

The honest flip side: this is not a blanket “use local models for government” claim. When a workload needs frontier-level reasoning and the data can legally sit in an authorized cloud, a FedRAMP-authorized cloud frontier model (for example, Claude on AWS GovCloud) is often the better answer: more capable, and far less infrastructure to maintain. The right architecture usually has both: a private on-prem tier for the most sensitive, bounded work, and an authorized cloud tier for the hardest reasoning.

(We’re describing how we reason about this, not asserting any specific authorization or certified offering.)

The takeaway

Pick the model that fits the constraint, not the one that tops the benchmark. For most sensitive-data work the binding constraint is the data boundary, and that, far more than a few points of benchmark IQ, is what should decide where your inference runs.

The posts in our DGX Spark series are the engineering proof underneath this, starting with running gpt-oss-120b on a single DGX Spark: what it actually takes to run capable open models on hardware you own.