Vllm on Conselara Labs https://conselara.dev/tags/vllm/ Recent content in Vllm on Conselara Labs Hugo en Thu, 14 May 2026 00:00:00 +0000 Building a Two-Node Ray Cluster for Distributed LLM Inference on DGX Spark https://conselara.dev/notes/two-node-ray-cluster-dgx-spark/ Thu, 14 May 2026 00:00:00 +0000 https://conselara.dev/notes/two-node-ray-cluster-dgx-spark/ Step-by-step setup for a two-node Ray cluster running Qwen3-235B across two DGX Sparks, including RoCE verification, model sync, NCCL configuration, and the specific gotchas that cause silent failures. vLLM on DGX Spark: What the SM121 Architecture Actually Requires https://conselara.dev/notes/vllm-dgx-spark-sm121-gotchas/ Wed, 13 May 2026 00:00:00 +0000 https://conselara.dev/notes/vllm-dgx-spark-sm121-gotchas/ Hard-won config rules for running vLLM on DGX Spark GB10 (SM121). Covers broken backends, unified memory limits, quantization traps, multi-node NCCL, and flags that silently destroy throughput. DGX Spark Benchmark Results: vLLM on SM121 https://conselara.dev/notes/dgx-spark-benchmarks/ Sat, 09 May 2026 00:00:00 +0000 https://conselara.dev/notes/dgx-spark-benchmarks/ Measured throughput and latency for Qwen3-235B-A22B (two-node) and gpt-oss-120b (single node) on DGX Spark GB10 hardware with vLLM 0.19.0. DGX Spark Model Comparison: What Fits and What Runs (SM121, 128 GB) https://conselara.dev/notes/dgx-spark-model-comparison/ Sat, 09 May 2026 00:00:00 +0000 https://conselara.dev/notes/dgx-spark-model-comparison/ A quick-reference comparison of open-weight models for a single DGX Spark GB10 — what fits in 128 GB unified memory, expected throughput, and SM121 compatibility notes. vLLM Model Selection for DGX Spark (SM121) https://conselara.dev/notes/dgx-spark-model-selection/ Sat, 09 May 2026 00:00:00 +0000 https://conselara.dev/notes/dgx-spark-model-selection/ How to choose a model for the DGX Spark GB10 (SM121). Covers architecture compatibility, quantization format requirements, and what to expect from each option. Running Qwen3.5-122B on a Single DGX Spark https://conselara.dev/notes/dgx-spark-qwen35-122b/ Tue, 05 May 2026 00:00:00 +0000 https://conselara.dev/notes/dgx-spark-qwen35-122b/ Qwen3.5-122B-A10B runs at 51 tok/s on a single DGX Spark in NVFP4 quantization. This post covers the SM121 constraints, required vLLM flags, and what to expect.