Skip to content

Model configuration and API

Model Controls

Ollama, vLLM and vLLM-Omni share one local-model form for both Create and Edit. Model source, hardware, runtime and memory sections expose the selected engine's supported controls; one collapsed Advanced section contains shared CPU controls and engine-specific options. External providers keep their provider form. Existing activations retain their saved engine, source and hardware target when edited; only changed fields are submitted with the original revision fence. Polling updates availability without replacing unsaved input.

With automatic NVIDIA pool assignment the calculator and slider describe one physical GPU, not total pool VRAM. A numeric budget above the known per-card physical limit disables Save/Add and is also rejected by the API, even with memory-risk acceptance. An invalid older saved budget remains visible for an explicit correction; opening Edit does not silently replace it. An explicit multi-GPU split retains its separate group-total control. See memory accounting.

The Name field validates local and external models before submission: 1–63 lowercase letters, numbers or hyphens, starting and ending with a letter or number. Inline feedback explains an invalid or already-used name and blocks creation. Surrounding whitespace is removed before saving; names remain fixed when editing. The API also validates the name when creating a resource.

Create → Location: Local → Inference Engine: (Experimental) vLLM-Omni selects the catalog's experimental Omni profiles for NVIDIA CUDA, AMD ROCm, Intel XPU and CPU. GPU choices use schedulable Kubernetes resources, not chip/driver or Strix-Halo allowlists. Shared mode uses one slot; exclusive mode offers the one-/two-GPU stage plans. CPU does not request a GPU. A matching runtime image is still required; CPU/XPU need an explicit image in Advanced. Context and concurrent sessions use the common runtime section. The memory section contains system RAM and the GPU fraction; Advanced contains the optional image and thinker offloading as well as CPU reservation/limit. RAM estimates are advisory; actual node capacity and slot checks stay. Model source offers HF search and direct repository paths. Quantization, architecture, stage completeness and config.json are not pre-approved. Any valid HF reference can be tried without an online configuration check; the runtime reports actual load failures in Status/Logs. Selection is not a hardware/model support guarantee. The selected repository is saved in local.url and retained on edit. Location distinguishes only Local and External; vLLM-Omni is offered alongside the other local engines when the catalog advertises a Realtime profile. It still persists as engine: VLLM with local.realtime, not a new backend engine type. Ordinary KV precision, tensor/pipeline parallelism, MiB VRAM controls and vision-attention overrides are not mapped into the shipped duplex stage plan; the API rejects these fields for Omni. The shared interface does not change that runtime contract. Installed Realtime models reuse Edit, Logs, Start/Stop, Restart and Remove. Speech testing uses LiteLLM's existing Realtime Playground, not a new dashboard conversation UI. See Realtime, including the pinned development-runtime and hardware acceptance boundaries.

Local model creation and editing include one collapsed Advanced section. Its CPU subsection is independent of memory controls. Blank fields use the engine defaults advertised by the compute-target catalog. Users can set a CPU reservation and optional limit in logical cores; limit 0 removes the CPU quota. Use automatic CPU settings removes stored overrides. Saving only CPU changes preserves RAM/VRAM and model identity, and may restart the model Pod. See the CPU scheduling policy for defaults. An unschedulable model shows Kubernetes' actual scheduling reason (for example, Insufficient cpu) instead of only waiting for a ready replica.

For vLLM on AMD, the Deployment subsection of Advanced exposes the Vision attention backend. It defaults to vLLM Triton attention for new AMD models; Automatic (vLLM default), PyTorch SDPA + AOTriton, and FlashAttention (AMD Triton) remain selectable. The compute-target catalog owns the options and eligible targets. Info icons explain runtime dependencies; selecting an option is not a successful hardware/model test. Editing saves only the changed selection and may restart the runtime. Switching engine or hardware in Create clears the AMD-specific draft. This setting does not change the memory reservation, context, KV precision, or text decoder backend. See vision attention deployment for mappings and the distinction between an omitted setting and explicit Auto.

Local and external models are runtime requests stored as ModelActivation resources in namespace ai-system.

The Create button beside Installed Models opens the model form directly in that section. The form starts with a persistent location dropdown containing Local and External. The external choice reveals the existing provider form. For a local model, inference engine and hardware appear as additional dropdowns above the model form. Every completed selection remains visible and can be changed directly; there are no wizard steps or back buttons. Engine and hardware choices are filtered by current cluster capability. Unavailable CPU or accelerator targets are omitted instead of being presented as disabled choices. The engine dropdown sorts regular engines alphabetically first (Ollama, vLLM), followed by (Experimental) vLLM-Omni, also alphabetically. The first offered engine is the default for a new form unless the API advertises an available default engine. These are display labels only; catalog availability and persisted engine identifiers are unchanged.

The ordinary local Create form detects the Chat or Embedding task from the selected preset or registry metadata. It shows the result without a Type dropdown. If metadata is insufficient, Model task becomes a required manual choice; changing the model clears that choice. External creation and existing model editing retain their explicit type controls. Realtime uses its own catalog profile. Detection does not certify that an engine can load the model. See the API task contract.

Ordinary Create and Edit both offer optional Max Output Tokens; leaving it blank uses the runtime default. Clearing a stored limit sends an explicit reset. Omni does not expose this field because its current duplex adapter rejects it.

For vLLM, Model source offers three persistent choices:

  • Hugging Face search accepts a model name or prefix and provides normal full-width dropdowns, one below the other, for the matching model and its original or quantized artifacts. A stable model-family row provides Qwen, DeepSeek, GLM, Llama, and Gemma searches, while a second row comes from Hugging Face's live trendingScore order. Additional result pages can be loaded without losing the current selection.
  • Tested preset retains the allowlisted catalog flow and its Precision / Quantization dropdown.
  • Direct Hugging Face URL retains the existing hf:// escape hatch.

The discovery backend returns only public, non-gated, non-disabled repositories. Quantizations must declare a direct quantized relationship to the exact selected model; name similarity, adapter, fine-tune, and merge relationships do not qualify. Publisher, formats, parameter metadata, revision, trust classification, and runtime-compatibility guidance remain visible after selection. Revision displays the full SHA returned for the currently selected artifact, including when a different quantization is selected. It is discovery metadata, not a guarantee that the runtime download is pinned to that commit. If the registry returns no revision, the dashboard does not invent one. When Hugging Face publishes usedStorage, the dashboard also shows the repository download size. The artifact resolver and estimator read the selected repository's public config.json, including nested language-model configuration such as text_config; this is necessary because Hugging Face search and detail payloads can omit the advertised context limit. When the configuration advertises a maximum context, that value becomes the initial Context Size while remaining editable. If a quantization repository does not publish its own context metadata, discovery inherits the directly related base model's value and labels it accordingly. A new dynamic model starts with Max Num Seqs = 1. A dynamic result is labelled experimental until the chosen runtime has loaded it; the dashboard does not claim that arbitrary community artifacts are validated. Selecting an artifact writes its hf:// repository to a custom activation and reuses the same server-side memory estimator as a direct URL. GGUF and MLX repositories are excluded from the current vLLM selection because that path cannot select the required GGUF file/plugin and does not run MLX.

For Ollama, Model source offers three persistent choices:

  • Ollama Library provides stable Qwen, DeepSeek, GLM, Llama, Gemma, and Mistral family shortcuts, a live popular-model row, prefix search, and two full-width dropdowns for model and tag/quantization.
  • Tested preset retains the validated catalog variants.
  • Direct Ollama model reference retains the ollama://model:tag escape hatch.

The tag selection displays the download size and advertised context from the public Ollama tag page and starts with Max Num Seqs = 1. After selection, the registry manifest supplies exact model-layer bytes and the estimator reads only a bounded GGUF-header range for architecture dimensions. It can also refine the quantization from source metadata. Cloud-only tags are not offered. The public Ollama Library is an HTML interface rather than a documented catalog API, so the backend uses a bounded, cached adapter and leaves presets and direct references available when discovery fails. Hugging Face discovery remains separate because the current runtime contract pulls ollama:// registry models and cannot import an arbitrary Hugging Face GGUF repository automatically.

vLLM accepts hf:// model references; Ollama uses ollama:// references. Ordinary vLLM uses upstream 0.31.0 for all four target images, pinned by digest; vLLM-Omni is versioned separately. Ollama uses immutable 0.40.3 images for CPU, NVIDIA and AMD ROCm, retaining the same registry tags and saved model controls. See Ollama upgrades before changing an installed runtime. Registry search and memory estimates are not a runtime compatibility certification. Check model architecture, quantization and target hardware against the selected runtime when startup fails. cpu is available on a compatible Ready Linux node. nvidia-gpu, amd-gpu, and intel-gpu are shown only when the matching provider is Ready and Kubernetes reports its allocatable resource. Intel automatically resolves gpu.intel.com/xe or gpu.intel.com/i915. vLLM supports all four targets. Ollama supports CPU, NVIDIA, and AMD; Intel is absent from the Ollama choices because no validated KubeAI/Ollama Intel profile is bundled.

For an experimental AMD profile, the selected engine needs current host evidence and an adopted runtime configuration; GPU validation remains optional. Upstream provider readiness and an allocatable resource alone do not satisfy that additional gate.

Beside Context Size, the local form shows a KV Cache dropdown. Ollama offers F16, Q8, and Q4. vLLM offers model precision on every target and FP8 only for CUDA/NVIDIA and ROCm/AMD. The options come from the selected compute target's live API contract, so CPU and Intel never show an unsupported FP8 choice. Changing the cache format immediately reruns both the normal memory estimate and, when selected, the CPU-offloading estimate.

An Ollama pod is not sufficient by itself for a Ready model. KubeAI addresses the runtime with the ModelActivation name, while the downloaded registry tag can have a different name. The Magic Stick Operator therefore waits for the source download and verifies or repairs that runtime alias before the model is published to LiteLLM. During this phase the installed-model status remains Starting instead of exposing a route that would return 404 model not found.

The dashboard calls POST /api/models/estimate-memory for every supported vLLM/Ollama local combination: vLLM on CPU, NVIDIA, AMD, and Intel, plus Ollama on CPU, NVIDIA, and AMD. vLLM estimates use public HuggingFace weight and model-configuration metadata. Ollama estimates use the exact runtime-layer byte total from the public registry manifest and attention, recurrent-state, GQA, hybrid-layer, and context dimensions from the GGUF header. Only the first bounded header range is requested; model tensors are not downloaded by the estimator. If a registry proxy cannot serve that range, the API retains the conservative manifest-only calculation and reports that fallback explicitly.

Both the RAM and VRAM controls use unreserved memory (total memory - active model reservations) as their 100-percent slider maximum. The separate live free-memory value does not change that planning limit. Minimum and recommended values are marked on the same scale. If either estimate exceeds unreserved capacity, its marker remains visible in a gray overflow section to the right of the slider. The slider stays bounded; the numeric field also accepts larger budgets, with an explicit memory-risk warning before creation. For Split one model across several selected NVIDIA GPUs, Create and Edit show a Total VRAM budget (MiB), including total Minimum, Recommended and 100% markers. The maximum is the smallest selected card's unreserved budget, rounded down to 100 MiB per card, multiplied by the number of selected cards. The input divides that total equally into per-card reservations; each share must still fit its physical GPU. Replicate model copies retains a budget for each full copy, and a single GPU retains its individual budget. The React dashboard's collapsible Breakdown separates model weights, the base KV-cache estimate, recurrent state where applicable, hybrid-allocator safety, engine runtime components, recommendation headroom, and download size. Split-model GPU components are totals across the selected cards, matching the calculator; their explanations show the per-card formula and group multiplier. Host runtime for a split Pod and download bytes are counted once. Replication keeps full per-copy estimates. For vLLM hybrid models, the UI labels the architecture-derived KV value as Theoretical KV cache and shows the extra compatibility budget independently instead of presenting their sum as physical cache use. Ollama hybrid models show attention KV and their context-independent recurrent state separately. CPU vLLM runtime reserve is split into compile/warm-up headroom, the multimodal processor cache, and a quantization working copy when applicable. Download size is storage/network information and is explicitly not included in the memory total. Every displayed recommendation and selectable reservation is rounded upward to a 100 MiB planning step. The safe slider maximum is rounded down to the same step. Each summary/Breakdown value has an info (i) button: hover or focus previews its explanation; click/Enter pins it, and Escape, the close button, or an outside click dismisses it. Overlays show the formula, substituted current inputs, binary units and rounding. Cache explanations include bytes per token per sequence and assumed KV precision, independent of weight quantization. Quantized-cache overlays also show the exact format storage rule, the F16/native-16-bit baseline, and the resulting saving. Ollama Q8_0 uses 34 bytes per 32-value block and Q4_0 uses 18 bytes per block, including their two-byte scales; only attention cache changes. Runtime allowances and fallback heuristics are labelled as estimates, not measurements. The API supplies additive calculations entries containing formula, substitution, and notes; an older API shows an honest unavailable message instead of invented dimensions.

For CPU targets, the selected value is stored as spec.local.memoryRequiredMi. The operator rounds it up to a 16 MiB unit and turns it into the model pod's Kubernetes requests.memory. For accelerator targets, the selected VRAM remains scheduling/planning metadata; old Ollama models without an explicit offloading policy retain their automatic fit. Live memory metrics currently come from NVIDIA DCGM, so AMD and Intel estimates can show minimum, recommendation, and breakdown without an adjustable maximum until matching memory metrics are available. For CPU vLLM, the dashboard API derives spec.local.kvCacheMemoryBytes server-side from public model architecture metadata, context size, and maximum parallel sequences. The browser cannot supply an arbitrary runtime value. The operator passes that value to vLLM as --kv-cache-memory-bytes; 512 MiB remains only as a compatibility fallback for older or directly created resources that do not contain the derived field.

GPU models also show System RAM estimate / reservation with serving RAM, an estimated startup range and a conservative recommended budget. Startup includes transfer buffers and a reclaimable file-cache envelope; these are not additional permanent model weights or measured RSS. One split Pod has one RAM budget, while replication shows per-copy and combined recommendations. New multi-GPU/offloading configurations use this recommendation when available; saved or manually edited RAM is preserved. The recommendation button is explicit, and a value below the provisional startup minimum needs memory-risk acceptance. Single-GPU models without CPU offloading retain their existing resource profile and show advisory RAM figures rather than claiming that a new limit was applied. Unverified capacity or missing weight metadata stays Unknown.

Opening Edit keeps the saved RAM/VRAM budgets visible while the estimate loads. A failed estimate shows an actionable error and blocks saving; it does not replace the saved budget with a guessed value. Realtime compute choices identify both the node and CPU/GPU target, including when both are advertised on one node.

vLLM GPU cache estimates use local tensor-rank heads (including GQA/MLA replication) or pipeline-local layers. Recurrent state and QSA side caches are itemized. Ollama uses an estimated whole-layer placement envelope. Per-worker runtime reserve follows local shards, with uneven placement separated from weights; the total estimate is labelled Estimated even when checkpoint byte metadata is exact. Special N-gram/PLE host placement requires runtime-specific verification and is not automatically enabled by this calculator.

The installed-model card distinguishes KV requested from KV active. The second value remains pending until the operator observes a Ready replica created with the requested vLLM argument or Ollama environment. This confirms the applied pod configuration rather than claiming a separate engine-memory measurement; a runtime that rejects the mode never receives a misleading active label.

For NVIDIA GPU models, CPU offloading → Use additional system RAM is an opt-in setting below the VRAM control. Keep VRAM and host RAM separate: the new Host RAM reservation slider/number field includes offloaded weights and host runtime, while Use recommended RAM allocation adjusts only host RAM. Budgets use 100 MiB steps. The host ceiling is the largest eligible node's allocatable RAM after workload requests, with pending requests deducted conservatively; it is not the cluster-wide CPU gauge. Unknown capacity, uncertain estimates, or inadequate budgets show a warning, not a disabled Add Local Model button. Clicking the warning-styled button accepts the memory risk and stores spec.local.allowMemoryRisk: true. The API/controller then allow estimated under-reservation and unverifiable/oversubscribed capacity without raising the selected budgets or removing requests/limits. Pods may remain Pending, fail to load, or OOM/restart. Invalid inputs, unsupported hardware/engine combinations, missing metadata needed to derive an offloading plan and replica restrictions are still errors. Manual RAM choices survive recalculation, and changing engine or hardware resets opt-in so a previous policy cannot silently carry over.

The expanded breakdown separates estimated GPU/RAM weights and KV placement, host/GPU runtime, and headroom. vLLM offloads weights, not KV; Ollama always uses GPU-first auto-fit and chooses the exact maximum layer placement when loading. The proportional preflight split is not a VRAM hard limit. The installed model card shows the host reservation separately and displays Ollama's /api/ps RAM/VRAM buffer reports when available. Missing measurements remain explicitly unknown, never zero. These reports are not process RSS or the Kubernetes reservation. See CPU offloading behavior and limits.

The estimator prefers exact public Safetensors file sizes, checks the requested context against the model's advertised maximum, and adds a CPU-specific start-up envelope for working tensors, compilation/warm-up, and the default multimodal processor cache. Hybrid attention models are labelled Estimated and receive a conservative cache compatibility factor because current vLLM hybrid cache grouping can allocate more memory than the pure full-attention formula suggests. The shared API returns both values as theoreticalKvCacheMi and hybridAllocatorSafetyMi; runtime components are returned in runtimeDetails, while recommendedReserveMi is the separate recommendation headroom. CPU reservations below the computed minimum require the explicit memory-risk acceptance described above. API/CLI clients that omit local.allowMemoryRisk retain the stricter preflight checks. This applies to both CPU Ollama and CPU vLLM, on creation and editing: the minimum is rounded up to the same 100 MiB step as the form. An explicit Ollama risk acceptance retains the requested budget without requiring public registry metadata; it does not inject vLLM cache parameters or bypass invalid RAM values. Legacy Ollama definitions without an explicit RAM budget keep their runtime defaults.

Accelerator availability uses the Ready, schedulable node's allocatable vendor resource as the final runtime signal. An enabled vendor operator that is temporarily reported as Reconciling by Flux therefore does not hide NVIDIA, AMD, or Intel once Kubernetes already publishes the matching GPU resource. If the resource is still absent, the operator phase remains the blocking reason.

Above the installed-model list, the Models screen renders only a small Compute memory heading and one semicircular gauge per compute device. Ordinary CPU/discrete-GPU gauges have two rings: violet for unreserved memory (total - active model reservations), cyan for current availability. The full arc represents that reading's capacity. On unified-memory CPUs both rings use Linux-visible RAM; the unreserved value excludes system headroom. The center shows current availability, or the unreserved budget if live metrics are missing. A small legend and an info symbol replace inline explanation paragraphs. CPU totals are aggregated across Ready, schedulable appliance nodes, reservations come from active CPU models and GPU models with explicit CPU offloading via ModelActivation.status.memoryRequiredMi (falling back to the requested value), and current availability comes from fresh host /proc/meminfo MemAvailable samples when available. Ordinary nodes retain the Kubelet summary/metrics API fallback. Unified-memory nodes never use that fallback: driver-owned GPU allocations can be absent from Kubelet working-set accounting, overstating both CPU and shared GPU availability.

NVIDIA gauges use one DCGM record per physical GPU. With the default device plugin, Kubernetes exposes a whole-GPU request but not the chosen GPU UUID on the ModelActivation; the dashboard packs planned vramRequiredMi reservations deterministically across detected devices. That packing is not a binding. With opt-in NVIDIA DRA, local.gpuDevice binds the model to a verified card and its node UID. Its reservation is charged only to that card, never a sibling or a replacement node. The actually-free inner ring always comes from DCGM. local.gpuDevices extends this to a same-node NVIDIA group: one slot and the per-card VRAM reservation are charged to each selected UUID, not every GPU on the node. Eligibility uses DRA node identity and known positive physical gpuCapacityMi, with gpuCapacitySource: nvidia-dra-inventory. GPU models and capacities may differ; equal per-card budgets are bounded by the smallest unreserved budget and every card's physical capacity. DCGM's totalMi can differ between samples or from the inventory's capacity; it is not the grouping criterion. Missing inventory capacity blocks a group, while missing live readings alone do not. Live readings and planned reservations retain their separate memory meanings. In Split one model, the dialog shows the combined VRAM budget and converts it to the per-card local.vram/local.vramMi contract when saving. Existing saved per-card budgets need no migration. System RAM/CPU are reserved once for the Pod. In Replicate model copies, the full model runs on each selected GPU and system RAM/CPU are reserved per copy. The UI shows the combined host-RAM budget and the per-copy ready state; Logs can select an individual copy. NVIDIA GPUs shows all physical cards as checkboxes. Checked cards determine the selected count and saved identities. Polling never silently substitutes a missing or full card. Multi-GPU settings and vLLM parallelism appear under Advanced. See GPU sharing for supported combinations, memory semantics and current acceptance limits. ResourceSlice capacity without DCGM yields a known total but unknown free memory. AMD and Intel device-plugin resources are also listed individually. Until their installed operator supplies a compatible memory exporter, unavailable readings use a dashed ring and —, not an invented zero, total, percentage, or free value. This preserves an honest UI while keeping the response contract ready for additional vendor metric adapters.

An explicitly configured unified-memory AMD profile shows one GPU, not separate fixed/shared GPUs. Fresh, PCI-matched KFD evidence determines the ordinary allocation domain and GPU planning capacity. GTT/TTM bounds alone do not prove model capacity. Fixed GPU allocations are outside Linux RAM; only their host runtime requests count there. Dynamic GPU allocations consume Linux RAM and intersect its remaining budgets after safety headroom. Missing metrics and unverified GPU cgroup accounting remain explicit.

GPU gauges add an outer segmented model-slot ring (gold: free, grey: occupied) and exact free/total counts, independently of memory. Fully occupied GPUs remain visible but disabled in the Hardware selector, with a no free slots hint. For multiple NVIDIA device-plugin cards, an aggregate Node slots ring is shown once above their memory gauges; the runtime chooses the physical card. NVIDIA DRA instead gives each card its own ring and a NVIDIA GPUs checkbox list in Create / Edit with name, PCI address, VRAM and free slots. Editing an active model may reuse its own card's slot. Stopped models have no such credit. A lost card or changed node UID remains visibly blocked rather than selecting another card. The model form refreshes every 15 seconds; if its selected GPU becomes full, submission is disabled without clearing the form or switching hardware. The API also checks current slots before writing a local model; accepting a memory-estimate risk does not bypass slot exhaustion. CPU models are unchanged. See slot accounting for counting rules.

On unified-memory hosts, Models → Compute Memory keeps one GPU gauge with four memory rings inside the slot ring: dedicated unreserved (violet), dedicated free (cyan), shared unreserved (blue), shared free (green). The compact legend groups each pair under its own capacity. These are separate scales, not an additive model budget or two deployment targets.

Dedicated free uses PCI-matched AMD mem_info_vram_total - mem_info_vram_used; dedicated unreserved is attributed only to a confirmed firmware-reserved model allocation domain. Without live metrics, the free ring stays unknown and the center falls back to the applicable unreserved budget. Shared free is max(0, min(Linux MemAvailable, dynamic GPU ceiling - mem_info_gtt_used)). Linux availability already reflects those GPU allocations; they are not subtracted from MemAvailable a second time. This is remaining reported capacity, not a guarantee that any individual allocation will fit. For example, 38.2 GiB Linux availability, a 108 GiB shared ceiling and 78.7 GiB reported GTT usage yield min(38.2, 108 - 78.7) = 29.3 GiB shared free. A separate 95 GiB reservation leaves a 13 GiB GPU planning budget; it does not change that measured free value. Calculations use bytes before conversion to MiB; compact gauge labels round GiB for display. Shared unreserved is limited by remaining Linux budgets after system headroom and model requests; when the confirmed model domain is shared/GTT, remaining GPU reservations also constrain it. Missing pool metrics, unknown domains and non-matching pool IDs are not replaced with another node's values. The dynamic ceiling is part of Linux-visible RAM, not an additional bank or protected reserve. A confirmed shared/GTT domain remains valid when its Linux-capped capacity is smaller than the fixed firmware reservation; that comparison alone must not erase the unreserved budget. The host corroborates the domain using raw KFD/GTT evidence before applying capacity bounds. The lightweight host sampler refreshes every 30 seconds. The API requires a sample no older than 90 seconds matching the Node UID, boot ID and kernel; missing, invalid or stale counters stay unknown, never zero usage. Current sample time and the formula are available through the info symbol.

Hovering or focusing the info symbol previews the explanation. Clicking/tapping pins it open; Escape, the close button or an outside click dismisses it. The popup includes installed RAM, fixed GPU reservation, Linux-visible RAM, dynamic ceiling, driver capacity, data sources and accounting caveats. The full inventory remains in System → Hardware → GPU nodes; it is not a separate card on Models. Unknown driver capacity remains unknown. Only separately confirmed hardware controls change memory settings. See the inventory contract. CLI hardware output and the physical TUI's Models/Hardware views expose the same values and the same non-additive relationship. The dynamic ceiling and Kubernetes requests are not protected reservations against other processes; the UI does not promise guaranteed dynamic AI capacity.

The preset selector is populated from ConfigMap/magicstick-model-presets and shows only variants compatible with the selected engine and target. Each variant may declare defaultArtifact and artifacts[]; the artifact entries carry their own checkpoint/tag, precision, quantization method, and planning budget. The selected ID is stored in spec.local.artifact and displayed on the installed-model card. Existing resources without that field continue to use the variant default. For catalog presets, the URL is derived from the artifact and remains read-only; selecting Custom is the explicit path for a user-supplied URL. An artifact compatibilityNote, for example an FP8 hardware requirement, is shown directly below the dropdown. Such a note is a warning rather than a replacement for accelerator-generation acceptance testing. qwen2505bcpu remains the portable smoke preset. The expanded Qwen3.5, Qwen3.6, and Qwen3.8 catalog uses official Hugging Face checkpoints plus target-specific FP8/GPTQ/AWQ artifacts where the runtime supports them, and explicit Q4_K_M/Q8_0/BF16 Ollama tags on CPU, NVIDIA, and AMD. Qwen3.5 4B fills the useful capacity gap between the 2B and 9B presets. qwen3827b retains its validated single-NVIDIA-GPU AWQ default and adds official FP8 and BF16 alternatives. Selecting a preset and artifact fills its target-specific context, output-token, concurrency, memory, and runtime values before the activation is submitted. The model catalog propagates consumer limits into managed KubeOpenCode templates.

The default dashboard is GPU-neutral. External models do not require or activate GPU or KubeAI modules. CPU model creation remains available without any GPU driver and lets the operator install KubeAI on demand. An unavailable target is disabled with a concrete reason. The API enforces the same live availability check and engine/target compatibility matrix and returns HTTP 409 if a client attempts to bypass the UI. It resolves the engine-specific resource profile server-side instead of accepting arbitrary args, environment values, or profile names from the browser.

The Models screen treats unavailable device metrics as a neutral, per-device state. Runtime removal remains outside the memory display and is exposed only when no local model still depends on the automatically enabled runtime.

The System → System Status screen renders all three GPU providers even on a CPU-only appliance. Each card shows the pinned operator version, driver mode, detected and compatible nodes, management owner, allocatable resource count, phase, and the controller's non-sensitive explanation. NotRequired means no matching hardware was found and the vendor operator consumes no cluster resources; Installing lasts until the vendor resource becomes allocatable.

The Services screen uses this hardware lifecycle rather than the Flux revision of the operator module. For NVIDIA it keeps the service in Installing after the extended resource appears until the DCGM exporter can provide GPU telemetry. This prevents a successfully applied chart from being presented as an operational GPU before the Models screen can observe the device.

Catalog-only models are read-only in the Models screen. Remove actions are shown only for ModelActivation rows that the dashboard can delete.

Operators and administrators can Stop and Start every managed model from its installed-model card. This uses the existing spec.enabled flag for Ollama, vLLM, Realtime, and external providers, preserving the activation and all saved settings. Local runtimes are removed on Stop; their resources become available as the Pods and allocations are released. Start recreates the runtime with its saved settings through the normal readiness and scheduling checks. For external providers, Stop only withdraws the Magic Stick route; it does not shut down the remote service. Realtime also provides a Restart action. The button follows the desired enabled state, prevents duplicate requests, and waits for runtime removal before allowing Start. Writes carry the current configuration revision; errors remain visible without removing the model card. For ordinary Ollama/vLLM runtimes, Removing continues until the KubeAI Model and remaining runtime Pods are gone. A terminal Pod or a withdrawn catalog entry alone is not proof of completed cleanup; runtime shutdown errors must be investigated rather than hidden by removing finalizers or extending every timeout. See stopping and resuming models.

Operators can open Edit on every managed installed model. Local edits keep the model reference, inference engine, hardware target, name, and namespace fixed while exposing the applicable engine settings: context/output limits, sequence concurrency, KV-cache/memory/offloading controls for vLLM/Ollama, and Realtime stage settings. The memory estimate is recalculated against capacity with the model's own existing reservation removed, so an unchanged deployment is not counted twice. External edits cover provider parameters and can replace an API key; leaving the key blank preserves the existing Secret. Save changes remains disabled until a parameter differs. Saving reconciles the activation and can restart its runtime. The object identity and configuration generation captured when the dialog opens prevent an older form from overwriting a concurrent change or a recreated model. Status-only updates do not invalidate the form. Writes still use an atomic Kubernetes resource-version precondition; one status-only race can be retried without accepting a concurrent spec edit.

Administrators can open Logs on each local installed-model card. The dialog refreshes a bounded tail from every declared runtime and init container and also shows the previous container run after a restart. It remains useful while a model is starting; if the runtime controller has not created a Pod, or Kubernetes has no output yet, the dialog says so without turning that expected state into a dashboard failure. External providers have no local Pod and therefore no Logs button. Because model input and provider output can be sensitive, this API and control are restricted to magicstick-admin. The backend accepts only a ModelActivation name, validates the target namespace, uses KubeAI model labels plus its controller owner reference, strips terminal control sequences, and never accepts a browser-supplied namespace, Pod, or container name.

A local model first shows WaitingForPod while no model Pod exists. After two minutes without a Pod it becomes Degraded with reason ModelPodCreationStalled, and a hint to inspect KubeAI/controller admission errors. Retries continue; the status recovers automatically when a Pod appears. This timeout does not apply to downloads or engine startup in an existing Pod. An owned Pod that has permanently failed is automatically replaced with bounded backoff. Its original failure appears in the model message; after five failed recovery attempts the model stays Degraded until its configuration is reviewed. A local model with a Pod is shown as Starting until KubeAI reports a ready vLLM or Ollama replica. If a container in an exactly owned model Pod restarts unsuccessfully three or more times without readiness, the model instead becomes Degraded with reason ModelRuntimeCrashLoop and a hint to inspect runtime logs, startup-probe events, the model reference and registry access. This also covers an Ollama download failure that leaves the Pod in Running / CrashLoopBackOff. Explicit OOM, image-pull backoff and container-configuration failures are reported separately. Kubernetes retries continue; the status check does not delete Pods. Readiness clears the failure, and a replacement Pod starts with its own restart count. A healthy, slow download is not subject to a blanket startup timeout. The status message includes the ready-replica count, for example 0/1 replicas ready. Ready therefore means both that the local runtime is serving its health endpoint and that the generated catalog has published the model.