Shared GPU queues need an allocation-policy audit

Treat scheduling as an allocation policy

Inventory the decision before accepting a shared queue. Preserve the accelerator pools and resource flavors, ClusterQueues and LocalQueues, namespaces and teams, workload classes, priority classes, nominal quotas, borrowing and lending rules, preemption behavior, topology requirements, elastic workloads, historical-usage window, entry penalty, resource weights, LocalQueue weights, and every default inherited from the deployed Kueue version. Map each technical class to a declared business purpose, accountable owner, funding model, data and security boundary, deadline, consequence of delay, and alternative capacity path. A scheduler can apply a formula consistently without that formula representing the enterprise's intended priorities.

Separate equal opportunity to request capacity from equal allocation, equal wait, and value-sensitive service. A long training run, customer-facing inference recovery, safety evaluation, regulatory deadline, speculative experiment, and executive demonstration may all be labeled high priority while carrying different consequences. Record who may create or change those labels, which evidence supports an exception, how long an exception lasts, and what prevents teams from dividing work or changing requests to improve their position. If the policy cannot explain why one queued workload should precede another, do not let the implementation's default order become the organization's decision by accident.

Reproduce the queue order and challenge its edge cases

Build an acceptance set that exercises ordinary demand, a new tenant with no usage history, a team returning after idle time, a sustained heavy user, a small urgent workload behind a large reservation, simultaneous high-priority requests, capacity that satisfies only some topology constraints, borrowed quota, elastic scaling, preemption, and a failed or canceled job. For each case, preserve the submitted workload, queue and cohort state, quota reservation, admission and eviction events, priority, accumulated usage, decay and entry-penalty inputs, resource and queue weights, topology decision, start and completion time, and final disposition. Have an independent platform owner reproduce the expected order from the approved policy rather than merely observe that Kueue admitted something.

Test for starvation, priority inflation, workload splitting, repeated cancellation and resubmission, inaccurate resource requests, queue shopping, and topology choices that reserve unusable fragments. Challenge the chosen half-life and sampling interval with both bursts and sustained demand. Verify what occurs when configuration changes while work is queued, metrics or status are delayed, a resource flavor disappears, or a team crosses a quota boundary. A passing audit shows that ordering remains explainable and bounded under these conditions, that an unexpected result is detectable, and that a named owner can pause admissions or restore the prior configuration without losing the decision record.

Measure service and value without borrowing the case study

Establish a buyer-owned baseline using the same workload classes, accelerator types, regions, observation window, queue policy, and demand profile. Measure wait-time distributions by declared class and team, time from reservation to useful work, completion and failure, retries and preemptions, quota borrowing, accelerator utilization, stranded fragments, cost per completed workload, operator intervention, and the age and consequence of work that never starts. Keep provider-reported improvements from the AI21 account labeled as customer claims hosted by the provider. They may justify a test; they do not establish the same result for a different mix of models, clusters, budgets, or operating practices.

Tie any business-value claim to the workload's verified downstream outcome rather than scheduler activity alone. Faster admission can be irrelevant if a job fails, produces an unusable model, duplicates lower-value work, or delays a more consequential task. Conversely, lower aggregate utilization may be acceptable when it protects a critical recovery or isolation boundary. Preserve unavailable evidence as unavailable instead of scoring it as no delay or no cost. Report denominators and tails, not only averages, so a favorable fleet-level number cannot hide one team or protected workload that repeatedly absorbs the waiting time.

Give changes, exceptions, and appeals explicit authority

Version the allocation policy and its implementation together. Require a change record for quota, cohort membership, priority, decay, entry penalty, resource weight, queue weight, borrowing, preemption, topology, and elastic-workload behavior. Name who may propose, test, approve, deploy, observe, roll back, and independently review each change. Define an emergency override with scope, expiration, evidence, communication, and retrospective review. Give teams an appeal path for a disputed classification or unexplained delay, while preventing an appeal from silently rewriting the active queue. The authoritative record should connect the request, policy version, computed inputs, scheduler events, human exception, and final disposition.

Start with one bounded pool and representative workload set, then expand only after the evidence meets declared thresholds and affected owners accept the residual risk. Platform, finance, security, privacy, resilience, procurement, data, model, application, and business owners should retain their existing decisions; Admission Fair Sharing does not replace them. Reopen approval after a Kueue or Kubernetes version change, new accelerator or topology, material demand shift, new team or workload class, altered funding model, incident, unexplained starvation, or persistent exception. The CIO decision is not whether a queue can keep GPUs busy. It is whether the enterprise can defend, observe, correct, and authorize how scarce accelerator time is allocated.

Turn this source into a reviewable decision

For AI for CIOs, use this briefing as a dated decision record rather than a substitute for the source. Preserve Google Cloud, AI21 trains its models faster with AI Hypercomputer, the exact URL, the October 4, 2026 review date, the supported facts above, the editorial interpretation, the limitations, and any buyer-specific evidence. Link that record to the decisions most directly affected: Enterprise AI platform architecture; Operations and incident intelligence; AI portfolio economics; Data products and AI-ready information. State whether the source changes the scope, evidence requirement, control, sequence, or only the language used to describe the decision.

Before action, name the accountable owner, affected population and workflow, exact offering or configuration, source data and rights, human decision point, exception and appeal path, complete cost, expected benefit, failure and stop conditions, retained evidence, and next review date. Keep official facts, provider statements, buyer observations, representative tests, measured outcomes, editorial inferences, and unknowns visibly separate. Reopen the record when the source, offer, model, integration, data, policy, population, responsible person, or measured result changes.

Limitations and unknowns

Google Cloud is the provider and host for the October 2, 2026 AI21 customer account, and the reported operational changes and outcomes are not independently verified here. The official RSS feed establishes a 19:00:00Z publication time before the authoritative cutoff. Kueue's current documentation supports the described Admission Fair Sharing status, ordering logic, usage decay, entry penalty, configurable resource and LocalQueue weights, and exposed status. The sources do not establish a buyer's deployed version, queue and cohort configuration, workload identity, priority integrity, entitlement, topology, security and data boundary, starvation behavior, utilization, cost, model or application outcome, or organizational fairness. Verify current product and project documentation, exact manifests and defaults, representative queue-order tests, scheduler events and metrics, cost and workload records, exception and appeal evidence, and qualified platform, application, model, data, security, privacy, resilience, finance, procurement, regulatory, legal, and business-owner review before consequential adoption.