docs: spec for session-scoped server selection

2026-08-31 09:43:38 +03:00 · 2026-08-21 13:31:16 +02:00
parent c472cc48af
commit ad22264869
1 changed files with 121 additions and 0 deletions
@@ -0,0 +1,121 @@
+# Session-Scoped Server Selection — Design
+
+Date: 2026-08-21
+Status: Approved direction, pending spec review
+
+## Problem
+
+The plugin polls every llama-server URL it discovers (`resolveServerUrls` may
+return several when multiple providers match). Two independent servers serving
+different clients would both be polled every cycle, and the displayed rate
+mixes whichever reported last. The plugin should show stats for the server the
+*current session* actually uses.
+
+Separately confirmed (no action needed): `GET /slots?model=X` is read-only;
+the `model` parameter filters slot reporting and never loads or switches models.
+
+## Goal
+
+Each poll cycle, poll only the server backing the current route session's
+model provider — when that can be determined. Otherwise keep today's behavior.
+
+## Non-Goals
+
+- No UI changes.
+- No change to the explicit `server` option semantics ("replaces discovery
+  entirely" stays true; an explicit list has one URL, so selection is a no-op
+  there).
+- No new config options.
+
+## Design
+
+### New pure function in `src/stats.ts`
+
+```ts
+export function selectSessionUrls(
+  candidates: string[],
+  providers: unknown,
+  providerID: string | undefined,
+): string[]
+```
+
+Behavior:
+
+1. If `providerID` is undefined/empty or `candidates` is empty → return
+   `candidates` unchanged.
+2. Find the first entry of `providers` (array of objects) whose `id` equals
+   `providerID` (exact, case-sensitive — provider ids are stable identifiers).
+3. Take its `options.baseURL || options.base_url` (strings only), normalize
+   through `stripBaseUrlPath`.
+4. If the normalized URL appears in `candidates` (string equality) → return
+   `[thatUrl]`.
+5. Any miss (provider not found, no URL, URL not among candidates — e.g. an
+   ollama session) → return `candidates` unchanged.
+
+Rationale for step 5: falling back to the full candidate list preserves
+today's behavior whenever the session-to-server mapping is unknown. Scoping
+only ever *narrows*, never invents URLs.
+
+### Wiring in `tui.tsx`
+
+`pollMetrics` already resolves the route session every cycle for the model id
+(tui.tsx:188–193). Reuse that session object: capture `session?.model?.providerID`
+alongside `session?.model?.id`, then replace
+
+```ts
+for (const baseUrl of llamaServerUrls) {
+```
+
+with
+
+```ts
+const activeUrls = selectSessionUrls(llamaServerUrls, api.state.provider, sessionProviderID)
+if (!activeUrls.length) return
+for (const baseUrl of activeUrls) {
+```
+
+`llamaServerUrls` remains the startup-resolved candidate set; selection happens
+per cycle so session switches are honored immediately. The existing
+`!llamaServerUrls.length` early-return stays.
+
+### Minor: dedup candidates
+
+`resolveServerUrls` now returns deduplicated results
+(`[...new Set(...)]` applied to detected lists before returning). Closes the
+known double-polling gap flagged in the 1.3.0 review. The explicit-server path
+returns a single URL and needs no dedup.
+
+## Error Handling
+
+No new failure modes: selection operates on already-normalized strings; every
+guard path falls back to the unchanged candidate list. No warnings added
+(silently polling everything when scope is unknown matches current behavior).
+
+## Testing
+
+All tests target `src/stats.ts` pure functions via `test-backoff.ts`
+(`npx -y tsx test-backoff.ts`):
+
+- `selectSessionUrls` narrows to the matching provider's URL (id match,
+  baseURL variant)
+- same via `base_url` variant
+- URL normalization: candidate stored without `/v1`, provider baseURL with
+  `/v1` → still selects
+- provider not found → unchanged
+- provider found but URL not in candidates (ollama case) → unchanged
+- missing/empty providerID → unchanged
+- empty candidates → unchanged
+- non-array providers / malformed entries → unchanged
+- first-match wins if duplicate ids exist
+- dedup: two providers with identical baseURL yield one URL from
+  `resolveServerUrls`
+
+## Changes Summary
+
+- Modify: `src/stats.ts` — add `selectSessionUrls`; dedup detected lists in
+  `resolveServerUrls`
+- Modify: `tui.tsx` — capture `providerID`, call `selectSessionUrls` per cycle
+- Modify: `test-backoff.ts` — new tests above
+- Modify: `README.md` — one sentence in Server Discovery noting stats follow
+  the current session's provider when multiple llama-servers are detected
+- Version bump deferred to release time (as with 1.3.0)