diff --git a/AGENTS.md b/AGENTS.md deleted file mode 100644 index dfb44e5..0000000 --- a/AGENTS.md +++ /dev/null @@ -1,262 +0,0 @@ -# AGENTS.md - -## Structure - -- **`tui.tsx`** — only source file. Exports a `TuiPluginModule` as default. No build step; OpenCode compiles JSX at runtime. -- **`package.json`** — declares peer deps (`@opencode-ai/plugin`, `@opentui/core`, `@opentui/solid`, `solid-js`) and an `engines.opencode` constraint (`>=1.3.14`). The `exports` field maps `./tui` to `tui.tsx`. - -## Constraints - -- No tsconfig, no test runner, no lint/format. -- `node_modules` is gitignored. -- JSX uses `@opentui/solid` (`/** @jsxImportSource @opentui/solid */`). -- The plugin registers via `api.slots.register` to render into `session_prompt_right`. - -## TUI Plugin Loading - -OpenCode has TWO separate plugin loaders: -- **Server plugins** — configured in `opencode.jsonc` -- **TUI plugins** — configured in `~/.config/opencode/tui.json` under the `plugin` field - -Server plugins in `opencode.jsonc` do NOT load TUI plugins. TUI plugins must be listed in `tui.json`: -```json -{"plugin": ["oc-ls-stats@latest"]} -``` - -OpenCode's `file://` path loader does not resolve `package.json` exports field for TUI plugins. TUI plugins require npm-style module resolution (package published to npm registry or private npm-compatible registry like Forgejo). - -### Important: Plugin Installation - -- **npm plugins are installed using Bun at startup**, not npm -- Cached in `~/.cache/opencode/node_modules/` -- `~/.npmrc` is respected for private registry auth (Bun's native `.npmrc` support) -- For private registries, ensure `~/.npmrc` has the correct format: - ``` - registry=https://your-registry/api/npm/ - //your-registry/api/npm/:_authToken=YOUR_TOKEN - ``` - -### tui.json Format Limitations - -- Only plain strings are supported: `["plugin-name@latest"]` -- **Tuple format with options is NOT supported**: `["plugin", { "url": "..." }]` will be rejected -- Log shows: `skipping invalid tui config — Expected string | array, got {"url":"..."}` -- To pass config to TUI plugins, hardcode in the plugin code or use environment variables - -### Debugging Plugin Loading Issues - -- **Check logs**: `~/.local/share/opencode/log/` (most recent: `opencode.log`) -- **Plugin loading errors**: grep for `skipping invalid tui config` or `plugin` -- **Run with debug**: `opencode --log-level DEBUG` or `opencode --print-logs` -- **Clear cache**: `rm -rf ~/.cache/opencode` (removes all plugin caches) -- **Verify installed version**: check `~/.cache/opencode/packages/@latest/package.json` for pinned version - -## Prefill Tracking (llama-server) - -The plugin polls llama-server's `GET /slots` endpoint to track prompt processing (prefill) timing. - -### URL Discovery -- Falls back to reading OpenCode config from `api.state.path.config` -- Parses JSONC (strips `//`, `/* */` comments and trailing commas) -- Extracts `baseURL`/`base_url` from `provider.*.options` where value contains "localhost" -- Strips `/v1` path suffix to get the llama-server base URL -- Falls back to `http://localhost:8080` default URL - -### Slot Polling -- Polls `GET /slots?model=...` every 500ms (`SLOT_POLL_MS = 500`) -- Prefill detection: `is_processing === true` AND `next_token[0].n_decoded === 0` -- Model parameter is required by the `/slots` endpoint -- Falls back to `http://localhost:8080` default URL - -### Display Format -`PP xxxx tps | GEN xxxx tps` -Example: `PP 1247 tps | GEN 25 tps` - -### Display Rules -- PP: integer only, no suffix, 4-char fixed-width padded -- TPS: integer only, no suffix, 4-char fixed-width padded -- "----" shown when no value available -- PP only shown during active prefill (`n_decoded === 0`) -- GEN only shown during active generation (`n_decoded > 0`) - -### Limitations -- Slot state tracked globally (not per-session) — slot IDs don't map to OpenCode sessions -- Shows latest prefill data across all slots -- Requires llama-server to have `/slots` endpoint enabled (default) - -## Deployment - -- Forgejo npm registry: `https://git.sync.wtf/api/packages/troed/npm/` -- Package name: `oc-ls-stats` (llama-server stats, distinct from `oc-tps` on npmjs.org) -- Auth: token in `~/.npmrc` -- Git remotes: `origin` → Forgejo, `upstream` → GitHub -- To update: bump version in `package.json`, run `npm publish --registry https://git.sync.wtf/api/packages/troed/npm/`, clear cache (`rm -rf ~/.cache/opencode/packages/@troed`), restart OpenCode - -## Debugging: Plugin Not Loading / No Output - -### Symptom -Plugin installs from private registry but produces no output in OpenCode UI. - -### Root Causes Identified - -#### 1. Incorrect TUI Function Signature (CRITICAL) -**Error:** Type mismatch prevented plugin from loading. -```typescript -// ❌ WRONG - causes load failure -const tui: TuiPlugin = async (api, options) => { ... } - -// ✅ CORRECT -const tui: TuiPlugin = async (api) => { ... } -``` -The `TuiPlugin` type from `@opencode-ai/plugin/tui` expects exactly one parameter: `api`. - -#### 2. Malformed Regex in JSONC Parser -**Error:** Syntax error in character class regex. -```typescript -// ❌ WRONG - invalid regex syntax -text.indexOf(/[\s}])/), i + 1) -/[\}])]/.test(text[next]) - -// ✅ CORRECT -text.indexOf(/[\s}\]]/, i + 1) -/[\s}\]]/.test(text[next]) -``` -Regex literals with unescaped parentheses caused JavaScript parse errors. - -#### 3. Hardcoded Test Text Replaced Component -**Error:** Development debug code left in production. -```tsx -// ❌ WRONG - no actual output -return *** SCOPED v0.0.17 LOADED *** - -// ✅ CORRECT -return -``` - -#### 4. Extra Export in package.json -**Error:** Non-standard exports configuration. -```json -// ❌ WRONG - extra root export -{ - "exports": { - ".": { "import": "./tui.tsx" }, - "./tui": { "import": "./tui.tsx" } - } -} - -// ✅ CORRECT - match working plugin config -{ - "exports": { - "./tui": { "import": "./tui.tsx" } - } -} -``` - -### Verification Steps - -1. **Check plugin loads:** Look for `[oc-ls-stats] TUI plugin loaded!` in OpenCode logs -2. **Clear cache between tests:** `rm -rf ~/.cache/opencode` -3. **Verify installed version:** `cat ~/.cache/opencode/packages/@troed/oc-ls-stats@latest/node_modules/@troed/oc-ls-stats/package.json` -4. **Compare with working plugin:** Diff against cached `@troed/oc-tps@latest` to spot discrepancies - -### Resolution Timeline -- Confirmed OpenCode CAN load scoped packages from Forgejo (tested with `@troed/oc-tps`) -- Compared working `@troed/oc-tps` vs non-working `@troed/oc-ls-stats` line-by-line -- Fixed all 4 issues above in version 0.0.20 -- Published 0.0.20 to Forgejo registry - -## Prefill Rate (PP) Calculation — v0.0.40+ - -### Slot State Structure (from `GET /slots?model=...`) - -```json -{ - "id": 2, - "n_ctx": 160000, - "speculative": false, - "is_processing": true, - "n_prompt_tokens": 12345, - "n_prompt_tokens_processed": 12345, - "next_token": { - "has_next_token": true, - "has_new_line": false, - "n_remain": 31943, - "n_decoded": 0 - } -} -``` - -### Slot State Phases - -| Phase | `is_processing` | `n_remain` | `n_decoded` | `n_prompt_tokens` | -|-------|-----------------|------------|-------------|-------------------| -| Idle | `false` | — | — | stable | -| Prefill | `true` | `>0` (e.g. 31943) | `0` | incrementing | -| Transition | `true` | `>0` (e.g. 31999) | `0` | stable (final) | -| Generation | `true` | `>0` (decreasing) | `>0` (incrementing) | stable (final) | - -**Key insight:** `n_decoded === 0` is the definitive prefill indicator (when `is_processing === true`). `n_remain === -1` only occurs when `n_predict == -1` (unlimited generation), which is rare in practice. - -### PP Calculation Method - -PP is calculated from the per-slot `n_prompt_tokens` field during prefill: - -1. Detect prefill: slot with `is_processing === true` AND `next_token[0].n_decoded === 0` -2. On first detection: capture `slot.id` and `slot.n_prompt_tokens` as baseline -3. On subsequent polls: `delta = current_n_prompt_tokens - baseline_tokens`, `dt = (now - startAt) / 1000`, `PP = delta / dt` -4. Reset when no slot has `n_decoded === 0` (prefill ended) - -### Why NOT `llamacpp:prompt_tokens_total` from `/metrics` - -`llamacpp:prompt_tokens_total` is a **global counter across all slots**, not per-model. When multiple slots are active simultaneously (e.g., slot A prefilling while slot B generating), the delta includes tokens from all slots, producing wildly inflated PP values. - -**Example from debug data (`test-debug-log.jsonl`):** -- Line 15: slot 3 prefilling, `prompt_tokens_total` delta = 49 tokens in 1815ms → PP = 27 (correct but tiny delta) -- Line 31: slot 2 prefilling, slot 3 still generating, `prompt_tokens_total` delta = 13793 tokens in 668ms → PP = 20,648 (wrong — includes slot 3 tokens) -- Actual server console showed ~1200-1300 tokens/sec during prefill - -### Why NOT `llamacpp:prompt_tokens_seconds` from `/metrics` - -`llamacpp:prompt_tokens_seconds` is an **average since server start**, not an instantaneous rate. It smooths out all prefill events into a single running average, making it useless for displaying current prefill speed. - -### Slot Polling - -- Polls `GET /slots?model=...` every 500ms (`SLOT_POLL_MS = 500`) -- Model parameter is required by the `/slots` endpoint (returns error without it) -- Model discovered from: route session → OpenCode config provider parsing -- Falls back to `http://localhost:8080` default URL - -### Display - -PP only shown during active prefill (`n_decoded === 0`). Format: integer only, no suffix (e.g., "1247"). Fixed-width 4 chars, space-padded. Shows "----" when no value. - -### Why Progress Percentage Is Not Shown - -The `/slots` endpoint does not expose the **final prompt size** (`task->n_tokens()` from llama.cpp). During prefill, `n_prompt_tokens` and `n_prompt_tokens_processed` are always equal because every token is processed as it's added to the prompt. Progress percentage requires the ratio `n_prompt_tokens_processed / task->n_tokens()`, but `task->n_tokens()` is not returned in `/slots` JSON. - -**Required llama.cpp changes:** - -- **PR #14685** (merged as part of #14728): adds `prompt_progress` streaming to `/completion` endpoint with `total`, `cache`, `processed`, `time_ms`. This enables progress for streaming clients but not for `/slots` polling. -- **PR #23454** (merged May 2026): added `n_prompt_tokens`, `n_prompt_tokens_processed`, `n_prompt_tokens_cache` to `/slots` — but NOT the total prompt size. -- **Needed**: expose `n_prompt_tokens_total` (or `n_tokens` from `task->n_tokens()`) in `/slots` output so polling clients can compute `(n_prompt_tokens_processed / n_prompt_tokens_total) * 100`. - -### Why `n_remain === -1` Does NOT Indicate Prefill - -`n_remain === -1` only occurs when `n_predict == -1` (unlimited generation). When `n_predict` is set (e.g., 32000), `n_remain = n_predict - n_decoded`, which is always positive during prefill. Prefill detection must use `n_decoded === 0` instead. - -## Debugging: PP Value Wrong - -### Symptom -PP shows 3634 when llama-server console shows ~1200-1300 tokens/sec during prefill. - -### Root Cause (fixed in v0.0.40) -Used `llamacpp:prompt_tokens_total` from `/metrics` endpoint which is a global counter across all slots. Delta between polls included tokens from other active slots. - -### Fix -Track `n_prompt_tokens` from the slot data itself (per-slot field) during prefill state (`n_decoded === 0`). - -### Debug Log Location -`/tmp/oc-ls-stats-debug.log` — TUI plugin writes here because `console.log` in TUI doesn't reach main OpenCode log. - -### Debug Data -Sample debug log with slot states and metrics values: `test-debug-log.jsonl` diff --git a/test-debug-log.jsonl b/test-debug-log.jsonl deleted file mode 100644 index 9bf634e..0000000 --- a/test-debug-log.jsonl +++ /dev/null @@ -1,46 +0,0 @@ -{"metrics":{"predictedTps":24.9122,"promptTokensTotal":97244,"promptTokensSeconds":1111.56},"lastPromptTokensTotal":97244,"lastPromptTokensTime":1781383392976,"now":1781383395633} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":true,"next_token":{"has_next_token":true,"has_new_line":false,"n_remain":31999,"n_decoded":1}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9122,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97244,"lastPromptTokensTime":1781383395633,"now":1781383396983} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":true,"next_token":{"has_next_token":true,"has_new_line":false,"n_remain":31969,"n_decoded":31}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9122,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383396983,"now":1781383399028} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":true,"next_token":{"has_next_token":true,"has_new_line":true,"n_remain":31921,"n_decoded":79}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383399028,"now":1781383401001} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383401001,"now":1781383403001} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383403001,"now":1781383405013} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383405013,"now":1781383407016} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383407016,"now":1781383409019} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383409019,"now":1781383411028} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383411029,"now":1781383413038} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383413038,"now":1781383415040} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383415041,"now":1781383417039} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383417039,"now":1781383419039} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383419039,"now":1781383421051} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383421051,"now":1781383423053} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383423053,"now":1781383425058} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383425058,"now":1781383427073} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383427073,"now":1781383429085} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383429085,"now":1781383431099} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383431099,"now":1781383433110} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383433110,"now":1781383435194} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":true,"next_token":{"has_next_token":true,"has_new_line":false,"n_remain":31999,"n_decoded":1}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97632,"promptTokensSeconds":1094.26},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383435194,"now":1781383437159} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":true,"next_token":{"has_next_token":true,"has_new_line":true,"n_remain":31952,"n_decoded":48}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]} -{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97632,"promptTokensSeconds":1094.26},"lastPromptTokensTotal":97632,"lastPromptTokensTime":1781383437159,"now":1781383439126} -{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":true,"next_token":{"has_next_token":true,"has_new_line":true,"n_remain":31903,"n_decoded":97}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}