mirror of
https://git.sync.wtf/troed/oc-ls-stats.git
synced 2026-08-31 09:43:38 +03:00
untrack AGENTS.md and test-debug-log.jsonl
This commit is contained in:
Troed Sångberg
@@ -1,262 +0,0 @@
|
||||
# AGENTS.md
|
||||
|
||||
## Structure
|
||||
|
||||
- **`tui.tsx`** — only source file. Exports a `TuiPluginModule` as default. No build step; OpenCode compiles JSX at runtime.
|
||||
- **`package.json`** — declares peer deps (`@opencode-ai/plugin`, `@opentui/core`, `@opentui/solid`, `solid-js`) and an `engines.opencode` constraint (`>=1.3.14`). The `exports` field maps `./tui` to `tui.tsx`.
|
||||
|
||||
## Constraints
|
||||
|
||||
- No tsconfig, no test runner, no lint/format.
|
||||
- `node_modules` is gitignored.
|
||||
- JSX uses `@opentui/solid` (`/** @jsxImportSource @opentui/solid */`).
|
||||
- The plugin registers via `api.slots.register` to render into `session_prompt_right`.
|
||||
|
||||
## TUI Plugin Loading
|
||||
|
||||
OpenCode has TWO separate plugin loaders:
|
||||
- **Server plugins** — configured in `opencode.jsonc`
|
||||
- **TUI plugins** — configured in `~/.config/opencode/tui.json` under the `plugin` field
|
||||
|
||||
Server plugins in `opencode.jsonc` do NOT load TUI plugins. TUI plugins must be listed in `tui.json`:
|
||||
```json
|
||||
{"plugin": ["oc-ls-stats@latest"]}
|
||||
```
|
||||
|
||||
OpenCode's `file://` path loader does not resolve `package.json` exports field for TUI plugins. TUI plugins require npm-style module resolution (package published to npm registry or private npm-compatible registry like Forgejo).
|
||||
|
||||
### Important: Plugin Installation
|
||||
|
||||
- **npm plugins are installed using Bun at startup**, not npm
|
||||
- Cached in `~/.cache/opencode/node_modules/`
|
||||
- `~/.npmrc` is respected for private registry auth (Bun's native `.npmrc` support)
|
||||
- For private registries, ensure `~/.npmrc` has the correct format:
|
||||
```
|
||||
registry=https://your-registry/api/npm/
|
||||
//your-registry/api/npm/:_authToken=YOUR_TOKEN
|
||||
```
|
||||
|
||||
### tui.json Format Limitations
|
||||
|
||||
- Only plain strings are supported: `["plugin-name@latest"]`
|
||||
- **Tuple format with options is NOT supported**: `["plugin", { "url": "..." }]` will be rejected
|
||||
- Log shows: `skipping invalid tui config — Expected string | array, got {"url":"..."}`
|
||||
- To pass config to TUI plugins, hardcode in the plugin code or use environment variables
|
||||
|
||||
### Debugging Plugin Loading Issues
|
||||
|
||||
- **Check logs**: `~/.local/share/opencode/log/` (most recent: `opencode.log`)
|
||||
- **Plugin loading errors**: grep for `skipping invalid tui config` or `plugin`
|
||||
- **Run with debug**: `opencode --log-level DEBUG` or `opencode --print-logs`
|
||||
- **Clear cache**: `rm -rf ~/.cache/opencode` (removes all plugin caches)
|
||||
- **Verify installed version**: check `~/.cache/opencode/packages/<plugin>@latest/package.json` for pinned version
|
||||
|
||||
## Prefill Tracking (llama-server)
|
||||
|
||||
The plugin polls llama-server's `GET /slots` endpoint to track prompt processing (prefill) timing.
|
||||
|
||||
### URL Discovery
|
||||
- Falls back to reading OpenCode config from `api.state.path.config`
|
||||
- Parses JSONC (strips `//`, `/* */` comments and trailing commas)
|
||||
- Extracts `baseURL`/`base_url` from `provider.*.options` where value contains "localhost"
|
||||
- Strips `/v1` path suffix to get the llama-server base URL
|
||||
- Falls back to `http://localhost:8080` default URL
|
||||
|
||||
### Slot Polling
|
||||
- Polls `GET /slots?model=...` every 500ms (`SLOT_POLL_MS = 500`)
|
||||
- Prefill detection: `is_processing === true` AND `next_token[0].n_decoded === 0`
|
||||
- Model parameter is required by the `/slots` endpoint
|
||||
- Falls back to `http://localhost:8080` default URL
|
||||
|
||||
### Display Format
|
||||
`PP xxxx tps | GEN xxxx tps`
|
||||
Example: `PP 1247 tps | GEN 25 tps`
|
||||
|
||||
### Display Rules
|
||||
- PP: integer only, no suffix, 4-char fixed-width padded
|
||||
- TPS: integer only, no suffix, 4-char fixed-width padded
|
||||
- "----" shown when no value available
|
||||
- PP only shown during active prefill (`n_decoded === 0`)
|
||||
- GEN only shown during active generation (`n_decoded > 0`)
|
||||
|
||||
### Limitations
|
||||
- Slot state tracked globally (not per-session) — slot IDs don't map to OpenCode sessions
|
||||
- Shows latest prefill data across all slots
|
||||
- Requires llama-server to have `/slots` endpoint enabled (default)
|
||||
|
||||
## Deployment
|
||||
|
||||
- Forgejo npm registry: `https://git.sync.wtf/api/packages/troed/npm/`
|
||||
- Package name: `oc-ls-stats` (llama-server stats, distinct from `oc-tps` on npmjs.org)
|
||||
- Auth: token in `~/.npmrc`
|
||||
- Git remotes: `origin` → Forgejo, `upstream` → GitHub
|
||||
- To update: bump version in `package.json`, run `npm publish --registry https://git.sync.wtf/api/packages/troed/npm/`, clear cache (`rm -rf ~/.cache/opencode/packages/@troed`), restart OpenCode
|
||||
|
||||
## Debugging: Plugin Not Loading / No Output
|
||||
|
||||
### Symptom
|
||||
Plugin installs from private registry but produces no output in OpenCode UI.
|
||||
|
||||
### Root Causes Identified
|
||||
|
||||
#### 1. Incorrect TUI Function Signature (CRITICAL)
|
||||
**Error:** Type mismatch prevented plugin from loading.
|
||||
```typescript
|
||||
// ❌ WRONG - causes load failure
|
||||
const tui: TuiPlugin = async (api, options) => { ... }
|
||||
|
||||
// ✅ CORRECT
|
||||
const tui: TuiPlugin = async (api) => { ... }
|
||||
```
|
||||
The `TuiPlugin` type from `@opencode-ai/plugin/tui` expects exactly one parameter: `api`.
|
||||
|
||||
#### 2. Malformed Regex in JSONC Parser
|
||||
**Error:** Syntax error in character class regex.
|
||||
```typescript
|
||||
// ❌ WRONG - invalid regex syntax
|
||||
text.indexOf(/[\s}])/), i + 1)
|
||||
/[\}])]/.test(text[next])
|
||||
|
||||
// ✅ CORRECT
|
||||
text.indexOf(/[\s}\]]/, i + 1)
|
||||
/[\s}\]]/.test(text[next])
|
||||
```
|
||||
Regex literals with unescaped parentheses caused JavaScript parse errors.
|
||||
|
||||
#### 3. Hardcoded Test Text Replaced Component
|
||||
**Error:** Development debug code left in production.
|
||||
```tsx
|
||||
// ❌ WRONG - no actual output
|
||||
return <text fg="#ff0000" bg="#000000">*** SCOPED v0.0.17 LOADED ***</text>
|
||||
|
||||
// ✅ CORRECT
|
||||
return <SessionPromptRight api={api} sessionID={value.session_id} tracker={tracker} version={version} clock={clock} />
|
||||
```
|
||||
|
||||
#### 4. Extra Export in package.json
|
||||
**Error:** Non-standard exports configuration.
|
||||
```json
|
||||
// ❌ WRONG - extra root export
|
||||
{
|
||||
"exports": {
|
||||
".": { "import": "./tui.tsx" },
|
||||
"./tui": { "import": "./tui.tsx" }
|
||||
}
|
||||
}
|
||||
|
||||
// ✅ CORRECT - match working plugin config
|
||||
{
|
||||
"exports": {
|
||||
"./tui": { "import": "./tui.tsx" }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Verification Steps
|
||||
|
||||
1. **Check plugin loads:** Look for `[oc-ls-stats] TUI plugin loaded!` in OpenCode logs
|
||||
2. **Clear cache between tests:** `rm -rf ~/.cache/opencode`
|
||||
3. **Verify installed version:** `cat ~/.cache/opencode/packages/@troed/oc-ls-stats@latest/node_modules/@troed/oc-ls-stats/package.json`
|
||||
4. **Compare with working plugin:** Diff against cached `@troed/oc-tps@latest` to spot discrepancies
|
||||
|
||||
### Resolution Timeline
|
||||
- Confirmed OpenCode CAN load scoped packages from Forgejo (tested with `@troed/oc-tps`)
|
||||
- Compared working `@troed/oc-tps` vs non-working `@troed/oc-ls-stats` line-by-line
|
||||
- Fixed all 4 issues above in version 0.0.20
|
||||
- Published 0.0.20 to Forgejo registry
|
||||
|
||||
## Prefill Rate (PP) Calculation — v0.0.40+
|
||||
|
||||
### Slot State Structure (from `GET /slots?model=...`)
|
||||
|
||||
```json
|
||||
{
|
||||
"id": 2,
|
||||
"n_ctx": 160000,
|
||||
"speculative": false,
|
||||
"is_processing": true,
|
||||
"n_prompt_tokens": 12345,
|
||||
"n_prompt_tokens_processed": 12345,
|
||||
"next_token": {
|
||||
"has_next_token": true,
|
||||
"has_new_line": false,
|
||||
"n_remain": 31943,
|
||||
"n_decoded": 0
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Slot State Phases
|
||||
|
||||
| Phase | `is_processing` | `n_remain` | `n_decoded` | `n_prompt_tokens` |
|
||||
|-------|-----------------|------------|-------------|-------------------|
|
||||
| Idle | `false` | — | — | stable |
|
||||
| Prefill | `true` | `>0` (e.g. 31943) | `0` | incrementing |
|
||||
| Transition | `true` | `>0` (e.g. 31999) | `0` | stable (final) |
|
||||
| Generation | `true` | `>0` (decreasing) | `>0` (incrementing) | stable (final) |
|
||||
|
||||
**Key insight:** `n_decoded === 0` is the definitive prefill indicator (when `is_processing === true`). `n_remain === -1` only occurs when `n_predict == -1` (unlimited generation), which is rare in practice.
|
||||
|
||||
### PP Calculation Method
|
||||
|
||||
PP is calculated from the per-slot `n_prompt_tokens` field during prefill:
|
||||
|
||||
1. Detect prefill: slot with `is_processing === true` AND `next_token[0].n_decoded === 0`
|
||||
2. On first detection: capture `slot.id` and `slot.n_prompt_tokens` as baseline
|
||||
3. On subsequent polls: `delta = current_n_prompt_tokens - baseline_tokens`, `dt = (now - startAt) / 1000`, `PP = delta / dt`
|
||||
4. Reset when no slot has `n_decoded === 0` (prefill ended)
|
||||
|
||||
### Why NOT `llamacpp:prompt_tokens_total` from `/metrics`
|
||||
|
||||
`llamacpp:prompt_tokens_total` is a **global counter across all slots**, not per-model. When multiple slots are active simultaneously (e.g., slot A prefilling while slot B generating), the delta includes tokens from all slots, producing wildly inflated PP values.
|
||||
|
||||
**Example from debug data (`test-debug-log.jsonl`):**
|
||||
- Line 15: slot 3 prefilling, `prompt_tokens_total` delta = 49 tokens in 1815ms → PP = 27 (correct but tiny delta)
|
||||
- Line 31: slot 2 prefilling, slot 3 still generating, `prompt_tokens_total` delta = 13793 tokens in 668ms → PP = 20,648 (wrong — includes slot 3 tokens)
|
||||
- Actual server console showed ~1200-1300 tokens/sec during prefill
|
||||
|
||||
### Why NOT `llamacpp:prompt_tokens_seconds` from `/metrics`
|
||||
|
||||
`llamacpp:prompt_tokens_seconds` is an **average since server start**, not an instantaneous rate. It smooths out all prefill events into a single running average, making it useless for displaying current prefill speed.
|
||||
|
||||
### Slot Polling
|
||||
|
||||
- Polls `GET /slots?model=...` every 500ms (`SLOT_POLL_MS = 500`)
|
||||
- Model parameter is required by the `/slots` endpoint (returns error without it)
|
||||
- Model discovered from: route session → OpenCode config provider parsing
|
||||
- Falls back to `http://localhost:8080` default URL
|
||||
|
||||
### Display
|
||||
|
||||
PP only shown during active prefill (`n_decoded === 0`). Format: integer only, no suffix (e.g., "1247"). Fixed-width 4 chars, space-padded. Shows "----" when no value.
|
||||
|
||||
### Why Progress Percentage Is Not Shown
|
||||
|
||||
The `/slots` endpoint does not expose the **final prompt size** (`task->n_tokens()` from llama.cpp). During prefill, `n_prompt_tokens` and `n_prompt_tokens_processed` are always equal because every token is processed as it's added to the prompt. Progress percentage requires the ratio `n_prompt_tokens_processed / task->n_tokens()`, but `task->n_tokens()` is not returned in `/slots` JSON.
|
||||
|
||||
**Required llama.cpp changes:**
|
||||
|
||||
- **PR #14685** (merged as part of #14728): adds `prompt_progress` streaming to `/completion` endpoint with `total`, `cache`, `processed`, `time_ms`. This enables progress for streaming clients but not for `/slots` polling.
|
||||
- **PR #23454** (merged May 2026): added `n_prompt_tokens`, `n_prompt_tokens_processed`, `n_prompt_tokens_cache` to `/slots` — but NOT the total prompt size.
|
||||
- **Needed**: expose `n_prompt_tokens_total` (or `n_tokens` from `task->n_tokens()`) in `/slots` output so polling clients can compute `(n_prompt_tokens_processed / n_prompt_tokens_total) * 100`.
|
||||
|
||||
### Why `n_remain === -1` Does NOT Indicate Prefill
|
||||
|
||||
`n_remain === -1` only occurs when `n_predict == -1` (unlimited generation). When `n_predict` is set (e.g., 32000), `n_remain = n_predict - n_decoded`, which is always positive during prefill. Prefill detection must use `n_decoded === 0` instead.
|
||||
|
||||
## Debugging: PP Value Wrong
|
||||
|
||||
### Symptom
|
||||
PP shows 3634 when llama-server console shows ~1200-1300 tokens/sec during prefill.
|
||||
|
||||
### Root Cause (fixed in v0.0.40)
|
||||
Used `llamacpp:prompt_tokens_total` from `/metrics` endpoint which is a global counter across all slots. Delta between polls included tokens from other active slots.
|
||||
|
||||
### Fix
|
||||
Track `n_prompt_tokens` from the slot data itself (per-slot field) during prefill state (`n_decoded === 0`).
|
||||
|
||||
### Debug Log Location
|
||||
`/tmp/oc-ls-stats-debug.log` — TUI plugin writes here because `console.log` in TUI doesn't reach main OpenCode log.
|
||||
|
||||
### Debug Data
|
||||
Sample debug log with slot states and metrics values: `test-debug-log.jsonl`
|
||||
@@ -1,46 +0,0 @@
|
||||
{"metrics":{"predictedTps":24.9122,"promptTokensTotal":97244,"promptTokensSeconds":1111.56},"lastPromptTokensTotal":97244,"lastPromptTokensTime":1781383392976,"now":1781383395633}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":true,"next_token":{"has_next_token":true,"has_new_line":false,"n_remain":31999,"n_decoded":1}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9122,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97244,"lastPromptTokensTime":1781383395633,"now":1781383396983}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":true,"next_token":{"has_next_token":true,"has_new_line":false,"n_remain":31969,"n_decoded":31}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9122,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383396983,"now":1781383399028}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":true,"next_token":{"has_next_token":true,"has_new_line":true,"n_remain":31921,"n_decoded":79}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383399028,"now":1781383401001}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383401001,"now":1781383403001}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383403001,"now":1781383405013}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383405013,"now":1781383407016}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383407016,"now":1781383409019}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383409019,"now":1781383411028}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383411029,"now":1781383413038}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383413038,"now":1781383415040}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383415041,"now":1781383417039}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383417039,"now":1781383419039}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383419039,"now":1781383421051}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383421051,"now":1781383423053}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383423053,"now":1781383425058}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383425058,"now":1781383427073}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383427073,"now":1781383429085}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383429085,"now":1781383431099}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383431099,"now":1781383433110}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31893,"n_decoded":107}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97559,"promptTokensSeconds":1101.94},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383433110,"now":1781383435194}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":true,"next_token":{"has_next_token":true,"has_new_line":false,"n_remain":31999,"n_decoded":1}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97632,"promptTokensSeconds":1094.26},"lastPromptTokensTotal":97559,"lastPromptTokensTime":1781383435194,"now":1781383437159}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":true,"next_token":{"has_next_token":true,"has_new_line":true,"n_remain":31952,"n_decoded":48}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
{"metrics":{"predictedTps":24.9031,"promptTokensTotal":97632,"promptTokensSeconds":1094.26},"lastPromptTokensTotal":97632,"lastPromptTokensTime":1781383437159,"now":1781383439126}
|
||||
{"baseUrl":"http://localhost:8080","model":"qwen3635b3-ud-q6_k","slotsType":"object","slotsCtor":"Array","slotsKeys":["0","1","2","3"],"slotsLen":4,"slotsProto":"[object Array]","slot0":"{\"id\":0,\"n_ctx\":160000,\"speculative\":false,\"is_processing\":false}","isProcessing":[{"id":0,"is_processing":false},{"id":1,"is_processing":true,"next_token":{"has_next_token":true,"has_new_line":true,"n_remain":31903,"n_decoded":97}},{"id":2,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":30096,"n_decoded":1904}},{"id":3,"is_processing":false,"next_token":{"has_next_token":false,"has_new_line":false,"n_remain":31948,"n_decoded":52}}]}
|
||||
Reference in New Issue
Block a user