dsh-plugins/dsh-gitea

Watch

Fork 0

[Experiment] Lazy tool discovery: gitea_get_tools(category) loads schemas on demand #50

New issue

Closed

opened 2026-10-24 20:30:06 +00:00 by dsh · 2 comments

dsh commented

2026-10-24 20:30:06 +00:00

Owner

Origin

Operator's idea draft (idea.txt, 2026-10-24) after the payload measurement in #48 / #49. Original working title: [Idea/Feature] Toolset Redesign: Remove legacy bloat and implement Lazy-Loading (KISS).

The concept: expose only a meta-tool at boot (gitea_get_tools(category)); when the agent needs Gitea work it asks for a category (issues, wiki, actions, prs, repos, customize) and only those schemas enter the context. A customize category holds gitea_favorite(action, tool_name) (always-loaded favourites, a lazy-load blacklist) and gitea_lazy-tools(enable) plus a settings-UI toggle; with lazy mode disabled the plugin behaves exactly as today.

Original acceptance criteria from the draft

Deprecated aliases are deleted (verify payload reduction).
The default boot footprint is drastically reduced (ideally < 1 KB payload for the base plugin).
Agents can successfully dynamically load and execute tools using the gitea_get_tools meta-tool.
gitea_favorite keeps preferred tools in the always-loaded context.
Backward compatibility: if lazy-loading is disabled (or unsupported by the client), the plugin behaves exactly as it does today (loads everything).

What the harness actually does (DSH 0.2.1-rc.2, from the installed packages)

Registering a tool is what makes it visible; schemas are projected during system-prompt assembly (dsh-tools).
A tool that makes tools appear produces tool-addition records; the visible set changes and the request prefix changes, so KV cache invalidates "from the first changed schema token" (dsh-tools, dsh-llm projectToolUpdates).
deferLoading: true exists and is preserved through projection, but the docs are explicit: "declaring a deferred tool does not activate it" and provider enforcement is limited.
Session history is append-only: "newly visible content follows the reusable request prefix." A schema delivered as a tool result stays in history; nothing unloads it.

Item 4 is the crux: lazy loading here is additive, not replacing. A session that loads issues then prs carries base + issues + prs + the meta-tool, and never gets that memory back. The saving exists only for sessions that touch few categories.

What the data says (28 sessions, 1,020 unique calls)

Fact	Value
avg categories per active session	2.8 of 5
active sessions using ≤2 categories	6 / 16 (the good case)
active sessions using ≥4 categories	5 / 16 (one used 91 distinct tools)
sessions calling no gitea tool at all	12 / 28

So the optimistic case is real but narrow, and the pessimistic case can end up larger than today (meta-tool + favourites + every loaded category). The 12 dead sessions are better served by #49, which is the only lever that removes the payload entirely.

Measurement-first plan (spike, not a shippable design yet)

Prototype behind an opt-in config (preset: lazy), default unchanged.
Instrument: assembled tool-payload bytes and prompt tokens per step, with cache-hit tokens from the usage record.
Run 5 real sessions across our actual workflows (issue triage, PR review, release, wiki edit, CI/ops) and compare against full and core:
- total prompt tokens and cache-read tokens per step
- steps per task, tool errors / UNKNOWN_TOOL rate
- whether compaction triggers earlier
Post the table here and decide: ship, revise, or close as won't do — presets are cheaper.

Acceptance criteria (spike gate)

prototype exists behind opt-in config; default behaviour byte-identical
measurement table posted in this issue
explicit decision recorded, plus a follow-up issue if it ships
favourites, enable/disable toggle and settings UI are not built before the spike wins

Impact

If it works, the base footprint drops to a few hundred bytes for sessions that only file one issue — the draft's stretch goal. If it does not, we will have a measured "no" and stop wondering, which is worth the spike on its own.

Boundaries

No changes to canonical schemas; the meta-tool is an addition, not a rewrite.
No settings-UI work, no gitea_favorite, no gitea_lazy-tools before the spike decides.
Out of scope: changing how DSH presents tools — that is upstream's surface, not ours.

Dependencies

#48 (alias purge) and #49 (presets) are the cheaper levers and should land first. This experiment is only justified if core + presets still leave too much on the table. The cache-safe alternative is the dispatcher experiment.

## Origin Operator's idea draft (`idea.txt`, 2026-10-24) after the payload measurement in #48 / #49. Original working title: `[Idea/Feature] Toolset Redesign: Remove legacy bloat and implement Lazy-Loading (KISS)`. **The concept:** expose only a meta-tool at boot (`gitea_get_tools(category)`); when the agent needs Gitea work it asks for a category (`issues`, `wiki`, `actions`, `prs`, `repos`, `customize`) and only those schemas enter the context. A `customize` category holds `gitea_favorite(action, tool_name)` (always-loaded favourites, a lazy-load blacklist) and `gitea_lazy-tools(enable)` plus a settings-UI toggle; with lazy mode disabled the plugin behaves exactly as today. <details> <summary>Original acceptance criteria from the draft</summary> - [ ] Deprecated aliases are deleted (verify payload reduction). - [ ] The default boot footprint is drastically reduced (ideally < 1 KB payload for the base plugin). - [ ] Agents can successfully dynamically load and execute tools using the `gitea_get_tools` meta-tool. - [ ] `gitea_favorite` keeps preferred tools in the always-loaded context. - [ ] Backward compatibility: if lazy-loading is disabled (or unsupported by the client), the plugin behaves exactly as it does today (loads everything). </details> ## What the harness actually does (DSH 0.2.1-rc.2, from the installed packages) 1. Registering a tool is what makes it visible; schemas are projected during system-prompt assembly (`dsh-tools`). 2. A tool that makes tools appear produces `tool-addition` records; the visible set changes and the request prefix changes, so KV cache invalidates "from the first changed schema token" (`dsh-tools`, `dsh-llm` `projectToolUpdates`). 3. `deferLoading: true` exists and is preserved through projection, but the docs are explicit: *"declaring a deferred tool does not activate it"* and provider enforcement is limited. 4. Session history is append-only: *"newly visible content follows the reusable request prefix."* A schema delivered as a tool **result** stays in history; nothing unloads it. Item 4 is the crux: **lazy loading here is additive, not replacing.** A session that loads `issues` then `prs` carries base + issues + prs + the meta-tool, and never gets that memory back. The saving exists only for sessions that touch few categories. ## What the data says (28 sessions, 1,020 unique calls) | Fact | Value | |---|---| | avg categories per active session | 2.8 of 5 | | active sessions using ≤2 categories | 6 / 16 (the good case) | | active sessions using ≥4 categories | 5 / 16 (one used 91 distinct tools) | | sessions calling no gitea tool at all | 12 / 28 | So the optimistic case is real but narrow, and the pessimistic case can end up **larger** than today (meta-tool + favourites + every loaded category). The 12 dead sessions are better served by #49, which is the only lever that removes the payload entirely. ## Measurement-first plan (spike, not a shippable design yet) 1. Prototype behind an opt-in config (`preset: lazy`), default unchanged. 2. Instrument: assembled tool-payload bytes and prompt tokens per step, with cache-hit tokens from the `usage` record. 3. Run 5 real sessions across our actual workflows (issue triage, PR review, release, wiki edit, CI/ops) and compare against `full` and `core`: - total prompt tokens and cache-read tokens per step - steps per task, tool errors / `UNKNOWN_TOOL` rate - whether compaction triggers earlier 4. Post the table here and decide: ship, revise, or close as *won't do — presets are cheaper*. ## Acceptance criteria (spike gate) - [ ] prototype exists behind opt-in config; default behaviour byte-identical - [ ] measurement table posted in this issue - [ ] explicit decision recorded, plus a follow-up issue if it ships - [ ] favourites, enable/disable toggle and settings UI are **not** built before the spike wins ## Impact If it works, the base footprint drops to a few hundred bytes for sessions that only file one issue — the draft's stretch goal. If it does not, we will have a measured "no" and stop wondering, which is worth the spike on its own. ## Boundaries - No changes to canonical schemas; the meta-tool is an addition, not a rewrite. - No settings-UI work, no `gitea_favorite`, no `gitea_lazy-tools` before the spike decides. - Out of scope: changing how DSH presents tools — that is upstream's surface, not ours. ## Dependencies #48 (alias purge) and #49 (presets) are the cheaper levers and should land first. This experiment is only justified if `core` + presets still leave too much on the table. The cache-safe alternative is the dispatcher experiment.

dsh added the

labels

2026-10-24 20:30:06 +00:00

dsh referenced this issue

2026-10-24 20:30:15 +00:00

[Experiment] Two-tool surface: stable gitea dispatcher + gitea_help (9,734 B vs 65,159 B) #51

dsh commented

2026-10-24 20:45:19 +00:00

Author

Owner

Deferred by decision (2026-10-24), not closed: this is a measured no candidate until the dispatcher (#51) reports.

Rationale from the issue's own analysis, which holds up against the implementation now on master:

The mechanism is additive. Schemas delivered as tool results stay in append-only history, so a session that loads issues then prs carries base + issues + prs + the meta-tool and never gets the memory back. The window where it wins is sessions touching ≤2 categories.
The 12/28 sessions that never call a Gitea tool are now served better and cheaper: preset: none (or a narrow preset) removes the payload at mount with no runtime machinery, no tool-addition records, and no KV-cache invalidation. That was the strongest argument in this issue, and it is shipped in 0.8.5.
#51 answers the same question ("how small can the surface get") with a stable registry and schema delivery as an append-only tool result, i.e. without betting on tool-addition/deferLoading behaviour before DSH 1.0.

Plan: experiment/dispatcher is branched off v0.8.5 for #51. If the dispatcher spike wins on prompt tokens and first-try argument accuracy, this issue is moot (the dispatcher is the better cache-safe shape) and closes as won't do. If the dispatcher loses, the remaining case for lazy loading is the narrow ≤2-category window — and then this issue gets a timeboxed prototype behind preset: lazy to measure that window before anything else (favourites, toggles, settings UI) is built.

Keeping it open so the deferred spike is not mistaken for an accepted design.

Deferred by decision (2026-10-24), not closed: this is a measured *no* candidate until the dispatcher (#51) reports. Rationale from the issue's own analysis, which holds up against the implementation now on master: - The mechanism is **additive**. Schemas delivered as tool results stay in append-only history, so a session that loads `issues` then `prs` carries base + issues + prs + the meta-tool and never gets the memory back. The window where it wins is sessions touching ≤2 categories. - The 12/28 sessions that never call a Gitea tool are now served better and cheaper: `preset: none` (or a narrow preset) removes the payload at mount with no runtime machinery, no `tool-addition` records, and no KV-cache invalidation. That was the strongest argument in this issue, and it is shipped in 0.8.5. - #51 answers the same question ("how small can the surface get") with a **stable** registry and schema delivery as an append-only tool result, i.e. without betting on `tool-addition`/`deferLoading` behaviour before DSH 1.0. Plan: `experiment/dispatcher` is branched off `v0.8.5` for #51. If the dispatcher spike wins on prompt tokens *and* first-try argument accuracy, this issue is moot (the dispatcher is the better cache-safe shape) and closes as *won't do*. If the dispatcher loses, the remaining case for lazy loading is the narrow ≤2-category window — and then this issue gets a timeboxed prototype behind `preset: lazy` to measure that window before anything else (favourites, toggles, settings UI) is built. Keeping it open so the deferred spike is not mistaken for an accepted design.

dsh referenced this issue

2026-10-24 20:49:09 +00:00

[Experiment] Two-tool surface: stable gitea dispatcher + gitea_help (9,734 B vs 65,159 B) #51

dsh commented

2026-10-24 22:25:28 +00:00

Author

Owner

Closed as won't do — this is now #51's shape, and the operator has chosen the dispatcher.

The deciding argument is the one this issue itself identified: lazy loading is additive. Schemas delivered as results stay in append-only history, the registry changes when categories load (invalidating the prefix), and a session that touches four categories ends up larger than today. #51 keeps the registry stable and delivers schemas as append-only tool results, so it gets the same "don't pay for what you don't use" benefit without betting on tool-addition/deferLoading behaviour before DSH 1.0.

Also, the part of this issue that was independently valuable is already shipped: preset: none (and the narrow presets) in 0.8.5 removes the payload entirely for the 12/28 sessions that never call a Gitea tool, with zero runtime machinery.

The analysis in the body stays as the record of why the additive route was rejected; no prototype will be built. experiment/dispatcher carries the spike.

Closed as **won't do** — this is now #51's shape, and the operator has chosen the dispatcher. The deciding argument is the one this issue itself identified: lazy loading is *additive*. Schemas delivered as results stay in append-only history, the registry changes when categories load (invalidating the prefix), and a session that touches four categories ends up larger than today. #51 keeps the registry **stable** and delivers schemas as append-only tool results, so it gets the same "don't pay for what you don't use" benefit without betting on `tool-addition`/`deferLoading` behaviour before DSH 1.0. Also, the part of this issue that was independently valuable is already shipped: `preset: none` (and the narrow presets) in 0.8.5 removes the payload entirely for the 12/28 sessions that never call a Gitea tool, with zero runtime machinery. The analysis in the body stays as the record of why the additive route was rejected; no prototype will be built. `experiment/dispatcher` carries the spike.

dsh closed this issue

2026-10-24 22:48:32 +00:00

dsh referenced this issue

2026-10-24 22:49:45 +00:00

feat(tools): opt-in dispatcher surface — gitea + gitea_help (#51) #54