fix(dedupe): gitea_issue_duplicates misses an exact-title duplicate when the existing body is long #30

New issue

Closed

opened 2026-10-24 00:59:04 +00:00 by dsh · 1 comment

dsh commented

2026-10-24 00:59:04 +00:00

Owner

Symptom

gitea_issue_duplicates misses an exact-title duplicate when the existing issue has a long body. Searching for the literal title of #5 returns zero candidates, while gitea_issue_search with the same query returns #5.

Repro

gitea_issue_duplicates {
  owner:"dsh-plugins", repo:"dsh-gitea",
  title:"fix(tools): empty and unrecognised results render as a bare OK"
}
→ Unrecognised response
   {"candidates":[],"threshold":0.3,"total":2}

gitea_issue_search {q:"fix(tools): empty and unrecognised results render as a bare OK", state:"all", repo:"dsh-plugins/dsh-gitea"}
→ #5 fix(tools): empty and unrecognised results render as a bare OK

A short probe title on the lab repo did match (score 0.44), so the search itself works.

Observed vs expected

Observed: candidates: [] for a 100%-title match because the candidate body dominates the Jaccard score.
Expected: an exact-title match is a candidate (title similarity 1.0); bodies should refine, not dilute, a title match.

Root cause

lib/dup-detect.js compares combined = title + body (query) with issue.title + issue.body (candidate) using one Jaccard score over the concatenated text. A long existing body adds many tokens to the union and pulls the score below threshold.

Proposed fix

Score separately and combine, e.g. max(similarity(queryTitle, issueTitle), 0.5*bodySim + 0.5*titleSim); weight title more than body. Add a regression fixture with an exact title and a long body asserting a candidate at/above threshold. Keep the no-write guarantee.

Acceptance: the Repro returns #5 as a candidate; genuinely unrelated titles still score below threshold.

Duplicate check

Searched duplicate (state all) — no existing issue. #6/#8 touched label/issue generation, not duplicate detection.

Layer

dsh-gitea plugin/tool.

## Symptom `gitea_issue_duplicates` misses an exact-title duplicate when the existing issue has a long body. Searching for the literal title of #5 returns zero candidates, while `gitea_issue_search` with the same query returns #5. ## Repro ``` gitea_issue_duplicates { owner:"dsh-plugins", repo:"dsh-gitea", title:"fix(tools): empty and unrecognised results render as a bare OK" } → Unrecognised response {"candidates":[],"threshold":0.3,"total":2} gitea_issue_search {q:"fix(tools): empty and unrecognised results render as a bare OK", state:"all", repo:"dsh-plugins/dsh-gitea"} → #5 fix(tools): empty and unrecognised results render as a bare OK ``` A short probe title on the lab repo did match (`score 0.44`), so the search itself works. ## Observed vs expected - Observed: `candidates: []` for a 100%-title match because the candidate body dominates the Jaccard score. - Expected: an exact-title match is a candidate (title similarity 1.0); bodies should refine, not dilute, a title match. ## Root cause `lib/dup-detect.js` compares `combined = title + body` (query) with `issue.title + issue.body` (candidate) using one Jaccard score over the concatenated text. A long existing body adds many tokens to the union and pulls the score below `threshold`. ## Proposed fix Score separately and combine, e.g. `max(similarity(queryTitle, issueTitle), 0.5*bodySim + 0.5*titleSim)`; weight title more than body. Add a regression fixture with an exact title and a long body asserting a candidate at/above threshold. Keep the no-write guarantee. Acceptance: the Repro returns #5 as a candidate; genuinely unrelated titles still score below threshold. ## Duplicate check Searched `duplicate` (state all) — no existing issue. #6/#8 touched label/issue generation, not duplicate detection. ## Layer dsh-gitea plugin/tool.

dsh added the

area/forgejo

priority/next

type/bug

labels

2026-10-24 00:59:04 +00:00

dsh referenced this issue from a commit

2026-10-24 01:16:06 +00:00

fix(qa): branch defaults, webhook content type and integration wording (#19, #20, #21, #22, #29, #30, #35)

dsh referenced this issue

2026-10-24 01:16:09 +00:00

fix(qa): branch defaults, webhook content type and integration wording #40

dsh commented

2026-10-24 01:16:30 +00:00

Author

Owner

Fixed in 281eade (PR #40).

Duplicate scoring now computes title and body similarity separately: max(titleSim, 0.7*titleSim + 0.3*bodySim), so an exact-title match is not diluted by a long existing body. Added a readable formatter.

Live: searching the exact title of #5 → Likely duplicates (2 issue(s) searched, threshold 0.3): - #5 "…" (score 1.00; title 1.00, body 0.00).

Tests: test/qa-defaults.test.mjs.

Fixed in `281eade` (PR #40). Duplicate scoring now computes title and body similarity separately: `max(titleSim, 0.7*titleSim + 0.3*bodySim)`, so an exact-title match is not diluted by a long existing body. Added a readable formatter. Live: searching the exact title of #5 → `Likely duplicates (2 issue(s) searched, threshold 0.3): - #5 "…" (score 1.00; title 1.00, body 0.00)`. Tests: `test/qa-defaults.test.mjs`.

dsh closed this issue

2026-10-24 01:16:51 +00:00