Agents should fetch and view issue image attachments before starting work #405

Closed
opened 2026-11-06 20:30:46 -05:00 by cmoriarty · 1 comment
Owner

Problem

When an agent reads a Forgejo issue via the API, image attachments appear only as
markdown links in the issue body (![image](/attachments/<uuid>)) plus asset
metadata (assets[].browser_download_url). The pixels are not included in the
response. Unless the agent explicitly downloads and views the attachments,
anything conveyed only by a screenshot is invisible to it — and nothing in the
current workflow makes it do that.

Evidence

cmoriarty/scratch-vllm-fp8 issue #6 ("UI fixes") reported a visual bug (cards
cut off at the bottom) with three screenshots. The agent that worked the issue
missed the clipping bug while handling the text-only items fine. All three
attachments on the issue have download_count: 0 — nobody ever fetched them.

Proposed fix

Before starting work on an issue, the agent should:

  1. Read the issue's assets array.
  2. Download each attachment, authenticated with the Forgejo token:
    curl -sL -H "Authorization: token $FORGEJO_TOKEN" -o <name> <browser_download_url>
  3. View the downloaded images (vision) before planning or writing code.

This could be baked into the orchestrator's step prompt, or built into the
issue-reading tooling so that fetching an issue automatically pulls its
attachments into the agent's context.

Acceptance criteria

  • When an issue has image attachments, the agent's working context includes the
    actual image content, not just the markdown links.
  • Repro: file an issue whose only content is a screenshot of a visual bug; the
    agent should be able to describe the bug from the image alone.
## Problem When an agent reads a Forgejo issue via the API, image attachments appear only as markdown links in the issue body (`![image](/attachments/<uuid>)`) plus asset metadata (`assets[].browser_download_url`). The pixels are not included in the response. Unless the agent explicitly downloads and views the attachments, anything conveyed only by a screenshot is invisible to it — and nothing in the current workflow makes it do that. ## Evidence `cmoriarty/scratch-vllm-fp8` issue #6 ("UI fixes") reported a visual bug (cards cut off at the bottom) with three screenshots. The agent that worked the issue missed the clipping bug while handling the text-only items fine. All three attachments on the issue have `download_count: 0` — nobody ever fetched them. ## Proposed fix Before starting work on an issue, the agent should: 1. Read the issue's `assets` array. 2. Download each attachment, authenticated with the Forgejo token: `curl -sL -H "Authorization: token $FORGEJO_TOKEN" -o <name> <browser_download_url>` 3. View the downloaded images (vision) before planning or writing code. This could be baked into the orchestrator's step prompt, or built into the issue-reading tooling so that fetching an issue automatically pulls its attachments into the agent's context. ## Acceptance criteria - When an issue has image attachments, the agent's working context includes the actual image content, not just the markdown links. - Repro: file an issue whose only content is a screenshot of a visual bug; the agent should be able to describe the bug from the image alone.
Author
Owner

Shipped in PR #406 (merged 2026-11-06, 962ca318).

  • Issue.assets reads the forge's attachments; download_issue_assets fetches each image with the token, checks the bytes and saves it under .osf/brief-assets/, out of the PR diff.
  • brief_from_issue points each embedded screenshot at its saved file ("read it before you plan or write code") and lists attachments the text never references, so a screenshot-only issue is still a request.
  • script.brief runs the best-effort osf.forgejo brief-assets step; a screenshot that does not land is said so in the brief, and the step never fails the run.

Evidence: the PR's tests (fetch, verification, the brief's pointers, the wizard and webhook paths) and a green suite. Closing on that; the first run from a screenshot-only issue is the production check, and a miss there is a new ticket.

**Shipped in PR #406 (merged 2026-11-06, 962ca318).** - `Issue.assets` reads the forge's attachments; `download_issue_assets` fetches each image with the token, checks the bytes and saves it under `.osf/brief-assets/`, out of the PR diff. - `brief_from_issue` points each embedded screenshot at its saved file ("read it before you plan or write code") and lists attachments the text never references, so a screenshot-only issue is still a request. - `script.brief` runs the best-effort `osf.forgejo brief-assets` step; a screenshot that does not land is said so in the brief, and the step never fails the run. Evidence: the PR's tests (fetch, verification, the brief's pointers, the wizard and webhook paths) and a green suite. Closing on that; the first run from a screenshot-only issue is the production check, and a miss there is a new ticket.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#405
No description provided.