Hi! We build PiCode, an ADE that orchestrates real pi agents, and pi-agent-browser-native is our browser engine of choice. Thank you for the excellent wrapper — the artifact verification, refSnapshot guards and restore-key work are exceptional.
The idea. GUI hosts that render pi's event stream (PiCode, and any RPC client) currently show agent_browser as a generic tool pill with text. Hosts could render a live preview of the page while the agent works if the wrapper emitted, alongside screenshots it already takes, an optional, self-describing object in tool details:
{ "preview": { "image": "<data-uri or https URL>", "url": "https://…", "title": "…", "ts": 0 } }
- in
partialResult.details.preview on meaningful steps (via the existing onUpdate path) — the host replaces frames, latest wins;
- in the final result's
details.preview — so hosts that replay persisted tool results (pi session files keep details) still show the last frame after reload.
Why we think it belongs upstream: the wrapper knows when a navigation/step happened (pageChangeSummary, artifactVerification) and already owns screenshot capture/cost discipline; a host-side emitter would have to re-read artifact files and guess cadence. Downscaled JPEG (~≤200 KB) per step keeps the RPC JSONL pipe cheap.
What we shipped host-side (already merged): a tool-agnostic renderer — any tool whose details carry preview gets an inline frame in the conversation (study + ADR in our repo: docs/benchmarks/2026-09-02-live-browser-preview.md, docs/decisions/0057-tool-live-preview.md). We deliberately kept agent_browser-specific code out of our core; the contract is optional metadata in free-form details, so nothing breaks if you choose a different shape — we'd conform.
Questions:
- Would you accept a PR adding opt-in preview emission (e.g. a config flag, default off) at the points where
pageChangeSummary/screenshot artifacts are already produced?
- If you'd shape it differently (e.g. inside
details.artifacts entries, or a dedicated details.live object), we'll adapt our renderer — it reads one optional object.
Related context from our study: Cursor renders "screenshots and actions in the chat, as well as the browser window itself" inline; Browserbase/browser-use expose embeddable live views. Frame-per-step through the tool stream gets PiCode 90% of that with zero new transport.
Hi! We build PiCode, an ADE that orchestrates real
piagents, andpi-agent-browser-nativeis our browser engine of choice. Thank you for the excellent wrapper — the artifact verification, refSnapshot guards and restore-key work are exceptional.The idea. GUI hosts that render pi's event stream (PiCode, and any RPC client) currently show
agent_browseras a generic tool pill with text. Hosts could render a live preview of the page while the agent works if the wrapper emitted, alongside screenshots it already takes, an optional, self-describing object in tooldetails:{ "preview": { "image": "<data-uri or https URL>", "url": "https://…", "title": "…", "ts": 0 } }partialResult.details.previewon meaningful steps (via the existingonUpdatepath) — the host replaces frames, latest wins;details.preview— so hosts that replay persisted tool results (pi session files keepdetails) still show the last frame after reload.Why we think it belongs upstream: the wrapper knows when a navigation/step happened (
pageChangeSummary,artifactVerification) and already owns screenshot capture/cost discipline; a host-side emitter would have to re-read artifact files and guess cadence. Downscaled JPEG (~≤200 KB) per step keeps the RPC JSONL pipe cheap.What we shipped host-side (already merged): a tool-agnostic renderer — any tool whose details carry
previewgets an inline frame in the conversation (study + ADR in our repo:docs/benchmarks/2026-09-02-live-browser-preview.md,docs/decisions/0057-tool-live-preview.md). We deliberately keptagent_browser-specific code out of our core; the contract is optional metadata in free-form details, so nothing breaks if you choose a different shape — we'd conform.Questions:
pageChangeSummary/screenshot artifacts are already produced?details.artifactsentries, or a dedicateddetails.liveobject), we'll adapt our renderer — it reads one optional object.Related context from our study: Cursor renders "screenshots and actions in the chat, as well as the browser window itself" inline; Browserbase/browser-use expose embeddable live views. Frame-per-step through the tool stream gets PiCode 90% of that with zero new transport.