LLM Output Handling: XSS and IDOR Paths Hunters Should Test
Teams bolt a chat panel onto a product and treat the model as a trusted author. The model is not trusted. Whatever it emits—or whatever a tool returns into the prompt—still hits your browser and your API.
Two bug classes show up constantly: XSS in rendered answers, and IDOR around transcripts, attachments, and tool-fetched objects.
XSS from "helpful" formatting
Many UIs render Markdown or HTML from the assistant. Feed the model content that survives into the reply: a markdown image, a raw <script>, an SVG, or a link with javascript: if the sanitizer is weak.
Better proofs use your own data the model is asked to quote. Upload a file named with a payload, paste a profile field containing HTML, or put a canary tag in a ticket the bot summarizes. When the UI reflects that string inside dangerouslySetInnerHTML or an unsanitized Markdown pipeline, you have stored XSS with a realistic path.
Test both streaming and final message nodes. I've watched apps sanitize the completed bubble and leave the partial stream raw for a second—long enough for a handler to run in some frameworks.
CSP matters, but do not stop at "CSP blocks inline script." Event handlers in Markdown, vulnerable PDF viewers, or markdown-it plugins can still matter depending on sink. Also try the "copy as HTML" or "export conversation" features; those sometimes use a second renderer with weaker filters.
IDOR on conversations and tools
Chat features create new objects: threads, messages, tool traces, exported summaries. Enumerate ids. Request /api/chat/threads/{id} as user B while the thread belongs to A. Check share links that are "secret" but permanent. Check admin replay tools that fetch any transcript by uuid.
Tool-using agents add a second IDOR flavor. If the model can call getInvoice(id) and the backend honors the tool call without binding the invoice to the session user, prompt injection is optional—you may call the API the same way the tool does. Compare the tool HTTP path with the ordinary UI path; sometimes only one enforces ownership.
Attachment endpoints are easy wins. The bot says "I saved your file at /files/123"; that path must still authorize. Watch websocket replay of historical tool results too—authorization on the first fetch does not always apply to the cached event stream.
Keep the demo ethical
Use accounts you own. Plant canaries, not other customers' data. For XSS, a alert on a self-owned page or a unique string in document.title is enough. For IDOR, show A's canary invoice appearing in B's session.
What to tell triage
Separate model weirdness from application failure. The vulnerable sink is the HTML renderer or the missing user_id predicate—not "the LLM is unsafe." Recommend sanitize-on-output, strict Markdown allowlists, and server-side authorization on every thread, tool, and file id.
Document whether the XSS fires for other users viewing a shared thread. Self-XSS in a private chat is a weaker story than a stored payload that lands in a teammate's inbox view.
If the product markets an AI feature, assume output handling was rushed. That assumption pays.