How to set up docs Q&A for your DeFi protocol

The pitch is simple: give your support agent a URL, it crawls your docs, and from that point on it can answer user questions based on what you've written. No manual Q&A pairs, no taxonomy work, no training data, just your existing documentation.

The pitch is accurate, but it glosses over some things that matter. The quality of the answers is directly proportional to the quality of the source material. A crawler can only work with what's there.

What the crawler actually does

When you paste a documentation URL, the crawler fetches the page, extracts the text content, chunks it into overlapping segments, and stores those chunks as vector embeddings. When a user asks a question, the support agent retrieves the most relevant chunks and uses them to construct an answer.

This retrieval step (not the underlying language model) is where most docs Q&A systems fail. If the relevant content isn't in the index, the model will either hallucinate an answer or admit it doesn't know. The model can only work with what retrieval surfaces.

What kinds of docs work best

Not all documentation formats are equally indexable. HTML pages with clean semantic markup are ideal. The crawler can identify headings, paragraphs, code blocks, and lists, which gives it good signal for chunking, keeping related content together and avoiding splits in the middle of explanations.

Static site generators like Docusaurus, GitBook, and Nextra all produce well-structured HTML. If your docs are built with one of these, you're starting from a good place.

• Works well: HTML pages with semantic structure, Docusaurus, GitBook, Nextra, GitHub wikis, Notion (when published publicly)
• Works with caveats: dynamically rendered pages (React/Next with client-side routing); the crawler must execute JavaScript to see the content
• Works poorly: PDFs embedded in iframes, documentation behind authentication walls, pages that require wallet connection to view
• Does not work: PDFs without text layers (scanned documents), video content, images with text

What to include in your docs to help the AI

This is where protocol teams leave the most value on the table. Most DeFi documentation was written for humans who are actively trying to learn. But Q&A retrieval favors documentation written for humans who are confused and looking for a specific answer.

The practical difference: instead of a 2,000-word explanation of how your staking mechanism works, have a section that directly answers the questions users actually ask.

• Write explicit question-and-answer sections. 'Why did my transaction fail?' is a better heading than 'Transaction troubleshooting'
• Include the exact error messages users will see, followed by what they mean. Revert reasons like INSUFFICIENT_OUTPUT_AMOUNT should appear verbatim in your docs
• Document edge cases separately: what happens at the lock expiry boundary, what happens if gas spikes mid-transaction, what happens when liquidity is low
• Describe numbers explicitly: if your staking lock is 14 days, write '14 days' not 'two weeks'; units and precision matter for retrieval matching
• Add a dedicated FAQ page that consolidates the most common support questions. Even if they're answered elsewhere, a single high-density page retrieves well

Common pitfalls

PDFs and protected content

PDFs are a significant failure mode. A PDF linked from your docs page won't be indexed unless you explicitly add it as a separate source. And PDFs created by scanning physical documents, or exported from tools that embed text as images, are opaque to a text-based crawler. The content simply isn't there.

The same problem applies to documentation behind authentication. If users need to log in to read your audit report, the crawler can't read it either. Either publish audits publicly as HTML, or paste the content directly into an indexable page.

Dynamically rendered content

Single-page applications that render documentation client-side are a common trap. The crawler fetches the page URL and gets back an almost-empty HTML shell. The actual content loads via JavaScript after the initial response. Check what the crawler actually indexed: if it shows only your nav and footer text, your docs are rendering client-side and you need a crawler that executes JavaScript.

Stale content

The index is a snapshot. If you update your docs and don't re-index, the support agent answers from the old version. Set a re-crawl schedule. Weekly is reasonable for most protocols, daily if you're actively shipping changes. Some teams trigger re-indexing as part of their docs deployment pipeline, which is the cleanest solution.

Structuring your docs for retrieval, not just for reading

The single most impactful structural change is moving from narrative documentation to reference documentation. Narrative docs tell a story: they're great for onboarding users who read sequentially. Reference docs answer specific questions: they're what retrieval systems find.

You don't have to choose between the two. Keep your getting-started guides and conceptual explanations. Add a separate reference section that's structured around the questions users will ask. Think of it as a page your support agent can quote directly.

Write your reference docs as if you're writing the answer to a question, not explaining a concept. 'The lock period is 14 days from deposit date, after which you can withdraw without penalty' retrieves better than a paragraph explaining why lock periods exist.

Testing your docs Q&A before launch

Before pointing real users at a docs-powered support agent, test it against your own support history. Take your last 20 inbound questions and run them through the agent. If it answers fewer than 14 or 15 correctly, the gap is almost always in the docs, not the model.

For questions where the agent hallucinates or says it doesn't know, add the answer explicitly to your documentation. The feedback loop between support questions and documentation gaps is the most useful quality signal you have.