Add LLM metadata endpoints#1499
Conversation
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds shared LLMS prompt templates and a response factory, implements llms.txt / llms-full.txt GET+HEAD routes across apps, updates the docs generator to emit embedded docs JSON and llms outputs, adds an agent reminders component integrated into multiple MDX pages, and wires reminders into the MCP handler and configs. ChangesLLMs.txt Support and AI Documentation
Sequence Diagram(s)sequenceDiagram
participant Client
participant RouteHandler
participant SharedLLMS
participant ResponseFactory
Client->>RouteHandler: GET /llms.txt or /llms-full.txt
RouteHandler->>SharedLLMS: read llmsTxt / llmsFullTxt
RouteHandler->>ResponseFactory: createLlmsTextResponse(body)
ResponseFactory->>SharedLLMS: validate body, attach llmsTextHeaders
ResponseFactory-->>Client: 200 text/plain response
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR exposes
Confidence Score: 4/5Purely additive change that exposes new read-only public endpoints; no existing behavior is modified and no sensitive data is involved. The backend llms.txt/llms-full.txt routes bypass the createSmartRouteHandler wrapper that every other backend route uses, and the hosted-components Nitro routes respond to all HTTP methods rather than just GET and HEAD. Both are style/consistency gaps rather than functional failures for these static-content endpoints. apps/backend/src/app/llms.txt/route.ts and apps/backend/src/app/llms-full.txt/route.ts (SmartRouteHandler pattern); apps/hosted-components/src/server/routes/llms.txt.ts and llms-full.txt.ts (method guard). Important Files Changed
Sequence DiagramsequenceDiagram
participant Agent as LLM / AI Agent
participant App as Any App<br/>(backend/dashboard/mcp/skills/hosted)
participant Shared as stack-shared<br/>llms.ts
participant Content as remindersPrompt /<br/>skillSitePrompt
Agent->>App: GET /llms.txt
App->>Shared: createLlmsTextResponse(llmsTxt)
Shared->>Content: deindent remindersPrompt
Content-->>Shared: prompt string
Shared-->>App: Response (text/plain + cache/CORS headers)
App-->>Agent: 200 OK
Agent->>App: GET /llms-full.txt
App->>Shared: createLlmsTextResponse(llmsFullTxt)
Shared->>Content: deindent skillSitePrompt
Content-->>Shared: full prompt string
Shared-->>App: Response (text/plain + cache/CORS headers)
App-->>Agent: 200 OK
Prompt To Fix All With AIFix the following 2 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 2
apps/backend/src/app/llms.txt/route.ts:3-9
**Plain route handler instead of SmartRouteHandler**
The backend uses `createSmartRouteHandler` for all route handlers (see `apps/backend/src/app/api/latest/check-version/route.ts` and every route under `/api/`). Exporting bare `GET`/`HEAD` functions bypasses that wrapper, which means error handling, logging, and any middleware it provides won't apply to these endpoints. The same pattern is repeated in `apps/backend/src/app/llms-full.txt/route.ts`.
### Issue 2 of 2
apps/hosted-components/src/server/routes/llms.txt.ts:3-6
**Unguarded HTTP method handling in Nitro route**
The `fetch()` handler is called for every HTTP method (POST, DELETE, PATCH, etc.), not just GET and HEAD. Unlike the Next.js routes in the other apps, which only expose the methods they export, a Nitro `fetch` handler catches all traffic. A crawler or malformed request using POST/PUT will get `200 OK` with the full text body instead of `405 Method Not Allowed`. The same applies to `llms-full.txt.ts`.
Reviews (1): Last reviewed commit: "Simplify llms prompt wrapper" | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| export function GET() { | ||
| return createLlmsTextResponse(llmsTxt); | ||
| } | ||
|
|
||
| export function HEAD() { | ||
| return GET(); | ||
| } |
There was a problem hiding this comment.
The backend uses createSmartRouteHandler for all route handlers (see apps/backend/src/app/api/latest/check-version/route.ts and every route under /api/). Exporting bare GET/HEAD functions bypasses that wrapper, which means error handling, logging, and any middleware it provides won't apply to these endpoints. The same pattern is repeated in apps/backend/src/app/llms-full.txt/route.ts.
Rule Used: Use SmartRouteHandler for all API route handlers i... (source)
Learned From
stack-auth/stack-auth#825
stack-auth/stack-auth#472
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/backend/src/app/llms.txt/route.ts
Line: 3-9
Comment:
**Plain route handler instead of SmartRouteHandler**
The backend uses `createSmartRouteHandler` for all route handlers (see `apps/backend/src/app/api/latest/check-version/route.ts` and every route under `/api/`). Exporting bare `GET`/`HEAD` functions bypasses that wrapper, which means error handling, logging, and any middleware it provides won't apply to these endpoints. The same pattern is repeated in `apps/backend/src/app/llms-full.txt/route.ts`.
**Rule Used:** Use SmartRouteHandler for all API route handlers i... ([source](https://app.greptile.com/hexclave/-/custom-context?memory=ab376f82-1d3d-4684-a503-9fb6824b49c2))
**Learned From**
[stack-auth/stack-auth#825](https://github.com/stack-auth/stack-auth/pull/825)
[stack-auth/stack-auth#472](https://github.com/stack-auth/stack-auth/pull/472)
How can I resolve this? If you propose a fix, please make it concise.
There was a problem hiding this comment.
Addressed in eb5259843: the backend /llms.txt and /llms-full.txt routes now use createSmartRouteHandler.
| export default { | ||
| fetch() { | ||
| return createLlmsTextResponse(llmsTxt); | ||
| }, |
There was a problem hiding this comment.
The fetch() handler is called for every HTTP method (POST, DELETE, PATCH, etc.), not just GET and HEAD. Unlike the Next.js routes in the other apps, which only expose the methods they export, a Nitro fetch handler catches all traffic. A crawler or malformed request using POST/PUT will get 200 OK with the full text body instead of 405 Method Not Allowed. The same applies to llms-full.txt.ts.
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/hosted-components/src/server/routes/llms.txt.ts
Line: 3-6
Comment:
**Unguarded HTTP method handling in Nitro route**
The `fetch()` handler is called for every HTTP method (POST, DELETE, PATCH, etc.), not just GET and HEAD. Unlike the Next.js routes in the other apps, which only expose the methods they export, a Nitro `fetch` handler catches all traffic. A crawler or malformed request using POST/PUT will get `200 OK` with the full text body instead of `405 Method Not Allowed`. The same applies to `llms-full.txt.ts`.
How can I resolve this? If you propose a fix, please make it concise.There was a problem hiding this comment.
Addressed in eb5259843: the hosted-components /llms.txt and /llms-full.txt routes now use assertMethod(event, "GET", true) so non-GET/HEAD methods return 405.
There was a problem hiding this comment.
Pull request overview
Adds LLM-readable metadata surfaces (/llms.txt and /llms-full.txt) across public apps plus Mintlify docs, sharing a single helper and generator-produced content.
Changes:
- New shared
llmsTxt/llmsFullTxt/createLlmsTextResponsehelper in@stackframe/stack-shared, wired into/llms.txt+/llms-full.txtroutes in backend, dashboard, mcp, skills, and hosted-components. - Mintlify gets a generated
llms-full.txt, aHexclaveAgentReminderssnippet rendered on key docs pages, and a generateddocs-json.generated.tsthat replaces direct imports ofdocs.json. - CLI description and MCP server instructions updated with Hexclave agent guidance.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/stack-shared/src/ai/llms/llms.ts | New shared helper composing llmsTxt/llmsFullTxt and response factory |
| packages/stack-shared/src/ai/unified-prompts/skill-site-prompt-parts/docs-index.ts | Switch docs index source from docs.json to generated TS |
| packages/stack-shared/src/ai/unified-prompts/skill-site-prompt-parts/docs-json.generated.ts | Generated mirror of Mintlify docs.json |
| scripts/generate-setup-prompt-docs.ts | Generates docs-json.generated.ts, agent-reminders snippet, and llms-full.txt |
| apps/{backend,dashboard,mcp,skills}/src/app/llms.txt/route.ts, llms-full.txt/route.ts | Next route handlers serving shared LLM text |
| apps/hosted-components/src/server/routes/llms.txt.ts, llms-full.txt.ts | Nitro server routes for the hosted-components app |
| apps/hosted-components/vite.config.ts | Enables Nitro scanDirs: ['server'] to pick up new routes |
| apps/skills/package.json | Adds @stackframe/stack-shared workspace dep used by new routes |
| apps/mcp/src/mcp-handler.ts | Appends remindersPrompt to MCP server instructions |
| packages/stack-cli/src/index.ts | CLI description points humans to docs and agents to skill site |
| docs-mintlify/snippets/hexclave-agent-reminders.jsx | Generated agent-only reminders snippet |
| docs-mintlify/{index.mdx, guides/getting-started/ai-integration.mdx, guides/going-further/cli.mdx, guides/getting-started/setup.mdx} | Mount HexclaveAgentReminders on key pages |
| docs-mintlify/llms-full.txt | Generated full LLM text artifact |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| writeFileSyncIfChanged( | ||
| path.join(repoRoot, "packages/stack-shared/src/ai/unified-prompts/skill-site-prompt-parts/docs-json.generated.ts"), | ||
| deindent` | ||
| // This file is generated from docs-mintlify/docs.json. | ||
|
|
||
| const docsJson = ${JSON.stringify(JSON.parse(readFileSync(path.join(repoRoot, "docs-mintlify/docs.json"), "utf-8")), null, 2)} as const; | ||
|
|
||
| export default docsJson; | ||
| ` + "\n", | ||
| ); |
| export const llmsFullTxt = deindent` | ||
| # Hexclave | ||
|
|
||
| ${skillSitePrompt} | ||
| `; | ||
|
|
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 4e19809. Configure here.
| writeFileSyncIfChanged( | ||
| path.join(repoRoot, "docs-mintlify/llms-full.txt"), | ||
| llmsFullTxt + "\n", | ||
| ); |
There was a problem hiding this comment.
Script produces stale llms-full.txt after docs change
Medium Severity
llmsFullTxt is imported at module load time (line 7), which evaluates it from the existing docs-json.generated.ts. The script then writes an updated docs-json.generated.ts (line 280) from the live docs.json, but llmsFullTxt still holds the value computed from the old generated file. When llms-full.txt is written (line 499), it contains stale docs-index content. A second run of the script is needed to produce correct output.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit 4e19809. Configure here.
| 'server', | ||
| ], | ||
| }, | ||
| }), |
There was a problem hiding this comment.
Nitro scanDirs path unlikely to find route files
High Severity
scanDirs: ['server'] resolves relative to the Nitro/Vite root directory (apps/hosted-components/), pointing to a non-existent apps/hosted-components/server/ directory. The actual route files live at apps/hosted-components/src/server/routes/. The value likely needs to be 'src/server' for Nitro to discover the llms.txt and llms-full.txt routes.
Reviewed by Cursor Bugbot for commit 4e19809. Configure here.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
packages/stack-shared/src/ai/llms/llms.ts (1)
17-23: Verify the cache duration is intentional.The
Cache-Controlheader sets a 1-hour cache duration for LLM metadata. Confirm this aligns with expected update frequency for AI documentation.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/stack-shared/src/ai/llms/llms.ts` around lines 17 - 23, The Cache-Control value in the exported constant llmsTextHeaders currently uses "public, max-age=3600, s-maxage=3600" (1 hour); confirm this is the intended cache duration for LLM metadata and either adjust those values in llmsTextHeaders to the correct seconds (e.g., shorter if metadata updates often or longer if stable) or make the duration configurable (env var or config constant) and reference that config when setting Cache-Control to ensure the header aligns with expected update frequency.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/stack-shared/src/ai/llms/llms.ts`:
- Around line 25-29: The createLlmsTextResponse function must validate its body
parameter and throw early on invalid input; update createLlmsTextResponse to
check that body is a non-empty string (e.g., typeof body === "string" and body
!== "" && body !== null && body !== undefined) and throw a clear TypeError if
the check fails, then proceed to return new Response(body, { headers:
llmsTextHeaders }); this ensures the Response constructor never receives
"undefined" or "null" stringified values.
---
Nitpick comments:
In `@packages/stack-shared/src/ai/llms/llms.ts`:
- Around line 17-23: The Cache-Control value in the exported constant
llmsTextHeaders currently uses "public, max-age=3600, s-maxage=3600" (1 hour);
confirm this is the intended cache duration for LLM metadata and either adjust
those values in llmsTextHeaders to the correct seconds (e.g., shorter if
metadata updates often or longer if stable) or make the duration configurable
(env var or config constant) and reference that config when setting
Cache-Control to ensure the header aligns with expected update frequency.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 63b50717-b16b-4d22-8313-f784d6f7cf3d
📒 Files selected for processing (24)
apps/backend/src/app/llms-full.txt/route.tsapps/backend/src/app/llms.txt/route.tsapps/dashboard/src/app/llms-full.txt/route.tsapps/dashboard/src/app/llms.txt/route.tsapps/hosted-components/src/server/routes/llms-full.txt.tsapps/hosted-components/src/server/routes/llms.txt.tsapps/hosted-components/vite.config.tsapps/mcp/src/app/llms-full.txt/route.tsapps/mcp/src/app/llms.txt/route.tsapps/mcp/src/mcp-handler.tsapps/skills/package.jsonapps/skills/src/app/llms-full.txt/route.tsapps/skills/src/app/llms.txt/route.tsdocs-mintlify/guides/getting-started/ai-integration.mdxdocs-mintlify/guides/getting-started/setup.mdxdocs-mintlify/guides/going-further/cli.mdxdocs-mintlify/index.mdxdocs-mintlify/llms-full.txtdocs-mintlify/snippets/hexclave-agent-reminders.jsxpackages/stack-cli/src/index.tspackages/stack-shared/src/ai/llms/llms.tspackages/stack-shared/src/ai/unified-prompts/skill-site-prompt-parts/docs-index.tspackages/stack-shared/src/ai/unified-prompts/skill-site-prompt-parts/docs-json.generated.tsscripts/generate-setup-prompt-docs.ts
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
4 issues found across 24 files
Reply with feedback, questions, or to request a fix.
Fix all with cubic | Re-trigger cubic
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
|
Addressed CodeRabbit’s HEAD parsing nit in |
Adds LLM-readable metadata surfaces across public apps and Mintlify docs.
llms.txt/llms-full.txthelpers and endpoints for dashboard, backend, MCP, skills, and hosted componentsllms-full.txtand agents-only reminder snippets on key docs pagesgenerate-setup-prompt-docs, including freshdocs.json-basedllms-full.txtgenerationLink to Devin session: https://app.devin.ai/sessions/cd5eb97972444dbca08e22850bd4bdaf
Requested by: @N2D4