Forum verdict · build Solution request

RAG scope limiter for LLM agents to answer only forum-specific questions

Built for Discourse forum admins using AI bots.

“I’m setting up Discourse AI on my forum and have an AI Bot/Agent working with Gemini 2.5 Flash via LiteLLM and Google Vertex AI. The bot is working, but I have…”

The receipts — real demand

“I’m setting up Discourse AI on my forum and have an AI Bot/Agent working with Gemini 2.5 Flash via LiteLLM and Google Vertex AI. The bot is working, but I have a problem with its scope. For example, my forum is a busin…”

Full dossier

Unlock the full dossier — free

Every corroborating quote, the source receipts, and the community echo. One email, no payment.

5.8 / 10 · demand score
Pain 8
Willingness to pay 4
Feasibility 6
Specificity 7
Audience 7
Competition 8

Why this is a gap

Surfaced from a high-intensity complaint with clear willingness to pay and a specific, reachable audience.

The market

Discourse forum admins running AI bots (Gemini 2.5 Flash + LiteLLM stack) who need LLM responses confined to forum content only. No search volume given; demand appears driven by active Discourse + AI bot adoption, not broad viral interest.

Competition & the opening

Already owned an incumbent owns the exact job Moat 2/10 · no real moat Market 5/10 · a real vertical
Category giants · 8/10 vs LlamaIndex (metadata filters + retrieval scoping built-in)LangChain (document namespaces, retriever routing, and chain guardrails)Pinecone (namespace-based vector isolation per knowledge base)Weaviate (multi-tenant class scoping with RBAC)Vectara (corpus-level access control and query isolation)Mendable / Inkeep (domain-scoped RAG chatbots for docs/communities)

LlamaIndex, LangChain, Pinecone, Weaviate, Vectara, and Mendable/Inkeep all offer retrieval scoping, metadata filtering, or corpus-level access control. The gap: none of these are purpose-built *for Discourse admin workflows* — they require custom integration work to wire forum-specific isolation into an existing LLM agent stack. A Discourse-native RAG limiter (plugin or LiteLLM middleware) would

What's hard to build

Integrating tightly with Discourse's post/category/user permission model while also intercepting and validating LLM retrieval calls in real-time is hard. You need robust hooks into both Discourse's data layer and the LiteLLM/Gemini request pipeline, plus robust testing across different forum structures (hierarchies, private categories, group visibility).

Why now

LLM agents on forums need scope guardrails that existing RAG frameworks (LlamaIndex, LangChain) require manual setup to enforce, creating friction for non-engineers deploying Discourse AI.

How you'd monetize

freemium (basic scope rules free, advanced routing/multi-tenant $29/mo)