Imagine getting instant, AI-powered answers while browsing Odoo documentation—no more waiting for complete responses to load. That's exactly what I've achieved with the latest streaming API update to my Odoo Expert chatbot project.

I’m excited to share this major update to my Odoo Expert chatbot project: streaming API support. This feature lets me serve up real-time, AI-powered responses directly in my local Odoo documentation site—right at the top of the page, above the usual search results.

What is Odoo Expert?

Revolutionizing Odoo Documentation with RAG & LLM: Introducing the Odoo Expert
Introduce Odoo Expert, an RAG-powered documentation assistant that revolutionizes Odoo’s technical documentation. Learn how this open-source tool leverages AI to provide instant, accurate answers across multiple versions, solving the documentation accessibility challenge.
Fanyang Meng's BlogFanyang Meng

Odoo Expert is a documentation assistant that leverages Retrieval-Augmented Generation (RAG) to provide intelligent search capabilities across Odoo's technical documentation. It supports multiple Odoo versions and features an interactive chat interface powered by Large Language Models (LLMs).

GitHub - MFYDev/odoo-expert: RAG-powered documentation assistant that converts, processes, and provides semantic search capabilities for Odoo’s technical documentation. Supports multiple Odoo versions with an interactive chat interface powered by LLM models.
RAG-powered documentation assistant that converts, processes, and provides semantic search capabilities for Odoo's technical documentation. Supports multiple Odoo versions with an interactive c…
GitHubMFYDev

The Challenge: Response Time and User Experience

The original version of Odoo Expert provided accurate answers but had a significant limitation: users had to wait for the entire response to be generated before seeing any output. The /api/chat endpoint's loading time was notably slow due to the comprehensive processing required for each query. This delay disrupted the natural flow of documentation search and hindered user experience.

The Solution: Streaming API Implementation

To address this challenge, I developed a streaming solution that delivers response chunks in real-time, creating a more dynamic and responsive experience. Here's how it works:

Backend Implementation

I extended my chatbot’s backend with an endpoint (/api/stream) that streams text chunks in text/event-stream format. This addition means I can now continuously deliver chunks of the AI’s response as they become available. Under the hood, it retrieves relevant documentation chunks, prepares them for context, and then streams the answer—complete with markdown and real-time updates—right in the browser.

Now, if you are using Odoo Expert, you can simply make a POST request to the /api/stream  endpoint, to query the documentation and get AI-powered responses in streaming format.

curl -X POST "http://localhost:8000/api/stream" \
-H "Authorization: Bearer your-api-token" \
-H "Content-Type: application/json" \
-d '{
    "query": "How do I install Odoo?",
    "version": 180,
    "conversation_history": []
}'

For the detailed guide and change log, you are welcome to follow the guide in the GitHub repo.

Integrate it into the Odoo Documentation Website

Thankfully, Odoo's documentation website with its content is open sourced:

GitHub - odoo/documentation: Odoo documentation sources
Odoo documentation sources. Contribute to odoo/documentation development by creating an account on GitHub.
GitHubodoo

Because the open source, I can easily build the static HTML website by myself, and by changing the source code, I can test the integration with Odoo Expert. To dive deeply into the technical details, here is what I have done:

  1. New AI Response Section
    • I introduced a dedicated section (#ai-response-section) in search.html to display AI-powered insights. This section is placed above the default search results, ensuring it grabs the user’s attention.
    • A new CSS file (ai-response.css) styles the response box and keeps the code neatly organized.
  2. JavaScript for Real-Time Updates
    • I added a script to fetch the search query from the URL and detect the Odoo version.
    • The script calls the new /api/stream endpoint, listening to the server-sent events.
    • As text chunks arrive, the script renders them in real time with Marked.js, enabling markdown formatting on the fly.
  3. Version-Aware Replies
    • I used a straightforward approach to capture the correct Odoo version (e.g., 16.0, 17.0, 18.0) from the documentation page.
    • With this version passed along in the request, my chatbot only pulls content relevant to that specific Odoo release.

Results and Impact

The streaming implementation has transformed and revolutionized the documentation search experience as the video shown above:

  • Immediate response initiation instead of waiting for complete generation
  • Smooth, chat-like interaction with the documentation
  • Maintained accuracy while improving responsiveness
  • Version-specific results that match the user's context

Here's how the search process now works:

  1. User enters a search query
  2. AI response begins streaming immediately above search results
  3. Traditional search results remain available below
  4. All responses include citations and references to official documentation

Technical Highlights

  • StreamingResponse in FastAPI: I used this class to handle the server-sent events so the front end can receive data piece by piece.
  • Efficient Document Retrieval: My chatbot filters and retrieves the most relevant chunks from local Odoo docs, ensuring the conversation is grounded in actual Odoo documentation.
  • Simple to Integrate: By modifying search.html and adding minimal CSS/JS, I was able to incorporate the streaming feature smoothly into the existing docs interface.

Wrapping Up

The project is open source and welcomes contributions. Whether you're interested in improving the streaming implementation, enhancing the RAG system, or adding new features, visit my GitHub repository to get started.

By implementing streaming responses, Odoo Expert now provides a more natural and efficient documentation search experience. The combination of real-time responses and version-aware context makes it a powerful tool for Odoo developers and users alike.

Have questions or suggestions? Feel free to reach out through the GitHub repository or join the discussion in the issues section. Let's continue making Odoo documentation more accessible and user-friendly!