Provides Vaadin Documentation and help with development tasks
A sophisticated, hierarchically-aware Retrieval-Augmented Generation (RAG) system for Vaadin documentation that understands document structure, provides framework-specific filtering, and enables intelligent parent-child navigation through documentation sections.
This project provides an advanced RAG system with enhanced hybrid search that:
vaadin-documentation-services/
├── packages/
│ ├── core-types/ # Shared TypeScript interfaces
│ ├── 1-asciidoc-converter/ # AsciiDoc → Markdown + metadata extraction
│ ├── 2-embedding-generator/ # Markdown → Vector database with hierarchical chunking
│ └── mcp-server/ # MCP server with hierarchical navigation
├── package.json # Bun workspace configuration
└── PROJECT_PLAN.md # Complete project documentationflowchart TD
subgraph "Step 1: Documentation Processing"
VaadinDocs["📚 Vaadin Docs<br/>(AsciiDoc)"]
Converter["🔄 AsciiDoc Converter<br/>• Framework detection<br/>• URL generation<br/>• Markdown output"]
Processor["⚡ Embedding Generator<br/>• Hierarchical chunking<br/>• Parent-child relationships<br/>• OpenAI embeddings"]
end
subgraph "Step 2: Agent Integration"
Pinecone["🗄️ Pinecone Vector DB<br/>• Rich metadata<br/>• Hierarchical relationships<br/>• Framework tags"]
MCP["🤖 MCP Server<br/>• search_vaadin_docs<br/>• get_full_document<br/>• Full document retrieval"]
IDEs["💻 IDE Assistants<br/>• Context-aware search<br/>• Hierarchical exploration<br/>• Framework-specific help"]
end
VaadinDocs --> Converter
Converter --> Processor
Processor --> Pinecone
Pinecone <--> MCP
MCP <--> IDEs
classDef processing fill:#e1f5fe,stroke:#01579b,stroke-width:2px
classDef storage fill:#f3e5f5,stroke:#4a148c,stroke-width:2px
classDef api fill:#e8f5e8,stroke:#2e7d32,stroke-width:2px
classDef agent fill:#fff3e0,stroke:#e65100,stroke-width:2px
class VaadinDocs,Converter,Processor processing
class Pinecone storage
class MCP api
class IDEs agent# Clone and install dependencies
git clone https://github.com/vaadin/vaadin-documentation-services
cd vaadin-documentation-services
bun install# Create .env file with your API keys
echo "OPENAI_API_KEY=your_openai_api_key" > .env
echo "PINECONE_API_KEY=your_pinecone_api_key" >> .env
echo "PINECONE_INDEX=your_pinecone_index" >> .env# Convert AsciiDoc to Markdown with metadata
cd packages/1-asciidoc-converter
bun run convert
# Generate embeddings and populate vector database
cd ../2-embedding-generator
bun run generateThe MCP server is deployed and available remotely via HTTP transport at:
https://mcp.vaadin.com/
Configure your IDE assistant to use the Streamable HTTP transport:
import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";
const transport = new StreamableHTTPClientTransport(
new URL("https://mcp.vaadin.com/")
);packages/core-types/)Shared TypeScript interfaces used across all packages:
DocumentChunk: Core documentation chunk structureRetrievalResult: Search result with relevance scoringFramework: Type-safe framework definitionspackages/1-asciidoc-converter/)Converts Vaadin AsciiDoc documentation to Markdown with metadata:
cd packages/1-asciidoc-converter
bun run convert # Convert all documentation
bun run test # Run framework detection testspackages/2-embedding-generator/)Creates vector embeddings with hierarchical relationships:
cd packages/2-embedding-generator
bun run generate # Generate embeddings from Markdown
bun run test # Run chunking and relationship testspackages/mcp-server/)Model Context Protocol server for IDE assistant integration:
search_vaadin_docs and get_full_documentcd packages/mcp-server
bun run build # Build for distribution
bun run test # Run document-based tests
Available Tools:
search_vaadin_docs: Search with semantic and keyword matchingget_full_document: Retrieve complete documentation pagesget_vaadin_version: Get latest Vaadin version and release timestampEach package includes comprehensive test suites:
# Test individual packages
cd packages/1-asciidoc-converter && bun run test
cd packages/2-embedding-generator && bun run test
cd packages/mcp-server && bun run testThe MCP server is available at:
https://mcp.vaadin.com/https://mcp.vaadin.com/healthAutomated via GitHub Actions:
This project uses Bun workspaces for package management:
bun install # Install all dependencies
bun run build # Build all packages
bun run test # Test all packagespackages/core-types/packages/1-asciidoc-converter/ or packages/2-embedding-generator/packages/mcp-server/This project successfully delivered:
✅ Sophisticated RAG System: Replaced naive implementation with hierarchically-aware search
✅ Enhanced User Experience: Agents can now navigate from specific details to broader context
✅ Production Quality: Clean architecture, comprehensive testing, and error handling
✅ Framework Intelligence: Accurate Flow/Hilla content separation with common content inclusion
✅ Developer Integration: Seamless IDE assistant integration via MCP protocol
The system now provides intelligent, context-aware documentation search that understands the hierarchical structure of Vaadin documentation and enables sophisticated agent interactions.
MIT - See license file for details.
Built with ❤️ for the Vaadin developer community