Project: PostHog
AI Docs Chat
PostHog
An AI-powered documentation chat system that allows users to upload or link to their documentation and then ask questions about it, receiving contextually relevant AI-generated answers.
Project Demo
Project Metrics
Project Details
The AI Docs Chat is a complete end-to-end solution for reducing support ticket volume by providing immediate, accurate answers to product questions without human intervention. It uses vector search, semantic reranking, and AI response generation to help PostHog's growing support team handle increasing customer questions.
Business Value
- Customer Support Automation
- Support Team Efficiency
- Scaling Support
Key Features
Document Processing Pipeline
Web crawler using Jina.ai API to retrieve documentation, intelligent chunking to segment documents, and vector embedding generation using VoyageAI API.
Vector Search with Reranking
Two-stage retrieval with initial vector search followed by semantic reranking, context preservation with references between adjacent chunks, and multi-tenant support.
AI Response Generation
Claude integration for natural language responses, context-aware answers using relevant document chunks, and citation support with source URLs and titles.
Chat Interface
Customer selection dropdown for switching between contexts, conversation history maintenance, and confidence indicators for response reliability.
Technologies Used
Frontend
Backend
DevOps
Other
Challenges & Solutions
Efficient Document Chunking
Implemented intelligent chunking with 1000 token chunks and 200% overlap, with special handling for code blocks and examples. Used Jina.ai segmentation API with custom overlap implementation.
Vector Search Quality
Implemented two-stage retrieval with reranking: initial vector search to get top 20 candidates, Jina reranker to refine results based on semantic understanding, and selection of top 5 results for context generation.
API Rate Limits
Implemented batch processing with configurable concurrency, rate limiting, and resume capability for interrupted processes. Added retry logic with exponential backoff for API failures.
Key Learnings
- Vector Database Implementation: PostgreSQL with pgvector provides a powerful, cost-effective solution for vector operations without requiring specialized vector databases.
- Document Chunking Strategies: Technical documentation benefits from larger chunks (1000 tokens) with substantial overlap (200%) to preserve context.
- Two-Stage Retrieval: Combining vector search with semantic reranking dramatically improves result quality.
- Multi-Tenant Architecture: Customer_id partitioning enables building systems that scale to multiple customers while maintaining data isolation.