DocuChat — Multimodal PDF Q&A
CompletedTeam projectInternship CapstoneJul 2025
RAG assistant over complex PDFs (text, tables, figures) that highlights the cited regions on the page. Team project — I led PDF processing and integration.
my part: Led the PDF processing module and system integration.
About
DocuChat is a team-built RAG assistant from the Mindsane internship capstone. It answers questions over complex PDFs — text, tables and figures — and highlights the cited source regions on the page in a Gradio UI.
My part
I led the PDF processing module and system integration:
- Layout-aware extraction with Qwen-VL: element classification, captioning and normalized bounding boxes
- Pydantic/JSON schemas as the contract between modules
- Integration with retrieval over MiniLM embeddings in Pinecone (title, description and content indexes) with cross-encoder reranking, answered by a locally served Hermes 2 Pro (Llama 3) model via llama.cpp and LangChain
The other modules were built by my teammates.
Tech
- Qwen-VL
- Pydantic
- Pinecone
- MiniLM
- cross-encoder
- llama.cpp
- LangChain
- Gradio