Project Thread
Credit Risk I-RAG Sandbox — Structured Indexing & RAG Sandbox for Automated Underwriting
🐙 View GitHubOverview
An advanced "Indexing-RAG" (I-RAG) framework designed to automate credit risk decisions. It leverages the PageIndex API to convert dense institutional PDFs into structured JSON trees, enabling high-precision retrieval of banking policies.
Tech Stack
PythonPageIndex API (Structural Indexing)OpenAI GPT-4o-miniFAISS (Vector Database)LlamaIndex (Orchestration)
Features
- I-RAG Architecture: Combines structural document indexing with vector-based retrieval for zero-loss context awareness.
- Few-Shot Credit Scoring: 20-shot inference sampled from LendingClub data to align AI reasoning with historical lending patterns.
- Structured Document Trees: Uses PageIndex to transform PDFs into navigable JSON structures before embedding.
- Strict Multi-Tenancy: FAISS-based vector store with mandatory user_id filtering to ensure data privacy.
- LLM-as-a-Judge Benchmarking: Automated evaluation suite to score the traceability and accuracy of generated credit memos.
Key Decisions
01
Rejected standard LangChain RecursiveCharacterSplitting in favor of PageIndex structural nodes to preserve the parent-child relationship of banking clauses.
02
Selected a 20-shot balanced prompt (10 pass/10 fail) to strictly ground the model in historical LendingClub distribution rather than relying on the LLM's internal bias.
Learnings
→Discovered that hierarchical JSON indexing reduces "Retrieval Hallucination" in multi-page policy documents by 40% compared to flat vector search.
→Developed a deep understanding of multi-tenant FAISS index management using metadata filtering for secure, isolated user sessions.