🧠 AI Document Processing Suite

🚀 Transform Your Documents into Intelligent, Queryable Knowledge

---

📋 Table of Contents

Click to Navigate

• ✨ Features
• 🎯 Use Cases
• 🏗️ Architecture
• 🚀 Quick Start
• 📊 Performance
• 🛠️ Tech Stack
• 📁 Project Structure
• 💻 Usage Examples
• 🔧 Configuration
• 📈 Benchmarks
• 🤝 Contributing
• 📄 License

---

✨ Features

Feature	Description	Status
🔍 Smart OCR	Extract text from scanned PDFs with 98%+ accuracy	✅ Ready
📑 Auto Classification	Categorize documents using AI-powered analysis	✅ Ready
🧠 RAG Pipeline	Answer questions using document context	✅ Ready
📊 Multi-Model Support	Compare TinyLlama, Phi-2, Mistral performance	✅ Ready
🚄 Optimized Processing	GPU-accelerated with batch processing	✅ Ready
🔐 Secure Handling	PII detection and redaction capabilities	🚧 Coming

🎬 Demo

---

🎯 Use Cases

🏦 Financial Services

• Mortgage Processing: Extract key terms from loan documents
• Contract Analysis: Identify important clauses and conditions
• Compliance Checking: Ensure documents meet regulatory requirements

📋 Legal Operations

• Document Discovery: Search through large document sets
• Contract Review: Extract and analyze key terms
• Due Diligence: Automated document verification

🏢 Enterprise Solutions

• Invoice Processing: Extract line items and totals
• Report Generation: Summarize lengthy documents
• Knowledge Management: Build searchable document repositories

---

🏗️ Architecture

graph LR
    A[📄 PDF Input] --> B[🔍 OCR Engine]
    B --> C[📑 Classifier]
    C --> D[🧩 Chunking]
    D --> E[🔢 Embeddings]
    E --> F[💾 Vector Store]
    F --> G[🤖 LLM + RAG]
    G --> H[💬 Answer]
    
    style A fill:#e1f5fe
    style H fill:#c8e6c9

Loading

---

🚀 Quick Start

Prerequisites

Requirement	Version
Python	3.8+
CUDA	11.8+ (for GPU)
RAM	8GB minimum
Storage	10GB free

🔧 Installation

# Clone the repository
git clone https://github.com/ShamsRupak/ai-doc-processing-suite.git
cd ai-doc-processing-suite

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Download models
python scripts/download_models.py

🎮 Quick Demo

from doc_processor import DocumentPipeline

# Initialize pipeline
pipeline = DocumentPipeline(model="tinyllama")

# Process document
result = pipeline.process("data/sample_loan.pdf")

# Query the document
answer = pipeline.query("What is the interest rate?")
print(f"Answer: {answer}")

---

📊 Performance

⚡ Processing Speed

Document Type	Pages	Processing Time	Accuracy
Loan Agreement	10	2.3s	98.5%
Bank Statement	5	1.1s	99.2%
Contract	15	3.5s	97.8%

🧠 Model Comparison

---

🛠️ Tech Stack

Category	Technologies
Core Framework
OCR & Extraction
NLP & RAG
Models

---

📁 Project Structure

📦 ai-doc-processing-suite/
├── 📂 src/
│   ├── 🔍 ocr/
│   │   ├── extract_text.py
│   │   └── preprocess.py
│   ├── 📑 classification/
│   │   ├── classifier.py
│   │   └── models.py
│   ├── 🧠 retrieval/
│   │   ├── rag_pipeline.py
│   │   ├── embeddings.py
│   │   └── vector_store.py
│   └── 🤖 llm/
│       ├── model_loader.py
│       └── prompts.py
├── 📂 data/
│   ├── sample_loan.pdf
│   ├── bank_statement.pdf
│   └── contract.pdf
├── 📂 tests/
├── 📂 notebooks/
│   └── demo.ipynb
├── 📋 requirements.txt
├── 🔧 config.yaml
└── 📖 README.md

---

💻 Usage Examples

📄 Basic Document Processing

Click to expand code example

from src.ocr import extract_text
from src.classification import DocumentClassifier
from src.retrieval import RAGPipeline

# Extract text from PDF
text = extract_text("path/to/document.pdf")

# Classify document
classifier = DocumentClassifier()
doc_type = classifier.classify(text)
print(f"Document type: {doc_type}")

# Setup RAG pipeline
rag = RAGPipeline(model="tinyllama")
rag.add_document(text, metadata={"type": doc_type})

# Query document
response = rag.query("What are the key terms?")
print(response)

🔍 Advanced Queries

Click to expand code example

# Complex multi-document analysis
pipeline = DocumentPipeline(
    model="mistral-7b",
    chunk_size=512,
    overlap=128
)

# Process multiple documents
documents = [
    "loan_agreement.pdf",
    "property_appraisal.pdf",
    "income_verification.pdf"
]

for doc in documents:
    pipeline.add_document(doc)

# Cross-document queries
questions = [
    "What is the total loan amount?",
    "Compare the appraised value with the loan amount",
    "Verify the borrower's income"
]

for q in questions:
    answer = pipeline.query(q)
    print(f"Q: {q}\nA: {answer}\n")

---

🔧 Configuration

Create a config.yaml file:

# Model Configuration
model:
  name: "tinyllama"
  quantization: "8bit"
  max_tokens: 512

# OCR Settings
ocr:
  engine: "tesseract"
  language: "eng"
  dpi: 300

# RAG Configuration
rag:
  chunk_size: 512
  chunk_overlap: 128
  retriever_k: 5
  
# Performance
performance:
  batch_size: 32
  use_gpu: true
  cache_embeddings: true

---

📈 Benchmarks

🏆 Model Performance Comparison

Model	Accuracy	Speed (docs/min)	Memory (GB)	Cost
TinyLlama 1.1B	85%	45	2.5	Free
Phi-2 2.7B	92%	30	4.0	Free
Mistral 7B	96%	15	8.0	Free
GPT-3.5	98%	60	API	$$$

---

🤝 Contributing

We love contributions! Please see our Contributing Guide for details.

👥 Contributors

---

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

---

⭐ Star History

🙏 Acknowledgments

Built with ❤️ using amazing open-source tools and libraries.

⬆ back to top