Lesson about using LangChain's load_summarize_chain with Ollama and long texts to summarize
In this repo, we explore two approaches of text summarization: map-reduce and refine.
This repository demonstrates how to use LangChain's load_summarize_chain with Ollama to summarize long texts efficiently.
The map-reduce approach involves breaking down the text into smaller chunks, summarizing each chunk, and then combining these summaries into a final summary.
The refine approach iteratively improves the summary by refining it through multiple passes over the text.
To install the necessary dependencies, run:
pip install -r requirements.txtOllamato run local LLM APILlama 3.2-3Bfrom Meta, to use as AI brain. See on Ollama page.LangChainas framework for LLM apptiktokenlibrary to estimate token counts
Feel free to copy and distribute, but we appreciate you giving us credits.
👍 Like | 🔗 Share | 📢 Subscribe | 💬 Comments | ❓ Questions