How to use the Text Chunk Splitter — LangChain
1. Paste your document into the input box (or pick a sample).
2. Choose a splitter — Recursive (LangChain default) works best for most text.
3. Tune chunk size and overlap; chunks update instantly.
4. Export as JSON, JSONL, LangChain Documents, or plain text — or download the file.
What is it?
Recursive Character Text Splitter is LangChain's default and the most robust strategy. It tries each separator in order (e.g. \n\n → \n → space → character) and falls back to the next level only when a piece is still too large. This preserves semantic structure — paragraphs stay together whenever possible.
Chunk size controls the maximum size of each chunk (in characters or tokens). Overlap copies N characters/tokens from the end of one chunk into the start of the next, ensuring sentences that fall at boundaries still appear in full context somewhere.
Most production RAG systems use 256–512 tokens with 10–20% overlap. Smaller chunks give higher retrieval precision; larger chunks give more context per result. Markdown splitting is ideal for documentation. Sentence splitting works well for Q&A over factual content.
Example
Retrieval-Augmented Generation (RAG) combines LLMs with external knowledge bases. In a RAG pipeline, the query is embedded and used to search a vector database for relevant passages. These passages are injected into the model's context window.
[ "Retrieval-Augmented Generation (RAG) combines LLMs with external knowledge bases. In a RAG pipeline, the query is embedded and used to search a vector database for relevant passages.", "...used to search a vector database for relevant passages. These passages are injected into the model's context window." ]
Overlap of 50 chars means each chunk shares ~50 chars with the previous — preventing context loss at boundaries.
Benefits
Use LangChain's Recursive Character Splitter — the industry standard for RAG
The Recursive splitter uses a priority list of separators ( → → space → char) to preserve semantic units. Paragraphs stay together unless they're too large — exactly how LangChain's RecursiveCharacterTextSplitter works.
Export directly as LangChain Documents, JSON, or JSONL
Export chunks as LangChain Document objects ({page_content, metadata}) ready to pass to vectorstore.add_documents(). Also supports raw JSON array and JSONL for any other vector database pipeline.
Code-aware splitting with Python, JavaScript, and Markdown presets
Separator presets ensure chunk boundaries fall at class, function, and def boundaries — not inside a function body. Critical for code search and code-aware RAG applications.
Tune chunk size, overlap, and length function with instant live preview
See exactly how your document splits before writing a single line of code. Change chunk size from 128 to 1024 tokens and watch the chunks update instantly — no need to run a Python script to verify.
Visual statistics help you select the optimal chunking strategy
See chunk count, average size, min/max per chunk, and total token count at a glance. Quickly identify strategies that produce too few large chunks or too many tiny fragments.
Split Markdown documentation by headers for structured retrieval
The Markdown splitter creates one chunk per header section, keeping each topic self-contained. Each retrieved chunk starts with its header — so the LLM always knows which section it came from.
Frequently asked questions
What chunk size should I use for RAG?
256–512 tokens is the most common range. Smaller chunks (128 tokens) give higher retrieval precision. Larger chunks (1024 tokens) give more context per result. Always measure retrieval quality on your specific queries.
Why does LangChain use Recursive splitting by default?
Recursive splitting tries to keep semantically related text together by using a priority list of separators. It only falls back to finer-grained splitting (by character) when coarser splits produce chunks that are still too large. This preserves paragraph and sentence structure better than a single-separator approach.
What is the difference between character and token length?
Character length counts every character including spaces. Token length is an estimate based on 4 chars per token (the standard English approximation used by OpenAI). Token counting matters for API cost and context-window limits. Character counting is faster and deterministic.
What is LangChain Document format?
LangChain represents chunks as Document objects with a page_content field (the text) and a metadata dict. Exporting in this format lets you paste the JSON directly into LangChain pipelines: vectorstore.add_documents(json.load(f)).
How do I handle code files?
Switch to the Recursive splitter and pick the Python or JavaScript separator preset. These presets use code-aware separators (class, def, function, const) so chunk boundaries fall at natural code boundaries rather than the middle of a function body.