Meet Token Saver: An Open-Supply MCP Extension Utilizing Native Hybrid RAG to Reduce Claude PDF Token Prices 90-99%

AI builders, researchers, and professionals ceaselessly hit a irritating wall when analyzing massive paperwork with LLMs: the hidden, compounding value of context home windows. Pasting a 200-page PDF right into a chat isn’t a one-time cost. As a result of the dialog historical past is re-sent to the mannequin on each single flip, that huge doc is paid for once more with each follow-up query.

Marktechpost AI group simply launched Token Saver, an open-source Mannequin Context Protocol (MCP) extension for Claude Desktop (MIT licensed, at the moment at v1.0). It was developed at Marktechpost AI Media Inc by Arnav Rai (CS pupil at Rochester Institute of Expertise) throughout his internship at Marktechpost, supervised by Jean-marc Mommessin and Asif Razzaq. Token Saver basically adjustments how Claude interacts with native paperwork. By implementing a Native Hybrid RAG (system instantly in your machine, it lets you ask questions on huge PDFs with out ever importing the precise file to the mannequin.

Token consumption is slashed by 92% to 99%, privateness is assured, and setup requires precisely zero Python environments or terminal configurations.

The way it seems!

The Hidden Token Drain of Massive PDFs

Most customers assume that once they drop a PDF into Claude, it merely extracts the textual content. In actuality, Claude’s default conduct converts every web page into a picture to protect charts and layouts, whereas individually extracting textual content. Earlier than a single picture token is even counted, the textual content alone can run 1,500 to three,000 tokens per web page.

Whereas Immediate Caching and Claude Initiatives assist soften this blow, they don’t remedy the core problem: the complete doc nonetheless crosses the boundary to the supplier’s servers. Moreover, in case you solely want two related paragraphs from a 1,000-page textbook, forcing the LLM to look the complete 1,000 pages itself is each inefficient and liable to hallucination.

How Native Hybrid RAG Solves the Downside

Marktechpost’s Token Saver acts as an area MCP server : a light-weight background program in your machine that Claude can name as a device. The PDF by no means leaves your exhausting drive.

If you ask a query, Token Saver makes use of Native Hybrid RAG to search out the reply. Hybrid RAG is the gold normal for doc retrieval as a result of it combines two highly effective search strategies:

1. Key phrase Matching (BM25): Runs over SQLite’s built-in FTS5 search (weighted at 0.4) to search out actual terminology.
2. Semantic Search (Cosine Similarity): Makes use of an area all-MiniLM-L6-v2 embedding mannequin (weighted at 0.6) to know the which means of your query, fairly than simply matching actual phrases.

By mixing these two approaches domestically, the server searches the file and arms Claude solely the extremely related passages. Claude then synthesizes the reply, citing the precise web page numbers and offering a working tally of the tokens you simply saved.

The 8-Stage Processing Pipeline

Token Saver runs fully inside a single, long-running native course of. Earlier than any textual content reaches Claude, the native Hybrid RAG pipeline executes eight fast steps:

1. Extract: Makes use of pypdfium2 (with pypdf as a fallback) to tug textual content.
2. Chunk: Splits textual content into 180-word passages, overlapping by 40 phrases so context isn’t blindly minimize.
3. Rating (Hybrid RAG): Evaluates chunks utilizing the blended BM25 and native embedding mannequin.
4. Gate: Enforces a high quality threshold. Passages sharing no actual key phrases should move a semantic similarity flooring of 0.25 to qualify.
5. Deduplicate: Drops near-identical passages.
6. Trim: Narrows the passage down strictly to the sentences that reply the person’s immediate.
7. Finances: Caps the overall payload despatched to Claude at 8,000 characters.
8. Envelope: Wraps the retrieved textual content in a document-chunk component containing the supply file and exact web page quantity.

(Be aware: The embedding mannequin is non-obligatory however beneficial. If it fails to load, the system gracefully falls again to keyword-only matching).

The Numbers: 99% Financial savings at Scale

As a result of the Hybrid RAG pipeline caps the returned textual content slice, the token financial savings develop exponentially with the scale of the doc. Right here is how Token Saver carried out in benchmarking (measured by way of tiktoken, assuming one query per doc):

Disclaimer: Token counts use cl100k_base as a stand-in, and baselines assume the entire doc is charged on each search.

For professionals working repeatedly with courtroom opinions, technical requirements, monetary reviews, or dense textbooks, Token Saver eliminates the repetitive token tax.

Safety and Native Privateness

For enterprise customers and authorized professionals, information privateness is non-negotiable. Token Saver’s safety mannequin rests on three pillars:

• Zero Uploads: The doc by no means leaves your machine.
• Folder Allowlisting: You configure a particular folder for Token Saver. The server will actively refuse to learn recordsdata outdoors of this designated listing.
• Community Isolation: The server communicates with Claude Desktop over normal I/O (stdio) solely. There are not any listening community ports, drastically lowering the assault floor.

Mannequin Efficiency: Sonnet vs. Opus

Marktechpost’s Token Saver works throughout all three Claude 3.5 tiers, however testing revealed distinct behaviors:

• Claude 3.5 Sonnet (Really helpful): The wise default. It handles free file references completely, asks clarifying questions if two recordsdata have related names, and appropriately applies the retrieved web page numbers.
• Claude 3 Opus: Excels at advanced paperwork with a number of voices (e.g., authorized rulings with majority opinions and dissents). Opus is sensible sufficient to flag when it infers an attribution primarily based on context fairly than specific textual content.

Getting Began in Minutes

Not like many open-source AI instruments that require advanced Python environments and JSON configurations, Token Saver is packaged as a single .mcpb bundle.

1. Obtain token-saver-ccr.mcpb from the challenge’s GitHub Releases web page.
2. Open Claude Desktop -> Settings -> Extensions -> Set up extension.
3. Allow the extension, click on Configure, and choose a devoted folder the place you retain your reference PDFs.
4. On the first immediate, select At all times enable.

The extension is not going to run till a folder is chosen, as a result of that single selection serves three functions without delay.

That folder is value a second’s thought. It units the boundary of what may be learn, it’s what enables you to ask for a file by title as a substitute of typing a path, and it’s the record the server reveals Claude when a dialog begins. A small folder holding solely what you plan to ask about works significantly higher than pointing it in any respect of Paperwork.

Key Takeaways

• Huge Price Discount: By utilizing Native Hybrid RAG to move solely related paragraphs to the LLM, token prices drop by as much as 99%. The longer the chat, the extra you save.
• Verifiable Accuracy: As a result of Token Saver attaches actual web page numbers to each retrieved chunk, you possibly can immediately confirm Claude’s claims by opening your native PDF.
• Absolute Privateness: The boundary is your native machine. Your proprietary information isn’t uploaded to an AI supplier’s server.
• Frictionless Setup: No Python required. A easy .mcpb file set up will get you working in Claude Desktop immediately.

Try the GitHub Repo.

References: MCP Bundle format | all-MiniLM-L6-v2 | pdfkb-mcp | wesleygriffin/pdfrag | Corpora: Dobbs v. Jackson Ladies’s Well being Group; Berkshire Hathaway 2023 Annual Report

Arnav is at the moment a pupil at Rochester Institute of Expertise pursuing a Bachelor’s diploma in Pc Science and a minor in Economics with hands-on backend growth expertise, and is a contributor at Marktechpost, the place he writes about AI/ML analysis.

Jean-marc is a profitable AI enterprise govt .He leads and accelerates development for AI powered options and began a pc imaginative and prescient firm in 2006. He’s a acknowledged speaker at AI conferences and has an MBA from Stanford.

Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is dedicated to harnessing the potential of Synthetic Intelligence for social good. His most up-to-date endeavor is the launch of an Synthetic Intelligence Media Platform, Marktechpost, which stands out for its in-depth protection of machine studying and deep studying information that’s each technically sound and simply comprehensible by a large viewers. The platform boasts of over 2 million month-to-month views, illustrating its recognition amongst audiences.