Picture by Writer
# Introduction
Do you know that a big portion of helpful data nonetheless exists in unstructured textual content? For instance, analysis papers, scientific notes, monetary experiences, and so on. Extracting dependable and structured data from these texts has all the time been a problem. LangExtract is an open-source Python library (launched by Google) that solves this drawback utilizing massive language fashions (LLMs). You may outline what to extract by way of easy prompts and some examples, after which it makes use of LLMs (like Google’s Gemini, OpenAI, or native fashions) to drag out that data from paperwork of any size. One other factor that makes it helpful is its help for very lengthy paperwork (via chunking and multi-pass processing) and interactive visualization of outcomes. Let’s discover this library in additional element.
# 1. Putting in and Setting Up
To put in LangExtract domestically, first guarantee you might have Python 3.10+ put in. The library is accessible on PyPI. In a terminal or digital atmosphere, run:
For an remoted atmosphere, you could first create and activate a digital atmosphere:
python -m venv langextract_env
supply langextract_env/bin/activate # On Home windows: .langextract_envScriptsactivate
pip set up langextract
There are different choices from supply and utilizing Docker as properly which you could test from right here.
# 2. Setting Up API Keys (for Cloud Fashions)
LangExtract itself is free and open-source, however in the event you use cloud-hosted LLMs (like Google Gemini or OpenAI GPT fashions), you will need to provide an API key. You may set the LANGEXTRACT_API_KEY atmosphere variable or retailer it in a .env file in your working listing. For instance:
export LANGEXTRACT_API_KEY="YOUR_API_KEY_HERE"
or in a .env file:
cat >> .env << 'EOF'
LANGEXTRACT_API_KEY=your-api-key-here
EOF
echo '.env' >> .gitignore
On-device LLMs by way of Ollama or different native backends don’t require an API key. To allow OpenAI, you’ll run pip set up langextract[openai], set your OPENAI_API_KEY, and use an OpenAI model_id. For Vertex AI (enterprise customers), service account authentication is supported.
# 3. Defining an Extraction Process
LangExtract works by you telling it what data to extract. You do that by writing a transparent immediate description and supplying a number of ExampleData annotations that present what an accurate extraction appears like on pattern textual content. For example, to extract characters, feelings, and relationships from a line of literature, you would possibly write:
import langextract as lx
immediate = """
Extract characters, feelings, and relationships so as of look.
Use actual textual content for extractions. Don't paraphrase or overlap entities.
Present significant attributes for every entity so as to add context."""
examples = [
lx.data.ExampleData(
text="ROMEO. But soft! What light through yonder window breaks? ...",
extractions=[
lx.data.Extraction(
extraction_class="character",
extraction_text="ROMEO",
attributes={"emotional_state": "wonder"}
),
lx.data.Extraction(
extraction_class="emotion",
extraction_text="But soft!",
attributes={"feeling": "gentle awe"}
)
]
)
]
These examples (taken from LangExtract’s README) inform the mannequin precisely what sort of structured output is predicted. You may create comparable examples to your area.
# 4. Operating the Extraction
As soon as your immediate and examples are outlined, you merely name the lx.extract() perform. The important thing arguments are:
text_or_documents: Your enter textual content, or an inventory of texts, or perhaps a URL string (LangExtract can fetch and course of textual content from a Gutenberg or different URL).prompt_description: The extraction directions (a string).examples: An inventory ofExampleDatathat illustrate the specified output.model_id: The identifier of the LLM to make use of (e.g."gemini-2.5-flash"for Google Gemini Flash, or an Ollama mannequin like"gemma2:2b", or an OpenAI mannequin like"gpt-4o").- Different non-obligatory parameters:
extraction_passes(to re-run extraction for greater recall on lengthy texts),max_workers(to do parallel processing on chunks),fence_output,use_schema_constraints, and so on.
For instance:
input_text=""'JULIET. O Romeo, Romeo! wherefore artwork thou Romeo?
Deny thy father and refuse thy title;
Or, if thou wilt not, be however sworn my love,
And I will not be a Capulet.
ROMEO. Shall I hear extra, or shall I communicate at this?
JULIET. 'Tis however thy title that's my enemy;
Thou artwork thyself, although not a Montague.
What’s in a reputation? That which we name a rose
By some other title would odor as candy.'''
consequence = lx.extract(
text_or_documents=input_text,
prompt_description=immediate,
examples=examples,
model_id="gemini-2.5-flash"
)
This sends the immediate and examples together with the textual content to the chosen LLM and returns a Outcome object. LangExtract mechanically handles tokenizing lengthy texts into chunks, batching calls in parallel, and merging the outputs.
# 5. Dealing with Output and Visualization
The output of lx.extract() is a Python object (usually known as consequence) that comprises the extracted entities and attributes. You may examine it programmatically or put it aside for later. LangExtract additionally supplies helper features to avoid wasting outcomes: for instance, you may write the outcomes to a JSONL (JSON Strains) file (one doc per line) and generate an interactive HTML evaluation. For instance:
lx.io.save_annotated_documents([result], output_name="extraction_results.jsonl", output_dir=".")
html = lx.visualize("extraction_results.jsonl")
with open("viz.html", "w") as f:
f.write(html if isinstance(html, str) else html.knowledge)
This writes an extraction_results.jsonl file and an interactive viz.html file. The JSONL format is handy for big datasets and additional processing, and the HTML file highlights every extracted span in context (color-coded by class) for simple human inspection like this:
# 6. Supporting Enter Codecs
LangExtract is versatile about enter. You may provide:
- Plain textual content strings: Any textual content you load into Python (e.g. from a file or database) could be processed.
- URLs: As proven above, you may move a URL (e.g. a Mission Gutenberg hyperlink) as
text_or_documents="https://www.gutenberg.org/information/1513/1513-0.txt". LangExtract will obtain and extract from that doc. - Record of texts: Move a Python checklist of strings to course of a number of paperwork in a single name.
- Wealthy textual content or Markdown: Since LangExtract works on the textual content stage, you could possibly additionally feed in Markdown or HTML in the event you pre-process it to uncooked textual content. (LangExtract itself doesn’t parse PDFs or photographs, it’s good to extract textual content first.)
# 7. Conclusion
LangExtract makes it simple to show unstructured textual content into structured knowledge. With excessive accuracy, clear supply mapping, and easy customization, it really works properly when rule-based strategies fall brief. It’s particularly helpful for complicated or domain-specific extractions. Whereas there’s room for enchancment, LangExtract is already a powerful software for extracting grounded data in 2025.
Kanwal Mehreen is a machine studying engineer and a technical author with a profound ardour for knowledge science and the intersection of AI with medication. She co-authored the e book “Maximizing Productiveness with ChatGPT”. As a Google Technology Scholar 2022 for APAC, she champions variety and tutorial excellence. She’s additionally acknowledged as a Teradata Range in Tech Scholar, Mitacs Globalink Analysis Scholar, and Harvard WeCode Scholar. Kanwal is an ardent advocate for change, having based FEMCodes to empower ladies in STEM fields.
