Stop Pasting URLs Into ChatGPT: This Tool Turns Any Web Page Into LLM-Ready Text

You know the drill. You find a great article, a PDF report, or a documentation page, and you want your LLM to analyze it. So you copy the text, paste it in, hit the token limit, trim it down, and eventually give up. Or you try to feed it a URL, and the LLM either can't access it or returns a garbled mess of HTML tags and navigation menus. Reader from Jina AI solves this with a dead simple approach: just prepend a URL prefix, and it handles the rest.

Reader is an open-source service that converts any URL into clean, LLM-friendly markdown. It also offers a search endpoint that lets your LLM query the live web. You can use it right now, for free, in production, without signing up for anything.

What It Does

Reader does two things, and it does them through two simple API endpoints. The first is the read endpoint at https://r.jina.ai/. You take any URL, prepend that prefix, and Reader returns a markdown version of the page content. The second is the search endpoint at https://s.jina.ai/. You append a search query, and it returns web search results as markdown, giving your LLM access to current information without needing a separate search API key.

Under the hood, Reader is more sophisticated than it looks. For web pages, it uses either headless Chrome for full rendering or a lightweight curl-impersonate approach, and it picks intelligently between the two. For PDFs, it uses PDF.js to extract content. For Microsoft Office documents (Word, Excel, PowerPoint), it converts them through LibreOffice first. And for images, it runs them through a vision-language model to generate captions, so even text-only LLMs get enough context to reason about what's in the picture.

The repository contains the open-source branch of the code that powers r.jina.ai and s.jina.ai. It runs in stateless mode by default, with an optional bucket-cached mode using MinIO or S3-compatible storage if you run it with Docker Compose. The SaaS version uses MongoDB Atlas for storage, but that layer is stripped from the open-source code.

Why It's Cool

The idea is almost too simple, but that's exactly why it works well.

• Zero setup. You don't need an API key, an account, or a registration. You just construct a URL. That's it. For developers who want to quickly prototype an LLM pipeline or just test whether a source works with their model, this removes all friction.

• It handles the messy stuff. Web pages are full of noise—navigation bars, sidebars, cookie consent popups, ads. Reader strips all of that and returns just the content as clean markdown. It also handles PDFs and Office documents, which are notoriously annoying to parse programmatically. You can even POST binary files directly via the file body field, so you don't need to host them somewhere first.

• The search endpoint is a hidden gem. Most LLM applications struggle with knowledge cutoff dates. The search endpoint at s.jina.ai lets you query the live web and get results back in a format your LLM can actually use. It's a surprisingly elegant solution for grounding LLM outputs in current information without building a whole web search pipeline yourself.

• It's actively maintained by a company that uses it. Jina AI runs Reader as one of their core products. That means it's not a side project that might get abandoned. It's free, stable, and scalable, and they're transparent about rate limits. The open-source branch stays synchronized with the SaaS code, so you're not getting a stale version.

How to Try It

You don't need to install anything to try Reader. Just open your browser and visit these URLs:

For reading a page as markdown:

https://r.jina.ai/https://en.wikipedia.org/wiki/Artificial_intelligence

For searching the web:

https://s.jina.ai/Who%20will%20win%202024%20US%20presidential%20election%3F

If you want to run it locally, clone the repository and use Docker Compose:

git clone https://github.com/jina-ai/reader
cd reader
docker compose up

The project runs in stateless mode out of the box. If you want bucket caching, the Docker Compose setup includes MinIO. Check the repository's README for the full local development guide.

There's also a Google Colab notebook linked in the README if you want to experiment with full website fetching without setting up a local environment.

Final Thoughts

Reader is one of those tools that feels obvious in retrospect. It solves a specific, annoying problem—getting clean text from arbitrary URLs into LLMs—and it solves it with minimal ceremony. It's not trying to be a platform or a framework. It's just a utility that does one thing well.

This is best for anyone building LLM-powered applications that need to ingest web content. Whether you're working on a RAG pipeline, an agent that browses the web, or just experimenting with prompt engineering, Reader removes a bunch of boilerplate. And because it's open source, you can inspect exactly what it does, modify it, or run it yourself if you need to avoid external dependencies.

The repository is at github.com/jina-ai/reader. Give it a try with a URL you know is messy to parse. You might be surprised how clean the output is.

Follow @githubprojects for more developer tools and open source projects.