How to Run Local AI Web Scraping with Obscura and Ollama
A guide to pairing Obscura's Rust headless browser with a local Ollama model for offline web scraping and summarization.
What is Obscura and why does it matter for AI scraping?
Obscura is an open-source headless browser written in Rust that renders JavaScript-heavy pages without the overhead of Chrome. Most modern websites need real JavaScript execution to display content, which is why scraping pipelines usually reach for Puppeteer or Playwright driving a full headless Chrome instance. That approach works, but it costs you 200+ megabytes of RAM per instance, slow startup times, and a dependency chain that includes Node and a full browser binary. Obscura ships as a single self-contained binary with no Chrome, no Node, and no external dependencies. According to a demonstration by AI educator Fahad Mirza, it uses around 30 megabytes of memory, loads pages in about 85 milliseconds, and starts instantly.
That combination matters for anyone building AI data pipelines. Scraping at scale means running many browser instances in parallel, and every megabyte and millisecond compounds. A lighter, faster scraper means you can run more concurrent jobs on the same hardware, which is directly relevant to collecting web data for model fine-tuning, retrieval pipelines, or agent tooling.
TL;DR
- Obscura is a Rust-based headless browser that executes real JavaScript through V8 while using roughly 30 megabytes of memory and loading pages in about 85 milliseconds, far lighter than Chrome-based tools like Puppeteer or Playwright.
- It ships as a single binary with no Chrome, Node, or other runtime dependencies, so setup is just downloading and extracting an archive.
- The core workflow scrapes a page and outputs clean markdown, which can be piped directly into a local language model for summarization or analysis.
- A local Ollama model handles the reading, meaning the entire pipeline, from page fetch to summarization, runs offline with nothing leaving the machine.
- Obscura supports MCP (Model Context Protocol), letting an AI agent call it as a tool over standard input/output rather than relying on a manual scrape-then-pipe script.
- The tool is not fully hands-off, scraped markdown can include page headers, footers, and navigation clutter that need pre- or post-processing in a production pipeline.
- The architecture is layered into eight Rust crates, each handling one function (HTTP fetch, HTML parsing, JavaScript execution, session routing), with calls flowing strictly downward through the stack.
How does the Obscura and Ollama pipeline actually work?
The pipeline has two distinct halves that hand off through plain text. Obscura does the scraping half: it fetches a URL, runs any inline JavaScript through its embedded V8 engine, parses the resulting HTML, and converts the page into markdown. That markdown gets written to a file or piped directly as text.
The second half is a local model running through Ollama, an open-source runtime for serving language models on your own hardware. In the demonstrated setup, the markdown output from Obscura was piped straight into a Qwen 3.6 model with 27 billion parameters, running locally on a machine with an Nvidia RTX A6000 GPU (48 GB VRAM). The prompt asked the model to summarize a blog post in five bullet points. Because Qwen 3.6 is a reasoning model, it visibly “thinks” before returning the final summary, but the round trip from scrape to summary completed in well under a minute for a single blog post.
The key requirement on the model side is tool use support. Any model with tool-calling enabled can sit in this pipeline, not just Qwen. That flexibility matters because it means teams can swap in whatever locally-hosted model fits their VRAM budget and licensing needs, from smaller distilled models to larger dense ones, as long as Ollama can serve it and it supports tool calls.
What does the installation process look like?
Getting Obscura running does not require compiling Rust code or installing a toolchain. The binary is downloaded and extracted with a standard tar command using the xzf flags, and that’s the entire setup. A quick sanity check command confirms the binary runs correctly.
From there, basic usage is a single command: point Obscura at a URL, tell it to output markdown, and save the result to a file. In one test, scraping an entire personal website (multiple blog posts, navigation tabs, and summaries) produced a 267-line markdown file in roughly 15 seconds, with memory usage barely moving. Harmless warnings, like an inline script error from a theme’s dark mode toggle, did not stop the scrape from completing.
Piping that output into Ollama is just a shell pipe: Obscura’s markdown becomes the input to an ollama run command (or an equivalent API call) with a prompt appended, such as asking for a five-bullet summary. For repeated use, this can be wrapped into a small reusable script that accepts any URL as an argument, scrapes it with Obscura, and forwards the markdown to the local model automatically.
Why does the internal architecture stay so lightweight?
Other agents ship a demo. Remy ships an app.
Real backend. Real database. Real auth. Real plumbing. Remy has it all.
Obscura’s speed and low memory footprint trace back to how it’s structured internally. The project is split into eight separate Rust crates (packages), each responsible for one layer of the request lifecycle, and calls only flow downward through those layers, never sideways. A request starts when a Puppeteer-compatible client sends a WebSocket frame to Obscura’s CDP (Chrome DevTools Protocol) server. That server routes the request by session ID to a dispatcher, which invokes a page-navigation handler. The navigation layer then fans out to three lower-level packages: one handles the raw HTTP fetch, one parses the returned HTML into a tree structure, and one executes the page’s inline JavaScript through V8.
Because every page shares a single, single-threaded V8 isolate, a lock serializes JavaScript execution across concurrent sessions. This keeps the design simple and predictable, though it also means JavaScript-heavy workloads are processed one at a time within that isolate rather than in true parallel threads. The tradeoff favors a small, auditable codebase over maximum raw concurrency, which fits Obscura’s goal of staying lightweight rather than replacing every capability of a full browser engine.
Notably, Obscura exposes a Chrome DevTools Protocol compatible interface, so tools built for Puppeteer can often talk to it with minimal changes, even though there’s no actual Chrome underneath.
Is Obscura ready for production scraping pipelines?
It handles the core job, page fetch and JavaScript execution, well, and does so with a fraction of the resource cost of Chrome-based tools. But scraped markdown output is raw. It can include site navigation, footers, cookie banners, and other boilerplate mixed in with the actual content. Anyone deploying this in production needs a cleanup step, whether that’s rule-based stripping of known boilerplate patterns or a secondary pass through a model to extract just the relevant text.
The other consideration is scope. Obscura is a browser engine, not a full scraping framework with built-in rate limiting, proxy rotation, or anti-bot evasion. Teams running large-scale scraping operations against sites with active bot detection will still need to layer their own infrastructure around it. Where Obscura clearly wins is in scenarios where the priority is running many lightweight scrape jobs cheaply and quickly, such as gathering intranet documents or internal web pages for fine-tuning a custom model, a use case where Obscura’s low overhead per instance directly translates into being able to scrape more pages in less time on the same hardware.
What role does MCP play in this setup?
Obscura also speaks the Model Context Protocol (MCP), an open standard that lets AI agents call external tools in a structured way. Instead of manually running a scrape command and piping the output into a model, an agent can talk to Obscura’s MCP server directly over standard input/output (STDIO). In a demonstrated example, an agent issued an MCP call telling Obscura to navigate to a specific blog URL and return the page as markdown. Obscura confirmed the page title, executed the scrape, and returned the markdown directly to the calling agent, which could then forward it to a local model exactly as in the manual pipeline.
This matters because it turns Obscura from a command-line utility into a tool an autonomous agent can invoke on demand, deciding for itself when to fetch a page as part of a longer task, rather than requiring a human to script each scrape step manually.
Frequently Asked Questions
What is Obscura used for?
Obscura is a headless browser used to fetch and render JavaScript-heavy web pages, then convert them into clean markdown text. It’s commonly paired with local or cloud language models to summarize or analyze scraped content.
Plans first. Then code.
Remy writes the spec, manages the build, and ships the app.
Do I need Chrome or Node.js to run Obscura?
No. Obscura ships as a single self-contained binary written in Rust with no Chrome, Node, or other runtime dependencies required.
Can I use any model with Ollama in this pipeline?
Any model served through Ollama works as long as it supports tool use (tool calling). The demonstrated setup used a 27 billion parameter Qwen 3.6 model on a 48 GB VRAM GPU, but smaller or larger models can be substituted depending on available hardware.
Does this pipeline require an internet connection after setup?
No. Once Obscura scrapes a page and the model is running locally through Ollama, the entire process, from fetching content to generating a summary, happens on the local machine with no data sent externally.
How is Obscura different from Puppeteer or Playwright?
Puppeteer and Playwright drive a full headless Chrome instance, which is powerful but resource-heavy. Obscura executes JavaScript through an embedded V8 engine without running Chrome itself, resulting in far lower memory use and faster startup, though it’s a narrower tool without Chrome’s full feature set.
