How to Build an LLM Wiki Knowledge Base with Obsidian and Claude Code
Learn how to ingest YouTube transcripts, PDFs, and URLs into a cross-linked Obsidian knowledge base using Claude Code in under 5 minutes.
Stop Losing Knowledge You’ve Already Found
You’ve watched the video. Read the paper. Skimmed the thread. And then it’s gone — buried in a browser tab graveyard or a downloads folder you’ll never open again.
Building an LLM wiki knowledge base with Obsidian and Claude Code solves this. Instead of passively consuming content, you route it through a lightweight pipeline that extracts key ideas, writes them up as structured notes, and cross-links them to everything else you know. The result is a personal wiki that grows smarter the more you feed it.
This guide walks you through the full setup: ingesting YouTube transcripts, PDFs, and URLs into Obsidian using Claude Code, with automatic cross-linking between notes. The whole thing takes under five minutes once it’s running.
Why Obsidian and Claude Code Work So Well Together
Obsidian stores everything as plain Markdown files on your local machine. No proprietary format, no cloud lock-in, no monthly fee for core functionality. Notes live in a folder called a vault, and Obsidian renders [[wikilinks]] between them automatically.
Claude Code is Anthropic’s agentic coding environment — a terminal-based AI assistant that can read files, write files, run scripts, and chain together multi-step tasks. Unlike using Claude through a chat interface, Claude Code can actually reach into your filesystem and do things.
The combination works because:
- Obsidian’s Markdown files are trivially easy for an LLM to read and write
- Claude Code can handle multi-step tasks (fetch → summarize → format → link → save) without you babysitting each step
- The vault’s folder structure gives Claude Code enough context to understand what already exists, so it can make meaningful cross-links instead of generic ones
You don’t need to be a developer to use this setup. Basic terminal comfort is enough.
Prerequisites and Setup
Before you write a single line of configuration, make sure you have the following installed and ready.
What You’ll Need
- Obsidian — download from the official Obsidian website. Create a new vault or use an existing one.
- Claude Code — installed via npm (
npm install -g @anthropic-ai/claude-code) and authenticated with your Anthropic API key - Node.js (v18+) — required to run Claude Code and any helper scripts
- Optional but useful:
yt-dlpfor YouTube transcript extraction, andpdftotext(part of the Poppler utilities) for PDF parsing
Folder Structure to Create Inside Your Vault
Once your vault is open, create this basic structure:
vault/
├── inbox/ ← raw inputs land here temporarily
├── notes/ ← processed, linked notes live here
├── sources/ ← metadata and original excerpts
└── index.md ← your vault's master indexThis separation matters. The inbox folder acts as a staging area — Claude Code picks up files there, processes them, and moves the finished note to notes/. Keeping these separate prevents half-processed content from polluting your main knowledge graph.
Configure Claude Code for Vault Operations
Claude Code works through a CLAUDE.md file in your project directory. This file tells Claude Code how to behave — what conventions to follow, what tools to use, and what the project is trying to accomplish.
Write Your CLAUDE.md
Create a file called CLAUDE.md in your vault root with content like this:
# Knowledge Base Instructions
This is a personal wiki vault managed with Obsidian.
## Note Format
- Every note starts with a YAML frontmatter block (title, date, tags, source_url)
- Use ## for main sections, ### for subsections
- Add [[wikilinks]] to existing notes wherever a concept or topic overlaps
- End every note with a ## Related Notes section listing linked notes
## Ingestion Rules
- Summaries should be 400–700 words unless the source is short
- Extract 3–5 key takeaways as a bulleted list near the top
- Infer tags from content — don't ask me for them
- Save processed notes to /notes/ folder
- Update index.md with the new note title and a one-sentence description
## Cross-Linking
- Before saving, read the titles of all files in /notes/ to identify overlap
- Add [[wikilinks]] for any notes that share concepts, people, tools, or themes
- Don't create wikilinks to notes that don't exist yetThe cross-linking instruction is the most important part. By telling Claude Code to scan existing notes before saving, you get real connections rather than placeholder links that go nowhere.
Test the Basic Setup
In your terminal, navigate to your vault folder and run:
cd ~/path/to/your/vault
claudeThen type a simple test prompt:
Create a note in /notes/ called "test.md" about the concept of spaced repetition. Follow the CLAUDE.md format. Check if any existing notes in /notes/ are relevant and add wikilinks.Other agents start typing. Remy starts asking.
Scoping, trade-offs, edge cases — the real work. Before a line of code.
If Claude Code writes a properly formatted Markdown file with frontmatter and saves it to your /notes/ folder, you’re good. If it errors, check that your Anthropic API key is set correctly (ANTHROPIC_API_KEY in your environment).
Ingesting YouTube Transcripts
YouTube is one of the richest sources of technical content — conference talks, tutorials, interviews, course lectures. The problem is that video is passive and unsearchable. Turning transcripts into structured notes fixes both.
Extract the Transcript
Install yt-dlp if you haven’t:
pip install yt-dlpThen pull the transcript for any YouTube video:
yt-dlp --write-auto-sub --skip-download --sub-format vtt -o "inbox/%(title)s" "https://youtube.com/watch?v=VIDEO_ID"This downloads the auto-generated subtitle file in .vtt format. The file lands in your inbox/ folder with the video title as the filename.
Let Claude Code Process It
With the .vtt file in your inbox, open Claude Code in your vault and run:
Read the .vtt file in /inbox/, extract the spoken content (ignoring timestamps and formatting codes), summarize it as a knowledge base note following CLAUDE.md conventions, check /notes/ for related notes to link, then save it to /notes/ and update index.md.Claude Code will:
- Read the subtitle file
- Strip the VTT formatting to get clean spoken text
- Summarize, extract key points, and generate tags
- Scan your existing notes for cross-link opportunities
- Write the finished note and update your index
The whole process takes about 30–60 seconds depending on video length.
Batch Processing Multiple Videos
If you have a playlist or a folder full of .vtt files, you can ask Claude Code to loop through them:
Process every .vtt file in /inbox/ one at a time using the CLAUDE.md format. Save each as a note in /notes/, cross-link where relevant, and update index.md after each one.Claude Code handles this sequentially, which avoids rate limit issues and lets it use each newly created note as context for the next one.
Ingesting PDFs
Research papers, ebooks, and documentation PDFs are dense — exactly the kind of content that benefits most from LLM summarization.
Convert PDF to Text
Use pdftotext (install via Homebrew on Mac: brew install poppler):
pdftotext path/to/paper.pdf inbox/paper.txtThis gives Claude Code a clean text file to work with. LLMs generally handle plain text better than raw PDF parsing, which often produces garbled output from multi-column layouts.
Process the Text File
Read /inbox/paper.txt. Treat it as an academic paper or technical document. Summarize the core argument, methodology, and findings. Extract key takeaways. Add frontmatter with a source field noting it came from a PDF. Cross-link to any related notes in /notes/ and save the result to /notes/.For longer documents, Claude Code may chunk the content internally. If you notice truncation, you can explicitly ask it to focus on specific sections:
Focus on the abstract, introduction, and conclusion sections of /inbox/paper.txt. These contain the core claims. Summarize those sections specifically.Handling Scanned PDFs
Scanned PDFs without embedded text won’t convert cleanly with pdftotext. For those, you’ll need an OCR step first. Tools like tesseract (open source) or cloud OCR APIs can handle this. Once you have clean text output, the rest of the workflow is identical.
Ingesting URLs and Web Pages
For blog posts, documentation pages, and news articles, you can pull content directly from a URL.
Use Claude Code’s Built-in Fetch Capability
Claude Code can fetch web pages directly. Just give it the URL:
Fetch the content of [URL]. Ignore navigation menus, footers, and ads — focus on the main article body. Summarize it as a knowledge base note following CLAUDE.md, cross-link to related notes in /notes/, and save it to /notes/.Claude Code uses its built-in browser tool to retrieve the page content and strips the noise automatically. It’s not perfect for heavily JavaScript-rendered pages, but it works well for most standard web content.
For JavaScript-Heavy Sites
If a page fails to load properly, you can copy-paste the article text directly into Claude Code’s terminal prompt:
Here is the text content of an article I want to add to my knowledge base. [paste text]. Follow CLAUDE.md conventions, cross-link to related notes, and save to /notes/.Not elegant, but it works reliably for any content you can select and copy.
Building a Strong Cross-Link Graph
The real value of a knowledge base isn’t individual notes — it’s the connections between them. A note about transformer attention mechanisms that links to a note about the original “Attention Is All You Need” paper, which links to a note about scaled dot-product attention, is far more useful than three isolated summaries.
How Claude Code Generates Links
The key instruction in your CLAUDE.md is telling Claude Code to read existing note titles before saving anything new. You can strengthen this by also asking it to read a summary or the first paragraph of potentially related notes:
## Cross-Linking (enhanced)
- List all files in /notes/ before saving any new note
- For notes whose titles suggest overlap with the new content, read their first ## section
- Add [[wikilinks]] based on actual conceptual overlap, not just keyword matching
- Add a brief annotation in the ## Related Notes section explaining *why* each note is linkedThe annotation is particularly useful. A related notes section that says “[[Transformer Architecture]] — shares the multi-head attention concept described in Section 2” is far more useful than a bare wikilink.
Maintaining Your Index
Your index.md file becomes your vault’s table of contents. Update your CLAUDE.md to standardize how Claude Code writes to it:
## Index Updates
- Add each new note to index.md under the appropriate tag-based section
- If no section exists for the note's primary tag, create one
- Format: - [[Note Title]] — one sentence descriptionOver time, this index gives you a scannable overview of your entire knowledge base without having to open Obsidian’s graph view.
Where MindStudio Fits Into This Workflow
The Obsidian + Claude Code setup is powerful, but it’s manual — you’re running commands in a terminal, feeding files one by one, and managing the pipeline yourself.
If you want this running automatically — processing a YouTube playlist overnight, pulling PDFs from a Notion database, or summarizing a weekly digest of URLs from a Slack channel — that’s where MindStudio comes in.
MindStudio lets you build automated AI workflows without writing orchestration code. You could, for example, build an agent that:
- Monitors a Notion database for new rows tagged “to summarize”
- Fetches the URL or PDF attached to each row
- Sends the content to Claude (one of 200+ models available on the platform) with your summarization prompt
- Writes the formatted Markdown output to a file in your Obsidian vault via a file sync integration
- Updates the Notion row to mark it as processed
The entire workflow runs on a schedule or triggers automatically when new content is added — no terminal commands required.
For developers who want Claude Code itself to call MindStudio capabilities, the MindStudio Agent Skills Plugin provides an npm SDK that gives any AI agent access to 120+ typed methods — things like agent.searchGoogle() or agent.runWorkflow() — so Claude Code can trigger MindStudio pipelines programmatically as part of a larger agentic task.
You can start for free at mindstudio.ai.
Common Issues and Fixes
Claude Code isn’t respecting the CLAUDE.md format
Make sure CLAUDE.md is in the root of the folder where you launch Claude Code, not inside a subfolder. Claude Code reads it from the working directory on startup.
Cross-links point to notes that don’t exist
Add this explicit constraint to your CLAUDE.md: “Never create a [[wikilink]] unless the target file already exists in /notes/.” Claude Code sometimes hallucinates plausible note titles — this instruction curbs that.
VTT files produce garbled summaries
Auto-generated YouTube subtitles can be messy, especially for technical vocabulary. You can pre-process the VTT file with a simple script that removes timestamps and joins fragments, then feed the cleaned text to Claude Code.
PDFs with complex layouts produce wrong content order
Multi-column papers especially can come out in the wrong reading order after pdftotext. A quick fix: ask Claude Code to focus on extracting claims and concepts rather than following document flow linearly. It’s better at semantic extraction than sequential reading in these cases.
The workflow is too slow for large batches
For batches over 20 files, consider splitting them into smaller groups of five or ten. Claude Code handles sequential tasks well but can lose context coherence across very long sessions.
Frequently Asked Questions
How accurate are Claude’s summaries of technical content?
Claude performs well on technical content when given clear instructions about what to preserve — key claims, methodologies, numbers, terminology. For highly specialized domains (e.g., cutting-edge ML papers or medical research), always verify key claims against the source. The notes are a starting point for understanding, not a replacement for reading the original.
Do I need to keep my vault synced to the cloud for this to work?
No. Claude Code operates entirely locally. Your vault stays on your machine. If you use Obsidian Sync or iCloud Drive to back up your vault, that works alongside this workflow — Claude Code writes files to your local vault folder, and the sync service handles replication.
Can I use a different AI model instead of Claude?
The Claude Code tool is specifically built around Claude models. But the general approach — using an LLM to process content into Markdown notes — works with any model that has file system access or code execution capability. OpenAI’s Codex environment, or local models via tools like Continue, can approximate the same workflow with some adjustment.
How do I handle content in languages other than English?
Claude handles multilingual content well. Add a line to your CLAUDE.md: “If the source content is not in English, produce the note in English unless instructed otherwise.” Claude will translate and summarize in one pass. You can also specify other target languages.
Will this workflow work on Windows?
Yes, with minor adjustments. Claude Code runs on Windows via WSL2 (Windows Subsystem for Linux) or natively with some path adjustments. The yt-dlp and pdftotext tools are both cross-platform. Obsidian itself works natively on Windows with no changes.
How do I prevent duplicate notes from being created?
Add this to your CLAUDE.md: “Before saving any new note, check /notes/ for files with similar titles or the same source URL in frontmatter. If a match exists, update the existing note rather than creating a new one.” Claude Code isn’t perfect at this, but explicit instructions significantly reduce duplication.
Key Takeaways
- Obsidian + Claude Code creates a lightweight pipeline for turning raw content (YouTube videos, PDFs, URLs) into structured, cross-linked Markdown notes.
- The CLAUDE.md file is your control layer — it defines how notes are formatted, how cross-links are generated, and how the index is maintained.
- Cross-linking is the most valuable feature — configure Claude Code to scan existing notes before saving so it builds real connections, not placeholder links.
- Batch processing works — Claude Code can handle multiple files in sequence, using earlier notes as context when creating later ones.
- MindStudio extends the setup with automated triggers and scheduled workflows, so content flows into your knowledge base without manual intervention.
If you want to automate the full pipeline — from content discovery to note creation — try building an agent in MindStudio. It connects directly to Claude and the tools you already use, and you can have a working workflow running in under an hour.
