Strata Local AI on One RTX 3060 12GB: A $769 Budget Rig Test
Can Strata's local AI engine run on a single RTX 3060 12GB? Here's the hardware cost breakdown, performance numbers, and Hermes agent setup.
Can you run Strata’s local AI engine on a single RTX 3060 12GB?
Yes. A tested budget build using one RTX 3060 (12GB VRAM), 64GB of system RAM, and a dual Xeon E5 v4 era CPU ran Strata’s local AI engine successfully, including vision tasks and an agent harness. Decode speeds landed in the 33 to 49 tokens per second range depending on load, and the whole rig was priced at roughly $769 to $825 depending on which CPU you pick. It works, but with real tradeoffs around context window size and agent concurrency.
TL;DR
- A single RTX 3060 12GB can run Strata’s local AI engine end to end, including document vision tasks and a connected agent harness, on a machine with 64GB of RAM.
- Measured decode speed ranged from about 33 tokens per second early in a run up to 49.2 tokens per second average on a long reasoning task, with speculative decoding kicking in and speeding things up mid-run.
- The full budget build came in at $769 using a cheaper Xeon E5-2695 V4 CPU, or $825 with the higher core count E5-2696 V4, not counting the rest of the chassis and storage.
- Running the Hermes agent against this setup works, but the context window has to be capped around 64K tokens, and sub-agent spawning is effectively disabled because batch size defaults to one on a single GPU.
- Power draw peaked around 350 watts for the single GPU configuration, which matters if you’re budgeting for a dedicated circuit or a quieter home office rig.
- A two-GPU version of this build (two RTX 3060s) pushes the total cost to around $1,100, buying you more batch headroom for sub-agents but not necessarily a big jump in raw tokens per second.
- The vision stack worked well enough to transcribe handwritten notes and review technical documentation pages, suggesting the single-GPU setup is usable for real multimodal tasks, not just text chat.
Remy is new. The platform isn't.
Remy is the latest expression of years of platform work. Not a hastily wrapped LLM.
What hardware does this budget build actually use?
The test rig centers on a single RTX 3060 with 12GB of VRAM, paired with 64GB of system RAM and a server-grade CPU from Intel’s Xeon E5 v4 family. The build used the E5-2696 V4, a part that doesn’t appear on Intel’s standard ARK product page but shows up on the secondary market for about $26.95. A cheaper alternative, the E5-2695 V4, trades some core count (18 cores versus 22) for a price around $75 shipped, or as low as $20 depending on the listing.
The 64GB of RAM ran about $250 for the kit used in testing. The RTX 3060 itself was priced around $325 at time of testing, and GPU prices in this tier have been climbing. A single six-pin to eight-pin power adapter, needed for a one-GPU setup, adds about $5.
Add it up and you get two configurations:
- $825 using the higher core count E5-2696 V4
- $769 using the lower cost E5-2695 V4 instead
Both numbers assume you’re buying used server parts off eBay, which is where this entire build was sourced. If you want to run two RTX 3060s instead of one, expect the total to land closer to $1,100, mostly because of GPU price inflation rather than any other component.
How fast does it actually run?
Performance scaled with the task. On a long reasoning-heavy prompt that asked the model to compare two CPU specs and predict performance impact, decode speed started around 33 tokens per second and crept into the mid-40s as the response grew past 10,000 tokens. By the time the output crossed 16,000 tokens, speculative decoding kicked in and the speed increased further. The full response took 617 seconds of “thinking” and produced roughly 32,000 output tokens, averaging 49.2 tokens per second overall.
That same task reported a VRAM hit rate described as good, alongside PCIe traffic utilization around 56.2 percent, which is notably high given that this is a single-GPU setup with no NVLink or multi-GPU interconnect to worry about. In practice, that means the system is leaning on the PCIe bus to shuttle data between the GPU and system RAM efficiently, since 12GB of VRAM alone isn’t enough to hold the full model and its working context.
Prefill speed (how fast the model processes input before it starts generating) is where the model is more GPU-bound. On a simpler handwriting transcription task, prefill speed reportedly climbed into the 800s tokens per second range, since that workload didn’t require long, multi-step reasoning. GPU temperature during testing sat around 69°C, and power draw peaked near 350 watts for the single-card configuration.
Does the vision stack work for real tasks?
Built like a system. Not vibe-coded.
Remy manages the project — every layer architected, not stitched together at the last second.
Yes. The build had the vision stack enabled, and it was tested on two meaningfully different tasks. First, it reviewed technical documentation pages (an E5 v4 CPU family reference page and a GitHub repo) to compare hardware specs and predict performance differences. Second, it was given a photo of handwritten notes, a cake recipe scrawled on paper, and asked to transcribe it. It correctly read most of the content, including specific numbers like “350” and “325” for a 30 to 35 minute bake time, even with messy handwriting.
This matters for anyone considering a single-GPU local AI setup for practical use: document OCR, note transcription, and page summarization are common real-world workloads, and this test suggests a 12GB card isn’t too constrained to handle them when paired with enough system RAM.
How do you connect Hermes agent to this setup?
Hermes agent connects to Strata through a custom model endpoint rather than a cloud API. The setup process:
- In Hermes (CLI or desktop), set a custom model endpoint pointing to your local machine’s internal IP address over HTTP, not HTTPS, since this is an internal network connection.
- Specify the port the Strata instance is running on (9877 in this test) followed by
/v1. - Skip the API key prompt if you’re running entirely inside your own network. Exposing this setup to the public internet without additional security isn’t advisable.
- Let API mode autodetect the model name, and leave context window detection as-is, though you’ll likely need to manually cap it.
- Name the connection something identifiable (the test used “Strata 3060”) and consider switching the reasoning effort from extra-high down to medium once you’re running it inside an agent harness, since agents tend to make many smaller calls rather than one long reasoning pass.
Once connected, Hermes correctly detected and displayed the custom provider, confirming it was routing requests to the local Strata instance over the network rather than falling back to a cloud model.
What are the real limits of running agents on one GPU?
The big constraint is context window and concurrency, not raw speed. This single-3060 build was capped at a 64K token context window. Pushing further is possible in theory, but longer context windows demand more VRAM and RAM headroom than this configuration has to spare.
More importantly, sub-agent spawning doesn’t really work here. Batch size defaults to one on a single GPU with this RAM configuration, which means the system processes requests in sequence rather than in parallel. If an agent tries to delegate work to sub-agents, those sub-agent calls just queue up one after another rather than running concurrently, which defeats much of the point of having sub-agents in the first place.
To get batch sizes of two or four, which would allow genuine parallel sub-agent execution, you’d need either a second GPU or significantly more system RAM (128GB of DDR4 was mentioned as a rough threshold). Even then, pushing context windows toward 128K on this kind of hardware would likely introduce long compaction times, since the system has to periodically compress conversation history to stay within memory limits, and that process isn’t fast on this hardware tier.
Is this budget build worth it?
For a single person running local AI tasks, document review, vision transcription, and a reasonably capable agent harness, the single RTX 3060 setup earns its keep. It’s inexpensive relative to multi-GPU alternatives, draws manageable power, and handled every task thrown at it in testing, from comparing CPU architecture tradeoffs to rendering and explaining a custom-built medical imaging viewer.
The tradeoff is concurrency. If your workflow depends on agents spinning up sub-agents to parallelize work, this configuration will bottleneck you. Doubling up to two 3060s roughly doubles cost to around $1,100 but buys the batch headroom needed for that kind of workflow. For straightforward single-agent use, the one-GPU, $769-$825 build is a reasonable entry point into running local AI without renting cloud GPU time.
Frequently Asked Questions
What GPU and RAM do you need to run Strata locally on a budget?
The tested configuration used one RTX 3060 with 12GB VRAM and 64GB of system RAM, paired with a Xeon E5 v4 series CPU. This combination successfully ran text generation, vision tasks, and a connected agent harness, though with a capped 64K context window.
How many tokens per second can you expect from a single RTX 3060?
Performance varied by task. Decode speeds ranged from around 33 tokens per second up to a 49.2 token per second average on a long, reasoning-heavy response, with speculative decoding improving speed as the response grew longer. Prefill speed on simpler tasks reportedly reached into the 800s tokens per second.
Can you run sub-agents with only one GPU?
Not effectively. Batch size defaults to one on this single-GPU, 64GB RAM configuration, which means sub-agent calls queue sequentially rather than running in parallel. Genuine sub-agent concurrency requires batch sizes of two or four, which needs either a second GPU or substantially more system RAM.
Is it cheaper to build with one GPU or two?
One GPU is cheaper. The single-3060 build totaled around $769 to $825 depending on CPU choice. A two-GPU version of the same build runs closer to $1,100, reflecting both the added card cost and recent GPU price increases, in exchange for better agent concurrency support.
Does the vision stack work well on a 12GB card?
In testing, yes. The system successfully reviewed technical documentation pages and transcribed handwritten notes, including specific numbers on a recipe card, despite having only 12GB of VRAM to work with alongside the rest of the model’s memory footprint.
