The short answer
A GPU is optional for most AI agents. The deciding question is where model inference happens. Inference is the work of turning your prompt and context into the model's next response. If that happens in a provider's datacenter, your agent does not need a graphics card. A small laptop, home server, virtual private server, or cloud VM can run the surrounding agent loop.
You need a capable GPU when you want the model itself to run locally. That includes agents connected to local models through tools such as Ollama, llama.cpp, or vLLM. In that setup, the machine must hold model weights in memory and perform the calculations for every generated token. The hardware requirement belongs to the model, not to the fact that the software is an agent.
This distinction explains why a terminal agent such as Codex or Claude Code can run on a modest remote box while a local Llama model may need an expensive workstation. Both are AI systems. Only one is doing inference on that machine.
Two AI agent setups that look similar but need different hardware
An API-based agent has a lightweight local loop. It collects your request, sends context to a model provider, receives a tool call, runs that tool, and sends the result back. The local machine handles files, shell commands, memory, scheduling, and network traffic. The provider handles the matrix calculations that benefit from GPUs.
A local-model agent keeps the same loop but also runs the language model. That changes the hardware profile completely. Model weights must fit in RAM or GPU memory. Larger context windows use more memory. Faster generation needs more memory bandwidth and compute. A GPU is not technically mandatory because many models can run on a CPU, but CPU-only inference can be too slow for an agent that needs dozens of model turns to finish one task.
The practical split is simple:
| Agent setup | GPU required? | What the local machine does |
|---|---|---|
| Claude Code or Codex with a subscription login | No | Runs the CLI, repository tools, and shell commands |
| Agent using an Anthropic, OpenAI, Gemini, or OpenRouter API | No | Runs the agent loop and sends inference requests |
| Agent with browser automation | Usually no | Runs the agent plus Chromium, which mainly needs RAM and CPU |
| Agent using a small Ollama model on CPU | No, but often slow | Runs both the agent and local inference |
| Agent using a medium or large local model at useful speed | Usually yes | Runs the agent and GPU-accelerated inference |
What hardware does an API-based AI agent need?
For a single terminal-based agent that calls a cloud model, start with 2 vCPU, 4 GB of RAM, and roughly 40 GB of SSD storage. The agent process itself may use less. The extra room covers package installation, code builds, logs, Docker, and the operating system. A 1 vCPU, 2 GB box can work for light tasks, but it has little margin when a build or dependency install spikes memory use.
Browser automation changes the recommendation more than the agent does. Chromium can open several renderer processes, and modern pages are heavy. Give a browser-driving agent at least 4 GB of RAM, with 8 GB more comfortable for multiple tabs or larger web apps. A GPU can help render graphics, but it is not required for normal browser research, form filling, screenshots, or Playwright tasks.
Disk and reliability also matter. Agent sessions, repositories, browser downloads, container layers, and logs accumulate. SSD storage makes those operations feel faster. If the agent runs unattended, a process supervisor and basic monitoring are more useful than a graphics card. The AI agent VPS guide covers sizing, security, and restart setup in detail.
When a GPU is worth it
A GPU makes sense when local inference is a deliberate requirement. You may want private processing, no per-token API bill, operation without an internet connection, control over a specific open model, or very high sustained usage where owned hardware is cheaper over time. Those are valid reasons. They also come with more setup and capacity planning.
Start from the model you intend to run, then choose hardware. A small quantized model may fit in 8 GB of system memory and run on a CPU for experimentation. Larger models need much more memory, and GPU memory is usually the limiting number. If the full model does not fit, software may split work between GPU and system RAM, but generation slows. Context length, quantization, concurrent agents, and tool-call frequency all affect the real requirement.
Do not buy a GPU based only on a model's parameter count or a single benchmark. Check the exact model file size, required context, expected tokens per second, and whether the runtime supports your operating system and GPU. Then test the model on rented hardware before buying a workstation. A short rental is cheaper than discovering that a new card cannot hold your chosen model.
Why agents can feel slow even without local inference
A slow API-based agent does not automatically need a GPU. Its waiting time is usually elsewhere: model-provider latency, a large prompt, a slow web page, package installation, a code build, network round trips, or a tool that is waiting for another service. A graphics card on the agent machine will not speed up a model running in someone else's datacenter.
Measure before changing hardware. Watch CPU, memory, disk pressure, and network activity during the slow step. If CPU is idle while the agent waits for a response, local compute is not the bottleneck. If the machine swaps heavily while Chromium is open, add RAM. If a TypeScript build pins every core, add CPU. If Ollama is generating one token every few seconds, then local inference hardware is the issue and a GPU may help.
This is also why a remote agent can work well on a modest cloud VM. It needs enough capacity for the tools it operates, not enough capacity to train or serve the cloud model. The AI agent hosting guide compares home hardware, a raw VPS, serverless jobs, and managed hosting using this same distinction.
A decision checklist before you buy anything
Answer these questions in order:
- Will the model run locally or through an API? An API means no local inference GPU.
- What tools will the agent run? Browsers, builds, Docker, and data processing determine CPU and RAM needs.
- Does the workload need to stay private or offline? If yes, local inference may justify the hardware.
- Which exact model and context length will you use? Size hardware for that combination, not for the phrase AI agent.
- How many agents or model requests run at once? Concurrency raises memory and compute needs.
- Have you tested on rented hardware? Validate speed and memory before buying a GPU.
For most people starting with an agent, the sensible route is an API-based setup on hardware they already own or a small remote machine. Learn which tasks are valuable first. Move to local inference only when privacy, offline operation, or usage economics make the extra complexity worthwhile.
Run the agent without buying a GPU
If you want an always-on agent but do not want another machine at home, Hivra runs agents on private cloud VMs. The official Codex and Claude Code CLIs use your own subscription sign-in, so their models still run through the provider. Hermes can use cloud model providers while its tools, memory, and schedules run on the VM.
You can try one agent on the 7-day free trial with card required, $0 charged today. If it proves useful, compare the always-on plans on the pricing page. No GPU purchase is needed for these cloud-model setups. The better first investment is enough RAM for the tools your agent actually uses.