🔍 Read the full analysis: Which AI Tasks Should Run Locally, And Which Should Use The Cloud? on ThorstenMeyerAI.com
Get smart everyday buys delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
A November 19 review of Apple and Nvidia specifications and AI software documentation finds that hardware choice depends on the task: cloud-backed agents can run on modest, low-power computers, while local models are constrained by accessible memory and software support. Macs support local inference and some fine-tuning; Nvidia PCs have broader CUDA support, and cloud services remain necessary for leading closed models.
A new comparison of Apple and Nvidia hardware and AI software documentation says the choice between local computing and cloud services depends on the workload: a small computer can keep a cloud-backed agent running, while local model use depends heavily on accessible memory and tool compatibility. The report, published November 19, says Macs handle several inference and fine-tuning tasks, Nvidia PCs suit software built around CUDA, and leading closed models remain cloud-only because they cannot be downloaded.
The comparison distinguishes an always-on agent from the AI model it uses. If an agent sends requests to a cloud model, the local machine mainly runs the agent’s loop, files and tools. The report cites Apple’s published 4-watt idle figure for the base Mac mini as an example of a low-power computer that could handle that role; the figure is for idle operation, not a measure of power use while running a workload.
For local models, the report treats memory capacity as a key constraint. Apple lists Mac Studio configurations with up to 512 GB of unified memory on M5 Ultra, compared with 64 GB at most for the Mac mini. Nvidia’s GeForce RTX 5090 has 32 GB of on-card memory. A model that exceeds a GPU’s memory may need to be split between GPU memory and system memory, which can change how it runs.
The article also compares software support. It says common local tools, including LM Studio, Ollama, llama.cpp and Apple MLX, support Mac hardware. Nvidia has an advantage when software is designed first for CUDA: the report says vLLM ships with precompiled CUDA binaries, while Apple silicon support is available through the community-maintained vLLM-Metal plugin. Apple’s MLX tools support LoRA and QLoRA fine-tuning, and PyTorch’s Mac backend supports GPU training, though the report notes that some precision and workload requirements can limit Mac use.
Decision guide · local AI hardware
Which machine fits which AI job?
Five common jobs, one after another, with the machine that fits each best on today’s specifications.
First question: does the model run in the cloud, or on your own machine? If it runs in the cloud, the box only keeps the agent going.
The job
- 1An agent that calls cloud models, running around the clock
→ Mac mini - 2A private assistant with a small open model
→ Mac mini - 3Huge open models with hundreds of billions of parameters
→ Mac Studio - 4Tools built for Nvidia’s CUDA, training, serving many users
→ GPU PC - 5The strongest models and the hardest tasks
→ Cloud API
The machine
Mac mini
up to 64 GB · 4 W at idle
Mac Studio
up to 512 GB · 385 W at most
GPU PC
32 GB on the card · 575 W card
Cloud API
strongest models · per-token bills
Sources: Apple, Mac mini and Mac Studio technical specifications and power pages; Nvidia, GeForce RTX 5090 specifications; all retrieved November 19, 2026. Matching of jobs to machines: our reading, not a buying recommendation.
Graphic: Powered by Thorsten Meyer AI
Matching AI Workloads to Hardware
The distinction matters because buying a machine for “AI” can mean very different things. A computer that keeps an agent online while delegating model responses to a cloud service does not need the same memory or graphics hardware as a machine expected to load a large model locally. Treating those uses as interchangeable can lead to spending on capacity a workload does not use—or choosing a machine that cannot run the intended model or software.
The trade-offs are not captured by memory alone. The report says Apple publishes memory-bandwidth figures for its chips, but the cited Nvidia RTX 5090 specification page does not provide a directly comparable bandwidth figure. Neither company publishes a token-speed comparison between the products. So the report’s hardware figures can help screen for fit, but they do not establish which machine will be faster for a particular model or task.
Power and operating complexity also differ. The report cites Apple’s maximum whole-system draw of 385 watts for the M5 Ultra Mac Studio, compared with Nvidia’s 575-watt rating for the RTX 5090 card alone and a 1,000-watt recommended system power supply. These are different measures, not like-for-like system consumption figures. They nevertheless indicate why a desktop GPU setup may bring different power and build requirements than a compact Mac.
Products Worth Considering

Vari ComfortEdge 48×30 Inch Adjustable Electric Standing Desk

Apple Mac mini (2026) with M6 chip

Logitech MX Mechanical Mini for Mac Wireless Keyboard, Space Gray
As an affiliate, we earn on qualifying purchases.
Memory, Software and Cloud Limits
The report follows an earlier discussion of what models can fit on a Mac mini with 16 GB to 64 GB of memory. Its next step is to compare that computer with Mac Studio configurations, Nvidia graphics PCs and cloud APIs. It says the hardware specifications and tool documentation were checked on November 19, 2026; its decision matrix is an interpretation of those sources, not a product test or buying recommendation.
To estimate whether a model’s weights can fit, the report offers a rule of thumb: at 4-bit precision, each billion parameters requires roughly 0.5 GB for weights alone, before overhead and conversation context. On that arithmetic, a 70-billion-parameter model needs about 35 GB for weights, while a 671-billion-parameter model needs about 336 GB. These are estimates, not full system-memory requirements or guarantees that a model will run at a useful speed.
Apple’s unified-memory design gives its CPU and GPU access to the same memory pool, according to the company’s MLX documentation. By contrast, the RTX 5090 has 32 GB of dedicated GDDR7 memory. The report also says Mac clustering is possible but involves additional setup, including Thunderbolt 5 connections between each pair of Macs and a macOS Recovery setting. The base Mac mini with M6 has Thunderbolt 4, according to the source material.
“The CPU and GPU have direct access to the same memory pool.”
— Apple, as quoted in its MLX documentation
Performance Depends on the Workload
The comparison does not report controlled benchmark results for model speed, power use during active workloads, or total operating costs. There is no published token-speed comparison in the cited materials between the M5 Ultra, M5 Max and RTX 5090, and the report says performance depends on the model. A GPU PC may be faster when a model fits on its card, while a Mac Studio may hold a larger model in unified memory; those are assessments, not results from a head-to-head test.
Actual fit also depends on more than parameter count. The report’s 4-bit calculation covers model weights only and excludes overhead and conversation context. It does not establish how much memory every model or software configuration needs. Nor does the comparison test individual workflows, software versions or multi-user serving loads. Nvidia’s cited product page lists no RTX 5090 price, and the cost of a complete GPU PC depends on its other components.
Check Requirements Before Choosing
The practical next step for readers is to identify the specific model, precision, software and workload before choosing hardware. For an agent that relies on a cloud model, estimate what the local agent itself needs. For local inference or fine-tuning, compare the model’s memory requirements with the machine’s accessible memory and verify that the intended software supports the hardware. Teams considering a serving engine should check its current platform support rather than assume CUDA-first tools work natively on Apple silicon.
The source material does not announce a new benchmark or a product change; it presents a comparison based on specifications and documentation available on November 19. Which machine performs best remains a workload-specific question until comparable tests establish speed, power use and cost across the same models and settings.
Key Questions
Does an always-on AI agent need a powerful local model?
Not necessarily. If the agent calls a cloud model, the local computer runs the agent loop, files and tools rather than the model itself. The report cites a 4-watt idle figure for the base Mac mini, but that figure does not describe power use under active workloads.
How much memory does a local model need?
It depends on model size, precision, overhead and conversation context. The report’s estimate at 4-bit precision is about 0.5 GB per billion parameters for weights alone: roughly 35 GB for a 70-billion-parameter model, before additional memory needs.
Is a Mac Studio or an Nvidia PC better for local AI?
The report identifies different strengths rather than a universal winner. Mac Studio configurations offer more unified memory, while Nvidia PCs have broader support for CUDA-first tools. The source material provides no direct token-speed benchmark comparing them.
Can Macs fine-tune AI models?
Yes, for some tasks. The report says Apple’s MLX tools support LoRA and QLoRA, and PyTorch’s Mac backend supports GPU training. It also cautions that memory needs and technical requirements can limit which models and training jobs are practical.
Why do some AI tasks still need the cloud?
The report says leading closed models cannot be downloaded, so they cannot be run locally. Cloud services are also an option when a local computer does not have the memory or software support needed for a particular model or workload.
Source: ThorstenMeyerAI.com
