The Ultimate LLM Providers Shopping Guide

What Is the Most Cost-Effective Way to Run a Large Language Model (LLM)?

When leveraging Large Language Models, you face a key decision: where should you run them? Options have multiplied, each with its own trade-offs. Should you:

1. Purchase dedicated hardware to run an open-source LLM locally?
2. Use a cloud service like OpenAI or Google Gemini, or pay for inference on a platform that hosts open‑source LLMs such as Groq or DeepInfra?
3. Rent raw GPU power from a cloud provider like Modal to host an open‑source model yourself?

I’ve explored all three paths, and here are my findings.

Running Open‑Source LLMs Directly on Your Machine — or Why Apple Stock Is Going to Explode

If you want to run an LLM locally, you first need to pick a worthwhile open‑source model.

You want the smartest open‑source model you can run with the lowest hardware requirements.

To do that, look at leaderboards like:

• Artificial Analysis or
• Vellum

I prefer Artificial Analysis because it computes an “intelligence index,” a composite of multiple renowned benchmarks for each model.

As of now, the new gpt-oss-120B appears to be the cheapest to run on cloud providers while maintaining a strong score (61), placing it second in intelligence. If it’s cheaper to run in the cloud, it’s likely less computationally demanding—and the same will hold on your hardware too.

Now, gpt-oss-120B needs high‑end hardware like an NVIDIA A100 or H100 with 80 GB of VRAM (memory dedicated to the GPU).

The interesting part is that Apple’s architecture blends VRAM and RAM and elude this reliance on the GPU only.

Apple’s Secret Sauce: Unified Memory Architecture (UMA)

On traditional PCs, the CPU has its own memory (RAM), and the GPU has its own separate, faster memory (VRAM). When running an LLM, data must constantly be copied back and forth between these two separate memory pools, creating a significant bottleneck that slows everything down.

Apple’s M‑series chips, such as the M4 Max, use a different approach.

Shared Memory Pool: With UMA, the CPU, GPU, and Neural Engine (NPU) share a single, large pool of high‑speed memory. Your local RAM is this unified pool.

No Copying Needed: Because all components are using the same memory, there’s no need to duplicate or transfer huge amounts of data. The GPU can directly access the model data the CPU is using with minimal delay. This dramatically increases efficiency and speed.

That’s why some people have had pretty decent results (40 tokens/sec) running gpt-oss-120B on an M4 Max: https://www.reddit.com/r/LocalLLM/comments/1mix4yp/getting_40_tokenssec_with_latest_openai_120b/

For comparison, ChatGPT online delivers around 100 tokens/sec. That said, a 128 GB RAM MacBook Pro M4 Max currently sells for $6,399—not cheap—but for a company running LLMs locally and paying only for electricity, it can make sense. Apple may be behind in the LLM revolution, but it owns the hardware to make it scalable and affordable. Waiting for Wall Street to realize it (this is not financial advice).

I Tested GPT-OSS-20B on My MacBook Air M2 (24 GB)

Of course, my MacBook Air couldn’t run the largest GPT‑OSS model, but I did run the 20B version and got ~20 tokens/sec, which was okay.

If you want to try it, here’s the simplest way:

• Download LM Studio from here
• Open the app and click to download the GPT‑OSS‑20B model
• Click on the “Start chat” button
• You will see the chat interface
• Ask a question. I asked, “How to clean jewelry the right way?” and here is the token generation rate:
• You can see the tokens‑per‑second stats at the bottom:

⚠️ Quick Warning

Just a quick note: LM Studio or open‑webui are interesting for playing with a model, but they do not replace a chat interface like ChatGPT, Claude, or Gemini. LM Studio and open‑webui don’t provide the same agentic capabilities out-of-the-box (web search, web scraping, document parsing, etc.) that these proprietary platforms offer to optimize model behavior. You can try to configure those capabilities in open‑webui, but you’ll likely not reach the same level of quality and performance as those online platforms.

Running Open‑Source LLMs on Inference Providers

Artificial Analysis can help you find the best inference providers. Basically, you want to sort models by intelligence index and compute the price per intelligence‑index point to find the best deal, then check which providers offer those models. I computed this and summarized it in the table below:

Source: Artificial Analysis

You can immediately see the good options:

• GPT‑5‑mini: $0.01 per million tokens (OpenAI)
• gpt‑oss‑120B: $0.005 per million tokens (open source; multiple providers)

If you choose a proprietary model like GPT‑5‑mini, you can go to the provider’s website (platform.openai.com) and buy credits.

For gpt‑oss‑120B, you can use OpenRouter to see which providers offer it and their pricing:

Source: openrouter.ai

If you don’t want to switch providers when they change prices, you can use OpenRouter directly to handle that for you. Most models follow the OpenAI JSON format, so you can use the OpenAI SDK and Agents SDK across providers—even for non‑OpenAI models.

OpenRouter takes a 5.5% markup on the credits you buy to pay for this broker service.

Other platforms like

• Martian
• Portkey.ai

also exist and offer additional features.

Running Open‑Source Models on Rented GPU Infrastructure

Well, this is the most disappointing option. I tried it with Modal. I deployed the smaller gpt‑oss‑20B model on an H200 GPU and, a few minutes later, I burned through my $5 starter credits. In comparison, I ran multiple heavy experiments over a full day with GPT‑5‑mini on the OpenAI platform and it cost me only 9 cents.

With Modal, by contrast, I did 12 small prompts to the model for $5.40 🤑:

This is because Modal reserves your infrastructure until you explicitly stop it. But if you stop it right after a prompt, you’ll need to wait for the infrastructure to provision again before the next prompt. This is fast on Modal—thanks to significant R&D mitigating Docker and Kubernetes filesystem boot‑time overhead—but in my view it’s still too slow for production.

Conclusion

If you optimize purely for cost today, OpenAI wins. For proprietary models, GPT‑5‑mini is currently the cheapest at roughly $0.01 per million tokens. For open‑source models, the lowest prices (e.g., gpt‑oss‑120B around $0.005 per million tokens) are easily accessed through the OpenAI‑compatible ecosystem via gateways like OpenRouter, so you can standardize on the OpenAI SDK and still route to the cheapest providers.

Companies developing agentic workflows can lower their cost by using GPT-OSS-120B on a well equipped Mac Mini that could act an inference dev server in the local network too.

Note: prices and availability change frequently—always recheck before committing a stack.