🎧 Prefer to listen?

If you’ve been building with AI models and assumed you’re stuck with NVIDIA GPUs forever, a 20-person Parisian startup just changed that. ZML — endorsed by Turing Award winner Yann LeCun and backed by founders from Docker and Hugging Face — just dropped LLMD, a free inference server that runs open-source AI models at peak speed across NVIDIA, AMD, Google TPU, Apple Metal, and Intel Arc. No vendor lock-in, no licensing fees, no catch.

This matters because inference costs are quietly becoming the biggest expense for anyone running AI tools in production. We’ve covered free AI tools before — like how Google I/O 2026 dropped free AI tools for beginners — but those are consumer-facing. If you’re building anything that processes prompts repeatedly, the inference bill adds up fast. And if you’re locked into one hardware vendor, you’re paying whatever they decide to charge. ZML’s move is about breaking that dependency, and the other free tools from startups we’ve covered show this is becoming a pattern: serious infrastructure, released free, monetized later.

What LLMD actually does (in plain language)

When you use an AI tool like ChatGPT, Claude, or any app powered by a large language model, the model needs to process your prompt and generate a response. That processing step is called inference. Right now, most inference runs on NVIDIA GPUs because the software ecosystem — CUDA — is deeply entrenched. If you’re a company running AI models at scale, you’re probably using NVIDIA hardware, and switching to something else means rewriting your entire software stack.

LLMD changes that equation. It’s an inference server that translates model operations into whatever hardware you throw at it. The same model, running on AMD chips instead of NVIDIA. Or Google’s TPUs. Or even Apple Silicon if that’s what you have access to.

The key insight isn’t just “it works on different chips.” It’s that it works at full speed on different chips. Morin, ZML’s founder, told TechCrunch they’re achieving peak performance across hardware — and sometimes going faster than the native stack. That’s a bold claim, but the team has serious credentials. Morin was VP of Engineering at Zenly, which Snapchat acquired for nine figures.

Why solo builders should care

If you’re running your own AI-powered tool, chatbot, or automation pipeline, the economics look like this:

NVIDIA GPUs are expensive and scarce. Cloud providers charge premium rates for NVIDIA instances because demand outstrips supply. If your app processes 10,000 prompts a day, you’re feeling that cost.

Vendor lock-in compounds over time. Once your entire stack is tuned for NVIDIA, switching is expensive and risky. Providers know this. Your leverage decreases every month.

Alternative chips exist and are getting better. AMD’s latest GPUs, Google’s TPUs, and even Intel Arc can handle inference workloads well — but the software barrier has kept most developers from experimenting with them. LLMD removes that barrier.

What this means practically: if you’re building a side project that uses AI, you could potentially run inference on cheaper AMD hardware or even your local Apple Silicon Mac without rewriting anything. If you’re running a small AI-powered business, you could mix and match hardware based on price and availability instead of being locked into whatever your cloud provider offers.

The free strategy — what ZML is actually doing

LLMD is not open source. But it is free — for now. Morin’s reasoning is straightforward: “I’d rather measure and then generate revenue where it is most effective without hindering my growth stupidly because I have been too greedy from the get-go.”

This is the same playbook we’ve seen from other AI infrastructure startups. This free cleaning robot training data startup gave away its core product to build adoption before monetizing. ZML is doing the same thing at the infrastructure layer — get developers using LLMD, learn how they use it, then charge for premium features or enterprise support.

The cap table tells you who’s paying attention. ZML has backing from the founders of Docker, Hugging Face, and Yann LeCun (now with AMI Labs). The company raised $20 million on a lean team of 20 people — that’s efficient capital allocation, and it means they can move fast without burning through runway.

How this fits into the broader inference landscape

ZML isn’t alone in the inference space. The “inference gold rush” is real — Baseten was recently valued at $13 billion, Inferact came from the creators of vLLM, and RadixArk spun out of SGLang. Each takes a different angle on the same problem: making AI inference faster, cheaper, and more accessible.

What separates ZML is the hardware-agnostic approach. Most competitors are optimizing for NVIDIA because that’s where the market is. ZML is betting that the market will fragment as more chip makers enter the AI space — European companies like Axelera, Fractile, Kalray, and SiPearl are all building AI-specific chips. If you can run inference on any of them at peak speed, you’re positioned for a future where hardware choice matters.

This also connects to something we covered in our breakdown of OpenAI’s hardware ambitions — the companies building AI models are increasingly thinking about hardware. The inference layer is where the money flows, and whoever controls it controls a significant piece of the AI economy.

What this means for your stack right now

If you’re just using AI tools as a consumer — ChatGPT, Claude, Gemini — this doesn’t change your day-to-day. But if you’re building anything that processes AI prompts in production, here’s what to consider:

  • Audit your inference costs. If you’re spending more than $100/month on inference through a single cloud provider, you’re a candidate for testing alternatives.
  • Watch LLMD’s development. It just launched, so expect rapid iteration. ZML has more releases planned, and their track record suggests those will land quickly.
  • Don’t rewrite anything yet. LLMD is free and worth testing, but it’s early. The existing infrastructure works. This is about having options, not ripping out your current stack.
  • Consider hardware diversity as a competitive advantage. If your costs are lower because you can run on cheaper hardware, that’s margin you can reinvest or pass to customers.

If you’re feeling overwhelmed by the pace of AI tool releases, you’re not alone — our guide on escaping AI tool overwhelm covers how to evaluate what actually matters for your situation.

The bottom line

ZML’s LLMD is a free inference server that breaks the NVIDIA lock-in by running AI models at peak speed across any major chip. It’s early, it’s free, and it’s backed by people who know what they’re doing. For solo builders and small teams watching their inference costs climb, this is worth paying attention to.

If you’re just getting started with AI tools, begin with /start-here/ — no hardware required.