Llamafile

Llamafile turns LLMs into a single executable file. Whether you’re a developer or an end user, you simply choose the LLM you want to run, download its llamafile, and execute it. llamafile runs on six operating systems (Windows, macOS, Linux, OpenBSD, FreeBSD, and NetBSD), and generally requires no installation or configuration. It uses your fancy GPU, if you have one. Otherwise, it uses your CPU. It makes open LLMs usable on everyday consumer hardware, without any specialized knowledge or skill.

Our goal is to make open LLMs much more accessible to both developers and end users. We’re doing that by combining llama.cpp with Cosmopolitan Libc into one framework that collapses all the complexity of LLMs down to a single-file executable (called a “llamafile”) that runs locally on most computers, with no installation.

We believe that llamafile is a big step forward for access to open source AI. But there’s something even deeper going on here: llamafile is also driving what we at Mozilla call “local AI.”

Local AI is AI that runs on your own computer or device. Not in the cloud, or on someone else’s computer. Yours. This means it’s always available to you. You don’t need internet access to use a local AI. You can turn off your WiFi, and it will still work.