LocalAI
v2.23.0
Get help - FAQ Discussions Discord Documentation website
Quickstart Models Roadmap ? Demo ? Explorer ? Examples
LocalAI is the free, Open Source OpenAI alternative. LocalAI act as a drop-in replacement REST API that’s compatible with OpenAI (Elevenlabs, Anthropic... ) API specifications for local AI inferencing. It allows you to run LLMs, generate images, audio (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families. Does not require GPU. It is created and maintained by Ettore Di Giacinto.
Run the installer script:
curl https://localai.io/install.sh | sh
Or run with docker:
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-cpu# Alternative images:# - if you have an Nvidia GPU:# docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-aio-gpu-nvidia-cuda-12# - without preconfigured models# docker run -ti --name local-ai -p 8080:8080 localai/localai:latest# - without preconfigured models for Nvidia GPUs# docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12
To load models:
# From the model gallery (see available models with `local-ai models list`, in the WebUI from the model tab, or visiting https://models.localai.io)local-ai run llama-3.2-1b-instruct:q4_k_m# Start LocalAI with the phi-2 model directly from huggingfacelocal-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf# Install and run a model from the Ollama OCI registrylocal-ai run ollama://gemma:2b# Run a model from a configuration filelocal-ai run https://gist.githubusercontent.com/.../phi-2.yaml# Install and run a model from a standard OCI registry (e.g., Docker Hub)local-ai run oci://localai/phi-2:latest
Getting started
Oct 2024: examples moved to LocalAI-examples
Aug 2024: ? FLUX-1, P2P Explorer
July 2024: ? P2P Dashboard, LocalAI Federated mode and AI Swarms: #2723
June 2024: ? You can browse now the model gallery without LocalAI! Check out https://models.localai.io
June 2024: Support for models from OCI registries: #2628
May 2024: Decentralized P2P llama.cpp: #2343 (peer2peer llama.cpp!) Docs https://localai.io/features/distribute/
May 2024: Openvoice: #2334
May 2024: ? Function calls without grammars and mixed mode: #2328
May 2024: Distributed inferencing: #2324
May 2024: Chat, TTS, and Image generation in the WebUI: #2222
April 2024: Reranker API: #2121
Roadmap items: List of issues
Multimodal with vLLM and Video understanding: #3729
Realtime API #3714
Distributed, P2P Global community pools: #3113
WebUI improvements: #2156
Backends v2: #1126
Improving UX v2: #1373
Assistant API: #1273
Moderation endpoint: #999
Vulkan: #1647
Anthropic API: #1808
If you want to help and contribute, issues up for grabs: https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3A%22up+for+grabs%22
Text generation with GPTs (llama.cpp, gpt4all.cpp, ... and more)
Text to Audio
Audio to Text (Audio transcription with whisper.cpp)
Image generation with stable diffusion
OpenAI-alike tools API
Embeddings generation for vector databases
Constrained grammars
Download Models directly from Huggingface
Vision API
Reranker API
P2P Inferencing
Integrated WebUI!
Check out the Getting started section in our documentation.
Build and deploy custom containers:
https://github.com/sozercan/aikit
WebUIs:
https://github.com/Jirubizu/localai-admin
https://github.com/go-skynet/LocalAI-frontend
QA-Pilot(An interactive chat project that leverages LocalAI LLMs for rapid understanding and navigation of GitHub code repository) https://github.com/reid41/QA-Pilot
Model galleries
https://github.com/go-skynet/model-gallery
Other:
Helm chart https://github.com/go-skynet/helm-charts
VSCode extension https://github.com/badgooooor/localai-vscode-plugin
Terminal utility https://github.com/djcopley/ShellOracle
Local Smart assistant https://github.com/mudler/LocalAGI
Home Assistant https://github.com/sammcj/homeassistant-localai / https://github.com/drndos/hass-openai-custom-conversation / https://github.com/valentinfrlch/ha-gpt4vision
Discord bot https://github.com/mudler/LocalAGI/tree/main/examples/discord
Slack bot https://github.com/mudler/LocalAGI/tree/main/examples/slack
Shell-Pilot(Interact with LLM using LocalAI models via pure shell scripts on your Linux or MacOS system) https://github.com/reid41/shell-pilot
Telegram bot https://github.com/mudler/LocalAI/tree/master/examples/telegram-bot
Github Actions: https://github.com/marketplace/actions/start-localai
Examples: https://github.com/mudler/LocalAI/tree/master/examples/
LLM finetuning guide
How to build locally
How to install in Kubernetes
Projects integrating LocalAI
How tos section (curated by our community)
Run Visual studio code with LocalAI (SUSE)
Run LocalAI on Jetson Nano Devkit
Run LocalAI on AWS EKS with Pulumi
Run LocalAI on AWS
Create a slackbot for teams and OSS projects that answer to documentation
LocalAI meets k8sgpt
Question Answering on Documents locally with LangChain, LocalAI, Chroma, and GPT4All
Tutorial to use k8sgpt with LocalAI
If you utilize this repository, data in a downstream project, please consider citing it with:
@misc{localai,
author = {Ettore Di Giacinto},
title = {LocalAI: The free, Open source OpenAI alternative},
year = {2023},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {url{https://github.com/go-skynet/LocalAI}},Do you find LocalAI useful?
Support the project by becoming a backer or sponsor. Your logo will show up here with a link to your website.
A huge thank you to our generous sponsors who support this project covering CI expenses, and our Sponsor list:
LocalAI is a community-driven project created by Ettore Di Giacinto.
MIT - Author Ettore Di Giacinto [email protected]
LocalAI couldn't have been built without the help of great software already available from the community. Thank you!
llama.cpp
https://github.com/tatsu-lab/stanford_alpaca
https://github.com/cornelk/llama-go for the initial ideas
https://github.com/antimatter15/alpaca.cpp
https://github.com/EdVince/Stable-Diffusion-NCNN
https://github.com/ggerganov/whisper.cpp
https://github.com/saharNooby/rwkv.cpp
https://github.com/rhasspy/piper
This is a community project, a special thanks to our contributors! ?
