libre chat
v0.0.6
輕鬆配置和部署基於開源大型語言模型 (LLM)（例如 Mixtral 或 Llama 2）的完全自託管聊天機器人 Web 服務，無需具備機器學習知識。
docker pip套件？LangChain和llama.cpp提供支持，可在本地進行推理。有關如何使用 Libre Chat 的更多詳細信息，請查看vemonet.github.io/libre-chat上的文檔
警告
該項目正在開發中，請謹慎使用。
這些檢查點是我們計劃在未來開發的功能，如果您有任何意見或要求，請隨時在問題中告訴我們。
如果您只是想使用預先訓練的模型Mixtral-8x7B-Instruct快速部署它，您可以使用 docker：
docker run -it -p 8000:8000 ghcr.io/vemonet/libre-chat:main您可以使用環境變數配置部署。為此，使用docker compose和.env檔案更容易，首先建立docker-compose.yml檔案：
version : " 3 "
services :
libre-chat :
image : ghcr.io/vemonet/libre-chat:main
volumes :
# ️ Share folders from the current directory to the /data dir in the container
- ./chat.yml:/data/chat.yml
- ./models:/data/models
- ./documents:/data/documents
- ./embeddings:/data/embeddings
- ./vectorstore:/data/vectorstore
ports :
- 8000:8000並在與docker-compose.yml相同的資料夾中使用您的配置建立一個chat.yml檔案：
llm :
model_path : ./models/mixtral-8x7b-instruct-v0.1.Q2_K.gguf
model_download : https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF/resolve/main/mixtral-8x7b-instruct-v0.1.Q2_K.gguf
temperature : 0.01 # Config how creative, but also potentially wrong, the model can be. 0 is safe, 1 is adventurous
max_new_tokens : 1024 # Max number of words the LLM can generate
# Always use input for the human input variable with a generic agent
prompt_variables : [input, history]
prompt_template : |
Your are an assistant, please help me
{history}
User: {input}
AI Assistant:
vector :
vector_path : null # Path to the vectorstore to do QA retrieval, e.g. ./vectorstore/db_faiss
# Set to null to deploy a generic conversational agent
vector_download : null
embeddings_path : ./embeddings/all-MiniLM-L6-v2 # Path to embeddings used to generate the vectors, or use directly from HuggingFace: sentence-transformers/all-MiniLM-L6-v2
embeddings_download : https://public.ukp.informatik.tu-darmstadt.de/reimers/sentence-transformers/v0.2/all-MiniLM-L6-v2.zip
documents_path : ./documents # Path to documents to vectorize
chunk_size : 500 # Maximum size of chunks, in terms of number of characters
chunk_overlap : 50 # Overlap in characters between chunks
chain_type : stuff # Or: map_reduce, reduce, map_rerank. More details: https://docs.langchain.com/docs/components/chains/index_related_chains
search_type : similarity # Or: similarity_score_threshold, mmr. More details: https://python.langchain.com/docs/modules/data_connection/retrievers/vectorstore
return_sources_count : 2 # Number of sources to return when generating an answer
score_threshold : null # If using the similarity_score_threshold search type. Between 0 and 1
info :
title : " Libre Chat "
version : " 0.1.0 "
description : |
Open source and free chatbot powered by [LangChain](https://python.langchain.com) and [llama.cpp](https://github.com/ggerganov/llama.cpp)
examples :
- What is the capital of the Netherlands?
- Which drugs are approved by the FDA to mitigate Alzheimer symptoms?
- How can I create a logger with timestamp using python logging?
favicon : https://raw.github.com/vemonet/libre-chat/main/docs/docs/assets/logo.png
repository_url : https://github.com/vemonet/libre-chat
public_url : https://chat.semanticscience.org
contact :
name : Vincent Emonet
email : [email protected]
license_info :
name : MIT license
url : https://raw.github.com/vemonet/libre-chat/main/LICENSE.txt最後啟動你的聊天服務：
docker compose up軟體包需要 Python >=3.8，只需使用pipx或pip安裝即可：
pip install libre-chat您可以使用終端機輕鬆啟動新的聊天 Web 服務，包括 UI 和 API：
libre-chat start提供具體的設定檔：
libre-chat start config/chat-vectorstore-qa.yml重建向量庫：
libre-chat build --vector vectorstore/db_faiss --documents documents透過以下方式取得可用選項的完整清單：
libre-chat --help或者你可以在 python 腳本中使用這個套件：
import logging
import uvicorn
from libre_chat import ChatConf , ChatEndpoint , Llm
logging . basicConfig ( level = logging . getLevelName ( "INFO" ))
conf = ChatConf (
model_path = "./models/mixtral-8x7b-instruct-v0.1.Q2_K.gguf" ,
vector_path = None
)
llm = Llm ( conf = conf )
print ( llm . query ( "What is the capital of the Netherlands?" ))
# Create and deploy a FastAPI app based on your LLM
app = ChatEndpoint ( llm = llm , conf = conf )
uvicorn . run ( app )靈感來自：
由 Freepik - Flaticon 創建的駱駝圖標
