streamlx
1.0.0
Streamlit로 구축된 MLX 제공 LLM을 위한 간단한 채팅 인터페이스입니다.
git clone https://github.com/kurcontko/mlx-chat.git
cd mlx-chat
./start.sh
또는 수동으로:
# Terminal 1: Start MLX server
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
# Pick the most suitable model based on your VRAM:
# - mlx-community/Qwen2.5-Coder-32B-Instruct-4bit
# - mlx-community/Mistral-7B-Instruct-v0.3-8bit
mlx_lm.server --model MODEL_NAME --port 8080
# Terminal 2: Start Streamlit app
streamlit run src/main.py
.env
파일 만들기(선택 사항):
MLX_MODEL = mlx-community/Qwen2.5-Coder-32B-Instruct-4bit
MLX_PORT = 8080
STREAMLIT_PORT = 7860
requirements.txt
참조하세요.