AMchat

AI Source Code

1.0.0

Download

AMchat advanced mathematics large model

?HuggingFace | |

News

[2024.08.09] We released the Q8_0 quantization model AMchat-q8_0.gguf.

[2024.06.23] InternLM2-Math-Plus-20B model fine-tuning.

[2024.06.22] InternLM2-Math-Plus-1.8B model fine-tuning, open source small-scale data set.

[2024.06.21] Updated README, InternLM2-Math-Plus-7B model fine-tuning.

[2024.03.24] Top 12 in the 2024 Puyuan Large Model Series Challenge (Spring Competition), Innovation and Creativity Award.

[2024.03.14] The model is uploaded to HuggingFace.

[2024.03.08] Improved README, added catalog and technical route. Added README_en-US.md.

[2024.02.06] Docker deployment is supported.

[2024.02.01] The first version of AMchat is deployed online https://openxlab.org.cn/apps/detail/youngdon/AMchat

How to use

quick start

Download model

From ModelScope

Download of reference model.

pip install modelscope

 from modelscope . hub . snapshot_download import snapshot_download
model_dir = snapshot_download ( 'yondong/AMchat' , cache_dir = './' )

FromOpenXLab

See download model.

pip install openxlab

 from openxlab . model import download
download ( model_repo = 'youngdon/AMchat' , 
        model_name = 'AMchat' , output = './' )

local deployment

git clone https://github.com/AXYZdong/AMchat.git
python start.py

Docker deployment

docker run -t -i --rm --gpus all -p 8501:8501 guidonsdocker/amchat:latest bash start.sh

retrain

Environment setup

clone this project

git clone https://github.com/AXYZdong/AMchat.git
cd AMchat

Create a virtual environment

conda env create -f environment.yml
conda activate AMchat
pip install xtuner

XTuner fine-tuning

Prepare configuration file

 # 列出所有内置配置
xtuner list-cfg

mkdir -p /root/math/data
mkdir /root/math/config && cd /root/math/config

xtuner copy-cfg internlm2_chat_7b_qlora_oasst1_e3 .

Model download

mkdir -p /root/math/model

download.py

 import torch
from modelscope import snapshot_download , AutoModel , AutoTokenizer
import os
model_dir = snapshot_download ( 'Shanghai_AI_Laboratory/internlm2-math-7b' , cache_dir = '/root/math/model' )

Modify configuration file

A fine-tuned configuration file has been provided under the config folder in the warehouse. You can refer to internlm_chat_7b_qlora_oasst1_e3_copy.py . It can be used directly, please pay attention to modify the paths of pretrained_model_name_or_path and data_path .

 cd /root/math/config
vim internlm_chat_7b_qlora_oasst1_e3_copy.py

 # 修改模型为本地路径
- pretrained_model_name_or_path = 'internlm/internlm-chat-7b'
+ pretrained_model_name_or_path = './internlm2-math-7b'

# 修改训练数据集为本地路径
- data_path = 'timdettmers/openassistant-guanaco'
+ data_path = './data'

Start fine-tuning

xtuner train /root/math/config/internlm2_chat_7b_qlora_oasst1_e3_copy.py

PTH model converted to HuggingFace model

mkdir hf
export MKL_SERVICE_FORCE_INTEL=1
export MKL_THREADING_LAYER=GNU
xtuner convert pth_to_hf ./internlm2_chat_7b_qlora_oasst1_e3_copy.py 
                         ./work_dirs/internlm2_chat_7b_qlora_oasst1_e3_copy/epoch_3.pth 
                         ./hf

HuggingFace model merged into large language model

 # 原始模型参数存放的位置
export NAME_OR_PATH_TO_LLM=/root/math/model/Shanghai_AI_Laboratory/internlm2-math-7b

# Hugging Face格式参数存放的位置
export NAME_OR_PATH_TO_ADAPTER=/root/math/config/hf

# 最终Merge后的参数存放的位置
mkdir /root/math/config/work_dirs/hf_merge
export SAVE_PATH=/root/math/config/work_dirs/hf_merge

# 执行参数Merge
xtuner convert merge 
    $NAME_OR_PATH_TO_LLM 
    $NAME_OR_PATH_TO_ADAPTER 
    $SAVE_PATH 
    --max-shard-size 2GB

Demo

streamlit run web_demo.py --server.address=0.0.0.0 --server.port 7860

OpenXLab application deployment

You only need to Fork this repository, then create a new project on OpenXLab, associate the Fork repository with the new project, and you can deploy AMchat on OpenXLab.

Demo

AMchat and InternLM2-Math-7B's solution to the same problem on integration. AMchat answered correctly, InternLM2-Math-7B answered incorrectly.

Demo Demo

LMDeploy quantification

First install LMDeploy

pip install -U lmdeploy

Then convert the model to turbomind format

--dst-path: You can specify the converted model storage location.

lmdeploy convert internlm2-chat-7b  要转化的模型地址 --dst-path 转换后的模型地址

LMDeploy Chat Conversation

lmdeploy chat turbomind 转换后的turbomind模型地址

OpenCompass review

InstallOpenCompass

git clone https://github.com/open-compass/opencompass
cd opencompass
pip install -e .

Download the unzipped dataset

cp /share/temp/datasets/OpenCompassData-core-20231110.zip /root/opencompass/
unzip OpenCompassData-core-20231110.zip

Evaluation starts!

python run.py 
    --datasets math_gen 
    --hf-path 模型地址 
    --tokenizer-path tokenizer地址 
    --tokenizer-kwargs padding_side= ' left ' truncation= ' left '     trust_remote_code=True 
    --model-kwargs device_map= ' auto ' trust_remote_code=True 
    --max-seq-len 2048 
    --max-out-len 16 
    --batch-size 2  
    --num-gpus 1 
    --debug

LMDeploy & OpenCompass quantitative and quantitative evaluation

W4 quantitative evaluation

W4 Quantification

lmdeploy lite auto_awq 要量化的模型地址 --work-dir 量化后的模型地址

Convert to TurbMind

lmdeploy convert internlm2-chat-7b 量化后的模型地址  --model-format awq --group-size 128 --dst-path 转换后的模型地址

Evaluation config writing

 from mmengine . config import read_base
from opencompass . models . turbomind import TurboMindModel

with read_base ():
 # choose a list of datasets   
 from . datasets . ceval . ceval_gen import ceval_datasets 
 # and output the results in a choosen format
#  from .summarizers.medium import summarizer

datasets = [ * ceval_datasets ]

internlm2_chat_7b = dict (
     type = TurboMindModel ,
     abbr = 'internlm2-chat-7b-turbomind' ,
     path = '转换后的模型地址' ,
     engine_config = dict ( session_len = 512 ,
         max_batch_size = 2 ,
         rope_scaling_factor = 1.0 ),
     gen_config = dict ( top_k = 1 ,
         top_p = 0.8 ,
         temperature = 1.0 ,
         max_new_tokens = 100 ),
     max_out_len = 100 ,
     max_seq_len = 512 ,
     batch_size = 2 ,
     concurrency = 1 ,
     #  meta_template=internlm_meta_template,
     run_cfg = dict ( num_gpus = 1 , num_procs = 1 ),
)
models = [ internlm2_chat_7b ]

Evaluation starts!

python run.py configs/eval_turbomind.py -w 指定结果保存路径

KV Cache Quantitative Evaluation

Convert to TurbMind

lmdeploy convert internlm2-chat-7b  模型路径 --dst-path 转换后模型路径

Calculate and obtain quantization parameters

 # 计算
lmdeploy lite calibrate 模型路径 --calib-dataset ' ptb ' --calib-samples 128 --calib-seqlen 2048 --work-dir 参数保存路径
# 获取量化参数
lmdeploy lite kv_qparams 参数保存路径 转换后模型路径/triton_models/weights/ --num-tp 1

Change quant_policy to 4 and change the path in the above config
Evaluation starts!

python run.py configs/eval_turbomind.py -w 结果保存路径

The result files and evaluation data sets can be obtained in the results file in the same directory.

? Acknowledgments

Project members

Zhang Youdong - project leader (Datawhale member scholar Puyu practical camp teaching assistant is responsible for model training, OpenXlab application deployment, data collection, RAG content organization, InternLM2-Math-Plus fine-tuning planning)
Song Zhixue - project leader (Datawhale member scholar and Puyu practical camp teaching assistant is responsible for project planning, RAG framework)
Xiao Hongru - Project leader (Datawhale member Tongji University scholar and Puyu practical camp teaching assistant is responsible for data collection, data set sorting and enhancement, model quantification and evaluation, RAG inference and verification)
Cheng Hong (Teaching Assistant of Scholar·Puyu Practical Camp & Teaching Assistant of Datawhale Jingying InternLM2-Math-Plus-7B Model Fine-tuning & Deployment)
Mo Baoqi (Yuchai Engineering Research Institute InternLM2-Math-Plus-1.8B model fine-tuning)
Chen Fuyuan (Gansu University of Political Science and Law InternLM2-Math-Plus-20B model fine-tuning)
Gong Heyang (Ph.D. in Statistics, University of Science and Technology of China, LMDeploy model quantification)
Jie Rongyang (datawhale member Harbin Institute of Technology (Weihai) data collection RAG content compilation)
Peng Chen (Datawhale member data collection)
Wang Xinming (data collection)
Liu Zhiwen (data collection from Shandong Women's University, member of Datawhale)
Wang Ruiyue (Northeastern University data collection)
Chen Yihan (Datawhale member Beijing University of Posts and Telecommunications data collection)
guidons (Northeast University docker deployment)
eltociear (Board member at I-Tecnology Co., Ltd., add Japanese README)

special thanks

Thanks to the Shanghai Artificial Intelligence Laboratory for organizing the Scholar Puyu Practical Camp Learning Activity~

Thanks to OpenXLab for its computing power support for project deployment~

Thanks to Puyu Assistant for supporting the project~

Thanks to the Shanghai Artificial Intelligence Laboratory for launching the Scholar·Puyu Large Model Practical Camp, which provides valuable technical guidance and powerful computing power support for our project!

InternLM-tutorial , InternStudio , xtuner , InternLM-Math

Citation

 @misc { 2024AMchat ,
    title = { AMchat: A large language model integrating advanced math concepts, exercises, and solutions } ,
    author = { AMchat Contributors } ,
    howpublished = { url{https://github.com/AXYZdong/AMchat} } ,
    year = { 2024 }
}