Chat-Zhen Huan is a chat language model that imitates Zhen Huan's tone and is obtained by LoRA fine-tuning based on a large model using all the lines and sentences about Zhen Huan in the script of "The Legend of Zhen Huan".
Zhen Huan, the female lead and core heroine in the novel "The Legend of Zhen Huan" and the TV series "The Legend of Zhen Huan". Her original name was Zhen Yuhuan, but she changed her name to Zhen Huan because she thought the word jade was vulgar. She was the daughter of Zhen Yuandao, a Han Chinese. Later, she was given the surname Niu Hulu by Yongzheng. She raised the flag as the top three flags of Manchuria and was named "Niu Hulu·Zhen Huan". Participated in the draft with Shen Meizhuang and An Lingrong, and was selected because she looked like Queen Chunyuan. After entering the palace, facing Concubine Hua's constant pressure, Shen Meizhuang was wronged, and An Lingrong changed her heart. From a young and quiet girl in a quiet corner, she became a veteran in palace fighting who could cause bloody events. After Yongzheng discovered the ambitions of the Nian family, he ordered his father Zhen Yuandao to cut them off. Zhen Huan also used her serial tricks in the harem to help the emperor deal with his political opponents, so she was deeply loved by Yongzheng. After many twists and turns, the arrogant and domineering Concubine Hua was finally defeated. When Zhen Huan became a concubine, she was plotted against by Queen Yi Xiu and was rejected by the emperor. After giving birth to her daughter Long Yue, she became disheartened and asked herself to leave the palace to become a nun. However, he was admired by King Guo, and the two fell in love. After learning about King Guo's death, they immediately planned to meet Yongzheng again and return to the palace in glory. Afterwards, Zhen's father's unjust case was vindicated and Zhen's family was resurrected. She also gave birth to twins. She escaped Yi Xiu's assassination in various conspiracies such as blood tests, and finally brought down the queen who was behind the scenes by sacrificing her own fetus. . But Yongzheng forced Zhen Huan to poison Yunli to test Zhen Huan's sincerity, and asked Zhen Huan, who had already given birth to a child, to go to Zhungeer to get married. Zhen Huan then regarded the emperor as the most deserving object of destruction. In the finale, she said that "all human struggles arise from the injustice of rulers" and poisoned Yong Zheng. The fourth elder brother Hongli ascended the throne as Qianlong, and Zhen Huan was revered as the Holy Mother and Queen Mother. She had power over the government and the public, and spent her old age peacefully in Ruyi's Royal Love in the Palace.
Chat-Zhen Huan has realized the complete process of creating a personalized AI fine-tuning large model based on novels and scripts using "The Legend of Zhen Huan" as the starting point. By providing any novel or script and specifying the character, the complete process of this project can be run. Let each user create his or her own personalized AI that fits the character's personality and is highly intelligent based on his or her favorite novels and scripts.
Chat-Huanhuan model has a cumulative download volume of 15.6k, Modelscope address: Link
Chat-Huanhuan has accumulated 500 stars, huahuan-chat project address: Link , xlab-huanhuan-chat project address: Link
OK, then I will lead you to personally implement the training process of Chat-Zhen Huan step by step. Let us experience it together~
The basic environment of this article is as follows:
----------------
ubuntu 22.04
python 3.12
cuda 12.1
pytorch 2.3.0
----------------
This article assumes that learners have installed the above Pytorch (cuda) environment. If not, please install it yourself.
First, use pip to change sources to speed up downloading and installing dependent packages.
# 升级pip
python -m pip install --upgrade pip
# 更换 pypi 源加速库的安装
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
pip install modelscope==1.16.1
pip install transformers==4.43.1
pip install accelerate==0.32.1
pip install peft==0.11.1
pip install datasets==2.20.0First, we need to prepare the script data of "The Legend of Zhen Huan". Here we use the script data of "The Legend of Zhen Huan". We can check the format of the original data.
第2幕
（退朝，百官散去）
官员甲：咱们皇上可真是器重年将军和隆科多大人。
官员乙：隆科多大人，恭喜恭喜啊！您可是国家的大功臣啊！
官员丙：年大将军，皇上对你可是垂青有加呀！
官员丁：年大人，您可是皇上的股肱之臣哪！
苏培盛（追上年羹尧）：年大将军请留步。大将军——
年羹尧：苏公公，有何指教？
苏培盛：不敢。皇上惦记大将军您的臂伤，特让奴才将这秘制的金创药膏交给大人，叫您使用。
年羹尧（遥向金銮殿拱手）：臣年羹尧恭谢皇上圣恩！敢问苏公公，小妹今日在宫中可好啊？
苏培盛：华妃娘娘凤仪万千、宠冠六宫啊，大将军您放心好了。
年羹尧：那就有劳苏公公了。（转身离去）
苏培盛：应该的。
Each sentence has a character and corresponding lines, so this data can be easily processed into a dialogue form, as follows:
[
{"rloe":"官员甲", "content":"咱们皇上可真是器重年将军和隆科多大人。"},
{"rloe":"官员乙", "content":"隆科多大人，恭喜恭喜啊！您可是国家的大功臣啊！"},
{"rloe":"官员丙", "content":"年大将军，皇上对你可是垂青有加呀！"},
{"rloe":"官员丁", "content":"年大人，您可是皇上的股肱之臣哪！"},
{"rloe":"苏培盛", "content":"年大将军请留步。大将军——"},
...
]
Then we extract the dialogues of the characters we focus on to form QA question and answer pairs. For such data, we can use regular expressions or other methods to quickly extract and extract the dialogue of the characters we care about.
And in many cases, we don't have such excellent line format data. So we may need to extract the character's dialogue data from a large piece of text, and then convert it into the format we need.
For example, in "Journey to the West in Vernacular", we can see that his text looks like this. For such text, we need to use the ability of large models to extract the characters and the dialogue corresponding to the characters from the text. Then we filter out the dialogue we need for the Tachibana character.
You can extract dialogue from text with the help of a small tool: extract-dialogue .
......
原来孙悟空走了以后，有一个混世魔王独占了水帘洞，并且抢走了许多猴子猴孙。孙悟空听到这些以后，气得咬牙跺脚。他问清了混世魔王的住处，决定找混世魔王报仇，便驾着筋斗云，朝北方飞去。
不一会儿，孙悟空就来到混世魔王的水脏洞前，对门前的小妖喊到∶“你家那个狗屁魔王，多次欺负我们猴子。我今天来，要和那魔王比比高低！
”小妖跑进洞里，报告魔王。魔王急忙穿上铁甲，提着大刀，在小妖们的簇拥下走出洞门。
孙悟空赤手空拳，夺过了混世魔王的大刀，把他劈成了两半。然后，拔下一把毫毛咬碎喷了出去，毫毛变成许多小猴子，直杀进洞里，把所有的妖精全杀死，然后救出被抢走的小猴子，放了一把火烧了水脏洞。
......
chat-Original data of Zhen Huan: Biography of Zhen Huan
Original data of Journey to the West in vernacular: Journey to the West
Finally, it is organized into data in json format, as follows:
[
{
"instruction": "小姐，别的秀女都在求中选，唯有咱们小姐想被撂牌子，菩萨一定记得真真儿的——",
"input": "",
"output": "嘘——都说许愿说破是不灵的。"
},
{
"instruction": "这个温太医啊，也是古怪，谁不知太医不得皇命不能为皇族以外的人请脉诊病，他倒好，十天半月便往咱们府里跑。",
"input": "",
"output": "你们俩话太多了，我该和温太医要一剂药，好好治治你们。"
},
{
"instruction": "嬛妹妹，刚刚我去府上请脉，听甄伯母说你来这里进香了。",
"input": "",
"output": "出来走走，也是散心。"
}
]
Chat-Huan Huan's data: chat-Zhen Huan
Therefore, the general idea for processing data in this step is:
1. Extract characters and dialogue from raw data 2. Filter out the dialogue of the characters we care about 3. Convert the dialogue into the format we need
This step can also add data enhancement steps, such as using two or three pieces of data as examples and throwing them to LLM, so that it can generate data with similar styles. Alternatively, you can find some data sets of daily conversations and use RAG to generate some fixed character style conversation data. Here everyone can feel free to try it boldly!
This step may be all too familiar to everyone. In each model of self-llm , there will be a Lora fine-tuning module. We only need to process the data into the format we need, and then call our training script.
Here we choose the LLaMA3_1-8B-Instruct model for fine-tuning. First, we need to download the model, create a model_download.py file, and enter the following content:
import torch
from modelscope import snapshot_download , AutoModel , AutoTokenizer
import os
model_dir = snapshot_download ( 'LLM-Research/Meta-Llama-3.1-8B-Instruct' , cache_dir = '/root/autodl-tmp' , revision = 'master' )Note: Remember to modify
cache_dirto your model download path~
Second, prepare the training code. For students who are familiar with self-llm , this step may not be easier. Here I will place train.py in the current directory, and you can modify the data set path and model path.
Of course, you can also use the
lorafine-tuning tutorial inself-llm. Tutorial address: Link
Run the following commands on the command line:
python train.pyNote: Remember to modify the data set path and model path in
train.py~
Training will take about 20 ~ 30 minutes. After the training is completed, the lora model will be generated in the output directory. You can use the following code to test:
from transformers import AutoModelForCausalLM , AutoTokenizer
import torch
from peft import PeftModel
mode_path = './LLM-Research/Meta-Llama-3___1-8B-Instruct'
lora_path = './output/llama3_1_instruct_lora/checkpoint-699' # 这里改称你的 lora 输出对应 checkpoint 地址
# 加载tokenizer
tokenizer = AutoTokenizer . from_pretrained ( mode_path , trust_remote_code = True )
# 加载模型
model = AutoModelForCausalLM . from_pretrained ( mode_path , device_map = "auto" , torch_dtype = torch . bfloat16 , trust_remote_code = True ). eval ()
# 加载lora权重
model = PeftModel . from_pretrained ( model , model_id = lora_path )
prompt = "嬛嬛你怎么了，朕替你打抱不平！"
messages = [
{ "role" : "system" , "content" : "假设你是皇帝身边的女人--甄嬛。" },
{ "role" : "user" , "content" : prompt }
]
input_ids = tokenizer . apply_chat_template ( messages , tokenize = False , add_generation_prompt = True )
# print(input_ids)
model_inputs = tokenizer ([ input_ids ], return_tensors = "pt" ). to ( 'cuda' )
generated_ids = model . generate ( model_inputs . input_ids , max_new_tokens = 512 )
generated_ids = [
output_ids [ len ( input_ids ):] for input_ids , output_ids in zip ( model_inputs . input_ids , generated_ids )
]
response = tokenizer . batch_decode ( generated_ids , skip_special_tokens = True )[ 0 ]
print ( '皇上：' , prompt )
print ( '嬛嬛：' , response )皇上： 嬛嬛你怎么了，朕替你打抱不平！
嬛嬛： 皇上，臣妾不是故意的。
Next, we can use this Huanhuan model to have conversations~
Interested students can try to use other models in self-llm for fine-tuning and test your learning results!
Chat-Huanhuan was created when LLM first became popular last year. We felt that if we didn’t do something, we might miss a lot of interesting things. So I spent a lot of time working on this project with a few friends. In this project, we learned a lot and encountered many problems, but we solved them one by one. And Chat-Huanhuan also won an award, which brought the project to the attention of many people. Therefore, I think this project is very meaningful and interesting.
