llm toys
1.0.0
小型（7B 及以下）、生產就緒的微調法學碩士，適用於各種有用的任務。
支援的任務：釋義、改變段落語氣、從對話中產生摘要和主題，檢索增強 QA（WIP） 。
我們在量化 3B 和 7B 模型上微調 LoRA。 3B 模型針對特定任務進行微調，而 7B 模型則針對所有任務進行微調。
目標是能夠在非常適度的消費級硬體上微調和使用所有這些模型。
pip install llm-toys如果沒有啟用 CUDA 的 GPU，可能無法運作
如果您在使用bitsandbytes時遇到“已安裝的bitsandbytes版本是在沒有GPU支援的情況下編譯的”，請查看此處bitsandbytes-foundation/bitsandbytes#112
或嘗試
cp <path_to_your_venv>/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so <path_to_your_venv>/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda17.
請注意，我們使用的是來源目錄中的轉換器和 pef 包，而不是已安裝的套件。 4位元bitsandbytes量化僅適用於變壓器和peft的主要分支。一旦 Transformers 版本 4.31.0 和 peft 版本 0.4.0 發佈到 pypi，我們將使用發布的版本。
| 模型 | 尺寸 | 任務 | 科拉布 |
|---|---|---|---|
| llm-toys/RedPajama-INCITE-Base-3B-v1-釋義音 | 3B | 釋義、語氣變化 | 筆記本 |
| llm-toys/RedPajama-INCITE-Base-3B-v1-對話-摘要-主題 | 3B | 對話總結和主題生成 | 筆記本 |
| llm-toys/falcon-7b-釋義-語氣-對話-摘要-主題 | 7B | 釋義、語氣變化、對話摘要和話題生成 | 筆記本 |
from llm_toys . tasks import Paraphraser
paraphraser = Paraphraser ()
paraphraser . paraphrase ( "Hey, can yuo hepl me cancel my last order?" )
# "Could you kindly assist me in canceling my previous order?" paraphraser . paraphrase ( "Hey, can yuo hepl me cancel my last order?" , tone = "casual" )
# "Hey, could you help me cancel my order?"
paraphraser . paraphrase ( "Hey, can yuo hepl me cancel my last order?" , tone = "professional" )
# "I would appreciate guidance on canceling my previous order."
paraphraser . paraphrase ( "Hey, can yuo hepl me cancel my last order?" , tone = "witty" )
# "Hey, I need your help with my last order. Can you wave your magic wand and make it disappear?" from llm_toys . tasks import SummaryAndTopicGenerator
summary_topic_generator = SummaryAndTopicGenerator ()
summary_topic_generator . generate_summary_and_topic (
"""
#Person1#: I'm so excited for the premiere of the latest Studio Ghibli movie!
#Person2#: What's got you so hyped?
#Person1#: Studio Ghibli movies are pure magic! The animation, storytelling, everything is incredible.
#Person2#: Which movie is it?
#Person1#: It's called "Whisper of the Wind." It's about a girl on a magical journey to save her village.
#Person2#: Sounds amazing! I'm in for the premiere.
#Person1#: Great! We're in for a visual masterpiece and a heartfelt story.
#Person2#: Can't wait to be transported to their world.
#Person1#: It'll be an unforgettable experience, for sure!
""" . strip ()
)
# {"summary": "#Person1# is excited for the premiere of the latest Studio Ghibli movie.
# #Person1# thinks the animation, storytelling, and heartfelt story will be unforgettable.
# #Person2# is also excited for the premiere.",
# "topic": "Studio ghibli movie"} from llm_toys . tasks import GeneralTaskAssitant
from llm_toys . config import TaskType
gta = GeneralTaskAssitant ()
gta . complete ( TaskType . PARAPHRASE_TONE , "Hey, can yuo hepl me cancel my last order?" )
# "Could you assist me in canceling my previous order?"
gta . complete ( TaskType . PARAPHRASE_TONE , "Hey, can yuo hepl me cancel my last order?" , tone = "casual" )
# "Hey, can you help me cancel my last order?"
gta . complete ( TaskType . PARAPHRASE_TONE , "Hey, can yuo hepl me cancel my last order?" , tone = "professional" )
# "I would appreciate if you could assist me in canceling my previous order."
gta . complete ( TaskType . PARAPHRASE_TONE , "Hey, can yuo hepl me cancel my last order?" , tone = "witty" )
# "Oops! Looks like I got a little carried away with my shopping spree. Can you help me cancel my last order?"
chat = """
#Person1#: I'm so excited for the premiere of the latest Studio Ghibli movie!
#Person2#: What's got you so hyped?
#Person1#: Studio Ghibli movies are pure magic! The animation, storytelling, everything is incredible.
#Person2#: Which movie is it?
#Person1#: It's called "Whisper of the Wind." It's about a girl on a magical journey to save her village.
#Person2#: Sounds amazing! I'm in for the premiere.
#Person1#: Great! We're in for a visual masterpiece and a heartfelt story.
#Person2#: Can't wait to be transported to their world.
#Person1#: It'll be an unforgettable experience, for sure!
""" . strip ()
gta . complete ( TaskType . DIALOGUE_SUMMARY_TOPIC , chat )
# {"summary": "#Person1# tells #Person2# about the upcoming Studio Ghibli movie.
# #Person1# thinks it's magical and #Person2#'s excited to watch it.",
# "topic": "Movie premiere"} 釋義與語氣變化：包含段落及其釋義版本，以及休閒、專業、詼諧等不同語氣的段落。用於模仿來改寫和改變段落的語氣。數據是使用 gpt-35-turbo 產生的。也從 quora 問題和squad_2 資料集中選取了一小部分訓練段落樣本。
對話摘要和主題生成：包含對話及其摘要和主題。訓練資料是來自 Dialogsum 資料集訓練分割的約 1k 筆記錄。它還包含來自開發組的約 20 個樣本。在採樣中優先考慮具有較長摘要和主題的資料點。請注意，最終訓練資料中的一些（~30）主題是手動編輯的，因為原始標記的主題只是一個單詞，描述性不夠。
查看所有選項
python llm_toys/train.py --help訓練釋義和語氣變化模型
python llm_toys/train.py
--task_type paraphrase_tone
--model_name meta-llama/Llama-2-7b
--max_length 128
--batch_size 8
--gradient_accumulation_steps 1
--learning_rate 1e-4
--num_train_epochs 3
--eval_ratio 0.05在製品
對 Dialogsum 測試分組中的 500 筆記錄進行評估。
# llm-toys/RedPajama-INCITE-Base-3B-v1-dialogue-summary-topic
{ "rouge1" : 0.453 , "rouge2" : 0.197 , "rougeL" : 0.365 , "topic_similarity" : 0.888 }
# llm-toys/falcon-7b-paraphrase-tone-dialogue-summary-topic
{ 'rouge1' : 0.448 , 'rouge2' : 0.195 , 'rougeL' : 0.359 , 'topic_similarity' : 0.886 }
