LLaMa2lang
v0.5
该存储库包含方便的脚本，用于微调 LLaMa3-8B（或任何其他基础模型）以进行任何语言（非英语）的聊天。其背后的基本原理是，LLaMa3 主要是在英语数据上进行训练的，虽然它在一定程度上适用于其他语言，但与英语相比，其性能较差。
将微调的力量与 RAG 的力量相结合 - 查看 RAG 上的 RAG Me Up 存储库，它可以在使用 LLaMa2Lang 调整的模型之上使用。
pip install -r requirements.txt
# Translate OASST1 to target language
python translate.py m2m target_lang checkpoint_location
# Combine the checkpoint files into a dataset
python combine_checkpoints.py input_folder output_location
# Finetune
python finetune.py tuned_model dataset_name instruction_prompt
# Optionally finetune with DPO (RLHF)
python finetune_dpo.py tuned_model dataset_name instruction_prompt
# Run inference
python run_inference.py model_name instruction_prompt input
我们为特定语言调整基础模型（例如 LLaMa3）的过程如下：
以下内容已经过测试，但可能还有更多内容可以使用
上述过程可以在免费的Google Colab T4 GPU上完全运行。然而，最后一步只能在足够短的上下文窗口和最多 2 个批次的情况下才能成功运行。此外，对于任何给定语言，步骤 2 中的翻译总共需要大约 36 小时，因此如果您需要，则应分多个步骤运行想要坚持使用免费的 Google Colab GPU。
我们的第 5 步微调模型是使用vast.ai 上的 A40 执行的，每个模型的成本不到一美元，大约在 1.5 小时内完成。
确保 pytorch 已安装并适用于您的环境（最好使用 CUDA）：https://pytorch.org/get-started/locally/
克隆存储库并安装要求。
pip install -r requirements.txt
usage: translate.py [-h] [--quant8] [--quant4] [--base_dataset BASE_DATASET] [--base_dataset_text_field BASE_DATASET_TEXT_FIELD] [--base_dataset_lang_field BASE_DATASET_LANG_FIELD]
[--checkpoint_n CHECKPOINT_N] [--batch_size BATCH_SIZE] [--max_length MAX_LENGTH] [--cpu] [--source_lang SOURCE_LANG]
{opus,mbart,madlad,m2m,nllb,seamless_m4t_v2,towerinstruct} ... target_lang checkpoint_location
Translate an instruct/RLHF dataset to a given target language using a variety of translation models
positional arguments:
{opus,mbart,madlad,m2m,nllb,seamless_m4t_v2,towerinstruct}
The model/architecture used for translation.
opus Translate the dataset using HelsinkiNLP OPUS models.
mbart Translate the dataset using mBART.
madlad Translate the dataset using Google's MADLAD models.
m2m Translate the dataset using Facebook's M2M models.
nllb Translate the dataset using Facebook's NLLB models.
seamless_m4t_v2 Translate the dataset using Facebook's SeamlessM4T-v2 multimodal models.
towerinstruct Translate the dataset using Unbabel's Tower Instruct. Make sure your target language is in the 10 languages supported by the model.
target_lang The target language. Make sure you use language codes defined by the translation model you are using.
checkpoint_location The folder the script will write (JSONized) checkpoint files to. Folder will be created if it doesn't exist.
options:
-h, --help show this help message and exit
--quant8 Optional flag to load the translation model in 8 bits. Decreases memory usage, increases running time
--quant4 Optional flag to load the translation model in 4 bits. Decreases memory usage, increases running time
--base_dataset BASE_DATASET
The base dataset to translate, defaults to OpenAssistant/oasst1
--base_dataset_text_field BASE_DATASET_TEXT_FIELD
The base dataset's column name containing the actual text to translate. Defaults to text
--base_dataset_lang_field BASE_DATASET_LANG_FIELD
The base dataset's column name containing the language the source text was written in. Defaults to lang
--checkpoint_n CHECKPOINT_N
An integer representing how often a checkpoint file will be written out. To start off, 400 is a reasonable number.
--batch_size BATCH_SIZE
The batch size for a single translation model. Adjust based on your GPU capacity. Default is 10.
--max_length MAX_LENGTH
How much tokens to generate at most. More tokens might be more accurate for lengthy input but creates a risk of running out of memory. Default is unlimited.
--cpu Forces usage of CPU. By default GPU is taken if available.
--source_lang SOURCE_LANG
Source language to select from OASST based on lang property of dataset
如果您想要不同翻译模型的更多参数，请运行：
python translate.py [MODEL] -h
在指定上面列表中的通用参数之前，请务必先指定特定于型号的参数。调用示例：
# Using M2M with 4bit quantization and differen batch sizes to translate Dutch
python translate.py m2m nl ./output_nl --quant4 --batch_size 20
# Using madlad 7B with 8bit quantization for German with different max_length
python translate.py madlad --model_size 7b de ./output_de --quant8 --batch_size 5 --max_length 512
# Be sure to use target language codes that the model you use understands
python translate.py mbart xh_ZA ./output_xhosa
python translate.py nllb nld_Latn ./output_nl
HF_TOKEN环境变量。 usage: combine_checkpoints.py [-h] input_folder output_location
Combine checkpoint files from translation.
positional arguments:
input_folder The checkpoint folder used in translation, with the target language appended.
Example: "./output_nl".
output_location Where to write the Huggingface Dataset. Can be a disk location or a Huggingface
Dataset repository.
options:
-h, --help show this help message and exit
usage: finetune.py [-h] [--base_model BASE_MODEL] [--base_dataset_text_field BASE_DATASET_TEXT_FIELD] [--base_dataset_rank_field BASE_DATASET_RANK_FIELD] [--base_dataset_id_field BASE_DATASET_ID_FIELD] [--base_dataset_parent_field BASE_DATASET_PARENT_FIELD]
[--base_dataset_role_field BASE_DATASET_ROLE_FIELD] [--quant8] [--noquant] [--max_seq_length MAX_SEQ_LENGTH] [--num_train_epochs NUM_TRAIN_EPOCHS] [--batch_size BATCH_SIZE] [--threads_output_name THREADS_OUTPUT_NAME] [--thread_template THREAD_TEMPLATE]
[--padding PADDING]
tuned_model dataset_name instruction_prompt
Finetune a base instruct/chat model using (Q)LoRA and PEFT
positional arguments:
tuned_model The name of the resulting tuned model.
dataset_name The name of the dataset to use for fine-tuning. This should be the output of the combine_checkpoints script.
instruction_prompt An instruction message added to every prompt given to the chatbot to force it to answer in the target language. Example: "You are a generic chatbot that always answers in English."
options:
-h, --help show this help message and exit
--base_model BASE_MODEL
The base foundation model. Default is "NousResearch/Meta-Llama-3-8B-Instruct".
--base_dataset_text_field BASE_DATASET_TEXT_FIELD
The dataset's column name containing the actual text to translate. Defaults to text
--base_dataset_rank_field BASE_DATASET_RANK_FIELD
The dataset's column name containing the rank of an answer given to a prompt. Defaults to rank
--base_dataset_id_field BASE_DATASET_ID_FIELD
The dataset's column name containing the id of a text. Defaults to message_id
--base_dataset_parent_field BASE_DATASET_PARENT_FIELD
The dataset's column name containing the parent id of a text. Defaults to parent_id
--base_dataset_role_field BASE_DATASET_ROLE_FIELD
The dataset's column name containing the role of the author of the text (eg. prompter, assistant). Defaults to role
--quant8 Finetunes the model in 8 bits. Requires more memory than the default 4 bit.
--noquant Do not quantize the finetuning. Requires more memory than the default 4 bit and optional 8 bit.
--max_seq_length MAX_SEQ_LENGTH
The maximum sequence length to use in finetuning. Should most likely line up with your base model's default max_seq_length. Default is 512.
--num_train_epochs NUM_TRAIN_EPOCHS
Number of epochs to use. 2 is default and has been shown to work well.
--batch_size BATCH_SIZE
The batch size to use in finetuning. Adjust to fit in your GPU vRAM. Default is 4
--threads_output_name THREADS_OUTPUT_NAME
If specified, the threads created in this script for finetuning will also be saved to disk or HuggingFace Hub.
--thread_template THREAD_TEMPLATE
A file containing the thread template to use. Default is threads/template_fefault.txt
--padding PADDING What padding to use, can be either left or right.
6.1 [可选] 使用 DPO 进行微调（类似于 RLHF）
usage: finetune_dpo.py [-h] [--base_model BASE_MODEL] [--base_dataset_text_field BASE_DATASET_TEXT_FIELD] [--base_dataset_rank_field BASE_DATASET_RANK_FIELD] [--base_dataset_id_field BASE_DATASET_ID_FIELD] [--base_dataset_parent_field BASE_DATASET_PARENT_FIELD] [--quant8]
[--noquant] [--max_seq_length MAX_SEQ_LENGTH] [--max_prompt_length MAX_PROMPT_LENGTH] [--num_train_epochs NUM_TRAIN_EPOCHS] [--batch_size BATCH_SIZE] [--threads_output_name THREADS_OUTPUT_NAME] [--thread_template THREAD_TEMPLATE] [--max_steps MAX_STEPS]
[--padding PADDING]
tuned_model dataset_name instruction_prompt
Finetune a base instruct/chat model using (Q)LoRA and PEFT using DPO (RLHF)
positional arguments:
tuned_model The name of the resulting tuned model.
dataset_name The name of the dataset to use for fine-tuning. This should be the output of the combine_checkpoints script.
instruction_prompt An instruction message added to every prompt given to the chatbot to force it to answer in the target language. Example: "You are a generic chatbot that always answers in English."
options:
-h, --help show this help message and exit
--base_model BASE_MODEL
The base foundation model. Default is "NousResearch/Meta-Llama-3-8B-Instruct".
--base_dataset_text_field BASE_DATASET_TEXT_FIELD
The dataset's column name containing the actual text to translate. Defaults to text
--base_dataset_rank_field BASE_DATASET_RANK_FIELD
The dataset's column name containing the rank of an answer given to a prompt. Defaults to rank
--base_dataset_id_field BASE_DATASET_ID_FIELD
The dataset's column name containing the id of a text. Defaults to message_id
--base_dataset_parent_field BASE_DATASET_PARENT_FIELD
The dataset's column name containing the parent id of a text. Defaults to parent_id
--quant8 Finetunes the model in 8 bits. Requires more memory than the default 4 bit.
--noquant Do not quantize the finetuning. Requires more memory than the default 4 bit and optional 8 bit.
--max_seq_length MAX_SEQ_LENGTH
The maximum sequence length to use in finetuning. Should most likely line up with your base model's default max_seq_length. Default is 512.
--max_prompt_length MAX_PROMPT_LENGTH
The maximum length of the prompts to use. Default is 512.
--num_train_epochs NUM_TRAIN_EPOCHS
Number of epochs to use. 2 is default and has been shown to work well.
--batch_size BATCH_SIZE
The batch size to use in finetuning. Adjust to fit in your GPU vRAM. Default is 4
--threads_output_name THREADS_OUTPUT_NAME
If specified, the threads created in this script for finetuning will also be saved to disk or HuggingFace Hub.
--thread_template THREAD_TEMPLATE
A file containing the thread template to use. Default is threads/template_fefault.txt
--max_steps MAX_STEPS
The maximum number of steps to run DPO for. Default is -1 which will run the data through fully for the number of epochs but this will be very time-consuming.
--padding PADDING What padding to use, can be either left or right.
6.1 [可选] 使用 ORPO 进行微调（类似于 RLHF）
usage: finetune_orpo.py [-h] [--base_model BASE_MODEL] [--base_dataset_text_field BASE_DATASET_TEXT_FIELD] [--base_dataset_rank_field BASE_DATASET_RANK_FIELD] [--base_dataset_id_field BASE_DATASET_ID_FIELD] [--base_dataset_parent_field BASE_DATASET_PARENT_FIELD] [--quant8]
[--noquant] [--max_seq_length MAX_SEQ_LENGTH] [--max_prompt_length MAX_PROMPT_LENGTH] [--num_train_epochs NUM_TRAIN_EPOCHS] [--batch_size BATCH_SIZE] [--threads_output_name THREADS_OUTPUT_NAME] [--thread_template THREAD_TEMPLATE] [--max_steps MAX_STEPS]
[--padding PADDING]
tuned_model dataset_name instruction_prompt
Finetune a base instruct/chat model using (Q)LoRA and PEFT using ORPO (RLHF)
positional arguments:
tuned_model The name of the resulting tuned model.
dataset_name The name of the dataset to use for fine-tuning. This should be the output of the combine_checkpoints script.
instruction_prompt An instruction message added to every prompt given to the chatbot to force it to answer in the target language. Example: "You are a generic chatbot that always answers in English."
options:
-h, --help show this help message and exit
--base_model BASE_MODEL
The base foundation model. Default is "NousResearch/Meta-Llama-3-8B-Instruct".
--base_dataset_text_field BASE_DATASET_TEXT_FIELD
The dataset's column name containing the actual text to translate. Defaults to text
--base_dataset_rank_field BASE_DATASET_RANK_FIELD
The dataset's column name containing the rank of an answer given to a prompt. Defaults to rank
--base_dataset_id_field BASE_DATASET_ID_FIELD
The dataset's column name containing the id of a text. Defaults to message_id
--base_dataset_parent_field BASE_DATASET_PARENT_FIELD
The dataset's column name containing the parent id of a text. Defaults to parent_id
--quant8 Finetunes the model in 8 bits. Requires more memory than the default 4 bit.
--noquant Do not quantize the finetuning. Requires more memory than the default 4 bit and optional 8 bit.
--max_seq_length MAX_SEQ_LENGTH
The maximum sequence length to use in finetuning. Should most likely line up with your base model's default max_seq_length. Default is 512.
--max_prompt_length MAX_PROMPT_LENGTH
The maximum length of the prompts to use. Default is 512.
--num_train_epochs NUM_TRAIN_EPOCHS
Number of epochs to use. 2 is default and has been shown to work well.
--batch_size BATCH_SIZE
The batch size to use in finetuning. Adjust to fit in your GPU vRAM. Default is 4
--threads_output_name THREADS_OUTPUT_NAME
If specified, the threads created in this script for finetuning will also be saved to disk or HuggingFace Hub.
--thread_template THREAD_TEMPLATE
A file containing the thread template to use. Default is threads/template_fefault.txt
--max_steps MAX_STEPS
The maximum number of steps to run ORPO for. Default is -1 which will run the data through fully for the number of epochs but this will be very time-consuming.
--padding PADDING What padding to use, can be either left or right.
usage: run_inference.py [-h] model_name instruction_prompt input
Script to run inference on a tuned model.
positional arguments:
model_name The name of the tuned model that you pushed to Huggingface in the previous
step.
instruction_prompt An instruction message added to every prompt given to the chatbot to force
it to answer in the target language.
input The actual chat input prompt. The script is only meant for testing purposes
and exits after answering.
options:
-h, --help show this help message and exit
我如何知道为我的目标语言选择哪种翻译模型？
我们为您提供了benchmark.py脚本，该脚本有助于做出一些好的猜测（我们使用的数据集与训练 OPUS 模型的数据集相同，因此结果总是有利于 OPUS）。有关使用方法，请参阅下面该脚本的帮助。模型以 4 位量化加载，并在 OPUS 书籍子集的小样本上运行。
请务必使用基础数据集中最常见的语言作为源语言，使用目标翻译语言作为目标语言。例如，对于 OASST1，请确保至少运行en和es作为源语言。
usage: benchmark.py [-h] [--cpu] [--start START] [--n N] [--max_length MAX_LENGTH] source_language target_language included_models
Benchmark all the different translation models for a specific source and target language to find out which performs best. This uses 4bit quantization to limit GPU usage. Note:
the outcomes are indicative - you cannot assume corretness of the BLEU and CHRF scores but you can compare models against each other relatively.
positional arguments:
source_language The source language you want to test for. Check your dataset to see which occur most prevalent or use English as a good start.
target_language The source language you want to test for. This should be the language you want to apply the translate script on. Note: in benchmark, we use 2-character
language codes, in constrast to translate.py where you need to specify whatever your model expects.
included_models Comma-separated list of models to include. Allowed values are: opus, m2m_418m, m2m_1.2b, madlad_3b, madlad_7b, madlad_10b, madlad_7bbt, mbart,
nllb_distilled600m, nllb_1.3b, nllb_distilled1.3b, nllb_3.3b, seamless
options:
-h, --help show this help message and exit
--cpu Forces usage of CPU. By default GPU is taken if available.
--start START The starting offset to include sentences from the OPUS books dataset from. Defaults to 0.
--n N The number of sentences to benchmark on. Defaults to 100.
--max_length MAX_LENGTH
How much tokens to generate at most. More tokens might be more accurate for lengthy input but creates a risk of running out of memory. Default is 512.
我们已经创建并将继续创建大量数据集和模型。想帮助法学硕士的民主化吗？克隆存储库并为其他语言创建数据集和模型，然后创建 PR。
| 荷兰语理解Ling/oasst1_nl | 西班牙语理解Ling/oast1_es | 法语理解Ling/oast1_fr | 德语理解Ling/oast1_de |
| 加泰罗尼亚语 xaviviro/oasst1_ca | 葡萄牙语理解Ling/oasst1_pt | 阿拉伯语 HeshamHaroon/oasst-arabic | 意大利语理解Ling/oast1_it |
| 俄语理解Ling/oast1_ru | 印地语理解Ling/oasst1_hi | 中文理解Ling/oasst1_zh | 波兰基督徒/oast1_pl |
| 日语理解Ling/oast1_jap | 巴斯克 xezpeleta/oasst1_eu | 孟加拉语 UnderstandingLing/oasst1_bn | 土耳其语理解Ling/oast1_tr |
在使用这些模型之前，请确保您有权访问 Meta 的 LLaMa3-8B 模型并设置您的 HF_TOKEN。
| UnderstandingLing/Llama-3-8B-Instruct-nl 荷兰语 | UnderstandingLing/Llama-3-8B-Instruct-es 西班牙语 | 了解Ling/Llama-3-8B-Instruct-fr 法语 | UnderstandingLing/Llama-3-8B-Instruct-de 德语 |
| 了解Ling/Llama-3-8B-Instruct-pt 葡萄牙语 | UnderstandingLing/Llama-3-8B-Instruct-it 意大利语 | UnderstandingLing/Llama-3-8B-Instruct-hi 印地语 | 了解Ling/Llama-3-8B-Instruct-ru 俄语 |
| 荷兰语 UnderstandingLing/oasst1_nl_threads | 西班牙语 UnderstandingLing/oast1_es_threads | 法语理解Ling/oast1_fr_threads | 德语理解Ling/oast1_de_threads |
| 加泰罗尼亚语 xaviviro/oasst1_ca_threads | 葡萄牙语理解Ling/oast1_pt_threads | 阿拉伯语 HeshamHaroon/oasst-arabic_threads | 意大利语理解Ling/oast1_it_threads |
| 俄语理解Ling/oast1_ru_threads | 印地语理解Ling/oasst1_hi_threads | 中文理解Ling/oasst1_zh_threads | 波兰基督徒/oasst1_pl_threads |
| 日语理解Ling/oast1_jap_threads | 巴斯克 xezpeleta/oasst1_eu_threads | 孟加拉语 UnderstandingLing/oasst1_bn_threads | 土耳其语 UnderstandingLing/oast1_tr_threads |
| 了解Ling/llama-2-7b-chat-nl 荷兰语 | 了解Ling/llama-2-7b-chat-es 西班牙语 | 了解Ling/llama-2-7b-chat-fr 法语 | 了解Ling/llama-2-7b-chat-de 德语 |
| xaviviro/llama-2-7b-chat-ca 加泰罗尼亚语 | 了解Ling/llama-2-7b-chat-pt 葡萄牙语 | HeshamHaroon/llama-2-7b-chat-ar 阿拉伯语 | 了解Ling/llama-2-7b-chat-it 意大利语 |
| 了解Ling/llama-2-7b-chat-ru 俄语 | 了解Ling/llama-2-7b-chat-hi 印地语 | 了解Ling/llama-2-7b-chat-zh 中文 | chrystians/llama-2-7b-chat-pl-polish-polski 波兰语 |
| xezpeleta/llama-2-7b-chat-eu 巴斯克语 | UnderstandingLing/llama-2-7b-chat-bn 孟加拉语 | 了解Ling/llama-2-7b-chat-tr 土耳其语 |
| UnderstandingLing/Mistral-7B-Instruct-v0.2-nl 荷兰语 | UnderstandingLing/Mistral-7B-Instruct-v0.2-es 西班牙语 | UnderstandingLing/Mistral-7B-Instruct-v0.2-de 德语 |
| 了解Ling/llama-2-13b-chat-nl 荷兰语 | 了解Ling/llama-2-13b-chat-es 西班牙语 | 了解Ling/llama-2-13b-chat-fr 法语 |
| UnderstandingLing/Mixtral-8x7B-Instruct-nl 荷兰语 |
[INST] <
[INST] <[INST] Hoeveel inwoners heeft die stad? [/INST] 850 duizend inwoners (2023)
[INST] <[INST] Hoeveel inwoners heeft die stad? [/INST] 850 duizend inwoners (2023)[INST] In welke provincie ligt die stad? [/INST] In de provincie Noord-Holland
[INST] <
问：为什么要先翻译完整的 OASST1/2 数据集？只翻译排名最高的线程不是更快吗？
答：虽然通过首先创建线程然后翻译它们可以在吞吐时间方面获得相当多的收益，但我们向社区提供完整的 OASST1/2 翻译，因为我们相信它们本身就很有用。
问：与原版 LLaMa3 相比，微调的表现如何？
答：虽然我们没有正式的基准，但让 LLaMa3 始终说英语以外的另一种语言即使不是不可能，也是具有挑战性的。它产生的非英语语言通常在语法上是错误的。我们的微调并没有表现出这种行为。
Q：我可以使用其他框架进行微调吗？
答：是的，可以，我们使用 Axolotl 进行多 GPU 设置上的训练。
问：我可以混合使用不同的翻译模型吗？
答：当然，我们认为通过多个模型完成翻译甚至可能会提高性能。您可以通过提前停止翻译并通过使用不同的翻译模型重新运行翻译脚本从检查点继续来实现此目的。
我们正在积极寻找资金来实现人工智能的民主化并推进其应用。如果您想投资，请通过 [email protected] 联系我们。
