GPT2 ChineseChat NCNN

GPT2-ChineseChat-NCNN

What : GPT has many excellent models. Choosing a model has given me a headache for a long time. People tend to use the minGPT model for deployment, but I have not been able to train a Chinese model. So I had to choose a ready-made Chinese model. Considering that the dialogue format is better displayed, I chose the GPT2-chitchat project. (PS: After testing, the dialogue quality of this model is not high. It can only be used for fun. It mainly shows the work of putting GPT into ncnn)

Work goal : Use ncnn to deploy GPT2-chitchat, a Chinese conversation model based on GPT2. The goal is to provide demos for x86 and android.

exhibit

android: APK download(6666)

x86: (At present, it looks a bit normal, but also feels a bit nonsense)

Work content

• Pytorch model sorting and exporting
• x86 demo (PS: Because the model is too big, I split it into four and uploaded it to github, so I have to cat the four bins under assert into one)
• android demo (To compile, copy the bin model under x86 assert to android assert, the same)

current problem

1. The x86 project only relies on ncnn, but I modified one step in the ncnn source code to adapt to the calculation of the model. I am considering changing it to a model that can be used by native ncnn when making the Android version.
2. The current implementation of tokenizer is relatively casual. Common Chinese should be no problem. In English, word segmentation cannot be distinguished, and it can be translated letter by letter.

refer to

1. ncnn
2. GPT2-chitchat