makit akan lambda

LLM Di dalam Fungsi AWS Lambda

Daripada menggunakan layanan seperti ChatGPT, Amazon Bedrock, Google Bard - ada beberapa model bahasa besar sumber terbuka yang dapat digunakan untuk menjalankan chatbot secara lokal, GPT4ALL adalah salah satu opsinya.

Hal ini membuat saya bertanya-tanya apakah mungkin menjalankan LLM di dalam fungsi AWS Lambda. Ini biasanya merupakan fungsi yang kecil, cepat dijalankan, dan digerakkan oleh peristiwa. Mereka tidak dimaksudkan untuk digunakan untuk tugas-tugas pemrosesan yang rumit, tetapi mereka skalanya nol dan Anda hanya membayar untuk apa yang Anda gunakan. Memiliki LLM di dalam fungsi Lambda tampaknya merupakan eksperimen yang menyenangkan, dan merupakan cara untuk memiliki model yang dihosting yang tidak memerlukan proses yang berjalan lama.

Ringkasan

Repositori ini berisi kode untuk membuat fungsi AWS Lambda runtime kustom menggunakan gambar kontainer. Itu dapat dijalankan secara lokal di dalam Docker untuk pengujian juga. Fungsi ini berisi model LLM lengkap dan kode untuk menggunakan model tersebut, memungkinkan pembuatan teks dasar dari panggilan HTTP ke dalamnya.

Ini menggunakan:

• LangChain
• GPT4ALL
• Gambar Kontainer Python

Menyebarkan

1. Unduh model GPT4ALL. Untuk ini saya menggunakan gpt4all-falcon-newbpe-q4_0.gguf karena modelnya kecil (4GB...) yang memiliki respon bagus. Model lain seharusnya berfungsi, namun harus cukup kecil agar sesuai dengan batas memori Lambda.

2. Tempatkan file bin di dalam folder function , di sebelah file lambda_function.py.

3. Bangun image dengan buruh pelabuhan: docker build --platform linux/amd64 -t makgpt:test1 .

4. Setelah dibuat maka Anda dapat menjalankannya secara lokal: docker run -p 9000:8080 makgpt:test1

5. Setelah berjalan maka Anda dapat menjalankan pemanggilan terhadapnya di jendela terminal lain, seperti curl "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{"body":"{"action":"Set fire to a tree"}"}'

6. Dorong gambar ke ECR di dalam akun AWS Anda. Untuk melakukan ini, Anda dapat mengikuti panduan AWS di bagian Deploying the image di Panduan Pengembang. Langkah sederhananya adalah:

   • aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 111122223333.dkr.ecr.us-east-1.amazonaws.com
   • aws ecr create-repository --repository-name hello-world --image-scanning-configuration scanOnPush=true --image-tag-mutability MUTABLE
   • docker tag docker-image:test :latest
   • docker tag docker-image:test 111122223333.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest
   • docker push 111122223333.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest

7. Buat fungsi Lambda dari gambar ini, pastikan untuk memberikan memori yang cukup (disarankan 10 GB) dan batas waktu minimal 5 menit.

8. Untuk mempermudah, berikan URL Fungsi pada fungsi Lambda dan Anda dapat menekannya melalui HTTP.

9. Anda kemudian dapat mencobanya dengan URL seperti: curl "https://INSERTHERE.lambda-url.us-east-1.on.aws/" -d '{"body":"{"action":"Set fire to a tree"}"}'

Contoh

Fungsi ini disiapkan sebagai game petualangan teks dasar dengan prompt:

You are a Text-Based Fantasy RPG. The player is exploring the dark forest and you are the dungeon master. The player performs the action: {action} What happens next? You are talking to the player.

Orc yang marah

Badan Permintaan:

 {
  "action": "Throw your sword at the angry orc"
}

Tanggapan:

 "nAs the player throws their sword at the angry orc, it strikes true and lands squarely in its chest. The orc roars in pain and anger, charging towards the player with a fierce look in its eyes.nnThe player quickly dodges out of the way, but not before noticing something strange about the orc's eyes. They seem to be glowing with an otherworldly light, as if they are possessed by some dark force.nnWith the orc now dangerously close, the player decides to take a risk and try to reason with it. "Stop! I mean you no harm," they say, holding out their hands in a gesture of peace.nnTo their surprise, the orc seems to understand them and hesitates for a moment before continuing its attack. The player quickly grabs their sword and prepares to defend themselves if necessary.nnAs the fight continues, it becomes clear that this orc is not like any other they have faced before. Its movements are unnaturally quick and its strength is unmatched. But with each blow of the sword, the player gains a little more confidence in their abilities.nnFinally, after what seems like an eternity, the orc falls to the ground, defeated."

Waktu yang dibutuhkan: 1m 33s

Mantra Penyihir

Badan Permintaan

 {
  "action": "A wizard casts a spell on the player and turns them into a pencil"
}

Tanggapan:

 "nAs the wizard finishes casting his spell, the player suddenly finds themselves transformed into a pencil. They look down at their new form in confusion before noticing that they are no longer holding the wand they were using to cast spells.nnThe player tries to move around, but find that they are unable to do so without making a noise. They try to speak, but find that they cannot make any sounds with their new pencil body.nnSuddenly, they hear a voice from behind them. "Well, well, well. Look who we have here." The player turns around to see the wizard who cast the spell on them.nnThe wizard chuckles at the player's transformation and begins to speak in a mocking tone. "I bet you didn't expect this when you decided to mess with magic, did you?"nnThe player tries to respond, but finds that they are unable to do so without making any noise. They feel helpless and frustrated as they try to come up with a plan to escape their new form.nnWhat will the player do next?"

Waktu yang dibutuhkan: 1m 1s