bosquet

LLMOps untuk aplikasi berbasis Model Bahasa Besar

Bosquet memiliki misi untuk membuat pembuatan aplikasi AI menjadi sederhana. Semua aplikasi AI nontrivial perlu bekerja dengan template cepat yang kompleksitasnya berkembang dengan cepat, jendela konteks LLM yang terbatas memerlukan manajemen memori, dan agen diperlukan agar aplikasi AI dapat berinteraksi dengan dunia luar.

Bosquet menyediakan instrumen untuk bekerja dengan konsep aplikasi AI tersebut:

• LLM dan manajemen layanan Alat
• Pembuatan templat cepat melalui integrasi dengan perpustakaan templating Selmer yang luar biasa
• Rangkaian dan komposisi cepat melalui mesin pengolah grafik Pathom yang kuat
• Abstraksi definisi agen dan alat untuk interaksi dengan API eksternal
• Penanganan memori LLM
• Instrumen lain seperti cache respons panggilan (lihat dokumentasi)

Dokumentasi

Dokumentasi proyek lengkap (WIP)

Menggunakan

Rahasia seperti kunci disimpan dalam file secrets.edn dan parameter lokal disimpan di config.edn . Buat salinan file config.edn.sample dan config.edn.sample . Ubah sesuai kebutuhan.

CLI

Demo antarmuka baris perintah

Jalankan perintah berikut untuk mendapatkan opsi CLI

 clojure -M -m bosquet.cli

Atur model default dengan

 clojure -M -m bosquet.cli llms set --service openai --temperature 0 --model gpt-4

Jangan lupa untuk mengatur API KEY untuk layanan Anda (ubah 'openai' ke nama lain jika diperlukan)

 clojure -M -m bosquet.cli keys set openai

Dengan set itu, Anda dapat menjalankan beberapa generasi:

 clojure -M -m bosquet.cli "2+{{x}}="

Atau menggunakan file

 clojure -M -m bosquet.cli -p demo/play-writer-prompt.edn -d demo/play-writer-data.edn

Penyelesaian yang cepat

Penyelesaian cepat yang sederhana dapat dilakukan seperti ini.

 (require '[bosquet.llm.generator :refer [generate llm]])

(generate "When I was 6 my sister was half my age. Now I’m 70 how old is my sister?")
=>
"When you were 6, your sister was half your age, which means she was 6 / 2 = 3 years old.nSince then, there is a constant age difference of 3 years between you and your sister.nNow that you are 70, your sister would be 70 - 6 = 64 years old."}

Penyelesaian dari peta prompt

( require '[bosquet.llm :as llm])
( require '[bosquet.llm.generator :refer [generate llm]])

( generate
 llm/default-services
 { :question-answer " Question: {{question}}  Answer: {{answer}} "
  :answer          ( llm :openai )
  :self-eval       [ " Question: {{question}} "
                    " Answer: {{answer}} "
                    " "
                    " Is this a correct answer? "
                    " {{test}} " ]
  :test            ( llm :openai )}
 { :question " What is the distance from Moon to Io? " })
=>

{ :question-answer
 " Question: What is the distance from Moon to Io?  Answer: " ,
 :answer
 " The distance from the Moon to Io varies, as both are orbiting different bodies. On average, the distance between the Moon and Io is approximately 760,000 kilometers (470,000 miles). However, this distance can change due to the elliptical nature of their orbits. " ,
 :self-eval
 " Question: What is the distance from Moon to Io? n Answer: The distance from the Moon to Io varies, as both are orbiting different bodies. On average, the distance between the Moon and Io is approximately 760,000 kilometers (470,000 miles). However, this distance can change due to the elliptical nature of their orbits. nn Is this a correct answer? " ,
 :test
 " No, the answer provided is incorrect. The Moon is Earth's natural satellite, while Io is one of Jupiter's moons. Therefore, the distance between the Moon and Io can vary significantly depending on their relative positions in their respective orbits around Earth and Jupiter. " }

Mengobrol

( require '[bosquet.llm.wkk :as wkk])
( generate
    [[ :system " You are an amazing writer. " ]
     [ :user [ " Write a synopsis for the play: "
             " Title: {{title}} "
             " Genre: {{genre}} "
             " Synopsis: " ]]
     [ :assistant ( llm wkk/openai
                    wkk/model-params { :temperature 0.8 :max-tokens 120 }
                    wkk/var-name :synopsis )]
     [ :user " Now write a critique of the above synopsis: " ]
     [ :assistant ( llm wkk/openai
                    wkk/model-params { :temperature 0.2 :max-tokens 120 }
                    wkk/var-name     :critique )]]
    { :title " Mr. X "
    :genre " Sci-Fi " })
=>

#:bosquet{ :conversation
          [[ :system " You are an amazing writer. " ]
           [ :user
            " Write a synopsis for the play: n Title: Mr. X n Genre: Sci-Fi n Synopsis: " ]
           [ :assistant " In a futuristic world where technology ... " ]
           [ :user " Now write a critique of the above synopsis: " ]
           [ :assistant
            " The synopsis for the play " Mr. X " presents an intriguing premise ... " ]],
          :completions
          { :synopsis
           " In a futuristic world where technology has ... " ,
           :critique
           " The synopsis for the play " Mr. X " presents an intriguing premise set ... " }}

Pengembalian generasi :bosquet/conversation obrolan lengkap dengan bagian yang dihasilkan terisi, dan :bosquet/completions hanya berisi data yang dihasilkan