Ini adalah klien Async Scala yang tidak masuk akal untuk API OpenAI yang mendukung semua titik akhir dan param yang tersedia termasuk streaming , penyelesaian obrolan terbaru, visi , dan rutinitas suara (sebagaimana didefinisikan di sini), yang disediakan dalam satu, layanan nyaman yang disebut OpenAiservice. Panggilan yang didukung adalah:
Perhatikan bahwa agar konsisten dengan penamaan API OpenAI, nama fungsi layanan cocok dengan tepat judul/deskripsi titik akhir API dengan CamelCase. Juga, kami mengarahkan Lib untuk mandiri dengan ketergantungan sececepat yang dimungkinkan oleh karena itu kami akhirnya hanya menggunakan dua LIBS play-ahc-ws-standalone
dan play-ws-standalone-json
(di level atas). Selain itu, jika injeksi ketergantungan diperlukan, kami juga menggunakan Lib scala-guice
.
Tidak ada waktu untuk membaca tutorial yang panjang? Tentu, kami mendengarmu! Lihat contoh untuk melihat cara menggunakan lib dalam praktik.
Selain API OpenAI, perpustakaan ini juga mendukung penyedia yang kompatibel dengan API (lihat contoh) seperti:
Untuk informasi latar belakang, baca artikel tentang lib/klien di medium.
Coba juga klien Scala kami untuk database Pinecone Vector, atau gunakan kedua klien bersama -sama! Proyek demo ini menunjukkan cara menghasilkan dan menyimpan embeddings openai (dengan model text-embedding-ada-002
) ke dalam pinus dan menanyakannya sesudahnya. Kombo OpenAI + Pinecone biasanya digunakan untuk agen AI otonom, seperti babyagi dan autogpt.
✔️ PENTING : Ini adalah perpustakaan "terawat di komunitas" dan, dengan demikian, tidak ada hubungannya dengan perusahaan openai.
Versi Scala yang saat ini didukung adalah 2.12, 2.13 , dan 3 .
Untuk menginstal perpustakaan, tambahkan ketergantungan berikut ke build.sbt Anda
"io.cequence" %% "openai-scala-client" % "1.1.0"
atau ke pom.xml (jika Anda menggunakan maven)
<dependency>
<groupId>io.cequence</groupId>
<artifactId>openai-scala-client_2.12</artifactId>
<version>1.1.0</version>
</dependency>
Jika Anda ingin dukungan streaming, gunakan "io.cequence" %% "openai-scala-client-stream" % "1.1.0"
sebagai gantinya.
OPENAI_SCALA_CLIENT_API_KEY
dan secara opsional juga OPENAI_SCALA_CLIENT_ORG_ID
(jika Anda memilikinya)I. Mendapatkan OpenAiservice
Pertama, Anda perlu memberikan konteks eksekusi implisit serta AKKA Materizer, misalnya, sebagai
implicit val ec = ExecutionContext .global
implicit val materializer = Materializer ( ActorSystem ())
Kemudian Anda dapat memperoleh layanan dengan salah satu cara berikut.
Config
) val service = OpenAIServiceFactory ()
val config = ConfigFactory .load( " path_to_my_custom_config " )
val service = OpenAIServiceFactory (config)
val service = OpenAIServiceFactory (
apiKey = " your_api_key " ,
orgId = Some ( " your_org_id " ) // if you have one
)
val service = OpenAIServiceFactory .forAzureWithApiKey(
resourceName = " your-resource-name " ,
deploymentId = " your-deployment-id " , // usually model name such as "gpt-35-turbo"
apiVersion = " 2023-05-15 " , // newest version
apiKey = " your_api_key "
)
OpenAICoreService
Supping listModels
, createCompletion
, createChatCompletion
, dan createEmbeddings
Calls - Disediakan misalnya oleh Layanan FastChat yang berjalan di Port 8000 val service = OpenAICoreServiceFactory ( " http://localhost:8000/v1/ " )
OpenAIChatCompletionService
Menyediakan semata -mata createChatCompletion
val service = OpenAIChatCompletionServiceFactory .forAzureAI(
endpoint = sys.env( " AZURE_AI_COHERE_R_PLUS_ENDPOINT " ),
region = sys.env( " AZURE_AI_COHERE_R_PLUS_REGION " ),
accessToken = sys.env( " AZURE_AI_COHERE_R_PLUS_ACCESS_KEY " )
)
openai-scala-anthropic-client
dan ANTHROPIC_API_KEY
val service = AnthropicServiceFactory .asOpenAI()
openai-scala-google-vertexai-client
dan VERTEXAI_LOCATION
+ VERTEXAI_PROJECT_ID
val service = VertexAIServiceFactory .asOpenAI()
GROQ_API_KEY"
val service = OpenAIChatCompletionServiceFactory ( ChatProviderSettings .groq)
// or with streaming
val service = OpenAIChatCompletionServiceFactory .withStreaming( ChatProviderSettings .groq)
GROK_API_KEY"
val service = OpenAIChatCompletionServiceFactory ( ChatProviderSettings .grok)
// or with streaming
val service = OpenAIChatCompletionServiceFactory .withStreaming( ChatProviderSettings .grok)
FIREWORKS_API_KEY"
val service = OpenAIChatCompletionServiceFactory ( ChatProviderSettings .fireworks)
// or with streaming
val service = OpenAIChatCompletionServiceFactory .withStreaming( ChatProviderSettings .fireworks)
OCTOAI_TOKEN
val service = OpenAIChatCompletionServiceFactory ( ChatProviderSettings .octoML)
// or with streaming
val service = OpenAIChatCompletionServiceFactory .withStreaming( ChatProviderSettings .octoML)
TOGETHERAI_API_KEY
val service = OpenAIChatCompletionServiceFactory ( ChatProviderSettings .togetherAI)
// or with streaming
val service = OpenAIChatCompletionServiceFactory .withStreaming( ChatProviderSettings .togetherAI)
CEREBRAS_API_KEY
val service = OpenAIChatCompletionServiceFactory ( ChatProviderSettings .cerebras)
// or with streaming
val service = OpenAIChatCompletionServiceFactory .withStreaming( ChatProviderSettings .cerebras)
MISTRAL_API_KEY
val service = OpenAIChatCompletionServiceFactory ( ChatProviderSettings .mistral)
// or with streaming
val service = OpenAIChatCompletionServiceFactory .withStreaming( ChatProviderSettings .mistral)
val service = OpenAIChatCompletionServiceFactory (
coreUrl = " http://localhost:11434/v1/ "
)
atau dengan streaming
val service = OpenAIChatCompletionServiceFactory .withStreaming(
coreUrl = " http://localhost:11434/v1/ "
)
createCompletionStreamed
dan createChatCompletionStreamed
disediakan oleh OpenAistreamedServiceExtra (memerlukan lib openai-scala-client-stream
)) import io . cequence . openaiscala . service . StreamedServiceTypes . OpenAIStreamedService
import io . cequence . openaiscala . service . OpenAIStreamedServiceImplicits . _
val service : OpenAIStreamedService = OpenAIServiceFactory .withStreaming()
Demikian pula untuk layanan pelengkap obrolan
import io . cequence . openaiscala . service . OpenAIStreamedServiceImplicits . _
val service = OpenAIChatCompletionServiceFactory .withStreaming(
coreUrl = " https://api.fireworks.ai/inference/v1/ " ,
authHeaders = Seq (( " Authorization " , s " Bearer ${sys.env( " FIREWORKS_API_KEY " )} " ))
)
atau hanya jika streaming diperlukan
val service : OpenAIChatCompletionStreamedServiceExtra =
OpenAIChatCompletionStreamedServiceFactory (
coreUrl = " https://api.fireworks.ai/inference/v1/ " ,
authHeaders = Seq (( " Authorization " , s " Bearer ${sys.env( " FIREWORKS_API_KEY " )} " ))
)
openai-scala-guice
) class MyClass @ Inject () ( openAIService : OpenAIService ) {...}
Ii. Fungsi panggilan
Dokumentasi lengkap dari setiap panggilan dengan input dan pengaturan masing -masing disediakan di OpenAiservice. Karena semua panggilan tidak ada, mereka mengembalikan tanggapan yang dibungkus di Future
.
Ada proyek baru openai-scala-klien-contoh di mana Anda dapat menemukan banyak contoh siap pakai!
service.listModels.map(models =>
models.foreach(println)
)
service.retrieveModel( ModelId .text_davinci_003).map(model =>
println(model.getOrElse( " N/A " ))
)
val text = """ Extract the name and mailing address from this email:
|Dear Kelly,
|It was great to talk to you at the seminar. I thought Jane's talk was quite good.
|Thank you for the book. Here's my address 2111 Ash Lane, Crestview CA 92002
|Best,
|Maya
""" .stripMargin
service.createCompletion(text).map(completion =>
println(completion.choices.head.text)
)
val text = """ Extract the name and mailing address from this email:
|Dear Kelly,
|It was great to talk to you at the seminar. I thought Jane's talk was quite good.
|Thank you for the book. Here's my address 2111 Ash Lane, Crestview CA 92002
|Best,
|Maya
""" .stripMargin
service.createCompletion(
text,
settings = CreateCompletionSettings (
model = ModelId .gpt_4o,
max_tokens = Some ( 1500 ),
temperature = Some ( 0.9 ),
presence_penalty = Some ( 0.2 ),
frequency_penalty = Some ( 0.2 )
)
).map(completion =>
println(completion.choices.head.text)
)
val source = service.createCompletionStreamed(
prompt = " Write me a Shakespeare poem about two cats playing baseball in Russia using at least 2 pages " ,
settings = CreateCompletionSettings (
model = ModelId .text_davinci_003,
max_tokens = Some ( 1500 ),
temperature = Some ( 0.9 ),
presence_penalty = Some ( 0.2 ),
frequency_penalty = Some ( 0.2 )
)
)
source.map(completion =>
println(completion.choices.head.text)
).runWith( Sink .ignore)
Agar ini berfungsi, Anda perlu menggunakan OpenAIServiceStreamedFactory
dari openai-scala-client-stream
LIB.
val createChatCompletionSettings = CreateChatCompletionSettings (
model = ModelId .gpt_4o
)
val messages = Seq (
SystemMessage ( " You are a helpful assistant. " ),
UserMessage ( " Who won the world series in 2020? " ),
AssistantMessage ( " The Los Angeles Dodgers won the World Series in 2020. " ),
UserMessage ( " Where was it played? " ),
)
service.createChatCompletion(
messages = messages,
settings = createChatCompletionSettings
).map { chatCompletion =>
println(chatCompletion.choices.head.message.content)
}
val messages = Seq (
SystemMessage ( " You are a helpful assistant. " ),
UserMessage ( " What's the weather like in San Francisco, Tokyo, and Paris? " )
)
// as a param type we can use "number", "string", "boolean", "object", "array", and "null"
val tools = Seq (
FunctionSpec (
name = " get_current_weather " ,
description = Some ( " Get the current weather in a given location " ),
parameters = Map (
" type " - > " object " ,
" properties " - > Map (
" location " - > Map (
" type " - > " string " ,
" description " - > " The city and state, e.g. San Francisco, CA "
),
" unit " - > Map (
" type " - > " string " ,
" enum " - > Seq ( " celsius " , " fahrenheit " )
)
),
" required " - > Seq ( " location " )
)
)
)
// if we want to force the model to use the above function as a response
// we can do so by passing: responseToolChoice = Some("get_current_weather")`
service.createChatToolCompletion(
messages = messages,
tools = tools,
responseToolChoice = None , // means "auto"
settings = CreateChatCompletionSettings ( ModelId .gpt_3_5_turbo_1106)
).map { response =>
val chatFunCompletionMessage = response.choices.head.message
val toolCalls = chatFunCompletionMessage.tool_calls.collect {
case (id, x : FunctionCallSpec ) => (id, x)
}
println(
" tool call ids : " + toolCalls.map(_._1).mkString( " , " )
)
println(
" function/tool call names : " + toolCalls.map(_._2.name).mkString( " , " )
)
println(
" function/tool call arguments : " + toolCalls.map(_._2.arguments).mkString( " , " )
)
}
val messages = Seq (
SystemMessage ( " Give me the most populous capital cities in JSON format. " ),
UserMessage ( " List only african countries " )
)
val capitalsSchema = JsonSchema . Object (
properties = Map (
" countries " - > JsonSchema . Array (
items = JsonSchema . Object (
properties = Map (
" country " - > JsonSchema . String (
description = Some ( " The name of the country " )
),
" capital " - > JsonSchema . String (
description = Some ( " The capital city of the country " )
)
),
required = Seq ( " country " , " capital " )
)
)
),
required = Seq ( " countries " )
)
val jsonSchemaDef = JsonSchemaDef (
name = " capitals_response " ,
strict = true ,
structure = schema
)
service
.createChatCompletion(
messages = messages,
settings = DefaultSettings .createJsonChatCompletion(jsonSchemaDef)
)
.map { response =>
val json = Json .parse(messageContent(response))
println( Json .prettyPrint(json))
}
createChatCompletions
atau createChatFunCompletions
, ini membantu Anda memilih model yang tepat dan mengurangi biaya. Ini adalah fitur eksperimental dan mungkin tidak berfungsi untuk semua model. Membutuhkan lib openai-scala-count-tokens
.Contoh Cara Menghitung Token Pesan:
import io . cequence . openaiscala . service . OpenAICountTokensHelper
import io . cequence . openaiscala . domain .{ AssistantMessage , BaseMessage , FunctionSpec , ModelId , SystemMessage , UserMessage }
class MyCompletionService extends OpenAICountTokensHelper {
def exec = {
val model = ModelId .gpt_4_turbo_2024_04_09
// messages to be sent to OpenAI
val messages : Seq [ BaseMessage ] = Seq (
SystemMessage ( " You are a helpful assistant. " ),
UserMessage ( " Who won the world series in 2020? " ),
AssistantMessage ( " The Los Angeles Dodgers won the World Series in 2020. " ),
UserMessage ( " Where was it played? " ),
)
val tokenCount = countMessageTokens(model, messages)
}
}
Contoh cara menghitung token pesan saat fungsi terlibat:
import io . cequence . openaiscala . service . OpenAICountTokensHelper
import io . cequence . openaiscala . domain .{ BaseMessage , FunctionSpec , ModelId , SystemMessage , UserMessage }
class MyCompletionService extends OpenAICountTokensHelper {
def exec = {
val model = ModelId .gpt_4_turbo_2024_04_09
// messages to be sent to OpenAI
val messages : Seq [ BaseMessage ] =
Seq (
SystemMessage ( " You are a helpful assistant. " ),
UserMessage ( " What's the weather like in San Francisco, Tokyo, and Paris? " )
)
// function to be called
val function : FunctionSpec = FunctionSpec (
name = " getWeather " ,
parameters = Map (
" type " - > " object " ,
" properties " - > Map (
" location " - > Map (
" type " - > " string " ,
" description " - > " The city to get the weather for "
),
" unit " - > Map ( " type " - > " string " , " enum " - > List ( " celsius " , " fahrenheit " ))
)
)
)
val tokenCount = countFunMessageTokens(model, messages, Seq (function), Some (function.name))
}
}
✔️ PENTING : Setelah Anda selesai menggunakan layanan, Anda harus menutupnya dengan menelepon service.close
. Kalau tidak, sumber daya/utas yang mendasarinya tidak akan dirilis.
AKU AKU AKU. Menggunakan adaptor
Adaptor untuk Layanan OpenAI (penyelesaian obrolan, inti, atau penuh) disediakan oleh OpenAiserviceAdapters. Adaptor digunakan untuk mendistribusikan beban antara beberapa layanan, coba lagi kesalahan sementara, rute, atau memberikan fungsionalitas tambahan. Lihat contoh untuk detail lebih lanjut.
Perhatikan bahwa adaptor dapat digabungkan secara sewenang -wenang/ditumpuk.
val adapters = OpenAIServiceAdapters .forFullService
val service1 = OpenAIServiceFactory ( " your-api-key1 " )
val service2 = OpenAIServiceFactory ( " your-api-key2 " )
val service = adapters.roundRobin(service1, service2)
val adapters = OpenAIServiceAdapters .forFullService
val service1 = OpenAIServiceFactory ( " your-api-key1 " )
val service2 = OpenAIServiceFactory ( " your-api-key2 " )
val service = adapters.randomOrder(service1, service2)
val adapters = OpenAIServiceAdapters .forFullService
val rawService = OpenAIServiceFactory ()
val service = adapters.log(
rawService,
" openAIService " ,
logger.log
)
val adapters = OpenAIServiceAdapters .forFullService
implicit val retrySettings : RetrySettings = RetrySettings (maxRetries = 10 ).constantInterval( 10 .seconds)
val service = adapters.retry(
OpenAIServiceFactory (),
Some (println(_)) // simple logging
)
class MyCompletionService @ Inject () (
val actorSystem : ActorSystem ,
implicit val ec : ExecutionContext ,
implicit val scheduler : Scheduler
)( val apiKey : String )
extends RetryHelpers {
val service : OpenAIService = OpenAIServiceFactory (apiKey)
implicit val retrySettings : RetrySettings =
RetrySettings (interval = 10 .seconds)
def ask ( prompt : String ) : Future [ String ] =
for {
completion < - service
.createChatCompletion(
List ( MessageSpec ( ChatRole . User , prompt))
)
.retryOnFailure
} yield completion.choices.head.message.content
}
val adapters = OpenAIServiceAdapters .forFullService
// OctoAI
val octoMLService = OpenAIChatCompletionServiceFactory (
coreUrl = " https://text.octoai.run/v1/ " ,
authHeaders = Seq (( " Authorization " , s " Bearer ${sys.env( " OCTOAI_TOKEN " )} " ))
)
// Anthropic
val anthropicService = AnthropicServiceFactory .asOpenAI()
// OpenAI
val openAIService = OpenAIServiceFactory ()
val service : OpenAIService =
adapters.chatCompletionRouter(
// OpenAI service is default so no need to specify its models here
serviceModels = Map (
octoMLService - > Seq ( NonOpenAIModelId .mixtral_8x22b_instruct),
anthropicService - > Seq (
NonOpenAIModelId .claude_2_1,
NonOpenAIModelId .claude_3_opus_20240229,
NonOpenAIModelId .claude_3_haiku_20240307
)
),
openAIService
)
val adapters = OpenAIServiceAdapters .forCoreService
val service = adapters.chatToCompletion(
OpenAICoreServiceFactory (
coreUrl = " https://api.fireworks.ai/inference/v1/ " ,
authHeaders = Seq (( " Authorization " , s " Bearer ${sys.env( " FIREWORKS_API_KEY " )} " ))
)
)
Wen Scala 3?
Feb 2023. Anda benar; Kami memilih bulan terpendek untuk melakukannya :) Selesai!
Saya mendapat pengecualian timeout. Bagaimana cara mengubah pengaturan batas waktu?
Anda dapat melakukannya dengan memberikan param timeouts
ke OpenAIServiceFactory
atau, jika Anda menggunakan file konfigurasi sendiri, maka Anda cukup menambahkannya di sana sebagai:
openai-scala-client {
timeouts {
requestTimeoutSec = 200
readTimeoutSec = 200
connectTimeoutSec = 5
pooledConnectionIdleTimeoutSec = 60
}
}
Saya mendapat pengecualian seperti com.typesafe.config.ConfigException$UnresolvedSubstitution: openai-scala-client.conf @ jar:file:.../io/cequence/openai-scala-client_2.13/0.0.1/openai-scala-client_2.13-0.0.1.jar!/openai-scala-client.conf: 4: Could not resolve substitution to a value: ${OPENAI_SCALA_CLIENT_API_KEY}
. Apa yang harus saya lakukan?
Atur env. variabel OPENAI_SCALA_CLIENT_API_KEY
. Jika Anda tidak memiliki satu daftar di sini.
Semuanya terlihat keren. Saya ingin mengobrol dengan Anda tentang penelitian dan pengembangan Anda?
Cukup tembak kami email di [email protected].
Perpustakaan ini tersedia dan diterbitkan sebagai open source di bawah ketentuan lisensi MIT.
Proyek ini open-source dan menyambut setiap kontribusi atau umpan balik (di sini).
Pengembangan perpustakaan ini telah didukung oleh - cequence.io - The future of contracting
Dibuat dan dikelola oleh Peter Banda.