openai scala client
Version 1.1.0
Este es un cliente de Async Scala sin sentido para la API de OpenAI que admite todos los puntos finales y parámetros disponibles, incluida la transmisión , las rutinas de finalización , visión y voz más recientes (como se define aquí), proporcionados en un servicio único y conveniente llamado OpenAiservice. Las llamadas compatibles son:
Tenga en cuenta que para ser consistente con el nombre de la API de OpenAI, los nombres de funciones de servicio coinciden exactamente con los títulos/descripciones de punto final de la API con CamelCase. Además, apuntamos a Lib para que fuera autónomo con la menor cantidad de dependencias posibles, por lo tanto, terminamos usando solo dos libs play-ahc-ws-standalone y play-ws-standalone-json (en el nivel superior). Además, si se requiere inyección de dependencia, también usamos scala-guice lib también.
¿No hay tiempo para leer un largo tutorial? ¡Claro, te escuchamos! Consulte los ejemplos para ver cómo usar la lib en la práctica.
Además de la API de Operai, esta biblioteca también admite proveedores de API compatibles (ver ejemplos) como:
Para obtener información sobre antecedentes, lea un artículo sobre Lib/Client en Medium.
¡También pruebe nuestro cliente Scala para la base de datos Vector de Pinecone, o use ambos clientes juntos! Este proyecto de demostración muestra cómo generar y almacenar incrustaciones de OpenAI (con el modelo text-embedding-ada-002 ) en Pinecone y consultarlos después. El combo Operai + Pinecone se usa comúnmente para agentes autónomos de IA, como Babyagi y AutoGpt.
✔️ IMPORTANTE : Esta es una biblioteca "mantenida por la comunidad" y, como tal, no tiene relación con OpenAI Company.
Las versiones Scala actualmente compatibles son 2.12, 2.13 y 3 .
Para instalar la biblioteca, agregue la siguiente dependencia a su build.sbt
"io.cequence" %% "openai-scala-client" % "1.1.0"
o a pom.xml (si usa Maven)
<dependency>
<groupId>io.cequence</groupId>
<artifactId>openai-scala-client_2.12</artifactId>
<version>1.1.0</version>
</dependency>
Si desea soporte de transmisión, use "io.cequence" %% "openai-scala-client-stream" % "1.1.0" en su lugar.
OPENAI_SCALA_CLIENT_API_KEY y opcionalmente también OPENAI_SCALA_CLIENT_ORG_ID (si tiene uno)I. Obtener OpenaIservice
Primero debe proporcionar un contexto de ejecución implícito, así como Akka Materializer, por ejemplo, como
implicit val ec = ExecutionContext .global
implicit val materializer = Materializer ( ActorSystem ())Luego puede obtener un servicio de una de las siguientes maneras.
Config . val service = OpenAIServiceFactory () val config = ConfigFactory .load( " path_to_my_custom_config " )
val service = OpenAIServiceFactory (config) val service = OpenAIServiceFactory (
apiKey = " your_api_key " ,
orgId = Some ( " your_org_id " ) // if you have one
) val service = OpenAIServiceFactory .forAzureWithApiKey(
resourceName = " your-resource-name " ,
deploymentId = " your-deployment-id " , // usually model name such as "gpt-35-turbo"
apiVersion = " 2023-05-15 " , // newest version
apiKey = " your_api_key "
)OpenAICoreService Supporting listModels , createCompletion , createChatCompletion y createEmbeddings Llamadas, proporcionadas por ejemplo, el servicio FastChat que se ejecuta en el puerto 8000 val service = OpenAICoreServiceFactory ( " http://localhost:8000/v1/ " )OpenAIChatCompletionService que proporciona únicamente createChatCompletion val service = OpenAIChatCompletionServiceFactory .forAzureAI(
endpoint = sys.env( " AZURE_AI_COHERE_R_PLUS_ENDPOINT " ),
region = sys.env( " AZURE_AI_COHERE_R_PLUS_REGION " ),
accessToken = sys.env( " AZURE_AI_COHERE_R_PLUS_ACCESS_KEY " )
)openai-scala-anthropic-client liber y ANTHROPIC_API_KEY val service = AnthropicServiceFactory .asOpenAI()openai-scala-google-vertexai-client LIB y VERTEXAI_LOCATION + VERTEXAI_PROJECT_ID val service = VertexAIServiceFactory .asOpenAI()GROQ_API_KEY" val service = OpenAIChatCompletionServiceFactory ( ChatProviderSettings .groq)
// or with streaming
val service = OpenAIChatCompletionServiceFactory .withStreaming( ChatProviderSettings .groq)GROK_API_KEY" val service = OpenAIChatCompletionServiceFactory ( ChatProviderSettings .grok)
// or with streaming
val service = OpenAIChatCompletionServiceFactory .withStreaming( ChatProviderSettings .grok)FIREWORKS_API_KEY" val service = OpenAIChatCompletionServiceFactory ( ChatProviderSettings .fireworks)
// or with streaming
val service = OpenAIChatCompletionServiceFactory .withStreaming( ChatProviderSettings .fireworks)OCTOAI_TOKEN val service = OpenAIChatCompletionServiceFactory ( ChatProviderSettings .octoML)
// or with streaming
val service = OpenAIChatCompletionServiceFactory .withStreaming( ChatProviderSettings .octoML)TOGETHERAI_API_KEY val service = OpenAIChatCompletionServiceFactory ( ChatProviderSettings .togetherAI)
// or with streaming
val service = OpenAIChatCompletionServiceFactory .withStreaming( ChatProviderSettings .togetherAI)CEREBRAS_API_KEY val service = OpenAIChatCompletionServiceFactory ( ChatProviderSettings .cerebras)
// or with streaming
val service = OpenAIChatCompletionServiceFactory .withStreaming( ChatProviderSettings .cerebras)MISTRAL_API_KEY val service = OpenAIChatCompletionServiceFactory ( ChatProviderSettings .mistral)
// or with streaming
val service = OpenAIChatCompletionServiceFactory .withStreaming( ChatProviderSettings .mistral) val service = OpenAIChatCompletionServiceFactory (
coreUrl = " http://localhost:11434/v1/ "
)o con transmisión
val service = OpenAIChatCompletionServiceFactory .withStreaming(
coreUrl = " http://localhost:11434/v1/ "
)createCompletionStreamed createChatCompletionStreamed openai-scala-client-stream import io . cequence . openaiscala . service . StreamedServiceTypes . OpenAIStreamedService
import io . cequence . openaiscala . service . OpenAIStreamedServiceImplicits . _
val service : OpenAIStreamedService = OpenAIServiceFactory .withStreaming()Del mismo modo para un servicio de completación de chat
import io . cequence . openaiscala . service . OpenAIStreamedServiceImplicits . _
val service = OpenAIChatCompletionServiceFactory .withStreaming(
coreUrl = " https://api.fireworks.ai/inference/v1/ " ,
authHeaders = Seq (( " Authorization " , s " Bearer ${sys.env( " FIREWORKS_API_KEY " )} " ))
)o solo si se requiere transmisión
val service : OpenAIChatCompletionStreamedServiceExtra =
OpenAIChatCompletionStreamedServiceFactory (
coreUrl = " https://api.fireworks.ai/inference/v1/ " ,
authHeaders = Seq (( " Authorization " , s " Bearer ${sys.env( " FIREWORKS_API_KEY " )} " ))
)openai-scala-guice ) class MyClass @ Inject () ( openAIService : OpenAIService ) {...}II. Funciones de llamadas
La documentación completa de cada llamada con sus respectivas entradas y configuraciones se proporciona en OpenAiservice. Dado que todas las llamadas son asíncronas, devuelven respuestas envueltas en Future .
¡Hay un nuevo proyecto OpenAi-Scala-Client-Examples donde puedes encontrar muchos ejemplos listos para usar!
service.listModels.map(models =>
models.foreach(println)
) service.retrieveModel( ModelId .text_davinci_003).map(model =>
println(model.getOrElse( " N/A " ))
) val text = """ Extract the name and mailing address from this email:
|Dear Kelly,
|It was great to talk to you at the seminar. I thought Jane's talk was quite good.
|Thank you for the book. Here's my address 2111 Ash Lane, Crestview CA 92002
|Best,
|Maya
""" .stripMargin
service.createCompletion(text).map(completion =>
println(completion.choices.head.text)
) val text = """ Extract the name and mailing address from this email:
|Dear Kelly,
|It was great to talk to you at the seminar. I thought Jane's talk was quite good.
|Thank you for the book. Here's my address 2111 Ash Lane, Crestview CA 92002
|Best,
|Maya
""" .stripMargin
service.createCompletion(
text,
settings = CreateCompletionSettings (
model = ModelId .gpt_4o,
max_tokens = Some ( 1500 ),
temperature = Some ( 0.9 ),
presence_penalty = Some ( 0.2 ),
frequency_penalty = Some ( 0.2 )
)
).map(completion =>
println(completion.choices.head.text)
) val source = service.createCompletionStreamed(
prompt = " Write me a Shakespeare poem about two cats playing baseball in Russia using at least 2 pages " ,
settings = CreateCompletionSettings (
model = ModelId .text_davinci_003,
max_tokens = Some ( 1500 ),
temperature = Some ( 0.9 ),
presence_penalty = Some ( 0.2 ),
frequency_penalty = Some ( 0.2 )
)
)
source.map(completion =>
println(completion.choices.head.text)
).runWith( Sink .ignore) Para que esto funcione, debe usar OpenAIServiceStreamedFactory desde openai-scala-client-stream LIB.
val createChatCompletionSettings = CreateChatCompletionSettings (
model = ModelId .gpt_4o
)
val messages = Seq (
SystemMessage ( " You are a helpful assistant. " ),
UserMessage ( " Who won the world series in 2020? " ),
AssistantMessage ( " The Los Angeles Dodgers won the World Series in 2020. " ),
UserMessage ( " Where was it played? " ),
)
service.createChatCompletion(
messages = messages,
settings = createChatCompletionSettings
).map { chatCompletion =>
println(chatCompletion.choices.head.message.content)
} val messages = Seq (
SystemMessage ( " You are a helpful assistant. " ),
UserMessage ( " What's the weather like in San Francisco, Tokyo, and Paris? " )
)
// as a param type we can use "number", "string", "boolean", "object", "array", and "null"
val tools = Seq (
FunctionSpec (
name = " get_current_weather " ,
description = Some ( " Get the current weather in a given location " ),
parameters = Map (
" type " - > " object " ,
" properties " - > Map (
" location " - > Map (
" type " - > " string " ,
" description " - > " The city and state, e.g. San Francisco, CA "
),
" unit " - > Map (
" type " - > " string " ,
" enum " - > Seq ( " celsius " , " fahrenheit " )
)
),
" required " - > Seq ( " location " )
)
)
)
// if we want to force the model to use the above function as a response
// we can do so by passing: responseToolChoice = Some("get_current_weather")`
service.createChatToolCompletion(
messages = messages,
tools = tools,
responseToolChoice = None , // means "auto"
settings = CreateChatCompletionSettings ( ModelId .gpt_3_5_turbo_1106)
).map { response =>
val chatFunCompletionMessage = response.choices.head.message
val toolCalls = chatFunCompletionMessage.tool_calls.collect {
case (id, x : FunctionCallSpec ) => (id, x)
}
println(
" tool call ids : " + toolCalls.map(_._1).mkString( " , " )
)
println(
" function/tool call names : " + toolCalls.map(_._2.name).mkString( " , " )
)
println(
" function/tool call arguments : " + toolCalls.map(_._2.arguments).mkString( " , " )
)
} val messages = Seq (
SystemMessage ( " Give me the most populous capital cities in JSON format. " ),
UserMessage ( " List only african countries " )
)
val capitalsSchema = JsonSchema . Object (
properties = Map (
" countries " - > JsonSchema . Array (
items = JsonSchema . Object (
properties = Map (
" country " - > JsonSchema . String (
description = Some ( " The name of the country " )
),
" capital " - > JsonSchema . String (
description = Some ( " The capital city of the country " )
)
),
required = Seq ( " country " , " capital " )
)
)
),
required = Seq ( " countries " )
)
val jsonSchemaDef = JsonSchemaDef (
name = " capitals_response " ,
strict = true ,
structure = schema
)
service
.createChatCompletion(
messages = messages,
settings = DefaultSettings .createJsonChatCompletion(jsonSchemaDef)
)
.map { response =>
val json = Json .parse(messageContent(response))
println( Json .prettyPrint(json))
}createChatCompletions o createChatFunCompletions , esto le ayuda a seleccionar el modelo adecuado y reducir los costos. Esta es una característica experimental y puede no funcionar para todos los modelos. Requiere Lib openai-scala-count-tokens .Un ejemplo de cómo contar tokens de mensaje:
import io . cequence . openaiscala . service . OpenAICountTokensHelper
import io . cequence . openaiscala . domain .{ AssistantMessage , BaseMessage , FunctionSpec , ModelId , SystemMessage , UserMessage }
class MyCompletionService extends OpenAICountTokensHelper {
def exec = {
val model = ModelId .gpt_4_turbo_2024_04_09
// messages to be sent to OpenAI
val messages : Seq [ BaseMessage ] = Seq (
SystemMessage ( " You are a helpful assistant. " ),
UserMessage ( " Who won the world series in 2020? " ),
AssistantMessage ( " The Los Angeles Dodgers won the World Series in 2020. " ),
UserMessage ( " Where was it played? " ),
)
val tokenCount = countMessageTokens(model, messages)
}
}Un ejemplo de cómo contar los tokens de mensaje cuando se trata de una función:
import io . cequence . openaiscala . service . OpenAICountTokensHelper
import io . cequence . openaiscala . domain .{ BaseMessage , FunctionSpec , ModelId , SystemMessage , UserMessage }
class MyCompletionService extends OpenAICountTokensHelper {
def exec = {
val model = ModelId .gpt_4_turbo_2024_04_09
// messages to be sent to OpenAI
val messages : Seq [ BaseMessage ] =
Seq (
SystemMessage ( " You are a helpful assistant. " ),
UserMessage ( " What's the weather like in San Francisco, Tokyo, and Paris? " )
)
// function to be called
val function : FunctionSpec = FunctionSpec (
name = " getWeather " ,
parameters = Map (
" type " - > " object " ,
" properties " - > Map (
" location " - > Map (
" type " - > " string " ,
" description " - > " The city to get the weather for "
),
" unit " - > Map ( " type " - > " string " , " enum " - > List ( " celsius " , " fahrenheit " ))
)
)
)
val tokenCount = countFunMessageTokens(model, messages, Seq (function), Some (function.name))
}
} ✔️ IMPORTANTE : Después de que haya terminado de usar el servicio, debe cerrarlo llamando service.close . De lo contrario, los recursos/hilos subyacentes no se lanzarán.
Iii. Usando adaptadores
Los adaptadores para servicios de OpenAI (finalización de chat, núcleo o completo) son proporcionados por OpenaIServiceAdapters. Los adaptadores se utilizan para distribuir la carga entre múltiples servicios, volver a intentar los errores transitorios, la ruta o proporcionar funcionalidad adicional. Ver ejemplos para más detalles.
Tenga en cuenta que los adaptadores pueden combinarse/apilarse arbitrariamente.
val adapters = OpenAIServiceAdapters .forFullService
val service1 = OpenAIServiceFactory ( " your-api-key1 " )
val service2 = OpenAIServiceFactory ( " your-api-key2 " )
val service = adapters.roundRobin(service1, service2) val adapters = OpenAIServiceAdapters .forFullService
val service1 = OpenAIServiceFactory ( " your-api-key1 " )
val service2 = OpenAIServiceFactory ( " your-api-key2 " )
val service = adapters.randomOrder(service1, service2) val adapters = OpenAIServiceAdapters .forFullService
val rawService = OpenAIServiceFactory ()
val service = adapters.log(
rawService,
" openAIService " ,
logger.log
) val adapters = OpenAIServiceAdapters .forFullService
implicit val retrySettings : RetrySettings = RetrySettings (maxRetries = 10 ).constantInterval( 10 .seconds)
val service = adapters.retry(
OpenAIServiceFactory (),
Some (println(_)) // simple logging
) class MyCompletionService @ Inject () (
val actorSystem : ActorSystem ,
implicit val ec : ExecutionContext ,
implicit val scheduler : Scheduler
)( val apiKey : String )
extends RetryHelpers {
val service : OpenAIService = OpenAIServiceFactory (apiKey)
implicit val retrySettings : RetrySettings =
RetrySettings (interval = 10 .seconds)
def ask ( prompt : String ) : Future [ String ] =
for {
completion < - service
.createChatCompletion(
List ( MessageSpec ( ChatRole . User , prompt))
)
.retryOnFailure
} yield completion.choices.head.message.content
} val adapters = OpenAIServiceAdapters .forFullService
// OctoAI
val octoMLService = OpenAIChatCompletionServiceFactory (
coreUrl = " https://text.octoai.run/v1/ " ,
authHeaders = Seq (( " Authorization " , s " Bearer ${sys.env( " OCTOAI_TOKEN " )} " ))
)
// Anthropic
val anthropicService = AnthropicServiceFactory .asOpenAI()
// OpenAI
val openAIService = OpenAIServiceFactory ()
val service : OpenAIService =
adapters.chatCompletionRouter(
// OpenAI service is default so no need to specify its models here
serviceModels = Map (
octoMLService - > Seq ( NonOpenAIModelId .mixtral_8x22b_instruct),
anthropicService - > Seq (
NonOpenAIModelId .claude_2_1,
NonOpenAIModelId .claude_3_opus_20240229,
NonOpenAIModelId .claude_3_haiku_20240307
)
),
openAIService
) val adapters = OpenAIServiceAdapters .forCoreService
val service = adapters.chatToCompletion(
OpenAICoreServiceFactory (
coreUrl = " https://api.fireworks.ai/inference/v1/ " ,
authHeaders = Seq (( " Authorization " , s " Bearer ${sys.env( " FIREWORKS_API_KEY " )} " ))
)
)Wen Scala 3?
Febrero de 2023. Tienes razón; Elegimos el mes más corto para hacerlo :) ¡Hecho!
Tengo una excepción de tiempo de espera. ¿Cómo puedo cambiar la configuración de tiempo de espera?
Puede hacerlo pasando el parámetro timeouts de espera a OpenAIServiceFactory o, si usa su propio archivo de configuración, simplemente puede agregarlo allí como:
openai-scala-client {
timeouts {
requestTimeoutSec = 200
readTimeoutSec = 200
connectTimeoutSec = 5
pooledConnectionIdleTimeoutSec = 60
}
}
Obtuve una excepción como com.typesafe.config.ConfigException$UnresolvedSubstitution: openai-scala-client.conf @ jar:file:.../io/cequence/openai-scala-client_2.13/0.0.1/openai-scala-client_2.13-0.0.1.jar!/openai-scala-client.conf: 4: Could not resolve substitution to a value: ${OPENAI_SCALA_CLIENT_API_KEY} . ¿Qué tengo que hacer?
Establezca el env. Variable OPENAI_SCALA_CLIENT_API_KEY . Si no tiene un registro aquí.
Todo se ve genial. ¿Quiero charlar contigo sobre tu investigación y desarrollo?
Simplemente envíenos un correo electrónico a [email protected].
Esta biblioteca está disponible y publicada como código abierto bajo los términos de la licencia MIT.
Este proyecto es de código abierto y da la bienvenida a cualquier contribución o retroalimentación (aquí).
El desarrollo de esta biblioteca ha sido respaldado por - Cequence.io - The future of contracting
Creado y mantenido por Peter Banda.
