yomo下载 - yomo源码下载

yomo

yomo是一个开源 LLM 函数调用框架，用于构建地理分布式人工智能应用程序。基于 QUIC 传输协议和有状态无服务器架构构建，使您的 AI 应用程序低延迟、可靠、安全且简单。

我们关心：人工智能时代的客户体验

？特征

	特征
⚡️	通过在 QUIC 之上实施来保证低延迟 QUIC
？	每个数据包均设计有安全TLS v1.3
？	有状态无服务器让 GPU 无服务器速度提高 10 倍
？	地理分布式架构让人工智能推理更贴近最终用户
	Y3比实时编解码器更快

入门

让我们实现一个使用sfn-currency-converter调用的函数：

步骤 1. 安装 CLI

curl -fsSL https://get.yomo.run | sh

验证 CLI 是否安装成功

 yomo version

步骤2.启动服务器

将配置准备为my-agent.yaml

 name : ai-zipper
host : 0.0.0.0
port : 9000

auth :
  type : token
  token : SECRET_TOKEN

bridge :
  ai :
    server :
      addr : 0.0.0.0:8000 # # Restful API endpoint
      provider : openai # # LLM API Service we will use

    providers :
      azopenai :
        api_endpoint : https://<RESOURCE>.openai.azure.com
        deployment_id : <DEPLOYMENT_ID>
        api_key : <API_KEY>
        api_version : <API_VERSION>

      openai :
        api_key : sk-xxxxxxxxxxxxxxxxxxxxxxxxxxx
        model : gpt-4-1106-preview

      gemini :
        api_key : <GEMINI_API_KEY>

      cloudflare_azure :
        endpoint : https://gateway.ai.cloudflare.com/v1/<CF_GATEWAY_ID>/<CF_GATEWAY_NAME>
        api_key : <AZURE_API_KEY>
        resource : <AZURE_OPENAI_RESOURCE>
        deployment_id : <AZURE_OPENAI_DEPLOYMENT_ID>
        api_version : 2023-12-01-preview

启动服务器：

 yomo _LOG_LEVEL=debug yomo serve -c my-agent.yaml

步骤 3. 编写函数

首先，我们来定义这个函数的作用以及所需的参数是怎样的，这些将结合起来在调用LLM时进行提示。

 type Parameter struct {
	Domain string `json:"domain" jsonschema:"description=Domain of the website,example=example.com"`
}

func Description () string {
	return `if user asks ip or network latency of a domain, you should return the result of the giving domain. try your best to dissect user expressions to infer the right domain names`
}

func InputSchema () any {
	return & Parameter {}
}

创建一个有状态无服务器函数来获取域的 IP 和延迟：

 func Handler ( ctx serverless. Context ) {
	var msg Parameter
	ctx . ReadLLMArguments ( & msg )

	// get ip of the domain
	ips , _ := net . LookupIP ( msg . Domain )

	// get ip[0] ping latency
	pinger , _ := ping . NewPinger ( ips [ 0 ]. String ())
	pinger . Count = 3
	pinger . Run ()
	stats := pinger . Statistics ()

	val := fmt . Sprintf ( "domain %s has ip %s with average latency %s" , msg . Domain , ips [ 0 ], stats . AvgRtt )
	ctx . WriteLLMResult ( val )
}

最后我们来运行一下

$ yomo run app.go

time=2024-03-19T21:43:30.583+08:00 level=INFO msg= " connected to zipper " component=StreamFunction sfn_id=B0ttNSEKLSgMjXidB11K1 sfn_name=fn-get-ip-from-domain zipper_addr=localhost:9000
time=2024-03-19T21:43:30.584+08:00 level=INFO msg= " register ai function success " component=StreamFunction sfn_id=B0ttNSEKLSgMjXidB11K1 sfn_name=fn-get-ip-from-domain zipper_addr=localhost:9000 name=fn-get-ip-from-domain tag=16

完成了，我们来尝试一下

$ curl -i http://127.0.0.1:9000/v1/chat/completions -H " Content-Type: application/json " -d ' {
  "messages": [
    {
      "role": "system",
      "content": "You are a test assistant."
    },
    {
      "role": "user",
      "content": "Compare website speed between Nike and Puma"
    }
  ],
  "stream": false
} '

HTTP/1.1 200 OK
Content-Length: 944
Connection: keep-alive
Content-Type: application/json
Date: Tue, 19 Mar 2024 13:30:14 GMT
Keep-Alive: timeout=4
Proxy-Connection: keep-alive

{
  " Content " : " Based on the data provided for the domains nike.com and puma.com which include IP addresses and average latencies, we can infer the following about their website speeds:
  - Nike.com has an IP address of 13.225.183.84 with an average latency of 65.568333 milliseconds.
  - Puma.com has an IP address of 151.101.194.132 with an average latency of 54.563666 milliseconds.
  
  Comparing these latencies, Puma.com is faster than Nike.com as it has a lower average latency. 
  
  Please be aware, however, that website speed can be influenced by many factors beyond latency, such as server processing time, content size, and delivery networks among others. To get a more comprehensive understanding of website speed, you would need to consider additional metrics and possibly conductreal-time speed tests. " ,
  " FinishReason " : " stop "
}