yomo下載 - yomo原始碼下載

yomo

yomo是一個開源 LLM 函數呼叫框架，用於建立地理分散式人工智慧應用程式。基於 QUIC 傳輸協定和有狀態無伺服器架構構建，使您的 AI 應用程式低延遲、可靠、安全且簡單。

我們關心：人工智慧時代的客戶體驗

？特徵

	特徵
⚡️	透過在 QUIC 之上實施來確保低延遲 QUIC
？	每個資料包均設計有安全TLS v1.3
？	有狀態無伺服器讓 GPU 無伺服器速度提升 10 倍
？	地理分散式架構讓人工智慧推理更貼近最終用戶
	Y3比即時編解碼器更快

入門

讓我們實作一個使用sfn-currency-converter呼叫的函數：

步驟 1. 安裝 CLI

curl -fsSL https://get.yomo.run | sh

驗證 CLI 是否安裝成功

 yomo version

步驟2.啟動伺服器

將設定準備為my-agent.yaml

 name : ai-zipper
host : 0.0.0.0
port : 9000

auth :
  type : token
  token : SECRET_TOKEN

bridge :
  ai :
    server :
      addr : 0.0.0.0:8000 # # Restful API endpoint
      provider : openai # # LLM API Service we will use

    providers :
      azopenai :
        api_endpoint : https://<RESOURCE>.openai.azure.com
        deployment_id : <DEPLOYMENT_ID>
        api_key : <API_KEY>
        api_version : <API_VERSION>

      openai :
        api_key : sk-xxxxxxxxxxxxxxxxxxxxxxxxxxx
        model : gpt-4-1106-preview

      gemini :
        api_key : <GEMINI_API_KEY>

      cloudflare_azure :
        endpoint : https://gateway.ai.cloudflare.com/v1/<CF_GATEWAY_ID>/<CF_GATEWAY_NAME>
        api_key : <AZURE_API_KEY>
        resource : <AZURE_OPENAI_RESOURCE>
        deployment_id : <AZURE_OPENAI_DEPLOYMENT_ID>
        api_version : 2023-12-01-preview

啟動伺服器：

 yomo _LOG_LEVEL=debug yomo serve -c my-agent.yaml

步驟 3. 編寫函數

首先，我們來定義這個函數的作用以及所需的參數是怎樣的，這些將結合起來在呼叫LLM時進行提示。

 type Parameter struct {
	Domain string `json:"domain" jsonschema:"description=Domain of the website,example=example.com"`
}

func Description () string {
	return `if user asks ip or network latency of a domain, you should return the result of the giving domain. try your best to dissect user expressions to infer the right domain names`
}

func InputSchema () any {
	return & Parameter {}
}

建立一個有狀態無伺服器函數來取得網域的 IP 和延遲：

 func Handler ( ctx serverless. Context ) {
	var msg Parameter
	ctx . ReadLLMArguments ( & msg )

	// get ip of the domain
	ips , _ := net . LookupIP ( msg . Domain )

	// get ip[0] ping latency
	pinger , _ := ping . NewPinger ( ips [ 0 ]. String ())
	pinger . Count = 3
	pinger . Run ()
	stats := pinger . Statistics ()

	val := fmt . Sprintf ( "domain %s has ip %s with average latency %s" , msg . Domain , ips [ 0 ], stats . AvgRtt )
	ctx . WriteLLMResult ( val )
}

最後我們來運行一下

$ yomo run app.go

time=2024-03-19T21:43:30.583+08:00 level=INFO msg= " connected to zipper " component=StreamFunction sfn_id=B0ttNSEKLSgMjXidB11K1 sfn_name=fn-get-ip-from-domain zipper_addr=localhost:9000
time=2024-03-19T21:43:30.584+08:00 level=INFO msg= " register ai function success " component=StreamFunction sfn_id=B0ttNSEKLSgMjXidB11K1 sfn_name=fn-get-ip-from-domain zipper_addr=localhost:9000 name=fn-get-ip-from-domain tag=16

完成了，我們來試試一下

$ curl -i http://127.0.0.1:9000/v1/chat/completions -H " Content-Type: application/json " -d ' {
  "messages": [
    {
      "role": "system",
      "content": "You are a test assistant."
    },
    {
      "role": "user",
      "content": "Compare website speed between Nike and Puma"
    }
  ],
  "stream": false
} '

HTTP/1.1 200 OK
Content-Length: 944
Connection: keep-alive
Content-Type: application/json
Date: Tue, 19 Mar 2024 13:30:14 GMT
Keep-Alive: timeout=4
Proxy-Connection: keep-alive

{
  " Content " : " Based on the data provided for the domains nike.com and puma.com which include IP addresses and average latencies, we can infer the following about their website speeds:
  - Nike.com has an IP address of 13.225.183.84 with an average latency of 65.568333 milliseconds.
  - Puma.com has an IP address of 151.101.194.132 with an average latency of 54.563666 milliseconds.
  
  Comparing these latencies, Puma.com is faster than Nike.com as it has a lower average latency. 
  
  Please be aware, however, that website speed can be influenced by many factors beyond latency, such as server processing time, content size, and delivery networks among others. To get a more comprehensive understanding of website speed, you would need to consider additional metrics and possibly conductreal-time speed tests. " ,
  " FinishReason " : " stop "
}