ollama python

奧拉馬 Python 函式庫

Ollama Python 函式庫提供了將 Python 3.8+ 專案與 Ollama 整合的最簡單方法。

先決條件

• Ollama 應該已安裝並運行
• 拉取模型以與庫一起使用： ollama pull <model>例如ollama pull llama3.2
  • 有關可用型號的更多信息，請訪問 Ollama.com。

安裝

pip install ollama

用法

 from ollama import chat
from ollama import ChatResponse

response : ChatResponse = chat ( model = 'llama3.2' , messages = [
  {
    'role' : 'user' ,
    'content' : 'Why is the sky blue?' ,
  },
])
print ( response [ 'message' ][ 'content' ])
# or access fields directly from the response object
print ( response . message . content )

有關回應類型的更多信息，請參閱 _types.py。

串流回應

可以透過設定stream=True來啟用回應流。

 from ollama import chat

stream = chat (
    model = 'llama3.2' ,
    messages = [{ 'role' : 'user' , 'content' : 'Why is the sky blue?' }],
    stream = True ,
)

for chunk in stream :
  print ( chunk [ 'message' ][ 'content' ], end = '' , flush = True )

客製化客戶端

可以透過實例化ollama中的Client或AsyncClient來建立自訂客戶端。

所有額外的關鍵字參數都會傳遞到httpx.Client 。

 from ollama import Client
client = Client (
  host = 'http://localhost:11434' ,
  headers = { 'x-some-header' : 'some-value' }
)
response = client . chat ( model = 'llama3.2' , messages = [
  {
    'role' : 'user' ,
    'content' : 'Why is the sky blue?' ,
  },
])

非同步客戶端

AsyncClient類別用於發出非同步請求。它可以配置與Client類別相同的字段。

 import asyncio
from ollama import AsyncClient

async def chat ():
  message = { 'role' : 'user' , 'content' : 'Why is the sky blue?' }
  response = await AsyncClient (). chat ( model = 'llama3.2' , messages = [ message ])

asyncio . run ( chat ())

設定stream=True會修改函數以傳回Python非同步產生器：

 import asyncio
from ollama import AsyncClient

async def chat ():
  message = { 'role' : 'user' , 'content' : 'Why is the sky blue?' }
  async for part in await AsyncClient (). chat ( model = 'llama3.2' , messages = [ message ], stream = True ):
    print ( part [ 'message' ][ 'content' ], end = '' , flush = True )

asyncio . run ( chat ())

應用程式介面

Ollama Python 函式庫的 API 是圍繞著 Ollama REST API 設計的

聊天

 ollama . chat ( model = 'llama3.2' , messages = [{ 'role' : 'user' , 'content' : 'Why is the sky blue?' }])

產生

 ollama . generate ( model = 'llama3.2' , prompt = 'Why is the sky blue?' )

清單

 ollama . list ()

展示

 ollama . show ( 'llama3.2' )

創造

 modelfile = '''
FROM llama3.2
SYSTEM You are mario from super mario bros.
'''

ollama . create ( model = 'example' , modelfile = modelfile )

複製

 ollama . copy ( 'llama3.2' , 'user/llama3.2' )

刪除

 ollama . delete ( 'llama3.2' )

拉

 ollama . pull ( 'llama3.2' )

推

 ollama . push ( 'user/llama3.2' )

嵌入

 ollama . embed ( model = 'llama3.2' , input = 'The sky is blue because of rayleigh scattering' )

嵌入（批次）

 ollama . embed ( model = 'llama3.2' , input = [ 'The sky is blue because of rayleigh scattering' , 'Grass is green because of chlorophyll' ])

PS

 ollama . ps ()

錯誤

如果請求傳回錯誤狀態或在串流時偵測到錯誤，則會引發錯誤。

 model = 'does-not-yet-exist'

try :
  ollama . chat ( model )
except ollama . ResponseError as e :
  print ( 'Error:' , e . error )
  if e . status_code == 404 :
    ollama . pull ( model )