GeminiWithFiles
v2.0.2
Gemini API 正在持續成長。 2024年8月5日，我將 GeminiWithFiles 大幅更新為 v2.xx 透過這次大更新，版本從 v1 改為 v2。
如果您想使用GeminiWithFiles v1.xx，請參閱此處。
這是一個帶有文件的 Gemini API 的 Google Apps 腳本庫。
名為 GeminiWithFiles 的新 Google Apps 腳本庫簡化了大型語言模型 Gemini 的使用，以處理影像和 PDF 等非結構化資料。 GeminiWithFiles 可以上傳檔案、產生內容並同時從多個影像建立描述。這顯著減少了工作量並擴大了使用 Gemini 的可能性。
最近，Google AI 的大型語言模型 Gemini 透過將非結構化資料用作結構化數據，為各種任務帶來了新的可能性。這一點尤其重要，因為大量資訊以文字文件、圖像和影片等非結構化格式存在。
最近發布的 Gemini 1.5 API 顯著擴展了這些功能。它可以產生多達 100 萬個代幣的內容，與之前的版本相比大幅增加。此外，Gemini 1.5 現在可以處理多達 3,000 個影像文件，大大超過了 Gemini 1.0 的 16 個影像限制。參考號
雖然 Gemini 無法直接使用文件、表格和幻燈片等 Google 雲端硬碟格式，但有一些解決方法。現階段PDF資料可以直接使用Gemini API進行處理。使用此功能，這些 Google 文件檔案將轉換為 PDF 並與 Gemini API 一起使用。參考號
本報告引入了一個名為「GeminiWithFiles」的新 Google Apps 腳本庫，它簡化了此過程。 GeminiWithFiles 讓使用者可以使用 Gemini 的強大功能輕鬆上傳檔案並產生內容。它還可以透過一次 API 呼叫從多個圖像有效地創建描述，與單獨處理每個圖像（如我之前的報告中所示）相比，顯著減少了工作量。參考號
透過簡化流程和擴充功能，GeminiWithFiles 為跨不同領域的各種用例帶來了希望。該報告是上一份報告的擴展方法，旨在進一步降低處理 Gemini 和非結構化資料時的流程成本並提高效率。
我根據以下報告創建了這個庫。
該程式庫 GeminiWithFiles 可讓您透過易於使用的 API 與功能強大的文件處理和管理平台 Gemini 進行互動。以下是您可以使用該程式庫實現的目標：
文件管理：
內容上傳：
聊天記錄管理：
內容生成：
輸出規格：
為 Gemini API 產生的結果指定所需的輸出格式。
使用response_mime_type和JSON模式，可以控制輸出格式。參考號
為了測試該腳本，請執行以下步驟。
請造訪 https://makersuite.google.com/app/apikey 並建立您的 API 金鑰。屆時，請在API控制台啟用產生語言API。此 API 金鑰用於此範例腳本。
這個官方文件也可以看。參考號
請建立一個獨立的 Google Apps 腳本專案。當然，這個腳本也可以與容器綁定腳本一起使用。
並且，請開啟 Google Apps 腳本專案的腳本編輯器。
有 2 種使用 GeminiWithFiles 的模式。
如果您將此庫用作 Google Apps 腳本庫，請按如下方式將該庫安裝到您的 Google Apps 腳本專案中。
建立一個 Google Apps 腳本專案。或者，開啟您的 Google Apps 腳本專案。
安裝這個庫。
1dolXnIeXKz-BH1BlwRDaKhzC2smJcGyVxMxGYhaY2kqiLa857odLXrIC
如果您在自己的 Google Apps 腳本專案中使用此程式庫，請將腳本「classGeminiWithFiles.js」複製並貼上到您的 Google Apps 腳本專案中。這樣就可以使用腳本了。
“main.js”用於 Google Apps 腳本庫。因此，在這種模式中，您不需要使用它。
該庫使用以下 2 個範圍。
https://www.googleapis.com/auth/script.external_requesthttps://www.googleapis.com/auth/drive如果您想使用存取權令牌，請將 Google Cloud Platform 專案連結到 Google Apps 腳本專案。並且，請新增以下範圍。
https://www.googleapis.com/auth/generative-language另外，您可以在 https://ai.google.dev/api/rest 上查看 Gemini API 的官方文件。
| 方法 | 描述 |
|---|---|
| setFileIds(fileIds, asImage = false) | 設定檔案 ID。 |
| 設定斑點（斑點） | 設定斑點。 |
| withUploadedFilesByGenerateContent(檔案清單 = []) | 建立使用generateContent方法的物件。 |
| 上傳文件(n = 50) | 將檔案上傳到 Gemini。 |
| 取得文件列表() | 取得 Gemini 中的檔案清單。 |
| 刪除檔案（名稱，n = 50） | 從 Gemini 刪除檔案。 |
| 產生內容（對象） | 主要方法。透過 Gemini API 產生內容。 |
| setFileIdsOrUrlsWithResumableUpload(物件) | 超過 50 MB 的檔案可以上傳到 Gemini。 |
當您將 GeminiWithFiles 作為庫安裝到 Google Apps 腳本專案時，請使用下列腳本。
const g = GeminiWithFiles . geminiWithFiles ( object ) ;或者
當您直接將 Class GeminiWithFiles 的腳本複製並貼上到您的 Google Apps 腳本專案中時，請使用下列腳本。
const g = new GeminiWithFiles ( object ) ; object的值如下。
{Object} object API key or access token for using Gemini API.
{String} object.apiKey API key.
{String} object.accessToken Access token.
{String} object.model Model. Default is "models/gemini-1.5-pro-latest".
{String} object.version Version of API. Default is "v1beta".
{Boolean} object.doCountToken Default is false. If this is true, when Gemini API is requested, the token of request is shown in the log.
{Array} object.history History for continuing chat.
{Array} object.functions If you want to give the custom functions, please use this.
{String} object.response_mime_type In the current stage, only "application/json" can be used.
{String} object.responseMimeType In the current stage, only "application/json" can be used.
{Object} object.response_schema JSON schema for controlling the output format.
{Object} object.responseSchema JSON schema for controlling the output format.
{Number} object.temperature Control the randomness of the output.
{Object} object.systemInstruction Ref: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini.
{Boolean} object.exportTotalTokens When this is true, the total tokens are exported as the result value. At that time, the generated content and the total tokens are returned as an object.
{Boolean} object.exportRawData The default value is false. When this is true, the raw data returned from Gemini API is returned.
{Object} object.toolConfig The default is null. If you want to directly give the object of "toolConfig", please use this.
{Array} object.tools The default value is null. For example, when you want to use "codeExecution", please set `tools: [{ codeExecution: {}}]`.
{PropertiesService.Properties} object.propertiesService PropertiesService.getScriptProperties()
{Boolean} object.resumableUploadAsNewUpload When you want to upload the data with the resumable upload as new upload, please set this as true. The default is false.
當您想要使用response_mime_type時，請將jsonSchema提供給generateContent方法。現階段， response_mime_type只能使用"application/json" 。
當您想使用systemInstruction時，請確認官方文件 Ref.
Gemini 1.5 Flash 最新版 ( models/gemini-1.5-flash-latest ) 用作預設模型。當您想要使用 Gemini 1.5 Pro 最新版本 ( models/gemini-1.5-pro-latest const g = GeminiWithFiles.geminiWithFiles({ apiKey, model: "models/gemini-1.5-pro-latest" })一樣使用。
現階段，使用response_schema時，會自動使用response_mime_type: "application/json" 。
設定檔案 ID。文件ID的檔案上傳到Gemini。
在這種情況下，函數中使用了 async/await。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const folderId = "###" ; // Please set your folder ID including images.
let fileIds = [ ] ;
const files = DriveApp . getFolderById ( folderId ) . getFiles ( ) ;
while ( files . hasNext ( ) ) {
const file = files . next ( ) ;
fileIds . push ( file . getId ( ) ) ;
}
const g = GeminiWithFiles . geminiWithFiles ( { apiKey } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const res = g . setFileIds ( fileIds , false ) . uploadFiles ( ) ;
console . log ( res ) ;
}setFileIds的第一個和第二個參數分別是 String[]（Google Drive 上的檔案 ID）和布林值。如果第二個參數為 false，則將輸入的檔案 ID 作為原始資料上傳。如果第二個參數為true，則將輸入的檔案ID的檔案轉換為影像資料並上傳。第二個參數的預設值為 false。false例如setFileIds(fileIds, false) 。 設定斑點。 Blob 上傳到 Gemini。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const folderId = "###" ; // Please set your folder ID including images.
const blobs = [ ] ;
const files = DriveApp . getFolderById ( folderId ) . getFiles ( ) ;
while ( files . hasNext ( ) ) {
blobs . push ( files . next ( ) . getBlob ( ) ) ;
}
const g = GeminiWithFiles . geminiWithFiles ( { apiKey } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const res = g . setBlobs ( blobs ) . uploadFiles ( ) ;
console . log ( res ) ;
}setBlobs的第一個參數是 Blob[]。建立使用generateContent方法的物件。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const q = "###" ; // Please set your question.
const g = GeminiWithFiles . geminiWithFiles ( { apiKey } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const fileList = g . getFileList ( ) ;
const res = g
. withUploadedFilesByGenerateContent ( fileList )
. generateContent ( { q } ) ;
console . log ( res ) ;
}withUploadedFilesByGenerateContent只有一個參數。這是來自 getFileList 方法的值。上傳文件後您可以看到實際值。將檔案上傳到 Gemini。使用輸入的檔案 ID 或 blob 將檔案上傳到 Gemini。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const fileIds = [ "###fileId1###" , "###fileId2###" , , , ] ; // Please set your file IDs in this array.
const g = GeminiWithFiles . geminiWithFiles ( { apiKey } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const res = g . setFileIds ( fileIds , false ) . uploadFiles ( ) ;
console . log ( res ) ;
}在此腳本中， fileIds的檔案會與原始資料一起上傳到 Gemini。如果將setFileIds(fileIds, false)修改為setFileIds(fileIds, true) ，則檔案將作為圖片上傳到 Gemini。
直接使用Blob時，可以使用以下腳本。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const fileIds = [ "###fileId1###" , "###fileId2###" , , , ] ; // Please set your file IDs in this array.
const blobs = fileIds . map ( id => DriveApp . getFileById ( id ) . getBlob ( ) ) ;
const g = GeminiWithFiles . geminiWithFiles ( { apiKey } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const res = g . setBlobs ( blobs ) . uploadFiles ( ) ;
console . log ( res ) ;
} 取得 Gemini 中的檔案清單。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const g = GeminiWithFiles . geminiWithFiles ( { apiKey } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const res = g . getFileList ( ) ;
console . log ( res ) ;
} 從 Gemini 刪除檔案。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const g = GeminiWithFiles . geminiWithFiles ( { apiKey } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const names = g . getFileList ( ) . map ( ( { name } ) => name ) ;
if ( names . length == 0 ) return ;
g . deleteFiles ( names ) ;
console . log ( ` ${ names . length } files were deleted.` ) ;
}主要方法。透過 Gemini API 產生內容。更多範例腳本可以在下面的“範例腳本”部分中看到。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const g = GeminiWithFiles . geminiWithFiles ( { apiKey } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const res = g . generateContent ( { q : "What is Google Apps Script?" } ) ;
console . log ( res ) ;
}在此腳本中，內容是透過函數呼叫產生的。
當您想要使用response_mime_type時，請將jsonSchema提供給generateContent方法，如下所示。在這種情況下，透過僅提供 JSON 模式，該程式庫可以傳回有效的物件。您也可以在我的報告中查看有關response_mime_type的詳細資訊。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const g = GeminiWithFiles . geminiWithFiles ( {
apiKey ,
response_mime_type : "application/json" ,
} ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey, response_mime_type: "application/json" }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const jsonSchema = {
title : "5 popular cookie recipes" ,
description : "List 5 popular cookie recipes." ,
type : "array" ,
items : {
type : "object" ,
properties : {
recipe_name : {
description : "Names of recipe." ,
type : "string" ,
} ,
} ,
} ,
} ;
const res = g . generateContent ( { jsonSchema } ) ;
console . log ( res ) ;
}執行此腳本時，將獲得以下結果。
[
{ "recipe_name" : " Chocolate Chip Cookies " },
{ "recipe_name" : " Peanut Butter Cookies " },
{ "recipe_name" : " Oatmeal Cookies " },
{ "recipe_name" : " Sugar Cookies " },
{ "recipe_name" : " Snickerdoodle Cookies " }
]此方法可以上傳超過50MB的檔案。
從 v2.xx 開始，這可以實現。這是來自參考文獻和參考文獻。
從v2.0.3開始，當您使用此方法時，請將propertiesService: PropertiesService.getScriptProperties()包含到初始物件中，如下所示。因為，當在庫中使用PropertiesService.getScriptProperties()時，值將被放入庫中。當我創建Ref和Ref時，我認為腳本是透過複製和貼上而不是庫來使用的。因此，我在腳本中包含了PropertiesService.getScriptProperties() 。但我注意到，與 GeminiWithFiles 一起使用時，每個使用者都需要使用PropertiesService.getScriptProperties() 。所以，我修改了這個。
範例腳本如下。
function myFunction ( ) {
// This URL is from https://github.com/google/generative-ai-docs/blob/main/site/en/gemini-api/docs/prompting_with_media.ipynb
const url = "https://download.blender.org/peach/bigbuckbunny_movies/BigBuckBunny_320x180.mp4" ; // 64,657,027 bytes
const apiKey = "###" ; // Please set your API key.
const q = "Description this video." ;
const g = GeminiWithFiles . geminiWithFiles ( { apiKey , propertiesService : PropertiesService . getScriptProperties ( ) } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey, propertiesService: PropertiesService.getScriptProperties() }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const fileList = g . setFileIdsOrUrlsWithResumableUpload ( [ { url } ] ) . uploadFiles ( ) ;
Utilities . sleep ( 10000 ) ; // This might be required to be used because the state of the uploaded file might not be active.
const res = g . withUploadedFilesByGenerateContent ( fileList ) . generateContent ( { q } ) ;
console . log ( res ) ;
}當這個腳本運行時，可以在日誌中看到以下日誌。
- Get metadata
- Calculate chunks
- Get location
- Download and upload data.
- Now... 1/4
- Start downloading data with 0-16777215
- Finished downloading data with 0-16777215
- Start uploading data with 0-16777215
- Finished uploading data with 0-16777215
- Upload the next chunk.
- Now... 2/4
- Start downloading data with 16777216-33554431
- Finished downloading data with 16777216-33554431
- Start uploading data with 16777216-33554431
- Finished uploading data with 16777216-33554431
- Upload the next chunk.
- Now... 3/4
- Start downloading data with 33554432-50331647
- Finished downloading data with 33554432-50331647
- Start uploading data with 33554432-50331647
- Finished uploading data with 33554432-50331647
- Upload the next chunk.
- Now... 4/4
- Start downloading data with 50331648-64657026
- Finished downloading data with 50331648-64657026
- Start uploading data with 50331648-64657026
- Finished uploading data with 50331648-64657026
- Done.
- Now, the state of the uploaded files "url@https://download.blender.org/peach/bigbuckbunny_movies/BigBuckBunny_320x180.mp4$page@1$maxPage@1" is not active. So, it will wait until it is active. Please wait for 10 seconds. Retry (1/3)
- 1 uploaded files are used with generateCotent.
- The video is a cartoon that shows a large, white rabbit in a field. The rabbit is shown waking up from a nap and then is seen eating an apple. After eating the apple, the rabbit is approached by a bird. The rabbit is scared of the bird and tries to hide from it. The bird flies away. The rabbit is seen smiling and then a squirrel flies toward the rabbit. The squirrel is startled by the rabbit and flies away. The rabbit is then seen catching another squirrel with its vine and the scene ends with a close-up of the rabbit's face.
如果您的檔案很大且上傳的檔案狀態尚未變成“ACTIVE”，請測試以下腳本。
function myFunction ( ) {
// This URL is from https://github.com/google/generative-ai-docs/blob/main/site/en/gemini-api/docs/prompting_with_media.ipynb
const url = "https://download.blender.org/peach/bigbuckbunny_movies/BigBuckBunny_320x180.mp4" ; // 64,657,027 bytes
const apiKey = "###" ; // Please set your API key.
const q = "Description this video." ;
const g = GeminiWithFiles . geminiWithFiles ( { apiKey , propertiesService : PropertiesService . getScriptProperties ( ) } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey, propertiesService: PropertiesService.getScriptProperties() }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const fileList = g . setFileIdsOrUrlsWithResumableUpload ( [ { url } ] ) . uploadFiles ( ) ;
console . log ( JSON . stringify ( fileList ) ) ;
// Please copy the value of "fileList".
}這樣就可以上傳文件了。並且，您可以在等待足夠的時間將狀態變更為「ACTIVE」後使用上傳的檔案。上傳的文件可以如下使用。
