GeminiWithFiles
v2.0.2
Gemini API 正在持续增长。 2024年8月5日，我将 GeminiWithFiles 大幅更新为 v2.xx 通过这次大更新，版本从 v1 更改为 v2。
如果您想使用GeminiWithFiles v1.xx，请参阅此处。
这是带有文件的 Gemini API 的 Google Apps 脚本库。
名为 GeminiWithFiles 的新 Google Apps 脚本库简化了大型语言模型 Gemini 的使用，以处理图像和 PDF 等非结构化数据。 GeminiWithFiles 可以上传文件、生成内容并同时从多个图像创建描述。这显着减少了工作量并扩大了使用 Gemini 的可能性。
最近，Google AI 的大型语言模型 Gemini 通过将非结构化数据用作结构化数据，为各种任务带来了新的可能性。这一点尤其重要，因为大量信息以文本文档、图像和视频等非结构化格式存在。
最近发布的 Gemini 1.5 API 显着扩展了这些功能。它可以生成多达 100 万个代币的内容，与之前的版本相比大幅增加。此外，Gemini 1.5 现在可以处理多达 3,000 个图像文件，大大超过了 Gemini 1.0 的 16 个图像限制。参考号
虽然 Gemini 无法直接使用文档、表格和幻灯片等 Google 云端硬盘格式，但有一些解决方法。现阶段PDF数据可以直接使用Gemini API进行处理。使用此功能，这些 Google 文档文件将转换为 PDF 并与 Gemini API 一起使用。参考号
本报告引入了一个名为“GeminiWithFiles”的新 Google Apps 脚本库，它简化了此过程。 GeminiWithFiles 允许用户使用 Gemini 的强大功能轻松上传文件并生成内容。它还可以通过一次 API 调用从多个图像高效地创建描述，与单独处理每个图像（如我之前的报告中所示）相比，显着减少了工作量。参考号
通过简化流程和扩展功能，GeminiWithFiles 为跨不同领域的各种用例带来了希望。该报告是上一份报告的扩展方法，旨在进一步降低处理 Gemini 和非结构化数据时的流程成本并提高效率。
我根据以下报告创建了这个库。
该库 GeminiWithFiles 允许您通过易于使用的 API 与功能强大的文档处理和管理平台 Gemini 进行交互。以下是您可以使用该库实现的目标：
文件管理：
内容上传：
聊天记录管理：
内容生成：
输出规格：
为 Gemini API 生成的结果指定所需的输出格式。
使用response_mime_type和JSON模式，可以控制输出格式。参考号
为了测试该脚本，请执行以下步骤。
请访问 https://makersuite.google.com/app/apikey 并创建您的 API 密钥。届时，请在API控制台启用生成语言API。此 API 密钥用于此示例脚本。
这个官方文档也可以看。参考号
请创建一个独立的 Google Apps 脚本项目。当然，这个脚本也可以与容器绑定脚本一起使用。
并且，请打开 Google Apps 脚本项目的脚本编辑器。
有 2 种使用 GeminiWithFiles 的模式。
如果您将此库用作 Google Apps 脚本库，请按如下方式将该库安装到您的 Google Apps 脚本项目中。
创建一个 Google Apps 脚本项目。或者，打开您的 Google Apps 脚本项目。
安装这个库。
1dolXnIeXKz-BH1BlwRDaKhzC2smJcGyVxMxGYhaY2kqiLa857odLXrIC
如果您在自己的 Google Apps 脚本项目中使用此库，请将脚本“classGeminiWithFiles.js”复制并粘贴到您的 Google Apps 脚本项目中。这样就可以使用脚本了。
“main.js”用于 Google Apps 脚本库。因此，在这种模式中，您不需要使用它。
该库使用以下 2 个范围。
https://www.googleapis.com/auth/script.external_requesthttps://www.googleapis.com/auth/drive如果您想使用访问令牌，请将 Google Cloud Platform 项目链接到 Google Apps 脚本项目。并且，请添加以下范围。
https://www.googleapis.com/auth/generative-language另外，您可以在 https://ai.google.dev/api/rest 上查看 Gemini API 的官方文档。
| 方法 | 描述 |
|---|---|
| setFileIds(fileIds, asImage = false) | 设置文件 ID。 |
| 设置斑点（斑点） | 设置斑点。 |
| withUploadedFilesByGenerateContent(文件列表 = []) | 创建使用generateContent方法的对象。 |
| 上传文件(n = 50) | 将文件上传到 Gemini。 |
| 获取文件列表() | 获取 Gemini 中的文件列表。 |
| 删除文件（名称，n = 50） | 从 Gemini 删除文件。 |
| 生成内容（对象） | 主要方法。通过 Gemini API 生成内容。 |
| setFileIdsOrUrlsWithResumableUpload(对象) | 超过 50 MB 的文件可以上传到 Gemini。 |
当您将 GeminiWithFiles 作为库安装到 Google Apps 脚本项目时，请使用以下脚本。
const g = GeminiWithFiles . geminiWithFiles ( object ) ;或者
当您直接将 Class GeminiWithFiles 的脚本复制并粘贴到您的 Google Apps 脚本项目中时，请使用以下脚本。
const g = new GeminiWithFiles ( object ) ; object的值如下。
{Object} object API key or access token for using Gemini API.
{String} object.apiKey API key.
{String} object.accessToken Access token.
{String} object.model Model. Default is "models/gemini-1.5-pro-latest".
{String} object.version Version of API. Default is "v1beta".
{Boolean} object.doCountToken Default is false. If this is true, when Gemini API is requested, the token of request is shown in the log.
{Array} object.history History for continuing chat.
{Array} object.functions If you want to give the custom functions, please use this.
{String} object.response_mime_type In the current stage, only "application/json" can be used.
{String} object.responseMimeType In the current stage, only "application/json" can be used.
{Object} object.response_schema JSON schema for controlling the output format.
{Object} object.responseSchema JSON schema for controlling the output format.
{Number} object.temperature Control the randomness of the output.
{Object} object.systemInstruction Ref: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini.
{Boolean} object.exportTotalTokens When this is true, the total tokens are exported as the result value. At that time, the generated content and the total tokens are returned as an object.
{Boolean} object.exportRawData The default value is false. When this is true, the raw data returned from Gemini API is returned.
{Object} object.toolConfig The default is null. If you want to directly give the object of "toolConfig", please use this.
{Array} object.tools The default value is null. For example, when you want to use "codeExecution", please set `tools: [{ codeExecution: {}}]`.
{PropertiesService.Properties} object.propertiesService PropertiesService.getScriptProperties()
{Boolean} object.resumableUploadAsNewUpload When you want to upload the data with the resumable upload as new upload, please set this as true. The default is false.
当您想使用response_mime_type时，请将jsonSchema提供给generateContent方法。现阶段， response_mime_type只能使用"application/json" 。
当您想使用systemInstruction时，请确认官方文档 Ref.
Gemini 1.5 Flash 最新版 ( models/gemini-1.5-flash-latest ) 用作默认模型。当您想使用 Gemini 1.5 Pro 最新版本 ( models/gemini-1.5-pro-latest ) 时，请像const g = GeminiWithFiles.geminiWithFiles({ apiKey, model: "models/gemini-1.5-pro-latest" })一样使用。
现阶段，使用response_schema时，会自动使用response_mime_type: "application/json" 。
设置文件 ID。文件ID的文件上传到Gemini。
在这种情况下，函数中使用了 async/await。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const folderId = "###" ; // Please set your folder ID including images.
let fileIds = [ ] ;
const files = DriveApp . getFolderById ( folderId ) . getFiles ( ) ;
while ( files . hasNext ( ) ) {
const file = files . next ( ) ;
fileIds . push ( file . getId ( ) ) ;
}
const g = GeminiWithFiles . geminiWithFiles ( { apiKey } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const res = g . setFileIds ( fileIds , false ) . uploadFiles ( ) ;
console . log ( res ) ;
}setFileIds的第一个和第二个参数分别是 String[]（Google Drive 上的文件 ID）和布尔值。如果第二个参数为 false，则将输入的文件 ID 作为原始数据上传。如果第二个参数为true，则将输入的文件ID的文件转换为图像数据并上传。第二个参数的默认值为 false。false如setFileIds(fileIds, false) 。 设置斑点。 Blob 被上传到 Gemini。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const folderId = "###" ; // Please set your folder ID including images.
const blobs = [ ] ;
const files = DriveApp . getFolderById ( folderId ) . getFiles ( ) ;
while ( files . hasNext ( ) ) {
blobs . push ( files . next ( ) . getBlob ( ) ) ;
}
const g = GeminiWithFiles . geminiWithFiles ( { apiKey } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const res = g . setBlobs ( blobs ) . uploadFiles ( ) ;
console . log ( res ) ;
}setBlobs的第一个参数是 Blob[]。创建使用generateContent方法的对象。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const q = "###" ; // Please set your question.
const g = GeminiWithFiles . geminiWithFiles ( { apiKey } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const fileList = g . getFileList ( ) ;
const res = g
. withUploadedFilesByGenerateContent ( fileList )
. generateContent ( { q } ) ;
console . log ( res ) ;
}withUploadedFilesByGenerateContent只有一个参数。这是来自 getFileList 方法的值。上传文件后您可以看到实际值。将文件上传到 Gemini。使用输入的文件 ID 或 blob 将文件上传到 Gemini。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const fileIds = [ "###fileId1###" , "###fileId2###" , , , ] ; // Please set your file IDs in this array.
const g = GeminiWithFiles . geminiWithFiles ( { apiKey } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const res = g . setFileIds ( fileIds , false ) . uploadFiles ( ) ;
console . log ( res ) ;
}在此脚本中， fileIds的文件与原始数据一起上传到 Gemini。如果将setFileIds(fileIds, false)修改为setFileIds(fileIds, true) ，则文件将作为图片上传到 Gemini。
当直接使用Blob时，可以使用以下脚本。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const fileIds = [ "###fileId1###" , "###fileId2###" , , , ] ; // Please set your file IDs in this array.
const blobs = fileIds . map ( id => DriveApp . getFileById ( id ) . getBlob ( ) ) ;
const g = GeminiWithFiles . geminiWithFiles ( { apiKey } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const res = g . setBlobs ( blobs ) . uploadFiles ( ) ;
console . log ( res ) ;
} 获取 Gemini 中的文件列表。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const g = GeminiWithFiles . geminiWithFiles ( { apiKey } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const res = g . getFileList ( ) ;
console . log ( res ) ;
} 从 Gemini 删除文件。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const g = GeminiWithFiles . geminiWithFiles ( { apiKey } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const names = g . getFileList ( ) . map ( ( { name } ) => name ) ;
if ( names . length == 0 ) return ;
g . deleteFiles ( names ) ;
console . log ( ` ${ names . length } files were deleted.` ) ;
}主要方法。通过 Gemini API 生成内容。更多示例脚本可以在下面的“示例脚本”部分中看到。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const g = GeminiWithFiles . geminiWithFiles ( { apiKey } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const res = g . generateContent ( { q : "What is Google Apps Script?" } ) ;
console . log ( res ) ;
}在此脚本中，内容是通过函数调用生成的。
当您想使用response_mime_type时，请将jsonSchema提供给generateContent方法，如下所示。在这种情况下，通过仅提供 JSON 模式，该库可以返回有效的对象。您还可以在我的报告中查看有关response_mime_type的详细信息。
function myFunction ( ) {
const apiKey = "###" ; // Please set your API key.
const g = GeminiWithFiles . geminiWithFiles ( {
apiKey ,
response_mime_type : "application/json" ,
} ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey, response_mime_type: "application/json" }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const jsonSchema = {
title : "5 popular cookie recipes" ,
description : "List 5 popular cookie recipes." ,
type : "array" ,
items : {
type : "object" ,
properties : {
recipe_name : {
description : "Names of recipe." ,
type : "string" ,
} ,
} ,
} ,
} ;
const res = g . generateContent ( { jsonSchema } ) ;
console . log ( res ) ;
}运行此脚本时，将获得以下结果。
[
{ "recipe_name" : " Chocolate Chip Cookies " },
{ "recipe_name" : " Peanut Butter Cookies " },
{ "recipe_name" : " Oatmeal Cookies " },
{ "recipe_name" : " Sugar Cookies " },
{ "recipe_name" : " Snickerdoodle Cookies " }
]此方法可以上传超过50MB的文件。
从 v2.xx 开始，这可以实现。这是来自参考文献和参考文献。
从v2.0.3开始，当您使用此方法时，请将propertiesService: PropertiesService.getScriptProperties()包含到初始对象中，如下所示。因为，当在库中使用PropertiesService.getScriptProperties()时，值将被放入库中。当我创建Ref和Ref时，我认为脚本是通过复制和粘贴而不是库来使用的。因此，我在脚本中包含了PropertiesService.getScriptProperties() 。但我注意到，当与 GeminiWithFiles 一起使用时，每个用户都需要使用PropertiesService.getScriptProperties() 。所以，我修改了这个。
示例脚本如下。
function myFunction ( ) {
// This URL is from https://github.com/google/generative-ai-docs/blob/main/site/en/gemini-api/docs/prompting_with_media.ipynb
const url = "https://download.blender.org/peach/bigbuckbunny_movies/BigBuckBunny_320x180.mp4" ; // 64,657,027 bytes
const apiKey = "###" ; // Please set your API key.
const q = "Description this video." ;
const g = GeminiWithFiles . geminiWithFiles ( { apiKey , propertiesService : PropertiesService . getScriptProperties ( ) } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey, propertiesService: PropertiesService.getScriptProperties() }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const fileList = g . setFileIdsOrUrlsWithResumableUpload ( [ { url } ] ) . uploadFiles ( ) ;
Utilities . sleep ( 10000 ) ; // This might be required to be used because the state of the uploaded file might not be active.
const res = g . withUploadedFilesByGenerateContent ( fileList ) . generateContent ( { q } ) ;
console . log ( res ) ;
}当这个脚本运行时，可以在日志中看到如下日志。
- Get metadata
- Calculate chunks
- Get location
- Download and upload data.
- Now... 1/4
- Start downloading data with 0-16777215
- Finished downloading data with 0-16777215
- Start uploading data with 0-16777215
- Finished uploading data with 0-16777215
- Upload the next chunk.
- Now... 2/4
- Start downloading data with 16777216-33554431
- Finished downloading data with 16777216-33554431
- Start uploading data with 16777216-33554431
- Finished uploading data with 16777216-33554431
- Upload the next chunk.
- Now... 3/4
- Start downloading data with 33554432-50331647
- Finished downloading data with 33554432-50331647
- Start uploading data with 33554432-50331647
- Finished uploading data with 33554432-50331647
- Upload the next chunk.
- Now... 4/4
- Start downloading data with 50331648-64657026
- Finished downloading data with 50331648-64657026
- Start uploading data with 50331648-64657026
- Finished uploading data with 50331648-64657026
- Done.
- Now, the state of the uploaded files "url@https://download.blender.org/peach/bigbuckbunny_movies/BigBuckBunny_320x180.mp4$page@1$maxPage@1" is not active. So, it will wait until it is active. Please wait for 10 seconds. Retry (1/3)
- 1 uploaded files are used with generateCotent.
- The video is a cartoon that shows a large, white rabbit in a field. The rabbit is shown waking up from a nap and then is seen eating an apple. After eating the apple, the rabbit is approached by a bird. The rabbit is scared of the bird and tries to hide from it. The bird flies away. The rabbit is seen smiling and then a squirrel flies toward the rabbit. The squirrel is startled by the rabbit and flies away. The rabbit is then seen catching another squirrel with its vine and the scene ends with a close-up of the rabbit's face.
如果您的文件很大并且上传的文件状态尚未变为“ACTIVE”，请测试以下脚本。
function myFunction ( ) {
// This URL is from https://github.com/google/generative-ai-docs/blob/main/site/en/gemini-api/docs/prompting_with_media.ipynb
const url = "https://download.blender.org/peach/bigbuckbunny_movies/BigBuckBunny_320x180.mp4" ; // 64,657,027 bytes
const apiKey = "###" ; // Please set your API key.
const q = "Description this video." ;
const g = GeminiWithFiles . geminiWithFiles ( { apiKey , propertiesService : PropertiesService . getScriptProperties ( ) } ) ; // This is for installing GeminiWithFiles as a library.
// const g = new GeminiWithFiles({ apiKey, propertiesService: PropertiesService.getScriptProperties() }); // This is for directly copying and pasting Class GeminiWithFiles into your Google Apps Script project.
const fileList = g . setFileIdsOrUrlsWithResumableUpload ( [ { url } ] ) . uploadFiles ( ) ;
console . log ( JSON . stringify ( fileList ) ) ;
// Please copy the value of "fileList".
}这样就可以上传文件了。并且，您可以在等待足够的时间将状态更改为“ACTIVE”后使用上传的文件。上传的文件可以如下使用。
