吲哚美辛
V0.1.31
官方网站 • 文档 • Discord
新：订阅我们的邮件列表以获取更新和新闻！
Indox Retrieval Augmentation是一款创新应用程序，旨在简化从各种文档类型（包括文本文件、PDF、HTML、Markdown 和 LaTeX）中提取信息的过程。无论是结构化还是非结构化，Indox 都为用户提供了强大的工具集来高效提取相关数据。
Indox Retrieval Augmentation 是一款创新应用程序，旨在简化从各种文档类型（包括文本文件、PDF、HTML、Markdown 和 LaTeX）中提取信息的过程。无论是结构化还是非结构化，Indox 都为用户提供了强大的工具集来高效提取相关数据。其关键功能之一是能够智能地对主要块进行聚类以形成更强大的分组，从而提高所提取信息的质量和相关性。 Indox 专注于适应性和以用户为中心的设计，旨在提供面向未来的功能，并计划在即将发布的版本中提供更多功能。与我们一起探索 Indox 如何彻底改变您的文档处理工作流程，为您的数据检索需求带来清晰度和组织性。
| ？型号支持 | 实施的 | 描述 |
|---|---|---|
| Ollama（例如 Llama3） | ✅ | 由 Ollama 提供支持的本地嵌入和 LLM 模型 |
| 抱脸 | ✅ | 由 HuggingFace 提供支持的本地嵌入和 LLM 模型 |
| 米斯特拉尔 | ✅ | Cohere 的嵌入和 LLM 模型 |
| 谷歌（例如双子座） | ✅ | Google 的嵌入和生成模型 |
| OpenAI（例如GPT4） | ✅ | OpenAI 的嵌入和生成模型 |
| 通过 Indox Api 支持的模型 | 实施的 | 描述 |
|---|---|---|
| 开放人工智能 | ✅ | 来自 Indox Api 的嵌入和 LLm OpenAi 模型 |
| 米斯特拉尔 | ✅ | 来自 Indox Api 的嵌入和 LLm Mistral 模型 |
| 人择 | Indox Api 的嵌入和 LLm 人择模型 |
| ？装载机和分离机 | 实施的 | 描述 |
|---|---|---|
| 简单的PDF | ✅ | 导入 PDF |
| 非结构化IO | ✅ | 通过非结构化导入数据 |
| 集群负载和拆分 | ✅ | 加载 pdf 和文本。添加额外的聚类层 |
| RAG特点 | 实施的 | 描述 |
|---|---|---|
| 混合搜索 | 语义搜索与关键词搜索相结合 | |
| 语义缓存 | ✅ | 根据语义保存和检索结果 |
| 集群提示 | ✅ | 检索较小的块并进行聚类和汇总 |
| 代理抹布 | ✅ | 如果需要，生成更可靠的答案、对上下文和网络搜索进行排名 |
| 高级查询 | 基于LLM评估的任务委派 | |
| 重新排名 | ✅ | 根据上下文对结果重新排序以改进结果 |
| 可定制的元数据 | 自由控制元数据 |
| ？酷炫奖金 | 实施的 | 描述 |
|---|---|---|
| Docker 支持 | Indox 可通过 Docker 部署 | |
| 可定制的前端 | Indox 的前端可通过前端完全定制 |
| ☑️ 示例 | 在 Colab 中运行 |
|---|---|
| Indox API (OpenAi) | |
| 米斯特拉尔（使用非结构化） | |
| OpenAi（使用集群分割） | |
| 拥抱脸部模型（米斯特拉尔） | |
| 奥拉马 | |
| 使用 IndoxJudge 进行评估 |
以下命令将安装最新的稳定版 inDox
pip install Indox
要安装最新的开发版本，您可以运行
pip install git+https://github.com/osllmai/inDox@master
克隆存储库并导航到目录：
git clone https://github.com/osllmai/inDox.git
cd inDox安装所需的 Python 包：
pip install -r requirements.txt如果您在本地 IDE 中运行此项目，请创建一个 Python 环境以确保所有依赖项都得到正确管理。您可以按照以下步骤设置名为indox的虚拟环境：
python -m venv indox indox S cripts a ctivatepython3 -m venv indox
source indox/bin/activate激活虚拟环境后，通过运行以下命令安装所需的依赖项：
pip install -r requirements.txtpip install indox
pip install openai
pip install chromadb如果您在本地 IDE 中运行此项目，请创建一个 Python 环境以确保所有依赖项都得到正确管理。您可以按照以下步骤设置名为indox的虚拟环境：
python -m venv indoxindox_judge S cripts a ctivatepython3 -m venv indox
2. **Activate the virtual environment:**
```bash
source indox/bin/activate
激活虚拟环境后，通过运行以下命令安装所需的依赖项：
pip install -r requirements.txt首先，您需要从环境中加载 API 密钥。
import os
from dotenv import load_dotenv
load_dotenv ()
OPENAI_API_KEY = os . environ [ 'OPENAI_API_KEY' ]从 Indox 包中导入必要的类。
from indox import IndoxRetrievalAugmentation from indox . llms import OpenAi from indox . embeddings import OpenAiEmbedding创建 IndoxRetrievalAugmentation 的实例。
Indox = IndoxRetrievalAugmentation () openai_qa = OpenAiQA ( api_key = OPENAI_API_KEY , model = "gpt-3.5-turbo-0125" )
openai_embeddings = OpenAiEmbedding ( model = "text-embedding-3-small" , openai_api_key = OPENAI_API_KEY ) file_path = "sample.txt"在本节中，我们利用unstructured库来加载文档并按标题将它们分割成块。此方法有助于将文档组织成可管理的部分以进行进一步处理。
from indox . data_loader_splitter import UnstructuredLoadAndSplit loader_splitter = UnstructuredLoadAndSplit ( file_path = file_path )
docs = loader_splitter . load_and_chunk () Starting processing...
End Chunking process.
将文档块存储在向量存储中对于实现高效的检索和搜索操作至关重要。通过将文本数据转换为向量表示并将其存储在向量存储中，您可以执行快速相似性搜索和其他基于向量的操作。
from indox . vector_stores import ChromaVectorStore
db = ChromaVectorStore ( collection_name = "sample" , embedding = embed_openai )
Indox . connect_to_vectorstore ( db )
Indox . store_in_vectorstore ( docs ) 2024-05-14 15:33:04,916 - INFO - Anonymized telemetry enabled. See https://docs.trychroma.com/telemetry for more information.
2024-05-14 15:33:12,587 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-05-14 15:33:13,574 - INFO - Document added successfully to the vector store.
Connection established successfully.
query = "how cinderella reach her happy ending?" retriever = indox . QuestionAnswer ( vector_database = db , llm = openai_qa , top_k = 5 )
retriever . invoke ( query ) 2024-05-14 15:34:55,380 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-05-14 15:35:01,917 - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
'Cinderella reached her happy ending by enduring mistreatment from her step-family, finding solace and help from the hazel tree and the little white bird, attending the royal festival where the prince recognized her as the true bride, and ultimately fitting into the golden shoe that proved her identity. This led to her marrying the prince and living happily ever after.'
retriever . context ["from the hazel-bush. Cinderella thanked him, went to her mother'snngrave and planted the branch on it, and wept so much that the tearsnnfell down on it and watered it. And it grew and became a handsomenntree. Thrice a day cinderella went and sat beneath it, and wept andnnprayed, and a little white bird always came on the tree, and ifnncinderella expressed a wish, the bird threw down to her what shennhad wished for.nnIt happened, however, that the king gave orders for a festival",
'worked till she was weary she had no bed to go to, but had to sleepnnby the hearth in the cinders. And as on that account she alwaysnnlooked dusty and dirty, they called her cinderella.nnIt happened that the father was once going to the fair, and hennasked his two step-daughters what he should bring back for them.nnBeautiful dresses, said one, pearls and jewels, said the second.nnAnd you, cinderella, said he, what will you have. Father',
'face he recognized the beautiful maiden who had danced withnnhim and cried, that is the true bride. The step-mother andnnthe two sisters were horrified and became pale with rage, he,nnhowever, took cinderella on his horse and rode away with her. Asnnthey passed by the hazel-tree, the two white doves cried -nnturn and peep, turn and peep,nnno blood is in the shoe,nnthe shoe is not too small for her,nnthe true bride rides with you,nnand when they had cried that, the two came flying down and',
"to send her up to him, but the mother answered, oh, no, she isnnmuch too dirty, she cannot show herself. But he absolutelynninsisted on it, and cinderella had to be called. She firstnnwashed her hands and face clean, and then went and bowed downnnbefore the king's son, who gave her the golden shoe. Then shennseated herself on a stool, drew her foot out of the heavynnwooden shoe, and put it into the slipper, which fitted like annglove. And when she rose up and the king's son looked at her",
'slippers embroidered with silk and silver. She put on the dressnnwith all speed, and went to the wedding. Her step-sisters and thennstep-mother however did not know her, and thought she must be annforeign princess, for she looked so beautiful in the golden dress.nnThey never once thought of cinderella, and believed that she wasnnsitting at home in the dirt, picking lentils out of the ashes. Thennprince approached her, took her by the hand and danced with her.']
.----------------. .-----------------. .----------------. .----------------. .----------------.
| .--------------. || .--------------. || .--------------. || .--------------. || .--------------. |
| | _____ | || | ____ _____ | || | ________ | || | ____ | || | ____ ____ | |
| | |_ _| | || ||_ |_ _| | || | |_ ___ `. | || | .' `. | || | |_ _||_ _| | |
| | | | | || | | | | | || | | | `. | || | / .--. | || | / / | |
| | | | | || | | | | | | || | | | | | | || | | | | | | || | > `' < | |
| | _| |_ | || | _| |_ |_ | || | _| |___.' / | || | `--' / | || | _/ /'` _ | |
| | |_____| | || ||_____|____| | || | |________.' | || | `.____.' | || | |____||____| | |
| | | || | | || | | || | | || | | |
| '--------------' || '--------------' || '--------------' || '--------------' || '--------------' |
'----------------' '----------------' '----------------' '----------------' '----------------' 
