吲哚美辛
V0.1.31
官方網站 • 文件 • Discord
新：訂閱我們的郵件清單以獲取更新和新聞！
Indox Retrieval Augmentation是一款創新應用程序，旨在簡化從各種文件類型（包括文字檔案、PDF、HTML、Markdown 和 LaTeX）中提取資訊的過程。無論是結構化或非結構化，Indox 都為使用者提供了強大的工具集來高效提取相關資料。
Indox Retrieval Augmentation 是一款創新應用程序，旨在簡化從各種文件類型（包括文字檔案、PDF、HTML、Markdown 和 LaTeX）中提取資訊的過程。無論是結構化或非結構化，Indox 都為使用者提供了強大的工具集來高效提取相關資料。其關鍵功能之一是能夠智慧地將主要區塊進行聚類以形成更強大的分組，從而提高所提取資訊的品質和相關性。 Indox 專注於適應性和以用戶為中心的設計，旨在提供面向未來的功能，並計劃在即將發布的版本中提供更多功能。與我們一起探索 Indox 如何徹底改變您的文件處理工作流程，為您的資料檢索需求帶來清晰度和組織性。
| ？型號支援 | 實施的 | 描述 |
|---|---|---|
| Ollama（例如 Llama3） | ✅ | 由 Ollama 提供支援的本地嵌入和 LLM 模型 |
| 抱臉 | ✅ | 由 HuggingFace 提供支援的本地嵌入和 LLM 模型 |
| 米斯特拉爾 | ✅ | Cohere 的嵌入與 LLM 模型 |
| Google（例如雙子座） | ✅ | Google 的嵌入與生成模型 |
| OpenAI（例如GPT4） | ✅ | OpenAI 的嵌入與生成模型 |
| 透過 Indox Api 支援的模型 | 實施的 | 描述 |
|---|---|---|
| 開放人工智慧 | ✅ | 來自 Indox Api 的嵌入和 LLm OpenAi 模型 |
| 米斯特拉爾 | ✅ | 來自 Indox Api 的嵌入和 LLm Mistral 模型 |
| 人擇 | Indox Api 的嵌入與 LLm 人擇模型 |
| ？裝載機和分離機 | 實施的 | 描述 |
|---|---|---|
| 簡單的PDF | ✅ | 匯入 PDF |
| 非結構化IO | ✅ | 透過非結構化導入數據 |
| 叢集負載和拆分 | ✅ | 載入 pdf 和文字。增加額外的聚類層 |
| RAG特點 | 實施的 | 描述 |
|---|---|---|
| 混合搜尋 | 語意搜尋與關鍵字搜尋結合 | |
| 語意緩存 | ✅ | 根據語義保存和檢索結果 |
| 集群提示 | ✅ | 檢索較小的區塊並進行聚類和匯總 |
| 代理抹布 | ✅ | 如果需要，產生更可靠的答案、對上下文和網路搜尋進行排名 |
| 進階查詢 | 基於LLM評估的任務委派 | |
| 重新排名 | ✅ | 根據上下文對結果重新排序以改善結果 |
| 可自訂的元數據 | 自由控制元數據 |
| ？酷獎金 | 實施的 | 描述 |
|---|---|---|
| Docker 支援 | Indox 可透過 Docker 部署 | |
| 可自訂的前端 | Indox 的前端可透過前端完全客製化 |
| ☑️ 範例 | 在 Colab 中運行 |
|---|---|
| Indox API (OpenAi) | |
| 米斯特拉爾（使用非結構化） | |
| OpenAi（使用叢集分割） | |
| 擁抱臉部模型（米斯特拉爾） | |
| 奧拉馬 | |
| 使用 IndoxJudge 進行評估 |
以下命令將安裝最新的穩定版 inDox
pip install Indox
要安裝最新的開發版本，您可以執行
pip install git+https://github.com/osllmai/inDox@master
克隆儲存庫並導航到目錄：
git clone https://github.com/osllmai/inDox.git
cd inDox安裝所需的 Python 套件：
pip install -r requirements.txt如果您在本機 IDE 中執行此項目，請建立一個 Python 環境以確保所有相依性都正確管理。您可以依照下列步驟設定名為indox的虛擬環境：
python -m venv indox indox S cripts a ctivatepython3 -m venv indox
source indox/bin/activate啟動虛擬環境後，透過執行以下命令安裝所需的依賴項：
pip install -r requirements.txtpip install indox
pip install openai
pip install chromadb如果您在本機 IDE 中執行此項目，請建立一個 Python 環境以確保所有相依性都正確管理。您可以依照下列步驟設定名為indox的虛擬環境：
python -m venv indoxindox_judge S cripts a ctivatepython3 -m venv indox
2. **Activate the virtual environment:**
```bash
source indox/bin/activate
啟動虛擬環境後，透過執行以下命令安裝所需的依賴項：
pip install -r requirements.txt首先，您需要從環境中載入 API 金鑰。
import os
from dotenv import load_dotenv
load_dotenv ()
OPENAI_API_KEY = os . environ [ 'OPENAI_API_KEY' ]從 Indox 套件中導入必要的類別。
from indox import IndoxRetrievalAugmentation from indox . llms import OpenAi from indox . embeddings import OpenAiEmbedding建立 IndoxRetrievalAugmentation 的實例。
Indox = IndoxRetrievalAugmentation () openai_qa = OpenAiQA ( api_key = OPENAI_API_KEY , model = "gpt-3.5-turbo-0125" )
openai_embeddings = OpenAiEmbedding ( model = "text-embedding-3-small" , openai_api_key = OPENAI_API_KEY ) file_path = "sample.txt"在本節中，我們利用unstructured庫來載入文件並按標題將它們分割成區塊。此方法有助於將文件組織成可管理的部分以進行進一步處理。
from indox . data_loader_splitter import UnstructuredLoadAndSplit loader_splitter = UnstructuredLoadAndSplit ( file_path = file_path )
docs = loader_splitter . load_and_chunk () Starting processing...
End Chunking process.
將文檔區塊儲存在向量儲存中對於實現高效的檢索和搜尋操作至關重要。透過將文字資料轉換為向量表示並將其儲存在向量儲存中，您可以執行快速相似性搜尋和其他基於向量的操作。
from indox . vector_stores import ChromaVectorStore
db = ChromaVectorStore ( collection_name = "sample" , embedding = embed_openai )
Indox . connect_to_vectorstore ( db )
Indox . store_in_vectorstore ( docs ) 2024-05-14 15:33:04,916 - INFO - Anonymized telemetry enabled. See https://docs.trychroma.com/telemetry for more information.
2024-05-14 15:33:12,587 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-05-14 15:33:13,574 - INFO - Document added successfully to the vector store.
Connection established successfully.
query = "how cinderella reach her happy ending?" retriever = indox . QuestionAnswer ( vector_database = db , llm = openai_qa , top_k = 5 )
retriever . invoke ( query ) 2024-05-14 15:34:55,380 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2024-05-14 15:35:01,917 - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
'Cinderella reached her happy ending by enduring mistreatment from her step-family, finding solace and help from the hazel tree and the little white bird, attending the royal festival where the prince recognized her as the true bride, and ultimately fitting into the golden shoe that proved her identity. This led to her marrying the prince and living happily ever after.'
retriever . context ["from the hazel-bush. Cinderella thanked him, went to her mother'snngrave and planted the branch on it, and wept so much that the tearsnnfell down on it and watered it. And it grew and became a handsomenntree. Thrice a day cinderella went and sat beneath it, and wept andnnprayed, and a little white bird always came on the tree, and ifnncinderella expressed a wish, the bird threw down to her what shennhad wished for.nnIt happened, however, that the king gave orders for a festival",
'worked till she was weary she had no bed to go to, but had to sleepnnby the hearth in the cinders. And as on that account she alwaysnnlooked dusty and dirty, they called her cinderella.nnIt happened that the father was once going to the fair, and hennasked his two step-daughters what he should bring back for them.nnBeautiful dresses, said one, pearls and jewels, said the second.nnAnd you, cinderella, said he, what will you have. Father',
'face he recognized the beautiful maiden who had danced withnnhim and cried, that is the true bride. The step-mother andnnthe two sisters were horrified and became pale with rage, he,nnhowever, took cinderella on his horse and rode away with her. Asnnthey passed by the hazel-tree, the two white doves cried -nnturn and peep, turn and peep,nnno blood is in the shoe,nnthe shoe is not too small for her,nnthe true bride rides with you,nnand when they had cried that, the two came flying down and',
"to send her up to him, but the mother answered, oh, no, she isnnmuch too dirty, she cannot show herself. But he absolutelynninsisted on it, and cinderella had to be called. She firstnnwashed her hands and face clean, and then went and bowed downnnbefore the king's son, who gave her the golden shoe. Then shennseated herself on a stool, drew her foot out of the heavynnwooden shoe, and put it into the slipper, which fitted like annglove. And when she rose up and the king's son looked at her",
'slippers embroidered with silk and silver. She put on the dressnnwith all speed, and went to the wedding. Her step-sisters and thennstep-mother however did not know her, and thought she must be annforeign princess, for she looked so beautiful in the golden dress.nnThey never once thought of cinderella, and believed that she wasnnsitting at home in the dirt, picking lentils out of the ashes. Thennprince approached her, took her by the hand and danced with her.']
.----------------. .-----------------. .----------------. .----------------. .----------------.
| .--------------. || .--------------. || .--------------. || .--------------. || .--------------. |
| | _____ | || | ____ _____ | || | ________ | || | ____ | || | ____ ____ | |
| | |_ _| | || ||_ |_ _| | || | |_ ___ `. | || | .' `. | || | |_ _||_ _| | |
| | | | | || | | | | | || | | | `. | || | / .--. | || | / / | |
| | | | | || | | | | | | || | | | | | | || | | | | | | || | > `' < | |
| | _| |_ | || | _| |_ |_ | || | _| |___.' / | || | `--' / | || | _/ /'` _ | |
| | |_____| | || ||_____|____| | || | |________.' | || | `.____.' | || | |____||____| | |
| | | || | | || | | || | | || | | |
| '--------------' || '--------------' || '--------------' || '--------------' || '--------------' |
'----------------' '----------------' '----------------' '----------------' '----------------' 
