NeumAI
neumai 0.0.40
主页 |文档 |博客 |不和谐|叽叽喳喳
Neum AI 是一个数据平台，可帮助开发人员利用数据通过检索增强生成 (RAG) 将大型语言模型置于上下文中，这包括从文档存储和 NoSQL 等现有数据源中提取数据，将内容处理为向量嵌入，并将向量嵌入引入到用于相似性搜索的矢量数据库。
它为您提供了全面的 RAG 解决方案，可以根据您的应用程序进行扩展，并减少集成数据连接器、嵌入模型和矢量数据库等服务所花费的时间。
您可以通过电子邮件 ([email protected])、discord 或安排与我们通话联系我们的团队。
立即在dashboard.neum.ai 上注册。请参阅我们的快速入门以开始使用。
Neum AI Cloud 支持大规模分布式架构，通过矢量嵌入运行数百万个文档。有关完整的功能集，请参阅：云与本地
安装neumai包：
pip install neumai要创建您的第一个数据管道，请访问我们的快速入门。
在较高级别上，管道由一个或多个用于从中提取数据的源、一个用于对内容进行矢量化的嵌入连接器以及一个用于存储所述矢量的接收器连接器组成。通过这段代码，我们将制作所有这些并运行管道：
from neumai . DataConnectors . WebsiteConnector import WebsiteConnector
from neumai . Shared . Selector import Selector
from neumai . Loaders . HTMLLoader import HTMLLoader
from neumai . Chunkers . RecursiveChunker import RecursiveChunker
from neumai . Sources . SourceConnector import SourceConnector
from neumai . EmbedConnectors import OpenAIEmbed
from neumai . SinkConnectors import WeaviateSink
from neumai . Pipelines import Pipeline
website_connector = WebsiteConnector (
url = "https://www.neum.ai/post/retrieval-augmented-generation-at-scale" ,
selector = Selector (
to_metadata = [ 'url' ]
)
)
source = SourceConnector (
data_connector = website_connector ,
loader = HTMLLoader (),
chunker = RecursiveChunker ()
)
openai_embed = OpenAIEmbed (
api_key = "<OPEN AI KEY>" ,
)
weaviate_sink = WeaviateSink (
url = "your-weaviate-url" ,
api_key = "your-api-key" ,
class_name = "your-class-name" ,
)
pipeline = Pipeline (
sources = [ source ],
embed = openai_embed ,
sink = weaviate_sink
)
pipeline . run ()
results = pipeline . search (
query = "What are the challenges with scaling RAG?" ,
number_of_results = 3
)
for result in results :
print ( result . metadata ) from neumai . DataConnectors . PostgresConnector import PostgresConnector
from neumai . Shared . Selector import Selector
from neumai . Loaders . JSONLoader import JSONLoader
from neumai . Chunkers . RecursiveChunker import RecursiveChunker
from neumai . Sources . SourceConnector import SourceConnector
from neumai . EmbedConnectors import OpenAIEmbed
from neumai . SinkConnectors import WeaviateSink
from neumai . Pipelines import Pipeline
website_connector = PostgresConnector (
connection_string = 'postgres' ,
query = 'Select * from ...'
)
source = SourceConnector (
data_connector = website_connector ,
loader = JSONLoader (
id_key = '<your id key of your jsons>' ,
selector = Selector (
to_embed = [ 'property1_to_embed' , 'property2_to_embed' ],
to_metadata = [ 'property3_to_include_in_metadata_in_vector' ]
)
),
chunker = RecursiveChunker ()
)
openai_embed = OpenAIEmbed (
api_key = "<OPEN AI KEY>" ,
)
weaviate_sink = WeaviateSink (
url = "your-weaviate-url" ,
api_key = "your-api-key" ,
class_name = "your-class-name" ,
)
pipeline = Pipeline (
sources = [ source ],
embed = openai_embed ,
sink = weaviate_sink
)
pipeline . run ()
results = pipeline . search (
query = "..." ,
number_of_results = 3
)
for result in results :
print ( result . metadata ) from neumai . Client . NeumClient import NeumClient
client = NeumClient (
api_key = '<your neum api key, get it from https://dashboard.neum.ai' ,
)
client . create_pipeline ( pipeline = pipeline )如果您有兴趣将 Neum AI 部署到您自己的云中，请通过[email protected] 联系我们。
我们在 GitHub 上发布了一个示例后端架构，您可以将其用作起点。
如需最新列表，请访问我们的文档
我们的路线图随着询问而不断发展，因此如果有任何遗漏，请随时提出问题或向我们发送消息。
连接器
搜索
可扩展性
实验性的
Neum AI 的其他工具可以在这里找到：
