首页>编程相关>其他源码
下载

语义搜索-rs

Rust 中的 gRPC 服务器语义搜索演示

目标

  • 具有成本效益的十亿级矢量搜索，单位延迟（< 10 ms）
  • 无状态向量搜索器
  • 软硬件层面优化，有效利用资源

非目标

  • 分布式和分片索引构建和搜索
  • 利用任何矢量数据库或引擎产品
  • 实现各种索引算法

待办事项

  • 从该项目中删除src/hnsw_index ( hnswlib-rs ) 并将其从 crate 中导入。
    • PR 1：使用std::simd距离实现（ packed_simd_2 ）
    • PR 2：更多并行化
    • PR 3：格式化和重构
  • 修改 hnswlib-rs
    • 解决构建问题
    • 使用 SIMD 重新实现距离函数（更有效，支持更多类型，速度提高 2 倍）
    • 支持 SIMD 量化和 i8 向量搜索
    • 更多并行化
  • 将嵌入和搜索部分作为不同的微服务分开

建筑学

特征

  • gRPC服务器
  • 动态批量推理（模型和搜索）
  • 在 GPU 上实时推理语言模型。
    • 型号信息：hf - Mini-LM L12 v2
  • 矢量量化（后量化）
    • 将 f32 重新调整为 i8
  • ANN（修改后的 hnswlib-rs）
    • 多线程插入和搜索
    • SIMD加速距离计算（重新实现）
    • 富裕新南威尔士州指数

数据

  • 输入：查询。字符串列表。
  • 输出：前 k 个索引。 int32 的二维列表。

要求

  • Intel（现代）CPU（支持 AVX2、FMA 指令）
    • 某些距离测量不支持 M1 Silicon 或 AMD CPU。
  • 铁锈（每晚）
  • libtorch 2.0
  • 原始缓冲区

跑步

建立索引

从给定文档中提取嵌入并构建索引并将其保存到本地磁盘。 

cargo +nightly run --release --features progress --bin embedding

如果您想要量化，请将quantize传递给参数。 

cargo +nightly run --release --features progress --bin embedding quantize

gRPC服务器

构建并运行 gRPC 服务器（用于模型和搜索推理）。 

make server

gRPC 客户端

构建并运行 gRPC 客户端。客户端将根据给定的参数开始对服务器进行基准测试。 

make client

您还可以更改参数。例如./client num_users num_requests bs k 

cargo +nightly run --release --bin client 1 1000 128 10

例子

使用给定的查询运行示例。 （必须有ag_news数据集的内置索引） 

cargo +nightly run --release --features progress --bin main " query "

基准测试

  • GPU：GTX 1060 6G（CUDA 11.8、CuDNN 8.8.0）
  • CPU：i7-7700K（4核8线程，未超频）
  • 信息
    • 使用AVX2 、 FMA标志编译
    • 索引 : HNSW
    • 嵌入尺寸：384
    • 嵌入数据类型：f32、i8
    • 距离测量：L2、余弦
    • k : 10
    • 文档数量：127.6K 个文档
    • 热身11次

％延迟与 Rust 版本有点不同。在最近的版本（1.73.0 nightly）中，速度变得更慢（2.275 ms -> 3.000 ms）。

% i8 以 1.73.0 为基准。

p距离废话要求k类型意思是p95 p99 p99.9最大限度QPS
F32 L2 1 10k 10全部的4.833 毫秒5.426 毫秒6.037 毫秒9.578 毫秒12.411 毫秒207
模型4.507 毫秒4.994 毫秒5.505 毫秒8.979 毫秒12.094 毫秒
搜索0.203毫秒0.308毫秒0.354 毫秒0.449 毫秒0.567 毫秒4926
32 1k 10全部的10.403 毫秒10.951 毫秒11.625 毫秒16.794 毫秒16.794 毫秒3076
模型9.211 毫秒9.684 毫秒10.226 毫秒14.796 毫秒14.796 毫秒
搜索0.981 毫秒1.183 毫秒1.386 毫秒2.386 毫秒2.386 毫秒32620
128 1k 10全部的32.220 毫秒32.602 毫秒32.862 毫秒33.440 毫秒33.440 毫秒3973
模型29.120 毫秒29.411 毫秒29.653 毫秒30.333 毫秒30.333 毫秒
搜索2.855 毫秒3.007 毫秒3.154 毫秒3.452 毫秒3.452 毫秒44834
科斯128 1k 10全部的31.547 毫秒31.890 毫秒32.087 毫秒33.479 毫秒33.479 毫秒4128
模型28.992 毫秒29.229 毫秒29.438 毫秒30.936 毫秒30.936 毫秒
搜索2.275 毫秒2.444 毫秒2.595 毫秒2.722 毫秒2.722 毫秒56264
i8科斯128 1k 10全部的30.896 毫秒31.271 毫秒31.483 毫秒31.786 毫秒31.786 毫秒4142
模型28.816 毫秒29.123 毫秒29.286 毫秒29.598 毫秒29.598 毫秒4442
搜索1.764 毫秒1.934 毫秒2.006 毫秒2.308 毫秒2.308 毫秒72560

搜索延迟

  • 经过大批量测试（超过 1024）
  • 在两种类型的CPU上进行了测试。
    • i7-7700K，4核8线程（未超频）
    • Macbook Pro 16 英寸 2019，i9-9880H 8 核 16 线程（基本时钟 2.3 GHz）
  • 量化向量比 f32 向量快约 2 ~ 40% ，并节省约4 倍内存
  • int8 余弦距离有点奇怪，但我只想按数据类型比较 SIMD 性能。
p距离废话要求k意思是p50 p95 p99 p99.9最大限度QPS
F32因斯1024 2k 10 10.969 毫秒10.856 毫秒12.078 毫秒12.384 毫秒14.817 毫秒18.024 毫秒93354
2048 2k 10 21.829 毫秒21.617 毫秒23.952 毫秒24.514 毫秒25.892 毫秒26.171 毫秒93820
4096 2k 10 43.538 毫秒43.199 毫秒46.303 毫秒48.374 毫秒50.194 毫秒57.159 毫秒94079
8192 2k 10 86.234 毫秒85.684 毫秒90.645 毫秒93.850 毫秒101.188 毫秒102.607 毫秒94997
i8因斯1024 2k 10 9.102 毫秒9.022 毫秒10.043 毫秒10.222 毫秒11.297 毫秒11.602 毫秒112506
2048 2k 10 18.084 毫秒17.867 毫秒19.799 毫秒20.548 毫秒22.749 毫秒29.330 毫秒113249
4096 2k 10 35.772 毫秒35.488 毫秒38.246 毫秒40.089 毫秒41.745 毫秒43.144 毫秒114503
8192 2k 10 71.447 毫秒70.790 毫秒76.420 毫秒78.610 毫秒94.783 毫秒122.490 毫秒114659
p距离废话要求k意思是p50 p95 p99 p99.9最大限度QPS
F32因斯1024 2k 10 9.459 毫秒9.033 毫秒11.412 毫秒11.842 毫秒12.753 毫秒12.819 毫秒108252
2048 2k 10 19.620 毫秒19.708 毫秒21.072 毫秒22.023 毫秒24.704 毫秒24.881 毫秒104383
4096 2k 10 38.269 毫秒38.237 毫秒40.505 毫秒42.475 毫秒45.439 毫秒45.691 毫秒107030
8192 2k 10 75.385 毫秒74.877 毫秒80.448 毫秒84.589 毫秒89.629 毫秒89.887 毫秒108668
i8因斯1024 2k 10 7.003 毫秒6.700 毫秒8.270 毫秒8.668 毫秒9.655 毫秒10.011 毫秒146223
2048 2k 10 14.388 毫秒14.419 毫秒15.637 毫秒16.067 毫秒19.237 毫秒20.231 毫秒142341
4096 2k 10 27.894 毫秒27.815 毫秒29.818 毫秒31.213 毫秒39.733 毫秒40.387 毫秒146842
8192 2k 10 54.674 毫秒54.348 毫秒57.844 毫秒61.486 毫秒100.327 毫秒135.911 毫秒149834

示例

  • 数据集：ag_news
  • 查询： Asia shares drift lower as investors factor in Fed rate hike.

余弦距离

 top 1 | id : 92048, dist : 0.3335874
Asian Shares Lower as Oil Fall Stalls  SINGAPORE (Reuters) - Asian share markets saw light selling  in line with a slowing retreat in oil prices on Tuesday, while  the dollar steadied as officials in Europe and Japan fretted  about the U.S. currency's recent decline.
top 2 | id : 65687, dist : 0.38142937
Asian Shares Lower on Earnings, Steel  SINGAPORE (Reuters) - Asian share markets staged a  broad-based retreat on Wednesday, led by steelmakers amid  warnings of price declines, but also enveloping technology and  financial stocks on worries that earnings may disappoint.
top 3 | id : 15361, dist : 0.39869058
Asian Stocks Mostly Lower on Tech Worries  SINGAPORE (Reuters) - Asian shares were mostly lower on  Tuesday, pulled down by technology stocks following worries  about a sales forecast from bellwether Intel Corp. and  unexpectedly flat industrial output in Japan.
top 4 | id : 63234, dist : 0.40184337
Asian Stocks Slip as Oil Moves Above $55 Asian share markets fell on Monday after oil hit a record above $55 a barrel, with investors cautious ahead of earnings reports from bellwether firms IBM and Texas Instruments later in the day.
top 5 | id : 76590, dist : 0.4137212
Asian Stocks Ease After China Rate Rise  SINGAPORE (Reuters) - China's surprise interest rate rise  weighed down Asian stocks on Friday as investors sold shares of  miners, shippers and other firms whose fortunes have been  closely linked to the country's rapid growth.
top 6 | id : 65572, dist : 0.4192986
Asian Shares Lower on Earnings  SINGAPORE (Reuters) - Asian share markets slumped on  Wednesday, led by South Korea's POSCO Co. Ltd. and other  steelmakers amid warnings of a drop in prices, and as investors  worried that third-quarter earnings may disappoint.
top 7 | id : 14210, dist : 0.41932273
Asian Stocks Mostly Flat to Lower  SINGAPORE (Reuters) - Asian shares were mostly flat to  lower on Monday as investors kept a wary eye on stabilizing oil  prices and looked past signs Japanese consumers were beginning  to spend more freely.
top 8 | id : 14904, dist : 0.4203742
European Shares Drift Down in Thin Trade European stocks were slightly lower by mid-day in holiday-hit trading on Monday, with crude oil rises weighing on shares, though Sanofi-Aventis gained on the result of trials of an anti-obesity drug.
top 9 | id : 76632, dist : 0.4244548
Asian Stocks Ease After China Rate Rise (Reuters) Reuters - China's surprise interest rate riseweighed down Asian stocks on Friday as investors sold shares ofminers, shippers and other firms whose fortunes have beenclosely linked to the country's rapid growth.
top 10 | id : 15265, dist : 0.42584515
Asian Stocks Mostly Lower on Tech Worries (Reuters) Reuters - Asian shares were mostly lower onTuesday, pulled down by technology stocks following worriesabout a sales forecast from bellwether Intel Corp. andunexpectedly flat industrial output in Japan.

L2距离

 top 1 | id : 92048, dist : 0.667175
Asian Shares Lower as Oil Fall Stalls  SINGAPORE (Reuters) - Asian share markets saw light selling  in line with a slowing retreat in oil prices on Tuesday, while  the dollar steadied as officials in Europe and Japan fretted  about the U.S. currency's recent decline.
top 2 | id : 65687, dist : 0.76285875
Asian Shares Lower on Earnings, Steel  SINGAPORE (Reuters) - Asian share markets staged a  broad-based retreat on Wednesday, led by steelmakers amid  warnings of price declines, but also enveloping technology and  financial stocks on worries that earnings may disappoint.
top 3 | id : 15361, dist : 0.7973814
Asian Stocks Mostly Lower on Tech Worries  SINGAPORE (Reuters) - Asian shares were mostly lower on  Tuesday, pulled down by technology stocks following worries  about a sales forecast from bellwether Intel Corp. and  unexpectedly flat industrial output in Japan.
top 4 | id : 63234, dist : 0.80368686
Asian Stocks Slip as Oil Moves Above $55 Asian share markets fell on Monday after oil hit a record above $55 a barrel, with investors cautious ahead of earnings reports from bellwether firms IBM and Texas Instruments later in the day.
top 5 | id : 76590, dist : 0.82744247
Asian Stocks Ease After China Rate Rise  SINGAPORE (Reuters) - China's surprise interest rate rise  weighed down Asian stocks on Friday as investors sold shares of  miners, shippers and other firms whose fortunes have been  closely linked to the country's rapid growth.
top 6 | id : 65572, dist : 0.83859724
Asian Shares Lower on Earnings  SINGAPORE (Reuters) - Asian share markets slumped on  Wednesday, led by South Korea's POSCO Co. Ltd. and other  steelmakers amid warnings of a drop in prices, and as investors  worried that third-quarter earnings may disappoint.
top 7 | id : 14210, dist : 0.83864534
Asian Stocks Mostly Flat to Lower  SINGAPORE (Reuters) - Asian shares were mostly flat to  lower on Monday as investors kept a wary eye on stabilizing oil  prices and looked past signs Japanese consumers were beginning  to spend more freely.
top 8 | id : 14904, dist : 0.8407483
European Shares Drift Down in Thin Trade European stocks were slightly lower by mid-day in holiday-hit trading on Monday, with crude oil rises weighing on shares, though Sanofi-Aventis gained on the result of trials of an anti-obesity drug.
top 9 | id : 77269, dist : 0.84736
Asian Markets Mixed on China Rate Hike (AP) AP - Asian financial markets showed a mixed reaction Friday to China's first interest rate rise in nine years, which analysts welcomed as a shift toward capitalist-style economic tools and away from central planning.
top 10 | id : 76632, dist : 0.8489096
Asian Stocks Ease After China Rate Rise (Reuters) Reuters - China's surprise interest rate riseweighed down Asian stocks on Friday as investors sold shares ofminers, shippers and other firms whose fortunes have beenclosely linked to the country's rapid growth.

带 i8 的余弦距离（未缩放） 

 top 1 | id : 70945, dist : 8807
Asian Shares Fall on Dollar; Gold, Oil Up (Reuters) Reuters - The U.S. dollar stumbled on Monday,setting multi-month lows against the yen and euro and tradingaround four-year lows against the Korean won and Singaporedollar, prompting investors to clip Asian share markets.
top 2 | id : 33140, dist : 10982
US Stock-Index Futures Decline; Citigroup, GE Slip in Europe US stock-index futures declined. Dow Jones Industrial Average shares including General Electric Co. slipped in Europe. Citigroup Inc.
top 3 | id : 33077, dist : 11208
US Stock-Index Futures Are Little Changed; Citigroup, GE Slip US stock-index futures were little changed. Dow Jones Industrial Average shares including General Electric Co. slipped in Europe. Citigroup Inc.
top 4 | id : 95897, dist : 11290
Red hot Google shares cooling  Google Inc. may be subject to the law of gravity, after all. Its shares, which more than doubled in the two months after the company's Aug. 19 initial public offering and traded as high as $201.60 on Nov. 3, have slipped about 17 percent over the past two weeks. They closed at $167.54 yesterday, down 2.88 percent for the day, ...
top 5 | id : 40354, dist : 11390
Tokyo stocks open slightly lower TOKYO - Stocks opened slightly lower Monday on the Tokyo Stock Exchange as declines in US technology shares last Friday prompted selling.
top 6 | id : 64025, dist : 12241
E*Trade Profit Rises as Expenses Drop (Reuters) Reuters - E*Trade Financial Corp. , anonline bank and brokerage, on Monday said its third-quarterprofit rose as lower expenses offset a drop in net revenue.
top 7 | id : 58035, dist : 12458
UK inflation rate fall continues The UK's inflation rate fell in September, thanks in part to a fall in the price of airfares, increasing the chance that interest rates will be kept on hold.
top 8 | id : 65573, dist : 12583
Yen Eases Against Dollar  TOKYO (Reuters) - The yen edged down against the dollar on  Wednesday as Japanese stock prices slid, but it remained within  striking distance of three-month highs on persistent concerns  about the health of the U.S. economy.
top 9 | id : 50470, dist : 12828
Tower Auto Sees Wider Loss, Shares Fall  CHICAGO (Reuters) - Auto parts maker Tower Automotive Inc.  &lt;A HREF="http://www.investor.reuters.com/FullQuote.aspx?ticker=TWR.N target=/stocks/quickinfo/fullquote"&gt;TWR.N&lt;/A&gt; said on Tuesday its third-quarter loss would be twice  as deep as previously expected because of lower vehicle  production in North America and higher steel costs.
top 10 | id : 1267, dist : 13020
Study: Global Warming Could Change Calif. (AP) AP - Global warming could cause dramatically hotter summers and a depleted snow pack in California, leading to a sharp increase in heat-related deaths and jeopardizing the water supply, according to a study released Monday.

参考

  • https://github.com/hora-search/hora
  • https://github.com/jean-pierreBoth/hnswlib-rs

维护者

@kozistr

展开
附加信息
相关应用
为您推荐
相关资讯 全部