android semantic search kit
v0.0.4
Menyematkan Demo:
Referensi: Alur Kerja GitHub
git submodule update --init1. siapkan NDK
Ganti NDK dengan jalur dan versi NDK lokal Anda (contoh macOS lokal saya)
export ANDROID_HOME= $HOME /Library/Android/sdk
export ANDROID_NDK_HOME= $HOME /Library/Android/sdk/ndk/26.1.10909125
export TOOLCHAIN= $ANDROID_NDK_HOME /toolchains/llvm/prebuilt/darwin-x86_64
export TARGET=aarch64-linux-android
export API=34
export AR= $TOOLCHAIN /bin/llvm-ar
export CC= $TOOLCHAIN /bin/ $TARGET$API -clang
export AS= $CC
export CXX= $TOOLCHAIN /bin/ $TARGET$API -clang++
export LD= $TOOLCHAIN /bin/ld
export RANLIB= $TOOLCHAIN /bin/llvm-ranlib
export STRIP= $TOOLCHAIN /bin/llvm-strip
export PATH= $PATH : $ANDROID_HOME /cmdline-tools/latest/bin
export PATH= $PATH : $TOOLCHAIN /bin2. atur target Rust
rustup target add armv7-linux-androideabi i686-linux-android arm-linux-androideabi x86_64-linux-android aarch64-linux-android3.konfigurasi tautan
Setelah itu tambahkan like berikut pada $HOME/.cargo/config (buat file config jika belum ada):
Ikuti: https://doc.rust-lang.org/beta/rustc/platform-support.html
PS: ingatlah untuk mengubah jalur ke NDK Anda
[ target . i686-linux-android ]
linker = " /Users/phodal/Library/Android/sdk/ndk/26.1.10909125/toolchains/llvm/prebuilt/darwin-x86_64/bin/i686-linux-android34-clang "
[ target . arm-linux-android ]
linker = " /Users/phodal/Library/Android/sdk/ndk/26.1.10909125/toolchains/llvm/prebuilt/darwin-x86_64/bin/arm-linux-android34-clang "
[ target . x86_64-linux-android ]
linker = " /Users/phodal/Library/Android/sdk/ndk/26.1.10909125/toolchains/llvm/prebuilt/darwin-x86_64/bin/x86_64-linux-android34-clang "
[ target . aarch64-linux-android ]
linker = " /Users/phodal/Library/Android/sdk/ndk/26.1.10909125/toolchains/llvm/prebuilt/darwin-x86_64/bin/aarch64-linux-android34-clang "4.membangun FFI
./gradlew cargoBuild
untuk iOS (contoh, tidak berfungsi)
rustup target add aarch64-apple-ios
cargo build --target=x86_64-apple-ios --release
5.jalankan aplikasi
Alat: https://github.com/huggingface/optimum
1.penyiapan
pip install optimum[exporters,onnxruntime]2.konversi onnx
optimum-cli export onnx -m Helsinki-NLP/opus-mt-zh-en --optimize O2 optus-mt-zh-en-onnx3.mengukur Onnx
optimum-cli onnxruntime quantize
--avx512
--onnx_model bert-tiny-onnx
-o quantized_bert-tiny-onnxContoh: https://github.com/unit-mesh/onnx-convert-test
Model Pengujian Lainnya:
Model Pencarian Di Mana Saja IntelliJ IDEA:
Model Gumpalan:
Caused by: ai.djl.engine.EngineException: Failed to load Huggingface native library.
at ai.djl.huggingface.tokenizers.jni.LibUtils.<clinit>(LibUtils.java:43)
at ai.djl.huggingface.tokenizers.jni.LibUtils.checkStatus(LibUtils.java:50)
at ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.newInstance(HuggingFaceTokenizer.java:170)
at org.unitmesh.llmpoc.embedding.STSemantic$Companion.create(STSemantic.kt:78)
at org.unitmesh.llmpoc.MainActivity.onCreate(MainActivity.kt:33)
at android.app.Activity.performCreate(Activity.java:8143)
at android.app.Activity.performCreate(Activity.java:8114)
at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1310)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3513)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3700)
at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:85)
at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2136)
at android.os.Handler.dispatchMessage(Handler.java:106)
at android.os.Looper.loop(Looper.java:236)
at android.app.ActivityThread.main(ActivityThread.java:8060)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:656)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:967)
Caused by: java.lang.IllegalStateException: Cannot copy jni files
at ai.djl.huggingface.tokenizers.jni.LibUtils.copyJniLibraryFromClasspath(LibUtils.java:108)
at ai.djl.huggingface.tokenizers.jni.LibUtils.loadLibrary(LibUtils.java:66)
at ai.djl.huggingface.tokenizers.jni.LibUtils.<clinit>(LibUtils.java:41)
at ai.djl.huggingface.tokenizers.jni.LibUtils.checkStatus(LibUtils.java:50)
at ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.newInstance(HuggingFaceTokenizer.java:170)
at org.unitmesh.llmpoc.embedding.STSemantic$Companion.create(STSemantic.kt:78)
at org.unitmesh.llmpoc.MainActivity.onCreate(MainActivity.kt:33)
at android.app.Activity.performCreate(Activity.java:8143)
at android.app.Activity.performCreate(Activity.java:8114)
at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1310)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3513)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3700)
at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:85)
at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2136)
at android.os.Handler.dispatchMessage(Handler.java:106)
at android.os.Looper.loop(Looper.java:236)
at android.app.ActivityThread.main(ActivityThread.java:8060)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:656)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:967)
Edisi: 2170
Solusi: Plugin Rust Android Gradle
Kesalahan saat membangun proyek Rust untuk Android (Flutter): arm-linux-androideabi-ranlib tidak ditemukan untuk kompilasi OpenSSL
https://stackoverflow.com/questions/75943717/error-building-rust-project-for-android-flutter-arm-linux-androideabi-ranlib
Perlu memperbarui versi Onnx Runtime
thread 'tokio-runtime-worker' panik saat 'memanggil Result::unwrap() pada nilai Err : CreateSession(Msg("Gagal memuat model dengan kesalahan: /Users/runner/work/1/s/onnxruntime/core/ graph/model_load_utils.h:56 void onnxruntime:: model_load_utils::ValidateOpsetForDomain(const std::unordered_map<std::string, int> &, const logging::Logger &, bool, const std::string &, int) ONNX Runtime hanya menjamin dukungan untuk model yang dicap dengan versi opset onnx yang dirilis secara resmi dalam pengembangan dan dukungan untuk ini terbatas. Skema operator dan atau fungsi lainnya dapat berubah sebelum rilis ONNX berikutnya dan dalam hal ini ONNX Runtime tidak akan menjamin kompatibilitas mundur saat ini untuk domain com.ms.internal.nhwc hingga opset 18.n"))', /Volumes/source/ai/Inference/inference_core/src/embed.rs:49:18
Masalah lainnya: penggunaan v1.9.0 untuk memuat ONNX Opset 15 gagal
Fungsi pemuatan: System.LoadLibrary("tokenizers");
Tidak ditemukan implementasi untuk org.unitmesh.tokenizer.huggingface.tokenizers.jni.TokenizersLibrary.createTokenizerFromString(java.lang.String) yang panjang (mencoba Java_org_unitmesh_tokenizer_huggingface_tokenizers_jni_TokenizersLibrary_createTokenizerFromString dan Java_org_unitmesh_tokenizer_huggingface_tokenizers_jni_TokenizersLibrary_createTokenizerFromString__Ljava_lang_String_2) - apakah perpustakaan dimuat, misalnya System.loadLibrary?
