transformers ruby
1.0.0
- หม้อแปลงที่ทันสมัยสำหรับทับทิม
สำหรับการอนุมานอย่างรวดเร็วตรวจสอบผู้แจ้งข่าว
ก่อนอื่นให้ติดตั้ง Torch.rb
จากนั้นเพิ่มบรรทัดนี้ไปยัง Gemfile ของแอปพลิเคชันของคุณ:
gem "transformers-rb" การฝัง
การฝังแบบเบาบาง
การวิ่งกลับมาใหม่
เอกสาร
sentences = [ "This is an example sentence" , "Each sentence is converted" ]
model = Transformers . pipeline ( "embedding" , "sentence-transformers/all-MiniLM-L6-v2" )
embeddings = model . ( sentences )เอกสาร
query = "How many people live in London?"
docs = [ "Around 9 Million people live in London" , "London is known for its financial district" ]
model = Transformers . pipeline ( "embedding" , "sentence-transformers/multi-qa-MiniLM-L6-cos-v1" )
query_embedding = model . ( query )
doc_embeddings = model . ( docs )
scores = doc_embeddings . map { | e | e . zip ( query_embedding ) . sum { | d , q | d * q } }
doc_score_pairs = docs . zip ( scores ) . sort_by { | d , s | - s }เอกสาร
sentences = [ "This is an example sentence" , "Each sentence is converted" ]
model = Transformers . pipeline ( "embedding" , "sentence-transformers/all-mpnet-base-v2" )
embeddings = model . ( sentences )เอกสาร
sentences = [ "This is an example sentence" , "Each sentence is converted" ]
model = Transformers . pipeline ( "embedding" , "sentence-transformers/paraphrase-MiniLM-L6-v2" )
embeddings = model . ( sentences )เอกสาร
query_prefix = "Represent this sentence for searching relevant passages: "
input = [
"The dog is barking" ,
"The cat is purring" ,
query_prefix + "puppy"
]
model = Transformers . pipeline ( "embedding" , "mixedbread-ai/mxbai-embed-large-v1" )
embeddings = model . ( input )เอกสาร
sentences = [ "That is a happy person" , "That is a very happy person" ]
model = Transformers . pipeline ( "embedding" , "thenlper/gte-small" )
embeddings = model . ( sentences )เอกสาร
doc_prefix = "passage: "
query_prefix = "query: "
input = [
doc_prefix + "Ruby is a programming language created by Matz" ,
query_prefix + "Ruby creator"
]
model = Transformers . pipeline ( "embedding" , "intfloat/e5-base-v2" )
embeddings = model . ( input )เอกสาร
query_prefix = "Represent this sentence for searching relevant passages: "
input = [
"The dog is barking" ,
"The cat is purring" ,
query_prefix + "puppy"
]
model = Transformers . pipeline ( "embedding" , "BAAI/bge-base-en-v1.5" )
embeddings = model . ( input )เอกสาร
query_prefix = "Represent this sentence for searching relevant passages: "
input = [
"The dog is barking" ,
"The cat is purring" ,
query_prefix + "puppy"
]
model = Transformers . pipeline ( "embedding" , "Snowflake/snowflake-arctic-embed-m-v1.5" )
embeddings = model . ( input , pooling : "cls" )เอกสาร
docs = [ "The dog is barking" , "The cat is purring" , "The bear is growling" ]
model_id = "opensearch-project/opensearch-neural-sparse-encoding-v1"
model = Transformers :: AutoModelForMaskedLM . from_pretrained ( model_id )
tokenizer = Transformers :: AutoTokenizer . from_pretrained ( model_id )
special_token_ids = tokenizer . special_tokens_map . map { | _ , token | tokenizer . vocab [ token ] }
feature = tokenizer . ( docs , padding : true , truncation : true , return_tensors : "pt" , return_token_type_ids : false )
output = model . ( ** feature ) [ 0 ]
values , _ = Torch . max ( output * feature [ :attention_mask ] . unsqueeze ( - 1 ) , dim : 1 )
values = Torch . log ( 1 + Torch . relu ( values ) )
values [ 0 .. , special_token_ids ] = 0
embeddings = values . to_aเอกสาร
query = "How many people live in London?"
docs = [ "Around 9 Million people live in London" , "London is known for its financial district" ]
model = Transformers . pipeline ( "reranking" , "mixedbread-ai/mxbai-rerank-base-v1" )
result = model . ( query , docs )เอกสาร
query = "How many people live in London?"
docs = [ "Around 9 Million people live in London" , "London is known for its financial district" ]
model = Transformers . pipeline ( "reranking" , "BAAI/bge-reranker-base" )
result = model . ( query , docs ) การฝัง
embed = Transformers . pipeline ( "embedding" )
embed . ( "We are very happy to show you the ? Transformers library." )การวิ่งกลับมาใหม่
rerank = Informers . pipeline ( "reranking" )
rerank . ( "Who created Ruby?" , [ "Matz created Ruby" , "Another doc" ] )การรับรู้ที่มีชื่อ
ner = Transformers . pipeline ( "ner" )
ner . ( "Ruby is a programming language created by Matz" )การวิเคราะห์ความเชื่อมั่น
classifier = Transformers . pipeline ( "sentiment-analysis" )
classifier . ( "We are very happy to show you the ? Transformers library." )ตอบคำถาม
qa = Transformers . pipeline ( "question-answering" )
qa . ( question : "Who invented Ruby?" , context : "Ruby is a programming language created by Matz" )การสกัดคุณลักษณะ
extractor = Transformers . pipeline ( "feature-extraction" )
extractor . ( "We are very happy to show you the ? Transformers library." )การจำแนกรูปภาพ
classifier = Transformers . pipeline ( "image-classification" )
classifier . ( "image.jpg" )การแยกคุณสมบัติของภาพ
extractor = Transformers . pipeline ( "image-feature-extraction" )
extractor . ( "image.jpg" ) ห้องสมุดนี้เป็นไปตาม Transformers Python API ปัจจุบันมีการรองรับสถาปัตยกรรมโมเดลต่อไปนี้:
ดูการเปลี่ยนแปลง
ทุกคนได้รับการสนับสนุนเพื่อช่วยปรับปรุงโครงการนี้ นี่คือสองสามวิธีที่คุณสามารถช่วยได้:
เพื่อเริ่มต้นกับการพัฒนา:
git clone https://github.com/ankane/transformers-ruby.git
cd transformers-ruby
bundle install
bundle exec rake download:files
bundle exec rake test
