llano
1.0.0
Biarkan Model Berbahasa Besar Berfungsi Sebagai Tato Data Anno .
Ekstraktor informasi zero-shot/few-shot.
stabil
python -m pip install -U llanoUntuk pengguna Cina, url indeks dapat ditentukan untuk instalasi yang lebih cepat.
python -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple -U llanoterbaru
python -m pip install git+https://github.com/SeanLee97/llano.git Saat ini, mendukung Python3.8+ . Karena masa pakai Python 3.7 berakhir pada tanggal 27 Juni 2023, kami tidak lagi mendukungnya.
Tugas Pendukung:
| Nama Tugas | Bahasa Pendukung | Status |
|---|---|---|
| Ner | Inggris (EN), Mandarin Sederhana (ZH_CN) | ? |
| Klasifikasi Teks (Biner, MultiKelas) | Inggris (EN), Mandarin Sederhana (ZH_CN) | ? |
| Klasifikasi MultiLabel | Inggris (EN), Mandarin Sederhana (ZH_CN) | ? |
| Augmentasi Data | Inggris (EN), Mandarin Sederhana (ZH_CN) | ? |
| Ekstraksi Relasi | Inggris (EN), Mandarin Sederhana (ZH_CN) | ? |
| Peringkasan | ||
| Teks ke SQL |
from llano . config import Tasks , Languages , OpenAIModels , NERFormatter
from llano import GPTModel , GPTAnnotator
print ( 'All Supported Tasks:' , Tasks . list_attributes ())
print ( 'All Supported Languages:' , Languages . list_attributes ())
print ( 'All Supported NERFormatter:' , NERFormatter . list_attributes ())
print ( 'All Supported OpenAIModels:' , OpenAIModels . list_attributes ())
api_key = 'Your API Key'
model = GPTModel ( api_key , model = OpenAIModels . ChatGPT )
annotator = GPTAnnotator ( model ,
task = Tasks . NER ,
language = Languages . EN ,
label_mapping = {
"people" : 'PEO' ,
'location' : 'LOC' ,
'company' : 'COM' ,
'organization' : 'ORG' ,
'job' : 'JOB' })
doc = '''Elon Reeve Musk FRS (/ˈiːlɒn/ EE-lon; born June 28, 1971) is a business magnate and investor. He is the founder, CEO and chief engineer of SpaceX; angel investor, CEO and product architect of Tesla, Inc.; owner and CEO of Twitter, Inc.; founder of The Boring Company; co-founder of Neuralink and OpenAI; and president of the philanthropic Musk Foundation. '''
# w/o hint, w/o formatted result
ret = annotator ( doc )
# w/o hint, w/ formatted result
ret = annotator ( doc , formatter = NERFormatter . BIO )
# w/ hint, w/ formatted result
ret = annotator ( doc , hint = 'the entity type `job` is job title such as CEO, founder, boss.' , formatter = NERFormatter . BIO ) result adalah hasil anotasi. formatted_result adalah hasil yang diformat.
Tip: jika Anda ingin melatih model domain Anda, Anda dapat menggunakan hasil yang diformat.
{
"request" : {
"prompt" : " You are a NER (Named-entity recognition) system, please help me with the NER task. n Task: extract the entities and corresponding entity types from a given sentence. n Only support 5 entity types, including: people, location, company, organization, job. nn Explanation and examples: the entity type `job` is job title such as CEO, founder, boss. nn Output format: (entity, entity_type). nn Following is the given sentence: Elon Reeve Musk FRS (/ˈiːlɒn/ EE-lon; born June 28, 1971) is a business magnate and investor. He is the founder, CEO and chief engineer of SpaceX; angel investor, CEO and product architect of Tesla, Inc.; owner and CEO of Twitter, Inc.; founder of The Boring Company; co-founder of Neuralink and OpenAI; and president of the philanthropic Musk Foundation. n Output: "
},
"meta" : {
"role" : " assistant " ,
"prompt_tokens" : 195 ,
"completion_tokens" : 74 ,
"total_tokens" : 269 ,
"taken_time" : 4.87583
},
"response" : " nn ( " Elon Reeve Musk " , " people " ), ( " FRS " , " job " ), ( " SpaceX " , " company " ), ( " Tesla, Inc. " , " company " ), ( " Twitter, Inc. " , " company " ), ( " The Boring Company " , " organization " ), ( " Neuralink " , " organization " ), ( " OpenAI " , " organization " ), ( " Musk Foundation " , " organization " ) " ,
"result" : {
"text" : " Elon Reeve Musk FRS (/ˈiːlɒn/ EE-lon; born June 28, 1971) is a business magnate and investor. He is the founder, CEO and chief engineer of SpaceX; angel investor, CEO and product architect of Tesla, Inc.; owner and CEO of Twitter, Inc.; founder of The Boring Company; co-founder of Neuralink and OpenAI; and president of the philanthropic Musk Foundation. " ,
"entities" : [
[
0 ,
15 ,
" Elon Reeve Musk " ,
" PEO "
],
[
16 ,
19 ,
" FRS " ,
" JOB "
],
[
139 ,
145 ,
" SpaceX " ,
" COM "
],
[
192 ,
203 ,
" Tesla, Inc. " ,
" COM "
],
[
222 ,
235 ,
" Twitter, Inc. " ,
" COM "
],
[
248 ,
266 ,
" The Boring Company " ,
" ORG "
],
[
282 ,
291 ,
" Neuralink " ,
" ORG "
],
[
296 ,
302 ,
" OpenAI " ,
" ORG "
],
[
339 ,
354 ,
" Musk Foundation " ,
" ORG "
]
],
"formatted_result": "EtB-PEOnltI-PEOnotI-PEOnntI-PEOn tI-PEOnRtI-PEOnetI-PEOnetI-PEOnvtI-PEOnetI-PEOn tI-PEOnMtI-PEOnutI-PEOnstI-PEOnktI-PEOn tOnFtB-JOBnRtI-JOBnStI-JOBn tOn(tOn/tOnˈtOnitOnːtOnltOnɒtOnntOn/tOn tOnEtOnEtOn-tOnltOnotOnntOn;tOn tOnbtOnotOnrtOnntOn tOnJtOnutOnntOnetOn tOn2tOn8tOn,tOn tOn1tOn9tOn7tOn1tOn)tOn tOnitOnstOn tOnatOn tOnbtOnutOnstOnitOnntOnetOnstOnstOn tOnmtOnatOngtOnntOnatOnttOnetOn tOnatOnntOndtOn tOnitOnntOnvtOnetOnstOnttOnotOnrtOn.tOn tOnHtOnetOn tOnitOnstOn tOnttOnhtOnetOn tOnftOnotOnutOnntOndtOnetOnrtOn,tOn tOnCtOnEtOnOtOn tOnatOnntOndtOn tOnctOnhtOnitOnetOnftOn tOnetOnntOngtOnitOnntOnetOnetOnrtOn tOnotOnftOn tOnStB-COMnptI-COMnatI-COMnctI-COMnetI-COMnXtI-COMn;tOn tOnatOnntOngtOnetOnltOn tOnitOnntOnvtOnetOnstOnttOnotOnrtOn,tOn tOnCtOnEtOnOtOn tOnatOnntOndtOn tOnptOnrtOnotOndtOnutOnctOnttOn tOnatOnrtOnctOnhtOnitOnttOnetOnctOnttOn tOnotOnftOn tOnTtB-COMnetI-COMnstI-COMnltI-COMnatI-COMn,tI-COMn tI-COMnItI-COMnntI-COMnctI-COMn.tI-COMn;tOn tOnotOnwtOnntOnetOnrtOn tOnatOnntOndtOn tOnCtOnEtOnOtOn tOnotOnftOn tOnTtB-COMnwtI-COMnitI-COMnttI-COMnttI-COMnetI-COMnrtI-COMn,tI-COMn tI-COMnItI-COMnntI-COMnctI-COMn.tI-COMn;tOn tOnftOnotOnutOnntOndtOnetOnrtOn tOnotOnftOn tOnTtB-ORGnhtI-ORGnetI-ORGn tI-ORGnBtI-ORGnotI-ORGnrtI-ORGnitI-ORGnntI-ORGngtI-ORGn tI-ORGnCtI-ORGnotI-ORGnmtI-ORGnptI-ORGnatI-ORGnntI-ORGnytI-ORGn;tOn tOnctOnotOn-tOnftOnotOnutOnntOndtOnetOnrtOn tOnotOnftOn tOnNtB-ORGnetI-ORGnutI-ORGnrtI-ORGnatI-ORGnltI-ORGnitI-ORGnntI-ORGnktI-ORGn tOnatOnntOndtOn tOnOtB-ORGnptI-ORGnetI-ORGnntI-ORGnAtI-ORGnItI-ORGn;tOn tOnatOnntOndtOn tOnptOnrtOnetOnstOnitOndtOnetOnntOnttOn tOnotOnftOn tOnttOnhtOnetOn tOnptOnhtOnitOnltOnatOnntOnttOnhtOnrtOnotOnptOnitOnctOn tOnMtB-ORGnutI-ORGnstI-ORGnktI-ORGn tI-ORGnFtI-ORGnotI-ORGnutI-ORGnntI-ORGndtI-ORGnatI-ORGnttI-ORGnitI-ORGnotI-ORGnntI-ORGn.tOn tO"
},
} from llano . config import Tasks , Languages , OpenAIModels , NERFormatter
from llano import GPTModel , GPTAnnotator
print ( 'All Supported Tasks:' , Tasks . list_attributes ())
print ( 'All Supported Languages:' , Languages . list_attributes ())
print ( 'All Supported NERFormatter:' , NERFormatter . list_attributes ())
print ( 'All Supported OpenAIModels:' , OpenAIModels . list_attributes ())
api_keys = [ 'Your API Keys' ]
model = GPTModel ( api_keys , model = OpenAIModels . ChatGPT )
annotator = GPTAnnotator ( model ,
task = Tasks . NER ,
language = Languages . ZH_CN ,
label_mapping = {
'人名' : 'PEO' ,
'地名' : 'LOC' ,
'公司名' : 'COM' ,
'机构名' : 'ORG' ,
'身份' : 'ID' })
doc = '''埃隆·里夫·马斯克（Elon Reeve Musk） [107] ，1971年6月28日出生于南非的行政首都比勒陀利亚，企业家、工程师、慈善家、美国国家工程院院士。他同时兼具南非、加拿大和美国三重国籍。埃隆·马斯克本科毕业于宾夕法尼亚大学，获经济学和物理学双学位。1995年至2002年，马斯克与合伙人先后办了三家公司，分别是在线内容出版软件“Zip2”、电子支付“X.com”和“PayPal”。'''
ret = annotator ( doc ) # w/o hint, w/o formatter
ret = annotator ( doc , formatter = NERFormatter . BIO ) # w/o hint, w/ formatter
ret = annotator ( doc , hint = '身份表示从事职位的头衔或社会地位等，如：老板，董事长，作家，理事长等' , formatter = NERFormatter . BIO ) # w/o hint, w/ formatter{
"request" : {
"prompt" : "你是一个 NER 系统，请帮我完成中文 NER 任务。 n任务要求如下：找到句子中的实体，并返回实体及实体类型。 n支持的实体类型仅限5类：人名、地名、公司名、机构名、身份。 nn解释及示例：身份表示从事职位的头衔或社会地位等，如：老板，董事长，作家，理事长等nn输出格式要求：(实体, 实体类型)。 nn以下是输入句子：埃隆·里夫·马斯克（Elon Reeve Musk） [107] ，1971年6月28日出生于南非的行政首都比勒陀利亚，企业家、工程师、慈善家、美国国家工程院院士。他同时兼具南非、加拿大和美国三重国籍。埃隆·马斯克本科毕业于宾夕法尼亚大学，获经济学和物理学双学位。1995年至2002年，马斯克与合伙人先后办了三家公司，分别是在线内容出版软件“Zip2”、电子支付“X.com”和“PayPal”。 n输出： "
},
"meta" : {
"role" : " assistant " ,
"prompt_tokens" : 346 ,
"completion_tokens" : 103 ,
"total_tokens" : 449 ,
"taken_time" : 4.54531
},
"response" : " ('埃隆·里夫·马斯克', '人名'), ('南非', '地名'), ('比勒托利亚', '地名'), ('美国国家工程院院士', '身份'), ('宾夕法尼亚大学', '机构名'), ('Zip2', '公司名'), ('X.com', '公司名'), ('PayPal', '公司名') " ,
"result" : {
"text" : "埃隆·里夫·马斯克（Elon Reeve Musk） [107] ，1971年6月28日出生于南非的行政首都比勒陀利亚，企业家、工程师、慈善家、美国国家工程院院士。他同时兼具南非、加拿大和美国三重国籍。埃隆·马斯克本科毕业于宾夕法尼亚大学，获经济学和物理学双学位。1995年至2002年，马斯克与合伙人先后办了三家公司，分别是在线内容出版软件“Zip2”、电子支付“X.com”和“PayPal”。 " ,
"entities" : [
[
0 ,
9 ,
"埃隆·里夫·马斯克" ,
" PEO "
],
[
48 ,
50 ,
"南非" ,
" LOC "
],
[
73 ,
82 ,
"美国国家工程院院士" ,
" ID "
],
[
88 ,
90 ,
"南非" ,
" LOC "
],
[
113 ,
120 ,
"宾夕法尼亚大学" ,
" ORG "
],
[
173 ,
177 ,
" Zip2 " ,
" COM "
],
[
184 ,
189 ,
" X.com " ,
" COM "
],
[
192 ,
198 ,
" PayPal " ,
" COM "
]
],
"formatted_result": "埃tB-PEOn隆tI-PEOn·tI-PEOn里tI-PEOn夫tI-PEOn·tI-PEOn马tI-PEOn斯tI-PEOn克tI-PEOn（tOnEtOnltOnotOnntOn tOnRtOnetOnetOnvtOnetOn tOnMtOnutOnstOnktOn）tOn tOn[tOn1tOn0tOn7tOn]tOn tOn tOn，tOn1tOn9tOn7tOn1tOn年tOn6tOn月tOn2tOn8tOn日tOn出tOn生tOn于tOn南tB-LOCn非tI-LOCn的tOn行tOn政tOn首tOn都tOn比tOn勒tOn陀tOn利tOn亚tOn，tOn企tOn业tOn家tOn、tOn工tOn程tOn师tOn、tOn慈tOn善tOn家tOn、tOn美tB-IDn国tI-IDn国tI-IDn家tI-IDn工tI-IDn程tI-IDn院tI-IDn院tI-IDn士tI-IDn。tOn他tOn同tOn时tOn兼tOn具tOn南tB-LOCn非tI-LOCn、tOn加tOn拿tOn大tOn和tOn美tOn国tOn三tOn重tOn国tOn籍tOn。tOn埃tOn隆tOn·tOn马tOn斯tOn克tOn本tOn科tOn毕tOn业tOn于tOn宾tB-ORGn夕tI-ORGn法tI-ORGn尼tI-ORGn亚tI-ORGn大tI-ORGn学tI-ORGn，tOn获tOn经tOn济tOn学tOn和tOn物tOn理tOn学tOn双tOn学tOn位tOn。tOn1tOn9tOn9tOn5tOn年tOn至tOn2tOn0tOn0tOn2tOn年tOn，tOn马tOn斯tOn克tOn与tOn合tOn伙tOn人tOn先tOn后tOn办tOn了tOn三tOn家tOn公tOn司tOn，tOn分tOn别tOn是tOn在tOn线tOn内tOn容tOn出tOn版tOn软tOn件tOn“tOnZtB-COMnitI-COMnptI-COMn2tI-COMn”tOn、tOn电tOn子tOn支tOn付tOn“tOnXtB-COMn.tI-COMnctI-COMnotI-COMnmtI-COMn”tOn和tOn“tOnPtB-COMnatI-COMnytI-COMnPtI-COMnatI-COMnltI-COMn”tOn。tO"
}
}WIP
Kontribusi selalu diterima!
Selamat bergabung dengan komunitas kami!
