nebula siwi
1.0.0
Siwi (/ˈsɪwi/) adalah PoC Sistem Dialog Dengan Grafik Pengetahuan yang Didukung Database Grafik.
Untuk saat ini, ini adalah demo untuk bot dialog yang digerakkan oleh tugas (bukan tujuan umum) dengan KG (Grafik Pengetahuan) yang memanfaatkan Grafik Nebula dengan kumpulan data minimal/sampel dari Nebula Graph Manual/ NG中文手册.
Tip: Sekarang Anda dapat bermain dengan grafik online tanpa menginstal sendiri!
Taman Bermain Nebula | Taman Bermain Nebula - Tiongkok Daratan
Kueri yang didukung:
relation :
serving :
friendship :
Anda dapat mencobanya dari awal di sini: https://siwei.io/learn/nebula-101-siwi-kgqa/
Ini adalah salah satu saluran paling naif untuk domain tertentu/bot obrolan tujuan tunggal yang dibangun di atas Grafik Pengetahuan.
Backend (Siwi API) adalah server API berbasis Flask:
Server Flask API menerima pertanyaan di HTTP POST, dan memanggil bot API.
Di bagian bot API terdapat pengklasifikasi (Symentic Parsing, Intent Matching, Slot Filling), dan aktor pertanyaan (Panggil tindakan terkait untuk menanyakan Grafik Pengetahuan dengan maksud dan slot).
Grafik Pengetahuan dibangun di atas Database Grafik Sumber Terbuka: Grafik Nebula
Frontend adalah Aplikasi Halaman Tunggal VueJS (SPA):
┌────────────────┬──────────────────────────────────────┐
│ │ │
│ │ Speech │
│ ┌──────────▼──────────┐ │
│ │ Frontend │ Siwi, /ˈsɪwi/ │
│ │ Web_Speech_API │ A PoC of │
│ │ │ Dialog System │
│ │ Vue.JS │ With Graph Database │
│ │ │ Backed Knowledge Graph │
│ └──────────┬──────────┘ │
│ │ Sentence │
│ │ │
│ ┌────────────┼──────────────────────────────┐ │
│ │ │ │ │
│ │ │ Backend │ │
│ │ ┌──────────▼──────────┐ │ │
│ │ │ Web API, Flask │ ./app/ │ │
│ │ └──────────┬──────────┘ │ │
│ │ │ Sentence ./bot/ │ │
│ │ ┌──────────▼──────────┐ │ │
│ │ │ │ │ │
│ │ │ Intent matching, │ ./bot/classifier│ │
│ │ │ Symentic Processing │ │ │
│ │ │ │ │ │
│ │ └──────────┬──────────┘ │ │
│ │ │ Intent, Entities │ │
│ │ ┌──────────▼──────────┐ │ │
│ │ │ │ │ │
│ │ │ Intent Actor │ ./bot/actions │ │
│ │ │ │ │ │
│ └─┴──────────┬──────────┴───────────────────┘ │
│ │ Graph Query │
│ ┌──────────▼──────────┐ │
│ │ │ │
│ │ Graph Database │ Nebula Graph │
│ │ │ │
│ └─────────────────────┘ │
│ │
│ │
│ │
└───────────────────────────────────────────────────────┘
.
├── README.md
├── src
│ ├── siwi # Siwi-API Backend
│ │ ├── app # Web Server, take HTTP requests and calls Bot API
│ │ └── bot # Bot API
│ │ ├── actions # Take Intent, Slots, Query Knowledge Graph here
│ │ ├── bot # Entrypoint of the Bot API
│ │ ├── classifier # Symentic Parsing, Intent Matching, Slot Filling
│ │ └── test # Example Data Source as equivalent/mocked module
│ └── siwi_frontend # Browser End
│ ├── README.md
│ ├── package.json
│ └── src
│ ├── App.vue # Listening to user and pass Questions to Siwi-API
│ └── main.js
└── wsgi.pyBackendnya bergantung pada Grafik Nebula, Database Grafik Terdistribusi Sumber Terbuka.
Instal Nebula Graph di oneliner:
curl -fsSL nebula-up.siwei.io/install.sh | bashMuat kumpulan data pemain bola basket.
~ /.nebula-up/console.sh
nebula-console -addr graphd -port 9669 -user root -p nebula -e " :play basketballplayer "Instal dan jalankan.
# Install siwi backend
python3 -m build
# Configure Nebula Graph Endpoint
export NG_ENDPOINTS=127.0.0.1:9669
# Run Backend API server
gunicorn --bind :5000 wsgi --workers 1 --threads 1 --timeout 60Untuk OpenFunction/ KNative
docker build -t weygu/siwi-api .
docker run --rm --name siwi-api
--env=PORT=5000
--env=NG_ENDPOINTS=127.0.0.1:9669
--net=host
weygu/siwi-apiCobalah API Web:
$ curl -s --header " Content-Type: application/json "
--request POST
--data ' {"question": "What is the relationship between Yao Ming and Lakers?"} '
http://192.168.8.128:5000/query | jq
{
" answer " : " There are at least 23 relations between Yao Ming and Lakers, one relation path is: Yao Ming follows Shaquille O'Neal serves Lakers. "
}Panggil Bot Python API:
from nebula3 . gclient . net import ConnectionPool
from nebula3 . Config import Config
# define a config
config = Config ()
config . max_connection_pool_size = 10
# init connection pool
connection_pool = ConnectionPool ()
# if the given servers are ok, return true, else return false
ok = connection_pool . init ([( '127.0.0.1' , 9669 )], config )
# import siwi bot
from siwi . bot import bot
# instantiate a bot
b = bot . SiwiBot ( connection_pool )
# make the question query
b . query ( "Which team had Jonathon Simmons served?" )Maka responnya akan seperti ini:
In [ 4 ]: b . query (" Which team had Jonathon Simmons serv
...: ed ?")
[ DEBUG ] ServeAction intent : { 'entities' : { 'Jonathon Simmons' : 'player' }, 'intents' : ( 'serve' ,)}
[ DEBUG ] query for RelationshipAction :
USE basketballplayer ;
MATCH p = ( v ) - [ e : serve * 1 ] - > ( v1 ) WHERE id ( v ) == "player112"
RETURN p LIMIT 100 ;
[ 2021 - 07 - 02 02 : 59 : 36 , 392 ]: Get connection to ( '127.0.0.1' , 9669 )
Out [ 4 ]: 'Jonathon Simmons had served 3 teams. Spurs from 2015 to 2015; 76ers from 2019 to 2019; Magic from 2017 to 2017; 'Mengacu pada siwi_frontend
┌─────────────────────────────┐
│ kind: Ingress │ ┌───────────────────┐
│ path: / │ │ Pod │
│ -> siwi-frontend ────┼─────┤ siwi-frontend │
│ │ │ │
│ │ └───────────────────┘
│ │
│ path: /query │ ┌───────────────────────────────────┐
│ -> siwi-api ────┼─────┤ KNative Service │
│ KNative Serving │ │ serving-xxxx │
│ │ │ │
│ │ │ apiVersion: serving.knative.dev/v1│
│ │ │ kind: Service │
└─────────────────────────────┘ └─────────┬─────────────────────────┘
│
└────────────┐
│
┌───────────────────────────────────────────────────────┐ │
│apiVersion: core.openfunction.io/v1alpha1 │ │
│kind: Function │ │
│spec: │ │
│ version: "v1.0.0" │ │
│ image: "weygu/siwi-api:latest" │ │
│ imageCredentials: │ │
│ name: push-secret │ │
│ port: 8080 │ │
│ build: │ │
│ builder: openfunction/builder:v1 │ │
│ env: │ │
│ FUNC_NAME: "siwi_api" │ │
│ FUNC_TYPE: "http" │ │
│ FUNC_SRC: "main.py" │ │
│ srcRepo: │ │
│ url: "https://github.com/wey-gu/nebula-siwi.git" │ │
│ sourceSubPath: "src" │ │
│ serving: │ │
│ runtime: Knative ─────────────────────────────────┼──┘
│ params: │
│ NG_ENDPOINTS: "NEBULA_GRAPH_ENDPOINT" │
│ template: │ │
│ containers: │ │
│ - name: function │ │
│ imagePullPolicy: Always │ │
└───────────────────────────────────────┼───────────────┘
│
┌──────────┘
│
┌────────────────────────────┴───────────────────────────┐
│apiVersion:lapps.nebula-graph.io/v1alpha1 │
│kind: NebulaCluster │
│spec: │
│ graphd: │
│ config: │
│ system_memory_high_watermark_ratio: "1.0" │
│ image: vesoft/nebula-graphd │
│ replicas: 1 │
│... │
└────────────────────────────────────────────────────────┘
Diasumsikan kita memiliki k8s dengan OpenFunctions terinstal
Instal Grafik Nebula dengan installer nebula kubesphere-all-in-one di KubeSphere:
curl -sL nebula-kind.siwei.io/install-ks-1.sh | bashDapatkan Nebula Graph NodePort:
NEBULA_GRAPH_ENDPOINT= $( kubectl get svc nebula-graphd-svc-nodeport -o yaml -o jsonpath= ' {.spec.clusterIP}:{.spec.ports[0].port} ' )
echo $NEBULA_GRAPH_ENDPOINTMuat Set Data ke dalam cluster nebula:
wget https://docs.nebula-graph.io/2.0/basketballplayer-2.X.ngql
~ /.nebula-kind/bin/console -u root -p password --address= < nebula-graphd-svc-nodeport > --port=32669 -f basketballplayer-2.X.ngqlBuat siwi-api yang didukung oleh Openfunction:
cat siwi-api-function.yaml | sed " s/NEBULA_GRAPH_ENDPOINT/ $NEBULA_GRAPH_ENDPOINT /g " | kubectl apply -f -Dapatkan fungsi nebula-siwi dan Layanan KNative:
kubectl get functions nebula-siwi
FUNCTION= $( kubectl get functions nebula-siwi -o go-template= ' {{.status.serving.resourceRef}} ' )
kubectl get ksvc -l openfunction.io/serving= $FUNCTION
KSVC= $( kubectl get ksvc -l openfunction.io/serving= $FUNCTION -o=jsonpath= ' {.items[0].metadata.name} ' )
kubectl get revision -l serving.knative.dev/service= $KSVC
REVISION= $( kubectl get revision -l serving.knative.dev/service= $KSVC -o=jsonpath= ' {.items[0].metadata.name} ' )
echo $REVISIONVerifikasikan fungsi berfungsi dengan baik:
curl -s --header " Content-Type: application/json "
--request POST
--data ' {"question": "What is the relationship between Yao Ming and Lakers ?"} '
$( kubectl get ksvc -l openfunction.io/serving= $FUNCTION -o=jsonpath= ' {.items[0].status.url} ' ) /queryBuat sumber daya aplikasi siwi di K8s:
cat siwi-app.yaml | sed " s/REVISION/ $REVISION /g " | kubectl apply -f -Verifikasikan fungsi berfungsi dengan baik melalui ingress:
Di sini nodeport dengan port http 31059 digunakan sebagai titik akhir pengontrol masuknya.
curl -s --header " Content-Type: application/json "
--request POST
--data ' {"question": "how does Tim Duncan and Lakers connected?"} '
demo-siwi.local:31059/queryVerifikasi bagian depan:
curl $( kubectl get svc -l app=siwi -o=jsonpath= ' {.items[0].spec.clusterIP} ' )Verifikasi frontend di belakang ingress:
curl demo-siwi.local:31059Dapatkan semua sumber daya di aplikasi siwi:
kubectl get service,pod,ingress,function -l app=siwiDan seharusnya seperti ini:
[root@wey nebula-siwi] # kubectl get service,pod,ingress,function -l app=siwi
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/siwi-frontend-file ClusterIP 10.233.60.81 < none > 80/TCP 64m
NAME READY STATUS RESTARTS AGE
pod/siwi-frontend-file 1/1 Running 0 64m
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.networking.k8s.io/siwi-service < none > demo-siwi.local 80 59m
NAME BUILDSTATE SERVINGSTATE BUILDER SERVING AGE
function.core.openfunction.io/nebula-siwi Succeeded Running builder-sbfz6 serving-vvjvl 26h
[root@wey nebula-siwi] # kubectl get service,pod,ingress,function -l app=siwi
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/siwi-frontend-file ClusterIP 10.233.60.81 < none > 80/TCP 65m
NAME READY STATUS RESTARTS AGE
pod/siwi-frontend-file 1/1 Running 0 65m
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.networking.k8s.io/siwi-service < none > demo-siwi.local 80 59m
NAME BUILDSTATE SERVINGSTATE BUILDER SERVING AGE
function.core.openfunction.io/nebula-siwi Succeeded Running builder-sbfz6 serving-vvjvl 26hdocker build -t weygu/siwi-frontend . -f Dockerfile.froentend
docker push weygu/siwi-frontend
