ngoprek llm

Ngoprek LLM (Large Language Model)

Required: Node.js 18 or later.

The following demo can be run using GPT from OpenAI or a local LLM.

Method #1 : To access the API from OpenAI, you need an API key from OpenAI which is stored in an environment variable called OPENAI_API_KEY . Please read how to generate an API key and don't forget to use this API key safely!

Method #2 : Run LocalAI first and take a suitable model, for example LLama 2 7B, in GGML format. Then set the two environment variables OPENAI_API_BASE to refer to the server address of LocalAI and CHAT_MODEL to select the model name to use.

Example of detailed steps:

$ curl -OL https://github.com/go-skynet/LocalAI/releases/download/v1.23.2/local-ai-avx-Linux-x86_64
$ chmod +x ./local-ai-avx-Linux-x86_64
$ curl -OL https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/resolve/main/llama-2-7b-chat.ggmlv3.q4_0.bin
$ mv llama-2-7b-chat.ggmlv3.q4_0.bin models/
$ ./local-ai-avx-Linux-x86_64
$ export CHAT_MODEL= ' llama-2-7b-chat.ggmlv3.q4_0.bin '
$ export OPENAI_API_BASE= ' http://127.0.0.1:8080 '

JavaScript Version

Example of running a demo to complete a sentence ( completion ):

 $ node complete.js "Sukarno dan Hatta pada tahun 1945"

Example of running a demo to ask:

 $ node ask.js "Berapa jumlah penduduk Bandung?"

Example of running a demo to find detailed information:

 $ node query.js "Berapa koordinat geografis dari Bandung?"

 sequenceDiagram
  participant Client
  LLM participant
  participant Geocoder
  Client->>+LLM: "What are the geographic coordinates of Bandung?"
  LLM-->>+Geocoder: geocode("Bandung")
  Geocoder-->>+LLM: {"lat":-6.9,"long":107.6}
  LLM->>+Client: "Bandung is at latitude -6.9 and longitude 107.6."

 $ node query.js "Bagaimana suhu di ibukota Jawa Timur?"

 sequenceDiagram
  participant Client
  LLM participant
  participant Geocoder
  participant WeatherStation
  Client->>+LLM: "How is the temperature in the capital of East Java?"
  Note right of LLM: capital of East Java = Surabaya
  LLM-->>+Geocoder: geocode("Surabaya")
  Geocoder-->>+LLM: {"lat":-7.3,"long":112.7}
  LLM-->>+WeatherStation: WeatherStation(-7.3, 112.7)
  WeatherStation-->>+LLM: {"main": {"temp": 27}}
  LLM->>+Client: "The temperature in Surabaya is around 27°C"

Clojure Version

The demo version of Clojure can be run directly via the terminal as shown below, but it is better to understand and try it using the REPL (for example with Visual Studio Code + Calva or Vim/NeoVim + vim-iced plugin).

First, make sure the required modules are installed:

 $ npm install

Example of running a demo to complete a sentence ( completion ):

 $ npm run nbb complete.cljs "Ibukota Indonesia adalah"

Example of running a demo to ask:

 $ npm run nbb ask.cljs "Apa ibukota Jawa Timur?"

Example of running a demo to find detailed information:

 $ npm run nbb probe.cljs "Kapan saya terakhir ke Bandung?"