ai dapter
1.0.0
Adapter to connect AI LLM with real-time grounding sources and data
Listing major enhancements
| Release | Date | Key Features |
|---|---|---|
| 2.0.5 | Apr 01, 2024 | Major update: Supports ClaudeAI in addition to OpenAI and GoogleAI. |
| 2.0.0 | Jan 05, 2024 | Major update: Supports GoogleAI in addition to OpenAI. |
| 1.3.0 | Dec 16, 2023 | Improved prompts to utilize relevant data and generate relevant responses. Responses for all methods now provide runtime information. |
| 1.2.9 | Nov 03, 2023 | (1) Agent configuration can specify (optional) language key. When this is set, LLM always thinks and responds in specified language (irrespective of input language). Note: Translation quality may not be optimal in some cases, so thorough testing is strongly adviced. (2) Response from all-in-one method includes grounding data used by LLM to geenrate a response. This should improve reliability. |
| 1.2.8 | Oct 26, 2023 | Incorporated deep-dive prompting technique to improve quality of response. |
| 1.2.2 | Oct 18, 2023 | Improved LLM prompts and logic for API determination. |
| 1.1.1 | Oct 09, 2023 | Introduced LLM prompts to incorporate a flavor of constitutional AI to ensure user input is fair, responsibile, respectful, and humane. |
| 1.1.0 | Oct 08, 2023 | Introduced ability to build context/memory for follow-up questions. |
| 1.0.0 | Oct 08, 2023 | Initial launch. |
As multi-modal artificial intelligence models, LLMs have broad applicability for generating content with minimal instructions. Prompting techniques such as zero-shot or few-shot are popular amongst everyday users of chat applications built on top of such LLMs. That said, although the quality of the response is excellent, how much can we trust it? How do we know the model isn't "making up" (a.k.a. hallucinating) on-the-fly?
As a result, grounding LLMs by providing contextual data in combination with proper prompting techniques is very important. Using prompts with grounded information as context to help LLM generate a better response is a practice widely followed.
One such approach is Retrieval-Augmented Generation (RAG), which relies on storing and searching text embeddings provided to LLM along with the prompt. However, RAG relies on static information converted into text embeddings and storing them in graph databases (a.k.a. vector databases) so that relevant information can be retrieved from it and augmented via grounding to generate text/response.
The RAG pattern might raise the question of whether real-time data can be used along with LLMs to generate effective and reliable responses. The simple answer is, "Of course, yes!". But that means a lot more responsibilities on the shoulders of Application Developers. The developer needs to call the API to wrap the response within a prompt to have LLM generate a relevant response. But will calling the same API over and over for every question work? - Most probably not! How can we dynamically determine which API should be called and which parameters must be passed based on the question? That also sounds like a capability for LLM.
That's where AI-Dapter (read as, AI Adapter) comes into play. AI-Dapter accelerates the LLM-based application development process for developers, allowing them to focus only on applications while offloading the burden of following activities to LLM.
identifying the right API endpoints from a pre-defined API repository,
acquiring real-time data from the identified API endpoints,
generating a response using the LLM model of choice.
The developer experience is improved tremendously by plugging in the AI-Dapter framework within the application code and seamlessly using it as a black box to perform LLM-based responses to user's questions. Note that these user questions themselves might be close to a zero-shot prompt!
The actual work behind this black box includes:
the ability to integrate an LLM model of your choice with your credentials (Note: Currently, only OpenAI GPT models are supported),
an integrated engine that leverages user inputs to perform identification of -
API-based data-sources,
acquisition of real-time data from those APIs, and
generation of response grounded by real-time data using LLM.
Please submit feedback or new feature requests via GitHub Discussions.
Please submit your issues via GitHub Issues.
Run the following on the command line to install the AI-Dapter for your NodeJS project. Make sure you are within your project's root directory.
npm install ai-dapter --save
Assuming you have set up a typescript NodeJS project, import the AIDapter class as follows.
import AIDapter from "ai-adapter";
To initialize AI-Dapter, you must pass some mandatory and optional parameters to complete the setup. An example is shown below, followed by documentation of the supported parameters.
const ai = new AIDapter({
"app_name": "<>"
"provider": "OpenAI",
"model_name": "gpt-3.5-turbo-16k",
"endpoint": "https://api.openai.com/v1/chat/completions",
"authentication": {"api_key": "<>","org_id": "<>"
},
"temperature": "<>"}); List of supported parameters to initialize the setup.
| Parameter | Mandatory(M) / Optional(O) | Purpose | Possible values |
|---|---|---|---|
app_name | M | Short app name. | - |
provider | M | The provider of LLM Model. Note: Currently, only models provided directly by OpenAI are supported. | "OpenAI" |
model_name | O | Allows you to select any model released by the provider. We recommend using models that allow large token sizes, such as gpt-3.5-turbo-16k or gemini-pro or claude-3-haiku-20240307. | - |
endpoint | O | The endpoint from where the provider serves the LLM model. You may have to refer to provider-specific documentation. For example, the OpenAI chat completion model is served from the https://api.openai.com/v1/chat/completions endpoint, GoogleAI model is served from https://generativelanguage.googleapis.com/v1/models/gemini-pro:generateContent and ClaudeAI model is served from https://api.anthropic.com/v1/messages. | - |
authentication | M | Provide authentication details as specified by your provider. For example, since OpenAI requires an API Key and an Organization ID, those are provided under the api_key and org_id fields, as shown in the initialization example above. | - |
telemetry | O | Telemetry data collection. Default is true. | true/false |
AI-Dapter allows you to customize specific agent configurations, such as adding roles, personalities, etc. Along the same lines, you can customize particular data configurations, such as controlling the number of records from real-time API responses, passing additional context or grounding information to simulate follow-ups, etc.
The agent and configuration options are passed as a single object as follows. Refer to the appropriate sections below to look up and adjust these configuration values.
let options: AIDapterOptions = {
"agentConfig": { "role": "personal assistant"
},
"dataConfig": { "max_records": 7
}};Below are currently supported agent configuration parameters. Add these fields under the agentConfig object.
| Parameter | Mandatory(M) / Optional(O) | Purpose | Possible values |
|---|---|---|---|
role | M | Helps guide LLM's approach to user questions. For example, in the role of an army Sergeant, LLM may respond to question about current time as something like, "The current time is 08:01 AM EDT.", while a comedian who always tells a one-liner joke about my question may respond something like, "It's time for you to get a watch! Just kidding, it is currently 8:01 AM on October 7th, 2023 in the Eastern Daylight Time (EDT) timezone.". | - |
personality | O | Gives a personality to the tone of LLM's response. | - |
language | O | The language you want the agent to respond in irrespective of the language the user question is asked in. Default=English. | - |
expert_at | O | If LLM should assume they are experts in any specific area, such as healthcare or automobile engines, etc. | - |
max_words | O | Control how long or short should LLM's response be within. Default is less than 200 words. | Any number between 1 and 200 |
Below are currently supported data configuration parameters. Add these fields under the dataConfig object.
| Parameter | Mandatory(M) / Optional(O) | Purpose | Possible values |
|---|---|---|---|
max_records | M | Controls how many top records from a result-set obtained from real-time API call should be retailed. This parameter is essential to control the input token size as results from the API call are used in grounding. Note: Because only top X rows are retained, it is best to provide API endpoints that would include data sorted in descending order. Default=10 | Any number between 1 and 10 |
additional_context | O | Additional context can be provided when follow-up-like capabiltiies are expected. (see examples from 6.3. Generation of LLM response with grounding real-time data). | Must be an array with the structure as follows: [{ "question": "", "response_summary": "", "entities": { ... } }, { ... }] |
max_contexts | O | Retains latest X contexts. AI-Dapter will retain the last two elements of the additional_context array, assuming the latest context is always appended at the end of this array. | 1 or 2 |
AI-Dapter provides three capabilities as follows and, therefore, three methods to access these capabilities.
Recommendation: For the best developer experience, see 6.3. Generation of LLM response with grounding real-time data
AIDapter.getRealtimeSources(input: string, apiRepository: Types.APIRepository[]): Promise
Use this method if your objective is to obtain relevant API endpoints based on user questions.
| Parameter | Mandatory(M) / Optional(O) | Purpose | Data Type |
|---|---|---|---|
input | M | User question | Text |
apiRepository[] | M | Provide a complete API repository that would contain details about the API endpoint (method, url, headers, and data), placeholders used in the API endpoint along with validation instructions which LLM will use. | See 7. API Repository |
| Field | Purpose |
|---|---|
api_endpoints[] | array of all identified API endpoints. |
provider | Indicates which LLM provider was used. |
runtime | To track overall response time. Note that this will depend on LLM's API response time. |
// Import and initialize AI-Dapterimport AIDapter from "ai-adapter";const ai = new AIDapter({
"provider": "GoogleAI",
"authentication": {"api_key": "<>"
}});// Define the API repository from where an appropriate API will be identified, updated and returned.// Notice this example provides only one example of an API endpoint, but since this is an array, you should expect to provide multiple such API endpoints.const apiRepository = [
{"api_info": { "title": "Current time", "description": "Identify the appropriate Timezone area and location for a given location and get time at that location."},"api_endpoint": { "method": "GET", "url": "http://worldtimeapi.org/api/timezone/|area_location|", "headers": {"Content-Type": "application/json" }},"placeholders": [ {"placeholder": "|area_location|","validation_criteria": "An example of area_location is: America/New_York or Europe/London. Based on the valid location provided, determine appropriate area_location.","default": "America/New_York" }]
}];// This is the user's questionlet input = "what time is it in Mumbai?"// Now call the getRealtimeSources() method to obtain valid API endpointsai.getRealtimeSources(input, apiRepository)
.then((resp) => {console.log(resp);/* { "api_endpoints": [ { "api": { "method": "GET", "url": "https://worldtimeapi.org/api/timezone/Asia/Kolkata", "headers": { "Content-Type": "application/json" } }, "placeholders": [ { "placeholder": "[area_location]", "determined": true } ], "status": "OK" } ], "provider": "GoogleAI", "runtime": "2 seconds" } */
}).catch((err) => console.log(JSON.stringify(err, null, 4))); Notice that based on the user-provided city as Mumbai, LLM determined the appropriate value for placeholder area_location and returned an updated API endpoint.
AIDapter.getDataFromRealtimeSource(input: string, apiRepository: Types.APIRepository[], dataConfig?: Types.DataConfig | undefined): Promise
Use this method if your objective is to obtain data from relevant API endpoints based on user questions.
| Parameter | Mandatory(M) / Optional(O) | Purpose | Data Type |
|---|---|---|---|
input | M | User question | Text |
apiRepository[] | M | Array of API information, endpoints (method, url, headers, and data), and placeholders. | See 7. API Repository |
dataConfig | O | Configuration paramters to control data obtained from API calls. | See 5.2. Data Configuration |
| Field | Purpose |
|---|---|
api_results[] | Array of responses from all API calls. |
provider | Indicates which LLM provider was used. |
runtime | To track overall response time. Note that this will depend on LLM's API response time. |
// Import and initialize AI-Dapterimport AIDapter from "ai-adapter";const ai = new AIDapter({
"provider": "GoogleAI",
"authentication": {"api_key": "<>"
}});// Define the API repository from where an appropriate API will be identified, updated and returned.// Notice this example provides only one example of an API endpoint, but since this is an array, you should expect to provide multiple such API endpoints.const apiRepository = [
{"api_info": { "title": "Current time", "description": "Identify the appropriate Timezone area and location for a given location and get time at that location."},"api_endpoint": { "method": "GET", "url": "http://worldtimeapi.org/api/timezone/|area_location|", "headers": {"Content-Type": "application/json" }},"placeholders": [ {"placeholder": "|area_location|","validation_criteria": "An example of area_location is: America/New_York or Europe/London. Based on the valid location provided, determine appropriate area_location.","default": "America/New_York" }]
}];// This is the user's questionlet input = "what time is it in Mumbai?"// Data configurationlet dataConfig = { "max_records": 3 }// Now call the getDataFromRealtimeSource() method to obtain data from calling all relevant API endpoints based on user questionai.getDataFromRealtimeSource(question, apiRepository, dataConfig)
.then((resp) => {console.log(resp);/* { "api_results": [ { "api_sources": "worldtimeapi.org", "data": { "abbreviation": "IST", "client_ip": "50.126.214.61", "datetime": "2024-01-05T22:48:30.316887+05:30", "day_of_week": 5, "day_of_year": 5, "dst": false, "dst_from": null, "dst_offset": 0, "dst_until": null, "raw_offset": 19800, "timezone": "Asia/Kolkata", "unixtime": 1704475110, "utc_datetime": "2024-01-05T17:18:30.316887+00:00", "utc_offset": "+05:30", "week_number": 1 } } ], "provider": "GoogleAI", "runtime": "4 seconds" } */
}).catch((err) => console.log(JSON.stringify(err, null, 4))); Response from the API call. This data can be used for LLM grounding.
AIDapter.getLLMResponseFromRealtimeSources(input: string, apiRepository: Types.APIRepository[], options?: AIDapterOptions | undefined): Promise
Use this method if your objective is to obtain LLM responses based on user questions. This includes identifying relevant API endpoints, calling the identified APIs, and using that in the LLM prompt to receive the response from LLM.
Recommendation: Use this method to achieve maximum acceleration in your application development process.
| Parameter | Mandatory(M) / Optional(O) | Purpose | Data Type |
|---|---|---|---|
input | M | User question | Text |
apiRepository[] | M | Array of API information, endpoints (method, url, headers, and data), and placeholders. | See 7. API Repository |
options | O | Agent and Data Configuration | See 5. Agent & Data Configuration Options |
| Field | Purpose |
|---|---|
ai_response | LLM generated response. |
ai_status | Helps determine if the response was based on the availability of all required data elements to make successful API calls. Possible values: OK, FOLLOW-UP, or INCOMPLETE |
ai_context | This contains a short response summary and a list of entities. The idea behind this field is for use cases involving follow-up conversations. The entire object can be passed as additional_content within the dataConfig options when follow-up questions are to be submitted. |
provider | Indicates which LLM provider was used. |
runtime | To track overall response time. Note that this will depend on LLM's API response time. |
// Import and initialize AI-Dapterimport AIDapter from "ai-adapter";const ai = new AIDapter({
"provider": "GoogleAI",
"authentication": {"api_key": "<>"
}});// Define the API repository from where an appropriate API will be identified, updated and returned.// Notice this example provides only one example of an API endpoint, but since this is an array, you should expect to provide multiple such API endpoints.const apiRepository = [
{"api_info": { "title": "Current time", "description": "Identify the appropriate Timezone area and location for a given location and get time at that location."},"api_endpoint": { "method": "GET", "url": "http://worldtimeapi.org/api/timezone/|area_location|", "headers": {"Content-Type": "application/json" }},"placeholders": [ {"placeholder": "|area_location|","validation_criteria": "An example of area_location is: America/New_York or Europe/London. Based on the valid location provided, determine appropriate area_location.","default": "America/New_York" }]
}];// This is the user's questionlet input = "what time is it in Mumbai?"// AI-Dapter options which provide combined Agent configuration and Data Configurationlet options: AIDapterOptions = {
"agentConfig": { "role": "comedian who always tells a one-liner joke about my question" },
"dataConfig": { "max_records": 7 }};// Now call the getLLMResponseFromRealtimeSources() method to obtain an LLM response to the user question.// LLM response is based on a prompt that uses real-time data for grounding.ai.getLLMResponseFromRealtimeSources(question, apiRepository, options)
.then((resp) => {console.log(resp);/* { "ai_response": "In the vibrant city of Mumbai, where Bollywood dreams take flight and the aroma of street food fills the air, it's currently 22:50 on this fabulous Friday, the 5th of January, 2024. So, whether you're sipping chai at the Gateway of India or grooving to the beats in a local dance club, remember, time waits for no one, not even for the biggest Bollywood stars!", "ai_status": "OK", "ai_context": { "questions": "what time is it in Mumbai? What is the current date in Mumbai?", "entities": [], "data": [ { "abbreviation": "IST", "client_ip": "50.126.214.61", "datetime": "2024-01-05T22:50:51.261990+05:30", "day_of_week": 5, "day_of_year": 5, "dst": false, "dst_from": null, "dst_offset": 0, "dst_until": null, "raw_offset": 19800, "timezone": "Asia/Kolkata", "unixtime": 1704475251, "utc_datetime": "2024-01-05T17:20:51.261990+00:00", "utc_offset": "+05:30", "week_number": 1 } ], "sources": [ "worldtimeapi.org" ] }, "provider": "GoogleAI", "runtime": "6 seconds" } */
}).catch((err) => console.log(JSON.stringify(err, null, 4))); Notice that the user question is used first to identify relevant API from the provided API repository. This method also calls the identified APIs, collects their responses to ground the final LLM prompt, and returns the generated response.
Also, Note that the response contains the LLM-generated content within the ai_response field and context within the ai_context field. The entire context can be passed as dataConfig.additional_context along with follow-up questions.
An example shows how the context can be passed to enable follow-up conversations.
// As shown in the previous example, ai_context contains the following information:// -----------------------------------------------------------------------------// resp.ai_context: {// "questions": "what time is it in Mumbai? What is the current date in Mumbai?",// "entities": [],// "data": [// {// "abbreviation": "IST",// "client_ip": "50.126.214.61",// "datetime": "2024-01-05T22:50:51.261990+05:30",// "day_of_week": 5,// "day_of_year": 5,// "dst": false,// "dst_from": null,// "dst_offset": 0,// "dst_until": null,// "raw_offset": 19800,// "timezone": "Asia/Kolkata",// "unixtime": 1704475251,// "utc_datetime": "2024-01-05T17:20:51.261990+00:00",// "utc_offset": "+05:30",// "week_number": 1// }// ],// "sources": [// "worldtimeapi.org"// ]// }// -----------------------------------------------------------------------------// Append the above context into dataConfig.additional_context ...if(options.dataConfig[additional_context]){ 
