azure search openai javascript
1.0.0
This sample demonstrates a few approaches for creating ChatGPT-like experiences over your own data using the Retrieval Augmented Generation pattern. It uses Azure OpenAI Service to access the ChatGPT model (gpt-35-turbo), and Azure AI Search for data indexing and retrieval.
The repo includes sample data so it's ready to try end to end. In this sample application we use a fictitious company called Contoso Real Estate, and the experience allows its customers to ask support questions about the usage of its products. The sample data includes a set of documents that describe its terms of service, privacy policy and a support guide.
The application is made from multiple components, including:
Watch a video overview of the app
IMPORTANT: In order to deploy and run this sample, you'll need:
Microsoft.Authorization/roleAssignments/write permissions, such as Role Based Access Control Administrator, User Access Administrator, or Owner. If you don't have subscription-level permissions, they must be granted to you with RBAC for an existing resource group and deploy to that existing group.Microsoft.Resources/deployments/write permissions at a subscription level.Pricing may vary per region and usage. Exact costs cannot be estimated. You may try the Azure pricing calculator for the resources below.
azd down --purge.
There are multiple ways to successfully setup this project.
The easiest way to get started is with GitHub Codespaces that provides preconfigurations to setup all the tools for you. Read more below. Alternatively you can set up your local environment follwing the instructions below.
You can run this repo virtually by using GitHub Codespaces, which will open a web-based VS Code in your browser:
A similar option to Codespaces is VS Code Remote Containers, that will open the project in your local VS Code instance using the Dev Containers extension:
pwsh.exe from a PowerShell command. If this fails, you likely need to upgrade PowerShell.Then get the project code:
azd auth login
azd init -t azure-search-openai-javascript
Execute the following command, if you don't have any pre-existing Azure services and want to start from a fresh deployment.
azd up - This will provision Azure resources and deploy this sample to those resources, including building the search index based on the files found in the ./data folder.
eastus2. You can set a different location with azd env set AZURE_OPENAI_RESOURCE_GROUP_LOCATION {location}. Currently only a short list of locations is accepted. That location list is based on the OpenAI model availability table and may become outdated as availability changes.eastus2. You can set a different location with azd env set AZURE_WEBAPP_LOCATION {location}. Currently only a short list of locations is accepted. Note that Static Web App is a global service, and the location you choose will only affect the managed Functions App which is not used in this sample.It will look like the following:
NOTE: It can take 15+ minutes for the application to be fully deployed.
If you already have existing Azure resources, you can re-use those by setting azd environment values.
azd env set AZURE_RESOURCE_GROUP {Name of existing resource group}
azd env set AZURE_LOCATION {Location of existing resource group}
azd env set AZURE_OPENAI_SERVICE {Name of existing OpenAI service}
azd env set AZURE_OPENAI_RESOURCE_GROUP {Name of existing resource group that OpenAI service is provisioned to}
azd env set AZURE_OPENAI_CHATGPT_DEPLOYMENT {Name of existing ChatGPT deployment}. Only needed if your ChatGPT deployment is not the default 'chat'.azd env set AZURE_OPENAI_EMBEDDING_DEPLOYMENT {Name of existing GPT embedding deployment}. Only needed if your embeddings deployment is not the default 'embedding'.azd env set AZURE_SEARCH_SERVICE {Name of existing Azure AI Search service}
azd env set AZURE_SEARCH_SERVICE_RESOURCE_GROUP {Name of existing resource group with ACS service}
azd up step,
then run azd env set AZURE_SEARCH_SERVICE_LOCATION {Location of existing service}
azd env set AZURE_SEARCH_SERVICE_SKU {Name of SKU}. The free tier won't work as it doesn't support managed identity. (See other possible values)You can also use an existing Storage Account. See ./infra/main.parameters.json for list of environment variables to pass to azd env set to configure those existing resources.
Now you can run azd up, following the steps in Deploying from scratch above.
That will both provision resources and deploy the code.
If you've only changed the backend/frontend code in the app folder, then you don't need to re-provision the Azure resources. You can just run:
azd deploy
If you've changed the infrastructure files (infra folder or azure.yaml), then you'll need to re-provision the Azure resources. You can do that by running:
azd up
To give someone else access to a completely deployed and existing environment, either you or they can follow these steps:
azd init -t azure-search-openai-javascript or clone this repository.azd env refresh -e {environment name}
They will need the azd environment name, subscription ID, and location to run this command. You can find those values in your .azure/{env name}/.env file. This will populate their azd environment's .env file with all the settings needed to run the app locally.AZURE_PRINCIPAL_ID either in that .env file or in the active shell to their Azure ID, which they can get with az ad signed-in-user show../scripts/roles.ps1 or ./scripts/roles.sh to assign all of the necessary roles to the user. If they do not have the necessary permission to create roles in the subscription, then you may need to run this script for them. Once the script runs, they should be able to run the app locally.To clean up all the resources created by this sample:
azd down --purge
y
y
The resource group and all the resources will be deleted.
By default, the deployed Azure web app will have no authentication or access restrictions enabled, meaning anyone with routable network access to the web app can chat with your indexed data. You can require authentication to your Azure Entra ID by following the Add app authentication tutorial and set it up against the deployed web app.
To then limit access to a specific set of users or groups, you can follow the steps from Restrict your Azure Entra app to a set of users by changing "Assignment Required?" option under the Enterprise Application, and then assigning users/groups access. Users not granted explicit access will receive the error message -AADSTS50105: Your administrator has configured the application <app_name> to block users unless they are specifically granted ('assigned') access to the application.-
We recommend deploying additional security mechanisms. When applicable, consider setting up a VNet or setting up a Proxy Policy.
By default, the deployed search API will only allow requests from the same origin as the deployed web app origin. To enable CORS for a frontend hosted on a different origin, run:
azd env set ALLOWED_ORIGIN https://<your-domain.com>
azd up
You can only run locally after having successfully run the azd up command.
azd auth login
azd env get-values > .env to get the environment variables for the appaz login
npm start or run the "VS Code Task: Start App" to start the project locally.Once in the web app:
The Search API service implements the HTTP protocol for AI chat apps. It can be swapped with any service that implements the same protocol, like the Python backend client in this repository instead of the Node.js implementation featured in this repo.
To do so, follow these steps:
azd env get-values | grep WEBAPP_URI to get the URL.http://localhost:5173.https://<your_codespace_base_url>-5173.app.github.dev.azd env set ALLOWED_ORIGIN <your_frontend_url>.azd env get-values | grep BACKEND_URI.azd env set BACKEND_URI <your_backend_url>.If you want to use the deployed web app, run azd up to redeploy.
If you want to use the local web app on your machine or in Codespaces, run:
# Export the environment variable.
# The syntax may be different depending on your shell or if you're using Windows.
export BACKEND_URI=<your_backend_url>
# Start the app
npm start --workspace=webappThis sample is composed by two applications: a backend service and API, deployed to Azure Container Apps, and a frontend application, deployed to Azure Static Web Apps. By default, the deployed Azure Container App will have no authentication or access restrictions enabled, meaning anyone with routable network access to the container app can chat with your indexed data. You can require authentication to your Azure Entra ID by following the Add container app authentication tutorial and set it up against the deployed Azure Container App.
To limit access to a specific set of users or groups, you can follow the steps from Restrict your Azure Entra app to a set of users by changing "Assignment Required?" option under the Enterprise Application, and then assigning users/groups access. Users not granted explicit access will receive the error message -AADSTS50105: Your administrator has configured the application <app_name> to block users unless they are specifically granted ('assigned') access to the application.-
This sample is designed to be a starting point for your own production application, but you should do a thorough review of the security and performance before deploying to production. Here are some things to consider:
chatGptDeploymentCapacity and embeddingDeploymentCapacity parameters in infra/main.bicep to your account's maximum capacity.
You can also view the Quotas tab in Azure OpenAI studio to understand how much capacity you have.Standard_LRS SKU.
To improve your resiliency, we recommend using Standard_ZRS for production deployments, which you can specify using the sku property under the storage module in infra/main.bicep.Standard SKU with the free semantic search option, which gives you 1000 free queries a month. Assuming your app will experience more than 1000 questions, you should either change semanticSearch to "standard" or disable semantic search entirely in the request options. If you see errors about search service capacity being exceeded, you may find it helpful to increase the number of replicas by changing replicaCount in infra/core/search/search-services.bicep or manually scaling it from the Azure Portal.Chunking allows us to limit the amount of information we send to OpenAI due to token limits. By breaking up the content, it allows us to easily find potential chunks of text that we can inject into OpenAI. The method of chunking we use leverages a sliding window of text such that sentences that end one chunk will start the next. This allows us to reduce the chance of losing the context of the text.
To upload more documents, put them in the data/ folder and run ./scripts/index-data.sh or ./scripts/index-data.ps1.
Another popular repository for this use case is here: https://github.com/Microsoft/sample-app-aoai-chatGPT/
That repository is designed for use by customers using Azure OpenAI studio and Azure Portal for setup. It also includes azd support for folks who want to deploy it completely from scratch.
The primary differences:
Feature comparison:
| Feature | azure-search-openai-javascript | sample-app-aoai-chatGPT |
|---|---|---|
| RAG approach | Multiple approaches | Only via ChatCompletion API data_sources |
| Vector support | ✅ Yes | ✅ Yes |
| Data ingestion | ✅ Yes (MD) | ✅ Yes (PDF, TXT, MD, HTML) |
| Persistent chat history | No (browser tab only) | ✅ Yes, in CosmosDB |
Technology comparison:
| Tech | azure-search-openai-javascript | sample-app-aoai-chatGPT |
|---|---|---|
| Frontend | React/Lit | React |
| Backend | Node.js (Fastify) | Python (Flask) |
| Vector DB | Azure AI Search | Azure AI Search |
| Deployment | Azure Developer CLI (azd) | Azure Portal, az, azd |
Run these commands:
azd env set AZURE_OPENAI_CHATGPT_MODEL gpt-4You may also need to adjust the capacity in infra/main.bicep file, depending on how much TPM your account is allowed.
The chat tab uses the approach programmed in chat-read-retrieve-read.ts. The ask tab uses the approach programmed in ask-retrieve-then-read.ts. There is also another one /ask approach available, [using an agent](https://github.com/Azure-Samples/azure-search-openai-javascript/blob/main/packages/search/src/lib/approaches/ask-read-retrieve-read.ts.
The azd up command comes from the Azure Developer CLI, and takes care of both provisioning the Azure resources and deploying code to the selected Azure hosts.
The azd up command uses the azure.yaml file combined with the infrastructure-as-code .bicep files in the infra/ folder. The azure.yaml file for this project declares several "hooks" for the prepackage step and postprovision steps. The up command first runs the prepackage hook which installs Node dependencies and builds the React.JS-based JavaScript files. It then packages all the code (both frontend and backend services) into a zip file which it will deploy later.
Next, it provisions the resources based on main.bicep and main.parameters.json. At that point, since there is no default value for the OpenAI resource location, it asks you to pick a location from a short list of available regions. Then it will send requests to Azure to provision all the required resources. With everything provisioned, it runs the postprovision hook to process the local data and add it to an Azure AI Search index.
Finally, it looks at azure.yaml to determine the Azure host (Container Apps and Static Web Apps, in this case) and uploads the zip to Azure App Service. The azd up command is now complete, but it may take a few minutes for the app to be fully available and working after the initial deploy.
Related commands are azd provision for just provisioning (if infra files change) and azd deploy for just deploying updated app code.
Here are the most common failure scenarios and solutions:
The subscription (AZURE_SUBSCRIPTION_ID) doesn't have access to the Azure OpenAI service. Please ensure AZURE_SUBSCRIPTION_ID matches the ID specified in the OpenAI access request process.
You're attempting to create resources in regions not enabled for Azure OpenAI (e.g. East US 2 instead of East US), or where the model you're trying to use isn't enabled. See this matrix of model availability.
You've exceeded a quota, most often number of resources per region. See this article on quotas and limits.
You're getting "same resource name not allowed" conflicts. That's likely because you've run the sample multiple times and deleted the resources you've been creating each time, but are forgetting to purge them. Azure keeps resources for 48 hours unless you purge from soft delete. See this article on purging resources.
After running azd up and visiting the website, you see a '404 Not Found' in the browser. Wait 10 minutes and try again, as it might be still starting up. Then try running azd deploy and wait again. If you still encounter errors with the deployed app, consult these tips for debugging App Service app deployments and file an issue if the error logs don't help you resolve the issue.
You're getting an error 401 Principal does not have access to API/Operation while running the project locally or trying to deploy. That's likely because your environment variables include AZURE_TENANT_ID, AZURE_CLIENT_ID and AZURE_CLIENT_SECRET. You should either grant permissions to the related Service Principal or remove these variables from your environment to ensure normal access. For more details, please refer to Azure identity SDK.
Note: The documents used in this demo contain information generated using a language model (Azure OpenAI Service). The information contained in these documents is only for demonstration purposes and does not reflect the opinions or beliefs of Microsoft. Microsoft makes no representations or warranties of any kind, express or implied, about the completeness, accuracy, reliability, suitability or availability with respect to the information contained in this document. All rights reserved to Microsoft.
