Home>Programming related>Other source code
Download

Module: Azure OpenAI Private ChatGPT

Current Version 2.x

Version 2.x a complete rewrite of the module and is not backwards compatible with version 1.x.
New integrations and features have been added to the module to use the latest Azure OpenAI services and features such as GPT-4-1106, GPT-4-Vision and DALL-E-3. A new ChatBot UI / LibreChat has been added to the module to provide a complete solution.

Coming next - Feature development

v2.7.x (in development)

  • Privatise solution with Private endpoint support
  • Front Door and WAF support for public entrypoint
  • Azure AI Search/MeiliSearch Integration
  • Add additional support for Azure OpenAI DALL-E-3

v2.6.x (current)

  • AzureRM provider v4.x.x support added

v2.3.x, v2.4.x, v2.5.x

  • Dependency updates and tests

v2.2.x

  • Added User Violations Support
  • Updated documentation and examples
  • Added module outputs

v2.1.x

  • Custom domain support with managed certificates

Legacy Version 1.x

NOTE: Legacy version 1.x can be found in the legacy branch here

Version 1.x Documentation

Introduction

Under OpenAI's terms when using the public version of ChatGPT, any questions you pose—referred to as "prompts"—may contribute to the further training of OpenAI's Large Language Model (LLM). Given this, it's crucial to ask: Are you comfortable with this precious data flow leaving your organization? If you're a decision-maker or hold responsibility over your organization's security measures, what steps are you taking to ensure proprietary information remains confidential?

An effective solution lies in utilising a hosted version of the popular LLM on Azure OpenAI. While there are numerous advantages to Azure OpenAI, I'd like to spotlight two:

  • Data Privacy: By hosting OpenAI's models on Azure, your prompts will never serve as a source for training the LLM. It's simply a self-contained version running on Azure tailored for your use.

  • Enhanced Security: Azure OpenAI offers robust security measures, from the capability to secure specific endpoints to intricate role-based access controls. For a deeper dive, refer to this Microsoft Learn article.

While Azure OpenAI does come with a cost, it's highly affordable—often, a conversation costs under 10 cents. You can review Azure OpenAI's pricing details here.

Diagram

coming soon...

Examples

  • Public Deployment with Custom Domain and IP Whitelisting

Screens

Model Selection

Presets

Vision analysis

Contributing

Contributions are welcome. Please submit a pull request if you have any improvements or fixes. Make sure to follow the existing code style and add comments to your code explaining what it does.

License

This terraform module is licensed under the MIT License. See the LICENSE file for more details.

Support

If you encounter any issues or have any questions about this terraform module, please open an issue on GitHub. We'll do our best to respond as quickly as possible.

Acknowledgements

This terraform module was developed by Marcel Lupo as part of a project to explore the capabilities of Azure OpenAI models. We'd like to thank the OpenAI and Microsoft team for their incredible work and ongoing support of the AI community.

Requirements

Name Version
terraform >= 1.9.5
azurerm ~> 4.0
random ~> 3.0

Providers

Name Version
azurerm ~> 4.0
random ~> 3.0

Modules

No modules.

Resources

Name Type
azurerm_app_service_certificate_binding.libre_app_cert_binding resource
azurerm_app_service_custom_hostname_binding.hostname_binding resource
azurerm_app_service_managed_certificate.libre_app_cert resource
azurerm_cognitive_account.az_openai resource
azurerm_cognitive_deployment.az_openai_models resource
azurerm_cosmosdb_account.az_openai_mongodb resource
azurerm_dns_cname_record.cname_record resource
azurerm_dns_txt_record.domain_verification resource
azurerm_key_vault.az_openai_kv resource
azurerm_key_vault_secret.libre_app_creds_iv resource
azurerm_key_vault_secret.libre_app_creds_key resource
azurerm_key_vault_secret.libre_app_jwt_refresh_secret resource
azurerm_key_vault_secret.libre_app_jwt_secret resource
azurerm_key_vault_secret.openai_cosmos_uri resource
azurerm_key_vault_secret.openai_endpoint resource
azurerm_key_vault_secret.openai_primary_key resource
azurerm_linux_web_app.librechat resource
azurerm_resource_group.az_openai_rg resource
azurerm_role_assignment.kv_role_assigment resource
azurerm_role_assignment.librechat_app_kv_access resource
azurerm_service_plan.az_openai_asp resource
azurerm_subnet.az_openai_subnet resource
azurerm_virtual_network.az_openai_vnet resource
random_password.libre_app_creds_iv resource
random_password.libre_app_creds_key resource
random_password.libre_app_jwt_refresh_secret resource
random_password.libre_app_jwt_secret resource
azurerm_client_config.current data source

Inputs

Name Description Type Default Required
app_service_name Name of the Linux App Service Plan. string "openaiasp9000" no
app_service_sku_name The SKU name of the App Service Plan. string "B1" no
cosmosdb_automatic_failover Whether to enable automatic failover for the Cosmos DB account bool false no
cosmosdb_capabilities The capabilities for the Cosmos DB account list(string) 
[
  "EnableMongo",
  "MongoDBv3.4"
]
 no
cosmosdb_consistency_level The consistency level of the Cosmos DB account string "BoundedStaleness" no
cosmosdb_geo_locations The geo-locations for the Cosmos DB account 
list(object({
    location          = string
    failover_priority = number
  }))
 
[
  {
    "failover_priority": 0,
    "location": "uksouth"
  }
]
 no
cosmosdb_is_virtual_network_filter_enabled Whether to enable virtual network filtering for the Cosmos DB account bool true no
cosmosdb_kind The kind of Cosmos DB to create string "MongoDB" no
cosmosdb_max_interval_in_seconds The maximum staleness interval in seconds for the Cosmos DB account number 10 no
cosmosdb_max_staleness_prefix The maximum staleness prefix for the Cosmos DB account number 200 no
cosmosdb_name The name of the Cosmos DB account string "openaicosmosdb" no
cosmosdb_offer_type The offer type to use for the Cosmos DB account string "Standard" no
cosmosdb_public_network_access_enabled Whether to enable public network access for the Cosmos DB account bool true no
cosmosdb_virtual_network_subnets The virtual network subnets to associate with the Cosmos DB account (Service Endpoint). If networking is created as part of the module, this will be automatically populated. list(string) null no
dns_resource_group_name The Resource Group that contains the custom DNS Zone to use for the App Service string "dns-rg" no
kv_fw_allowed_ips value of key vault firewall allowed ip rules. list(string) [] no
kv_fw_bypass List of key vault firewall rules to bypass. string "AzureServices" no
kv_fw_default_action Default action for key vault firewall rules. string "Deny" no
kv_fw_network_subnet_ids The virtual network subnets to associate with the Cosmos DB account (Service Endpoint). If networking is created as part of the module, this will be automatically populated. list(string) null no
kv_name Name of the Key Vault to create (solution secrets). string "openaikv9000" no
kv_sku SKU of the Key Vault to create. string "standard" no
libre_app_allow_email_login Allow Email Login bool true no
libre_app_allow_registration Allow Registration bool true no
libre_app_allow_social_login Allow Social Login bool false no
libre_app_allow_social_registration Allow Social Registration bool false no
libre_app_allowed_ip_addresses Allowed IP Addresses. The CIDR notation of the IP or IP Range to match to allow. For example: 10.0.0.0/24 or 192.168.10.1/32 
list(object({
    ip_address = string
    priority   = number
    name       = string
    action     = string
  }))
 
[
  {
    "action": "Allow",
    "ip_address": "0.0.0.0/0",
    "name": "ip-access-rule1",
    "priority": 300
  }
]
 no
libre_app_allowed_subnets Allowed Subnets (By default the subnet the app service is deployed in is allowed access already as priority 100). Add any additionals here 
list(object({
    virtual_network_subnet_id = string
    priority                  = number
    name                      = string
    action                    = string
  }))
 
[
  {
    "action": "Allow",
    "name": "subnet-access-rule1",
    "priority": 200,
    "virtual_network_subnet_id": "subnet_id1"
  }
]
 no
libre_app_az_oai_api_key Azure OpenAI API Key string null no
libre_app_az_oai_api_version Azure OpenAI API Version string "2023-07-01-preview" no
libre_app_az_oai_dall3_api_version Azure OpenAI DALL-E API Version string "2023-12-01-preview" no
libre_app_az_oai_dall3_deployment_name Azure OpenAI DALL-E Deployment Name string "dall-e-3" no
libre_app_az_oai_instance_name Azure OpenAI Instance Name string null no
libre_app_az_oai_models Azure OpenAI Models. E.g. 'gpt-4-1106-preview,gpt-4,gpt-3.5-turbo,gpt-3.5-turbo-1106,gpt-4-vision-preview' string "gpt-4-1106-preview" no
libre_app_az_oai_use_model_as_deployment_name Azure OpenAI Use Model as Deployment Name bool true no
libre_app_custom_domain_create Create a custom domain and managed certificate for the App Service. bool false no
libre_app_custom_footer Add a custom footer for the App. string "Privately hosted chat app powered by Azure OpenAI and LibreChat." no
libre_app_debug_console Enable verbose server output in the console, though it's not recommended due to high verbosity. bool false no
libre_app_debug_logging LibreChat has central logging built into its backend (api). Log files are saved in /api/logs. Error logs are saved by default. Debug logs are enabled by default but can be turned off if not desired. bool false no
libre_app_debug_plugins Enable debug mode for Libre App plugins. bool false no
libre_app_docker_image The Docker Image to use for the App Service. string "ghcr.io/danny-avila/librechat-dev-api:latest" no
libre_app_domain_client To use locally, set DOMAIN_CLIENT and DOMAIN_SERVER to http://localhost:3080 (3080 being the port previously configured).When deploying to a custom domain, set DOMAIN_CLIENT and DOMAIN_SERVER to your deployed URL, e.g. https://mydomain.example.com string "http://localhost:3080" no
libre_app_domain_server To use locally, set DOMAIN_CLIENT and DOMAIN_SERVER to http://localhost:3080 (3080 being the port previously configured).When deploying to a custom domain, set DOMAIN_CLIENT and DOMAIN_SERVER to your deployed URL, e.g. https://mydomain.example.com string "http://localhost:3080" no
libre_app_enable_meilisearch Enable Meilisearch bool false no
libre_app_endpoints endpoints and models selection. E.g. 'openAI,azureOpenAI,bingAI,chatGPTBrowser,google,gptPlugins,anthropic' string "azureOpenAI" no
libre_app_host he server will listen to localhost:3080 by default. You can change the target IP as you want. If you want to make this server available externally, for example to share the server with others or expose this from a Docker container, set host to 0.0.0.0 or your external IP interface. string "0.0.0.0" no
libre_app_jwt_refresh_secret JWT Refresh Secret string null no
libre_app_jwt_secret JWT Secret string null no
libre_app_mongo_uri The MongoDB Connection String to connect to. string null no
libre_app_name Name of the LibreChat App Service. string "librechatapp9000" no
libre_app_plugins_creds_iv Libre App Plugins Creds IV string null no
libre_app_plugins_creds_key Libre App Plugins Creds Key string null no
libre_app_port The host port to listen on. number 3080 no
libre_app_public_network_access_enabled Whether or not public network access is enabled. Defaults to false. bool true no
libre_app_title Add a custom title for the App. string "PrivateGPT" no
libre_app_violations Configuration for violations 
object({
    enabled                      = bool
    ban_duration                 = number
    ban_interval                 = number
    login_violation_score        = number
    registration_violation_score = number
    concurrent_violation_score   = number
    message_violation_score      = number
    non_browser_violation_score  = number
    login_max                    = number
    login_window                 = number
    register_max                 = number
    register_window              = number
    limit_concurrent_messages    = bool
    concurrent_message_max       = number
    limit_message_ip             = bool
    message_ip_max               = number
    message_ip_window            = number
    limit_message_user           = bool
    message_user_max             = number
    message_user_window          = number
  })
 
{
  "ban_duration": 7200000,
  "ban_interval": 20,
  "concurrent_message_max": 2,
  "concurrent_violation_score": 1,
  "enabled": true,
  "limit_concurrent_messages": true,
  "limit_message_ip": true,
  "limit_message_user": false,
  "login_max": 7,
  "login_violation_score": 1,
  "login_window": 5,
  "message_ip_max": 40,
  "message_ip_window": 1,
  "message_user_max": 40,
  "message_user_window": 1,
  "message_violation_score": 1,
  "non_browser_violation_score": 20,
  "register_max": 5,
  "register_window": 60,
  "registration_violation_score": 1
}
 no
libre_app_virtual_network_subnet_id The ID of the subnet, used to allow access to the App Service (priority 100), e.g. cosmosdb, meilisearch etc. If networking is created as part of the module, this will be automatically populated if value is 'null'. string null no
librechat_app_custom_dns_zone_name The DNS Zone to use for the App Service. string "domain.com" no
librechat_app_custom_domain_name The custom domain to use for the App Service. string "privategpt" no
location Azure region where resources will be hosted. string "uksouth" no
oai_account_name The name of the OpenAI service. string "az-openai-account" no
oai_custom_subdomain_name The subdomain name used for token-based authentication. Changing this forces a new resource to be created. (normally the same as the account name) string "demo-account" no
oai_customer_managed_key type = object({
key_vault_key_id = (Required) The ID of the Key Vault Key which should be used to Encrypt the data in this OpenAI Account.
identity_client_id = (Optional) The Client ID of the User Assigned Identity that has access to the key. This property only needs to be specified when there're multiple identities attached to the OpenAI Account.
})		 
object({
    key_vault_key_id   = string
    identity_client_id = optional(string)
  })
 null no
oai_dynamic_throttling_enabled Whether or not dynamic throttling is enabled. Defaults to true. bool true no
oai_fqdns A list of FQDNs to be used for token-based authentication. Changing this forces a new resource to be created. list(string) [] no
oai_identity type = object({
type = (Required) The type of the Identity. Possible values are SystemAssigned, UserAssigned, SystemAssigned, UserAssigned.
identity_ids = (Optional) Specifies a list of User Assigned Managed Identity IDs to be assigned to this OpenAI Account.
})		 
object({
    type         = string
    identity_ids = optional(list(string))
  })
 
{
  "type": "SystemAssigned"
}
 no
oai_local_auth_enabled Whether local authentication methods is enabled for the Cognitive Account. Defaults to true. bool true no
oai_model_deployment type = list(object({
deployment_id = (Required) The name of the Cognitive Services Account Model Deployment. Changing this forces a new resource to be created.
model_name = {
model_format = (Required) The format of the Cognitive Services Account Deployment model. Changing this forces a new resource to be created. Possible value is OpenAI.
model_name = (Required) The name of the Cognitive Services Account Deployment model. Changing this forces a new resource to be created.
model_version = (Required) The version of Cognitive Services Account Deployment model.
}
sku = {
sku_name = (Required) The name of the SKU. Possible values include Standard, GlobalBatch, GlobalStandard and ProvisionedManaged.
sku_tier = (Optional) Possible values are Free, Basic, Standard, Premium, Enterprise. Changing this forces a new resource to be created.
sku_size = (Optional) The SKU size. When the name field is the combination of tier and some other value, this would be the standalone code. Changing this forces a new resource to be created.
sku_family = (Optional) If the service has different generations of hardware, for the same SKU, then that can be captured here. Changing this forces a new resource to be created.
sku_capacity = (Optional) Tokens-per-Minute (TPM). If the SKU supports sku out/in then the capacity integer should be included. If sku out/in is not possible for the resource this may be omitted. Default value is 1. Changing this forces a new resource to be created.
}
rai_policy_name = (Optional) The name of RAI policy. Changing this forces a new resource to be created.
}))		 
list(object({
    deployment_id   = string
    model_name      = string
    model_format    = string
    model_version   = string
    sku_name        = string
    sku_tier        = optional(string)
    sku_size        = optional(number)
    sku_family      = optional(string)
    sku_capacity    = optional(number)
    rai_policy_name = optional(string)
  }))
 [] no
oai_network_acls type = set(object({
default_action = (Required) The Default Action to use when no rules match from ip_rules / virtual_network_rules. Possible values are Allow and Deny.
ip_rules = (Optional) One or more IP Addresses, or CIDR Blocks which should be able to access the Cognitive Account.
virtual_network_rules = optional(set(object({
subnet_id = (Required) The ID of a Subnet which should be able to access the OpenAI Account.
ignore_missing_vnet_service_endpoint = (Optional) Whether ignore missing vnet service endpoint or not. Default to false.
})))
}))		 
set(object({
    default_action = string
    ip_rules       = optional(set(string))
    virtual_network_rules = optional(set(object({
      subnet_id                            = string
      ignore_missing_vnet_service_endpoint = optional(bool, false)
    })))
  }))
 null no
oai_outbound_network_access_restricted Whether or not outbound network access is restricted. Defaults to false. bool false no
oai_public_network_access_enabled Whether or not public network access is enabled. Defaults to false. bool true no
oai_sku_name SKU name of the OpenAI service. string "S0" no
oai_storage type = list(object({
storage_account_id = (Required) Full resource id of a Microsoft.Storage resource.
identity_client_id = (Optional) The client ID of the managed identity associated with the storage resource.
}))		 
list(object({
    storage_account_id = string
    identity_client_id = optional(string)
  }))
 [] no
resource_group_name Name of the resource group to create the OpenAI service / or where an existing service is hosted. string n/a yes
subnet_config A list of subnet configuration objects to create subnets in the virtual network. 
object({
    subnet_name                                   = string
    subnet_address_space                          = list(string)
    service_endpoints                             = list(string)
    private_endpoint_network_policies_enabled     = string
    private_link_service_network_policies_enabled = bool
    subnets_delegation_settings = map(list(object({
      name    = string
      actions = list(string)
    })))
  })
 
{
  "private_endpoint_network_policies_enabled": "Enabled",
  "private_link_service_network_policies_enabled": false,
  "service_endpoints": [
    "Microsoft.AzureCosmosDB",
    "Microsoft.Web"
  ],
  "subnet_address_space": [
    "10.4.0.0/24"
  ],
  "subnet_name": "app-cosmos-sub",
  "subnets_delegation_settings": {
    "app-service-plan": [
      {
        "actions": [
          "Microsoft.Network/virtualNetworks/subnets/action"
        ],
        "name": "Microsoft.Web/serverFarms"
      }
    ]
  }
}
 no
tags A map of key value pairs that is used to tag resources created. map(string) {} no
use_cosmosdb_free_tier Whether to enable the free tier for the Cosmos DB account. This needs to be false if another instance already uses free tier. bool true no
virtual_network_name Name of the virtual network where resources are attached. string "openai-vnet-9000" no
vnet_address_space value of the address space for the virtual network. list(string) 
[
  "10.4.0.0/24"
]
 no

Outputs

Name Description
app_service_default_hostname The default hostname of the App Service
app_service_id The ID of the App Service
app_service_name The name of the App Service
app_service_outbound_ip_addresses The outbound IP addresses of the App Service
app_service_plan_id The ID of the App Service Plan
app_service_plan_name The name of the App Service Plan
cognitive_deployment_ids The IDs of the OpenAI Cognitive Account Model Deployments
cognitive_deployment_names The names of the OpenAI Cognitive Account Model Deployments
cosmosdb_account_endpoint The endpoint used to connect to the Cosmos DB account
cosmosdb_account_id The ID of the Cosmos DB account
cosmosdb_account_name The name of the Cosmos DB account
cosmosdb_account_primary_key The primary master key for the Cosmos DB account
cosmosdb_account_secondary_key The secondary master key for the Cosmos DB account
key_vault_id The ID of the Key Vault
key_vault_name The name of the Key Vault
key_vault_uri The URI of the Key Vault
openai_endpoint The endpoint used to connect to the Cognitive Service Account.
openai_primary_key The primary access key for the Cognitive Service Account.
openai_secondary_key The secondary access key for the Cognitive Service Account.
openai_subdomain The subdomain used to connect to the Cognitive Service Account.
subnet_id The ID of the subnet
subnet_name The name of the subnet
virtual_network_id The ID of the virtual network
virtual_network_name The name of the virtual network
Expand
Additional Information
Related Applications
Recommended for You
Related Information All