[docs] How to host a azure open ai using docker

[EZHU331]

Description
This document aims to guide users on deploying Azure OpenAI Service within a Docker container. The current documentation lacks a comprehensive, step-by-step guide for setting up Azure OpenAI in a Dockerized environment, including authentication, configuration, and deployment processes.

Current documentation
Currently, there is limited or no specific guidance on hosting Azure OpenAI using Docker in the Azure OpenAI Service documentation.

Suggested changes
Add a dedicated section or guide that covers:

1. Prerequisites

   • Docker installation and setup.
   • Azure account with the necessary permissions for OpenAI service.
   • API keys for authentication.

2. Dockerfile Configuration

   • A sample Dockerfile to set up an environment with the required dependencies.
   • Instructions on installing necessary libraries such as the Azure SDK for Python, if applicable.

3. Environment Variables

   • Configuration of environment variables to securely pass Azure OpenAI API keys and other configuration settings.

4. Sample Docker Compose File

   • A docker-compose.yml file example to demonstrate how to set up multi-container applications if additional services are needed.

5. Running the Container

   • Step-by-step commands for building and running the Docker container.
   • Guidance on exposing necessary ports and accessing the service.

6. Testing and Verification

   • Instructions for verifying successful deployment, including sample API calls to Azure OpenAI.

7. Security Best Practices

   • Suggestions for securely handling API keys, using .env files, and avoiding hardcoding credentials.

Additional context
Adding this section will help users deploy Azure OpenAI Service using Docker more efficiently and securely, improving accessibility for development and production deployments. Screenshots or command line examples would be beneficial for each step.

Checklist

• I have checked that this issue hasn't already been reported.
• I have checked the latest version of the documentation to ensure this issue still exists.
• For simple typos or fixes, I have considered submitting a pull request instead.

[JosephCatrambone]

Thank you for the report. I think this makes sense. There is a little bit on using Azure with OpenAI here: https://www.guardrailsai.com/docs/how_to_guides/using_llms#azure-openai

That should cover the extra environment variables that one needs to set to use things with Azure. Additionally, for Docker deployments, there's this set of docs: https://www.guardrailsai.com/docs/how_to_guides/hosting_with_docker

I'm not sure if that will fully answer the questions you have, but it's perhaps a starting point while we figure out how to make the documentation clearer.

[EZHU331]

Given the exisiting documentation:
https://www.guardrailsai.com/docs/how_to_guides/hosting_with_docker
https://www.guardrailsai.com/docs/getting_started/guardrails_server

The below API only works for standard open ai; what would be the procedure to revise below code so it will work with azure open ai?

from openai import OpenAI

client = OpenAI(
base_url='http://127.0.0.1:8000/guards/gibberish_guard/openai/v1',
)

response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{
"role": "user",
"content": "Make up some gibberish for me please!"
}]
)

print(response.choices[0].message.content)
print(response.guardrails['validation_passed'])

[JosephCatrambone]

Assuming your Azure environment variables are set according to the linked documentation, it should only be a matter of

#Assumes "AZURE_API_KEY", "AZURE_API_BASE", and AZURE_API_VERSION" are set.
from guardrails import Guard
from guardrails.hub import GibberishText

guard = Guard().use(
    GibberishText, threshold=0.5, validation_method="sentence", on_fail="exception"
)

result = guard(
    model="azure/<your_deployment_name>",
    messages=[
        {"role":"user", "content":"Make up some gibberish for me, please?"}
    ],
)

[EZHU331]

Yes, this works with guard() but does not work when using as a server:
http://127.0.0.1:8000/guards/gibberish_guard**/openai/v1**

Any idea on how to modify this link, assume up on docker creation the env has "AZURE_API_KEY", "AZURE_API_BASE", and AZURE_API_VERSION"

[JosephCatrambone]

Aha! I think I see the difficulty. It's worth trying from openai import AzureOpenAI. I'm struggling a bit to test this because of resource availability, but in theory that should be API-compatible with OpenAI and would be the only real change.

from openai import AzureOpenAI

client = AzureOpenAI(
  azure_endpoint = 'http://127.0.0.1:8000/guards/gibberish_guard/openai/v1'
  azure_ad_token_provider="your token provider",
  api_version="2024-09-01-preview"
)

response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{
        "role": "user",
        "content": "Make up some gibberish for me please!"
    }]
)

I'll update this comment again if I can verify that this works.

EDIT: For reference, I'm running over Microsoft's Azure OpenAI Service here: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=python-secure%2Cglobal-standard%2Cstandard-chat-completions

[EZHU331]

Using 'http://127.0.0.1:8000/guards/gibberish_guard/openai/v1' will simply return the return the URL is not found in this setting.

I'm also trying to follow your guide for cloud environment setting, but the documentation is outdated:
https://www.guardrailsai.com/docs/how_to_guides/continuous_integration_continuous_deployment

This command is no longer available: guardrails create --template hub:template://guardrails/chatbot

The json file cannot be installed either:

{
  "name": "chatbot",
  "description": "guard to validate chatbot output",
  "template_version": "0.0.1",
  "namespace": "guardrails",
  "guards": [
    {
      "id": "chatbot",
      "name": "chatbot",
      "validators": [
        {
          "id": "guardrails/detect_pii",
          "on": "$",
          "onFail": "exception",
          "kwargs": {
            "pii_entities": ["PERSON"]
          }
        }
      ]
    }
  ]
}

 => [7/8] COPY chatbot.json /app/chatbot.json                                              0.0s 
 => ERROR [8/8] RUN guardrails create --template /app/chatbot.json                       195.1s 
------                                                                                          
 > [8/8] RUN guardrails create --template /app/chatbot.json:                                    
1.134 Installing...                                                                             
1.136 Installing hub://guardrails/detect_pii...                                                 
193.2 ERROR:guardrails-cli:Failed to install guardrails-grhub-detect-pii                        
193.2 Exit code: 1                                                                              
193.2 stderr: error: subprocess-exited-with-error
193.2   
193.2   × pip subprocess to install build dependencies did not run successfully.
193.2   │ exit code: 1
193.2   ╰─> [391 lines of output]
193.2       Looking in indexes: https://__token__:****@pypi.guardrailsai.com/simple, https://pypi.org/simple
193.2       Ignoring numpy: markers 'python_version < "3.9"' don't match your environment
193.2       Collecting setuptools
193.2         Using cached setuptools-75.5.0-py3-none-any.whl.metadata (6.8 kB)
193.2       Collecting cython<3.0,>=0.25
193.2         Using cached Cython-0.29.37-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl.metadata (3.1 kB)
193.2       Collecting cymem<2.1.0,>=2.0.2

[JosephCatrambone]

Would it be possible to see a copy/paste of your config.py file (with the auth token redacted)? It should really be only a matter of those env variables.

I can look into the issue with --template, but I think that might be worth separating out as a different issue.

[OrenRachmil]

I'm encountering the same issue when attempting to use Azure OpenAI models. However, the standard OpenAI models are functioning properly.

[JosephCatrambone]

My access to Azure has strangely disappeared, but I haven't lost site of this issue.

I was seeing the "can't connect" matter with the OpenAI client, even outside of Guardrails. I posted a comment on their help thread: https://learn.microsoft.com/en-us/answers/questions/1315650/unable-to-log-into-azure-ai-studio-after-approved

When I get my access back I'll set up a local repro and be able to check everything in mroe depth.