[Python] Add an image in prompt for AzureOpenAI gpt4-mini?

I am able to use the web interface of azure OpenAI studio in the chat playground to analyze images but I would like to do the same using python. It seems that it's not working and I could not (so far) find a reference online on how to include an image in the prompt. Could anyone please help or provide a reference on how I can do this? Do I actually need to upload my image or would it be possible for the model to read a local image. ?

import os
from openai import AzureOpenAI

client = AzureOpenAI(
api_key=os.getenv("AZURE_OPENAI_API_KEY"),
api_version=os.getenv("AZURE_API_VERSION"),
azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
)

deployment_name = "gpt-4o-mini"
# Send a completion call to generate an answer
print("Sending a test completion job")
#image local path
image_input = r"c:/users/..../image.jpeg"
prompt = "Tell me what do you see in this image  "

messages = [{"role": "user", "content": prompt}]
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
temperature=0,
)
generated_text = response.choices[0].message.content
print("Response:", generated_text)


Update: I uploaded also the image to the blob storage and I replaced the local path with the URL, I still get the same response from the model:

Response: It seems that I can't view images directly. However, if you describe the image or provide details about its content, I can help analyze it or provide insights based on your description!

Continue reading...