Adding Azure Search parameters to the payload in the AOAI Chat

[joergrau]

Describe the issue

Small product enhancement:
The Codeunit "AOAI Chat Completion Params" uses the Codeunit "AOAI Chat Compl Params Impl", which adds parameters to the payload:
procedure AddChatCompletionsParametersToPayload(var Payload: JsonObject)
begin
if GetMaxTokens() > 0 then
Payload.Add('max_tokens', GetMaxTokens());
Payload.Add('temperature', GetTemperature());
Payload.Add('presence_penalty', GetPresencePenalty());
Payload.Add('frequency_penalty', GetFrequencyPenalty());
end;

There is currently no option to specify Azure Search parameters. The parameters I would like to add are:
"azureSearchEndpoint"
"azureSearchKey"
"azureSearchIndexName"

Expected behavior

Adding additional procedures to set and get the new Azure Seach parameters. Adding the parameters to the payload if they were set.

Steps to reproduce

Currently no procedures available to set the Azure Search parameters.

Additional context

As discussed on the yammer group: https://www.yammer.com/dynamicsnavdev/#/threads/inGroup?type=in_group&feedId=132956700672&view=all

I will provide a fix for a bug

• I will provide a fix for a bug

[darjoo]

Hi @joergrau, this is something we'd accept :)
I need a bit more clarification about your implementation. Is this purely just adding the parameters to enable the it to perform Azure AI Search or would there be additions to the API/Objects that make it easier to work with the information returned as well?

-Darrick

[joergrau]

Hi @darjoo,
It is for adding more procedures for set/get the three parameters and adding them to the payload like this:

procedure AddChatCompletionsParametersToPayload(var Payload: JsonObject)
    begin
        if GetMaxTokens() > 0 then
            Payload.Add('max_tokens', GetMaxTokens());

        Payload.Add('temperature', GetTemperature());
        Payload.Add('presence_penalty', GetPresencePenalty());
        Payload.Add('frequency_penalty', GetFrequencyPenalty());

        if IsJsonMode() then
            Payload.Add('response_format', GetJsonResponseFormat());

        if GetAzureSearchEndpoint() <> '' then
            Payload.Add('azureSearchEndpoint', GetAzureSearchEndpoint());
        if GetAzureSearchKey() <> '' then
            Payload.Add('azureSearchKey', GetAzureSearchKey());
        if GetAzureSearchIndexName() <> '' then
            Payload.Add('azureSearchIndexName', GetAzureSearchIndexName());
    end;

So, following the same way as for the other parameters, e.g. temperature

[darjoo]

@joergrau
Just adding those parameters do not seem correct though. I have not tested and only reviewed the documentation and API specs.
It seems like it should be added through the DataSource key, with a specific type etc.

https://learn.microsoft.com/en-us/azure/ai-services/openai/references/on-your-data?tabs=python#data-source
https://github.com/Azure/azure-rest-api-specs/blob/main/specification/cognitiveservices/data-plane/AzureOpenAI/inference/stable/2024-02-01/inference.json

And even once that works, it would be better for all if we can work together to design how the output can be worked with. For example with Operation Response/Function Response. Otherwise, everyone who uses this will have to develop their own parsing mechanism which makes little sense when it should be part of the AI module.

[joergrau]

@darjoo I already did the coding as it is not much work. However, I struggle to get this tested as I can't do any testing around AI in a Docker container. I tried to follow the process explained here LOCAL_DEV_ENV.md get an error message:

when running the NewDevEnv.ps1 script.

What is the best way to deploy this updated System App and to try it out?

[JesperSchulz]

@mazhelez / @aholstrup1, could you look into the script issue?

[mazhelez]

@joergrau I see codeunit "Library - Password" was added recently. Are you maybe not running against the latest code or latest artifact?

[joergrau]

@mazhelez I synced my fork and updated my branch, but I got the same error.

[JesperSchulz]

@darjoo / @joergrau, where are we on this one?

[joergrau]

@JesperSchulz I'm waiting for your assistance. Alternatively, please advise on how I could deploy a new system app to test AI integration. The coding is done, I "just" need to test it.