6/20/2023 Admin

Implementing Recursive ChatGPT Function Calling in Blazor

Microsoft Blazor is a web framework that allows developers to build interactive web user interfaces using C#. In this blog post, we will explore how to implement recursive ChatGPT function calling in Microsoft Blazor.

One may ask, what is the difference between this, and having ChatGPT create .json like Blazor OpenAI Configurator? The difference is that ChatGPT can call the C# code rather than just provide the .json that indicated what needed to be updated.

Video

https://youtu.be/5W9kbtgFg-4

Rag Pattern VS Functions

As described in the article: Use a Poor Developers Vector Database to Implement The Retrieval Augmented Generation (RAG) pattern is a technique for building natural language generation systems that can retrieve and use relevant information from external sources.

The concept is to first retrieve a set of passages that are related to the search query, then use them to supply grounding to the prompt, to finally generate a natural language response that incorporates the retrieved information.

Using OpenAI Functions can achieve the same results, but rather than pushing grounding to ChatGPT it calls functions to pull in the grounding information.

In addition, ChatGPT can call these functions to perform external actions such as updating a database, sending an email, or triggering a service to turn off a light bulb.

ChatGPT is able to make multiple recursive function calls to obtain information and remains in control of the overall orchestration. This can eliminate the need for LangChain or other AI Agent frameworks.

Sample Application

The sample application behaves like the standard ChatGPT application in that it can provide responses in a conversation format and track what is discussed so you don’t have to repeat information previously covered in the conversation.

However, this application has three functions defined:

AddTodo – Allows a item to be added to the list
DeleteTodo – Allows an item to be deleted from the list
GetTodos – Retrieves the items in the list

The end user is able to instruct ChatGPT to maintain the list in normal conversational language. ChatGPT makes the decision when to call the functions.

Yes ChatGPT can manage a list without using functions, but since ChatGPT has a limited memory capacity, it would start to forget items. This method will effectively provide ChatGPT with unlimited memory.

The Code

We start with code adapted from the article: Create a C# OpenAI ChatGPT Plugin.

This provides a simple class that allows the TODO items to be maintained:

    public class TodosService
    {
        private static readonly List<string> _TODOS = new List<string>();
        public string AddTodo(string NewToDO)
        {
            _TODOS.Add(NewToDO);
            return $"{NewToDO} added";
        }
        public string GetTodos()
        {
            return JsonSerializer.Serialize<List<string>>(_TODOS);
        }
        public string DeleteTodo(int TodoIdxInt)
        {
            if (TodoIdxInt >= 0 && TodoIdxInt < _TODOS.Count)
            {
                _TODOS.RemoveAt(TodoIdxInt);
                return $"TODO {TodoIdxInt} deleted";
            }
            else
            {
                return "TODO not found";
            }
        }
    }
    public class ToDoAddRequest
    {
        public Todorequest TodoRequest { get; set; }
    }
    public class Todorequest
    {
        public string todo { get; set; }
    }
    public class ToDoRemoveRequest
    {
        public Todoindexrequest TodoIndexRequest { get; set; }
    }
    public class Todoindexrequest
    {
        public int todoIdx { get; set; }
    }

We open the Program.cs file and register the class as a Service:

    builder.Services.AddScoped<TodosService>();

The remaining code is contained in the Index.razor page.

Chat UI

The chat user interface was covered in the article: Build Your Own ChatGPT Client in Blazor.

CallChatGPT

When the end user enters a prompt and clicks the Call ChatGPT button, the following code runs:

            // Set Processing to true to indicate that the method is processing
            Processing = true;
            // Call StateHasChanged to refresh the UI
            StateHasChanged();
            // Clear any previous error messages
            ErrorMessage = "";
            // Create a new OpenAIClient object
            // with the provided API key and organization
            var api = 
            new OpenAIClient(new OpenAIAuthentication(ApiKey, Organization));
            // Create a collection of chatPrompts
            List<Message> chatPrompts = new List<Message>();
            // Add the existing Chat messages to chatPrompts
            chatPrompts = AddExistingChatMessags(chatPrompts);

This calls the AddExistingChatMessages method that retrieves all the previous saved messages in the conversation and strips out older messages so that we don’t exceed the amount of space that the ChatGPT model allows:

    private List<Message> AddExistingChatMessags(List<Message> chatPrompts)
    {
        // Create a new LinkedList of ChatMessages
        LinkedList<ChatMessage> ChatPromptsLinkedList = new LinkedList<ChatMessage>();
        // Loop through the ChatMessages and add them to the LinkedList
        foreach (var item in ChatMessages)
        {
            ChatPromptsLinkedList.AddLast(item);
        }
        // Set the current word count to 0
        CurrentWordCount = 0;
        // Reverse the chat messages to start from the most recent messages
        foreach (var item in ChatPromptsLinkedList.Reverse())
        {
            if (item.Prompt != null)
            {
                int promptWordCount = item.Prompt.Split(
                    new char[] { ' ', '\t', '\n', '\r' },
                    StringSplitOptions.RemoveEmptyEntries).Length;
                if (CurrentWordCount + promptWordCount >= 1000)
                {
                    // This message would cause the total to exceed 1000 words,
                    // so break out of the loop
                    break;
                }
                // Add the message to the chat prompts
                chatPrompts.Insert(
                    0, 
                    new Message(item.Role, item.Prompt, item.FunctionName));
                CurrentWordCount += promptWordCount;
            }
        }
        // Add the first message to the chat prompts to indicate the System message
        chatPrompts.Insert(0,
            new Message(
                Role.System,
                @"You are helpful Assistant that only outputs text or html never markdown.
                You never include links to articles or blog posts, only the name."
            )
        );
        return chatPrompts;
    }

Next it ads the current prompt:

            // Add the new message to chatPrompts
            chatPrompts.Add(new Message(Role.User, prompt));

Defining The Functions

The functions, available to ChatGPT, are defined using the following code:

var DefinedFunctions = new List<Function>
{
    new Function(
        "Todos_POST",
        @"Creates a new TODO item.
            Use this function to add a new TODO item to the list.".Trim(),
        new JsonObject
        {
            ["type"] = "object",
            ["properties"] = new JsonObject
            {
                ["TodoRequest"] = new JsonObject
                {
                    ["type"] = "object",
                    ["properties"] = new JsonObject
                    {
                        ["todo"] = new JsonObject
                        {
                            ["type"] = "string",
                            ["description"] = @"The TODO item to be added."
                        }
                    },
                    ["required"] = new JsonArray { "todo" }
                }
            },
            ["required"] = new JsonArray { "TodoRequest" }
        }),
    new Function(
        "Todos_GET",
        @"Retrieves the TODO list.
            Use this function to view the TODO list.".Trim(),
        new JsonObject
        {
            ["type"] = "object",
            ["properties"] = new JsonObject {}
        }),
    new Function(
        "Todos_DELETE",
        @"Deletes a specific TODO item from the list.
            Use this function to remove a TODO item from the list.".Trim(),
        new JsonObject
        {
            ["type"] = "object",
            ["properties"] = new JsonObject
            {
                ["TodoIndexRequest"] = new JsonObject
                {
                    ["type"] = "object",
                    ["properties"] = new JsonObject
                    {
                        ["todoIdx"] = new JsonObject
                        {
                            ["type"] = "integer",
                            ["description"] = @"The index of the TODO item to be deleted."
                        }
                    },
                    ["required"] = new JsonArray { "todoIdx" }
                }
            },
            ["required"] = new JsonArray { "TodoIndexRequest" }
        })
};

Call ChatGPT

The following code calls ChatGPT and adds the latest prompt to the ChatMessages list so that it can later be displayed in the Chat UI:

            // Call ChatGPT
            // Create a new ChatRequest object with the chat prompts and pass
            // it to the API's GetCompletionAsync method
            var chatRequest = new ChatRequest(
                chatPrompts,
                functions: DefinedFunctions,
                functionCall: "auto",
                model: "gpt-3.5-turbo-0613", // Must use this model or higher
                temperature: 0.0,
                topP: 1,
                frequencyPenalty: 0,
                presencePenalty: 0);
            var result = await api.ChatEndpoint.GetCompletionAsync(chatRequest);
            // Create a new Message object with the user's prompt and other
            // details and add it to the messages list
            ChatMessages.Add(new ChatMessage
                {
                    Prompt = prompt,
                    Role = Role.User,
                    Tokens = result.Usage.PromptTokens ?? 0
                });

Notice it passes the functions, defined earlier, to the functions property. The functionCall property is currently set to “auto” which means ChatGPT can decide if it wants to call a function in response to this prompt. This can be set to instruct ChatGPT to call a specific defined function, or not to call any functions at all.

Calling The Function

The following block of code processes the result from ChatGPT.

If ChatGPT wants to call a function, we call the ExecuteFunction method.

We do this in a While loop because ChatGPT may want to call multiple functions based on the prompt. For example, to add multiple items, it will call the Todos_POST function multiple times.

        // See if as a response ChatGPT wants to call a function
        if (result.FirstChoice.FinishReason == "function_call")
        {
            // Chat GPT wants to call a function
            // To allow ChatGPT to call multiple functions
            // We need to start a While loop
            bool FunctionCallingComplete = false;
            while (!FunctionCallingComplete)
            {
                // Call the function
                chatPrompts = ExecuteFunction(result, chatPrompts);
                // Get a response from ChatGPT (now that is has the results of the function)
                chatRequest = new ChatRequest(
                    chatPrompts,
                    functions: DefinedFunctions,
                    functionCall: "auto",
                    model: "gpt-3.5-turbo-0613", // Must use this model or higher
                    temperature: 0.0,
                    topP: 1,
                    frequencyPenalty: 0,
                    presencePenalty: 0);
                result = await api.ChatEndpoint.GetCompletionAsync(chatRequest);
                if (result.FirstChoice.FinishReason == "function_call")
                {
                    // Keep looping
                    FunctionCallingComplete = false;
                }
                else
                {
                    // Break out of the loop
                    FunctionCallingComplete = true;
                }
            }
        }
        else
        {
            // ChatGPT did not want to call a function                
        }

This is the ExecuteFunction method:

private List<Message> ExecuteFunction(
    ChatResponse ChatResponseResult, List<Message> ParamChatPrompts)
{
    // Get the arguments
    var functionArgs = 
    ChatResponseResult.FirstChoice.Message.Function.Arguments.ToString();
    // Get the function name
    var functionName = ChatResponseResult.FirstChoice.Message.Function.Name;
    // Variable to hold the function result
    string functionResult = "";
    //Use select case to call the function
    switch (functionName)
    {
        case "Todos_POST":
            var NewTODO = 
            JsonSerializer.Deserialize<ToDoAddRequest>(functionArgs);
            if (NewTODO != null)
            {
                functionResult = _TodosService.AddTodo(NewTODO.TodoRequest.todo);
            }
            break;
        case "Todos_GET":
            functionResult = _TodosService.GetTodos();
            break;
        case "Todos_DELETE":
            var DeleteTODO = 
            JsonSerializer.Deserialize<ToDoRemoveRequest>(functionArgs);
            if (DeleteTODO != null)
            {
                functionResult = 
                _TodosService.DeleteTodo(DeleteTODO.TodoIndexRequest.todoIdx);
            }
            break;
        default:
            break;
    }
    // Create a new Message object with the user's prompt and other
    // details and add it to the messages list
    ChatMessages.Add(new ChatMessage
        {
            Prompt = functionResult,
            Role = Role.Function,
            FunctionName = functionName,
            Tokens = ChatResponseResult.Usage.PromptTokens ?? 0
        });
    // Call ChatGPT again with the results of the function
    ParamChatPrompts.Add(
        new Message(Role.Function, functionResult, functionName)
    );
    return ParamChatPrompts;
}

Finally it adds ChatGPT’s response to the ChatMessages list so that it can be displayed in the Chat UI:

            // Create a new Message object with the response and other details
            // and add it to the messages list
            ChatMessages.Add(new ChatMessage
                {
                    Prompt = result.FirstChoice.Message,
                    Role = Role.Assistant,
                    Tokens = result.Usage.CompletionTokens ?? 0
                });
            // Update the total number of tokens used by the API
            TotalTokens = TotalTokens + result.Usage.TotalTokens ?? 0;