Story Book Cabin Blazor Godot

Story Book Cabin is an interactive Blazor Godot project that allows users to drag and drop characters into a virtual environment. Users can give natural language prompts to an AI, which interprets the instructions and moves characters accordingly.

You can play the game here at: https://storybookcabinpocblazor.azurewebsites.net

The Game

Drag and drop one or more characters  into the level and fill in the character info.

Give the AI a prompt such as “Alan (character name) likes to read” and the AI will understand the prompt and move Alan to the book shelf.

How It Works
1. User Action: The player drags characters into the scene and submits a prompt.
2. Data Transmission: Godot sends the prompt, game board state, and character data to the web application.
3. AI Processing: The web app formats the data and sends it to OpenAI for processing.
4. Receiving the Response: The AI responds with updated character positions.
5. Updating the Scene: Godot parses this response and moves the characters accordingly

Sending Game Data from Godot to the Web Application

func _on_button_pressed():
	var user_name = paramUserName
	var http_token = paramHTTPToken
	var user_text = promptLine.text
	var http_request = $HTTPRequest
	
	var game_board = Global.GlobalDic
	var character_info = Global.CharacterDic
	
	var url = WebApplicationURL + "ReceiveCallFromGodot/"
	var body_dict = {
		"userName": user_name,
		"hTTPToken": http_token,
		"userText": user_text,
		"gameBoard": game_board,
		"characterInfo": character_info
	}
	
	var json = JSON.new()
	var body = json.stringify(body_dict)
	var headers = ["Content-Type: application/json"]
	http_request.request_completed.connect(self._on_request_completed)
	var error = http_request.request(url, headers, HTTPClient.METHOD_POST, body)
	if error != OK:
		print("Error sending request: ", error)
	else:
		# Display the user's message in the chat area
		add_message(user_text, true)

When the prompt is sent, this function gathers and sends a collection of important data from Godot to the web application. The data includes:

1. User Prompt:

   • The actual text input from the user, representing the command or instruction they want the AI to process.

2. Game State:

   • Game Board Data: Represents the current layout of the game. The gameboard is a dictionary of tiles in the game. Each tile has coordinates, a type (empty, object, wall) determining if a character can enter the tile, and a description of the tile.

Ex: {"Type":"object","description":"Staircase down entry"},"(120, 8)":{"Type":"empty","description":"Empty area a character can occupy"},"(120, 88)":{"Type":"wall","description":"A wall"},

• Character Information: A dictionary of all characters in the level. Stores the current position of the character and all of the character info.

Ex: {"(-120, -24)":{"age":0,"description":"","name":"Alan","gender":"Male"}}

All this information is sent to the web application where it is processed and receives the Open AI response.

Processing Game Data with AI

private async Task<OpenAI.Chat.Message> GetOpenAIResponse(UserData userData)
{
    // Get the API key from the appsettings.json file
    var ApiKey = _openAIServiceOptions.ApiKey;
    // Create a new instance of OpenAIClient using the ApiKey
    var api = new OpenAIClient(new OpenAIAuthentication(ApiKey));
    // Serialize 
    string jsonGameBoard = JsonSerializer.Serialize(userData.GameBoard);
    string jsonCharacterInfo = JsonSerializer.Serialize(userData.CharacterInfo);
    StringBuilder sb = new StringBuilder();
    sb.AppendLine("Please examine the following JSON that represents the #CharacterInfo and their coordinates on the gameboard:");
    sb.AppendLine(jsonCharacterInfo);
    sb.AppendLine("");
    sb.AppendLine("Please examine the following JSON that represents the gameboard:");
    sb.AppendLine(jsonGameBoard);
    sb.AppendLine("");
    sb.AppendLine("Please do the following based on the #PlayerInstructions.:");
    sb.AppendLine("");
    sb.AppendLine("#1 - Only output an updated #CharacterInfo json object that updates the character coordinates on the gameboard ");
    sb.AppendLine("based on the #PlayerInstructions. ");
    sb.AppendLine("");
    sb.AppendLine("#2 - Only allow #CharacterInfo to put a character in a gameboard coordinate that is Type empty.");
    sb.AppendLine("");
    sb.AppendLine("#3 - A #CharacterInfo coordinate cannot occupy the same coordinate as another character.");
    sb.AppendLine("");
    sb.AppendLine("#4 - A #CharacterInfo can only be listed once in the JSON return");
    sb.AppendLine("");
    sb.AppendLine("#5 - Only update #CharacterInfo coordinates");
    sb.AppendLine("");
    sb.AppendLine("#Player Instructions: ");
    sb.AppendLine(userData.UserText);
    var messages = new List<OpenAI.Chat.Message>
    {
        new OpenAI.Chat.Message(Role.System, "You are a helpful assistant that responds in JSON format."),
        new OpenAI.Chat.Message(Role.User, sb.ToString())
    };
    // Create a new instance of the ChatRequest class, ensuring we ask for JSON format
    var chatRequest = new OpenAI.Chat.ChatRequest(
        messages,
        temperature: 0.1,
        model: Model.GPT4o
    );
    var chatResponse = await api.ChatEndpoint.GetCompletionAsync(chatRequest);
    // Return the JSON response
    return chatResponse.FirstChoice.Message;
}

This code handles the communication between the web application and OpenAI. When data is received from Godot (including the game board, character information, and user prompt), the function prepares this data, structures it into a detailed prompt, and sends it to the AI.

The AI is instructed to analyze the current game state and the user's prompt, then respond with an updated set of character positions in JSON format. The rules embedded in the prompt ensure that characters move logically, avoiding overlaps and only occupying valid spaces on the game board.

In summary, this code acts as the bridge that takes the current game state, applies the user's instructions, and uses AI to calculate the new positions of characters before sending the updated data back to Godot.

Updating Character Positions Based on AI Response

func _on_request_completed(result, response_code, headers, body):
	if response_code == 200:
		var body_string = body.get_string_from_utf8()
	
		var json = JSON.new()
		var AIResponse = json.parse_string(body_string)
		
		if "message" in AIResponse:
			AIResponse = AIResponse["message"]
		else:
			add_message("Can not get AIMessage", false)
			return
		
		print("Raw response body: ", AIResponse)
		var clean_json = AIResponse.replace("```json", "").replace("```", "").strip_edges()
		var parsed_data  = JSON.parse_string(clean_json)
		
		if parsed_data != null:
			add_message(AIResponse, false)
		else:
			print("Unexpected JSON structure")
		
		var num_keys = parsed_data.size()
		
		for i in range(num_keys):
			var coordKey = parsed_data.keys()[i - 1]
			var characterData = parsed_data[coordKey]
			var chrName = characterData.get("name", "")
			var found_character = level.get_node(chrName)
			
			if found_character:
				var coords = coordKey.trim_prefix("(").trim_suffix(")")
				var coordinates = coords.split(", ")
				var coordX = coordinates[0].to_float() 
				var coordY = coordinates[1].to_float() 
				found_character.global_position = Vector2(coordX, coordY) 
				print("Moved character Alan to new position")
			else:
				print("Character not found")

This function processes the AI's response to update character positions in the game. Once the response is received, it extracts the updated coordinates for each character. For every character listed:

1. Identifies the Character:
   It matches the character's name from the response with the corresponding character node in the game scene.

2. Parses the Coordinates:
   The new coordinates are extracted from the response and converted into position values that Godot can understand.

3. Moves the Character:
   The character’s position is updated on the game board using the new coordinates, reflecting the AI’s decisions based on the player's prompt.

In summary, this function ensures that characters in the game are repositioned accurately according to the AI-generated data.