10/19/2022 Admin

Blazor Video Creator


image

You can dynamically create videos with pictures and audio using Blazor.

The Application

image

When you run the application you will be presented with the UPLOAD IMAGE FILES button.

image

Select images from your computer and click the Open button.

image

Enter text for the video and click the Create Video File button.

image

The video will process.

The text will be turned into audio.

The pictures will be added to the video so that they evenly span the length of the audio.

The completed video will automatically download to your computer.

image

You can find the video file in the Downloads folder on your computer.

image

When you open the video file it will display the pictures and a voice will read the text.

Note: Open the video in a video player not a web browser like Chrome (it will not work).

Note: There is another project: carlfranklin/FFmpegBlazorDemo that does something similar, however, that code only works with Blazor WebAssembly. This works with Blazor Server.

How To Run The Code

image

To turn the text into audio you will need to obtain a Microsoft Speech Service key.

Go to: https://portal.azure.com/ and select Create a resource.

image

Search for Speech, select it and click Create.

image

After the service is created, select Keys and Endpoint and copy the Key and the Location/Region.

image

Download the project from the Downloads page on this site, unzip it, and open it in Visual Studio 2022 (or higher).

Open the appsettings.json file.

image

Update the SubscriptionKey and SpeechRegion properties with the values you copied earlier.

Hit F5 to build and run the project.

The Code

image

The project makes use of the FFMpegCore NuGet Package.

image

FFMpegCore provides a .Net Core wrapper around the FFMpeg project.

image

The executables were downloaded and added to the “bin” directory of the project, and the Build Action for each was set to Content and the Copy to Output Directory setting is set to Copy always.

image

Most of the code is contained in the Index.razor page.

								
									<
									RadzenUpload
									ChooseText="Upload Image Files"
									Multiple="true"
								
									Accept="image/*"
									AllowedExtensions=".png;"
								
									Url=@($"api/upload/multiple")
								
									Style="margin-bottom: 20px;height: 45px"
								
									Progress="@((args) => OnProgress(args))"
								
									Complete=@OnComplete
									/>
								

The Radzen Upload Control is used to allow the user to upload image files.

The upload is processed by the UploadController.cs file that creates a temporary folder, resizes the images, and returns the name of the temporary folder:

								
									using
									Microsoft.AspNetCore.Mvc;
								
									using
									System.Drawing;
								
									using
									System.Drawing.Imaging;
								

								
									namespace
									Speech
								
{
								
									[Route("api/[controller]")]
								
									[ApiController]
								
									public
									class
									UploadController : Controller
								
									{
								
									private
									readonly
									IWebHostEnvironment environment;
								

								
									public
									UploadController(IWebHostEnvironment environment)
								
									{
								
									this.environment = environment;
								
									}
								

								
									#region
									public
									async Task<IActionResult> MultipleAsync(IFormFile[] files)
								
									[HttpPost("[action]")]
								
									public
									async Task<IActionResult> MultipleAsync(
								
									IFormFile[] files)
								
									{
								
									try
								
									{
								
									// Create a 5 digit random number
								
									// to use a temp directory
								
									Random rnd =
									new
									Random();
								
									int
									randomNumber = rnd.Next(10000, 99999);
								
									string
									TempDirectory = randomNumber.ToString();
								
								
									string
									Processingpath =
								
									Path.Combine(
								
									environment.WebRootPath,
								
									"processing\\",
								
									TempDirectory);
								

								
									Directory.CreateDirectory(Processingpath);
								
								
									if
									(HttpContext.Request.Form.Files.Any())
								
									{
								
									foreach
									(var file
									in
									HttpContext.Request.Form.Files)
								
									{
								
									string
									FileName = file.FileName;
								

								
									string
									path =
								
									Path.Combine(
								
									Processingpath,
								
									FileName);
								

								
									using
									(var stream =
								
									new
									FileStream(path, FileMode.Create))
								
									{
								
									await file.CopyToAsync(stream);
								
									}
								
									}
								
								
									// Get all the files
								
									var AllFilesInDirectory =
								
									Directory.EnumerateFileSystemEntries(Processingpath);
								

								
									// Loop through picture and scale image
								
									foreach
									(var file
									in
									AllFilesInDirectory)
								
									{
								
									using
									(var image = Image.FromFile(file))
								
									{
								
									using
									(var newImage = ScaleImage(image, 500, 500))
								
									{
								
									try
								
									{
								
									var newImageName = Path.Combine(Processingpath,
								
									Path.GetFileNameWithoutExtension(file)
								
									+ "_resized" + ".png");
								
									newImage.Save(newImageName, ImageFormat.Png);
								
									}
								
									catch
								
									{
								
									continue;
								
									}
								
									}
								
									}
								
									}
								

								
									// Get all the non resized files
								
									var AllNonResizedFilesInDirectory =
								
									Directory.EnumerateFileSystemEntries(Processingpath)
								
									.Where(x => !x.Contains("_resized"));
								

								
									// Delete all the non resized files
								
									foreach
									(var file
									in
									AllNonResizedFilesInDirectory)
								
									{
								
									System.IO.File.Delete(file);
								
									}
								
									}
								

								
									// Let the index.razor page know the name of the temp directory
									
								
									// so it knows where to look for the images
								
									return
									StatusCode(200, TempDirectory);
								
									}
								
									catch
									(Exception ex)
								
									{
								
									return
									StatusCode(500, ex.Message);
								
									}
								
									}
								
									#endregion
								
									public
									static
									Image ScaleImage(Image image,
									int
									maxWidth,
									int
									maxHeight)
								
									{
								
									var newImage =
									new
									Bitmap(maxWidth, maxHeight);
								
								
									using
									(Graphics graphics = Graphics.FromImage(newImage))
								
									{
								
									graphics.CompositingQuality =
								
									System.Drawing.Drawing2D.CompositingQuality.HighQuality;
								
									graphics.InterpolationMode =
								
									System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;
								
									graphics.SmoothingMode =
								
									System.Drawing.Drawing2D.SmoothingMode.HighQuality;
								
									graphics.DrawImage(image, 0, 0, maxWidth, maxHeight);
								
									}
								

								
									return
									newImage;
								
									}
								
									}
								
}

The user is presented with a box to enter the text and process the video.

When the Create Video File button is clicked, the CreateVideoFile() method is triggered and it calls the CreateWAVFile() method that contacts the Microsoft Speech Service and creates the audio file:

								
async Task CreateWAVFile()
								
{
								
									Processing =
									true;
								
									StateHasChanged();
								

								
									try
								
									{
								
									Status = "";
								
									Error = "";
								
									using
									(var client =
									new
									HttpClient())
								
									{
								
									// Get a auth token
								
									client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", SubscriptionKey);
								
									UriBuilder uriBuilder =
									new
									UriBuilder(TokenUri);
								
									var result = await client.PostAsync(uriBuilder.Uri.AbsoluteUri,
									null);
								
									var AuthToken = await result.Content.ReadAsStringAsync();
								
									// Set Headers
								
									client.DefaultRequestHeaders.Authorization =
									new
									AuthenticationHeaderValue("Bearer", AuthToken);
								
									client.DefaultRequestHeaders.Add("User-Agent", "curl");
								
									client.DefaultRequestHeaders.Add("X-Microsoft-OutputFormat", "audio-16khz-128kbitrate-mono-mp3");
								

								
									var ssml = @"<speak version='1.0' xml:lang='en-US' xmlns='http://www.w3.org/2001/10/synthesis'
									";
								
									ssml = ssml + @"xmlns:mstts='http://www.w3.org/2001/mstts'>";
								
									ssml = ssml + @$"<voice name='en-US-JennyNeural'>{InputText}</voice></speak>";
								

								
									// Call the service
								
									HttpResponseMessage response =
								
									await client.PostAsync(new
									Uri(DestinationURL),
								
									new
									StringContent(ssml,
								
									Encoding.UTF8, "application/ssml+xml"));
								

								
									if
									(response.IsSuccessStatusCode)
								
									{
								
									//Read as a byte array
								
									var bytes = await response.Content.ReadAsByteArrayAsync().ConfigureAwait(false);
								
									var AudioFilePath = $"{Processingpath}MyWavFile.mp3";
								

								
									File.WriteAllBytes(AudioFilePath, bytes);
								
									}
								
									}
								
									}
								
									catch
									(Exception ex)
								
									{
								
									Error = ex.Message;
								
									}
								
									finally
								
									{
								
									StateHasChanged();
								
									}
								
}

The CreateWAVFile() method was triggered by the CreateVideoFile() method that uses FFMpegCore to create the video:

								
async Task CreateVideoFile()
								
{
								
									Status = "";
								
									Processing =
									true;
								
									StateHasChanged();
								

								
									try
								
									{
								
									await CreateWAVFile();
								

								
									var FileNames = Directory.EnumerateFileSystemEntries(Processingpath)
								
									.Where(x => x.Contains(".png"));
								

								
									// Path to locate the FFMpeg executable
								
									GlobalFFOptions.Configure(options => options.BinaryFolder = "./bin");
								

								
									//Get length of the audio file
								
									var audioInfo = await FFProbe.AnalyseAsync($"{Processingpath}MyWavFile.mp3");
								
									var AudioLength = audioInfo.Duration.TotalSeconds;
								

								
									var frames = CreateVideoFrames(FileNames, AudioLength);
								
									var videoFramesSource =
									new
									RawVideoPipeSource(frames) { FrameRate = 30 };
								
									var success = await FFMpegArguments
								
									.FromPipeInput(videoFramesSource)
								
									.OutputToFile($"{Processingpath}output.mp4",
								
									overwrite:
									true,
								
									options => options.WithVideoCodec("mpeg4"))
								
									.ProcessAsynchronously();
								

								
									Status = "Video File Created";
								
									StateHasChanged();
								

								
									FFMpeg.ReplaceAudio(
								
									$"{Processingpath}output.mp4",
								
									$"{Processingpath}MyWavFile.mp3",
								
									$"{Processingpath}outputaudio.mp4");
								

								
									Status = "Video file with audio Created";
								
									StateHasChanged();
								

								
									//Read as a stream and return a file
								
									var bytes = File.ReadAllBytes($"{Processingpath}outputaudio.mp4");
								

								
									// return file as a stream
								
									var stream =
									new
									MemoryStream(bytes);
								

								
									using
									var streamRef =
									new
									DotNetStreamReference(stream: stream);
								
									await JS.InvokeVoidAsync("downloadFileFromStream", "outputaudio.mp4", streamRef);
								

								
									// Delete all the files
								
									Directory.Delete(Processingpath,
									true);
								
									}
								
									catch
									(Exception ex)
								
									{
								
									Error = ex.Message;
								
									}
								
									finally
								
									{
								
									Processing =
									false;
								
									ProcessingComplete =
									true;
								
									StateHasChanged();
								
									}
								
}

Links

Azure Cognitive Services Speech
https://speech.microsoft.com/portal

Azure-Samples/cognitive-services-speech-sdk
https://github.com/Azure-Samples/cognitive-services-speech-sdk

FFMpegCore
https://github.com/rosenbjerg/FFMpegCore

Quickstart: Convert text to speech
https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/get-started-text-to-speech

carlfranklin/FFmpegBlazorDemo
https://github.com/carlfranklin/FFmpegBlazorDemo

swharden/Csharp-Data-Visualization
https://github.com/swharden/Csharp-Data-Visualization

An error has occurred. This application may no longer respond until reloaded. Reload 🗙