10/19/2022 Admin

Blazor Video Creator

You can dynamically create videos with pictures and audio using Blazor.

The Application

When you run the application you will be presented with the UPLOAD IMAGE FILES button.

Select images from your computer and click the Open button.

Enter text for the video and click the Create Video File button.

The video will process.

The text will be turned into audio.

The pictures will be added to the video so that they evenly span the length of the audio.

The completed video will automatically download to your computer.

You can find the video file in the Downloads folder on your computer.

When you open the video file it will display the pictures and a voice will read the text.

Note: Open the video in a video player not a web browser like Chrome (it will not work).

Note: There is another project: carlfranklin/FFmpegBlazorDemo that does something similar, however, that code only works with Blazor WebAssembly. This works with Blazor Server.

How To Run The Code

To turn the text into audio you will need to obtain a Microsoft Speech Service key.

Go to: https://portal.azure.com/ and select Create a resource.

Search for Speech, select it and click Create.

After the service is created, select Keys and Endpoint and copy the Key and the Location/Region.

Download the project from the Downloads page on this site, unzip it, and open it in Visual Studio 2022 (or higher).

Open the appsettings.json file.

Update the SubscriptionKey and SpeechRegion properties with the values you copied earlier.

Hit F5 to build and run the project.

The Code

The project makes use of the FFMpegCore NuGet Package.

FFMpegCore provides a .Net Core wrapper around the FFMpeg project.

The executables were downloaded and added to the “bin” directory of the project, and the Build Action for each was set to Content and the Copy to Output Directory setting is set to Copy always.

Most of the code is contained in the Index.razor page.

<RadzenUpload ChooseText="Upload Image Files" Multiple="true"
    Accept="image/*" AllowedExtensions=".png;"
    Url=@($"api/upload/multiple")
    Style="margin-bottom: 20px;height: 45px"
    Progress="@((args) => OnProgress(args))"
    Complete=@OnComplete />

The Radzen Upload Control is used to allow the user to upload image files.

The upload is processed by the UploadController.cs file that creates a temporary folder, resizes the images, and returns the name of the temporary folder:

using Microsoft.AspNetCore.Mvc;
using System.Drawing;
using System.Drawing.Imaging;
namespace Speech
{
    [Route("api/[controller]")]
    [ApiController]
    public class UploadController : Controller
    {
        private readonly IWebHostEnvironment environment;
        public UploadController(IWebHostEnvironment environment)
        {
            this.environment = environment;
        }
        #region public async Task<IActionResult> MultipleAsync(IFormFile[] files)    
        [HttpPost("[action]")]
        public async Task<IActionResult> MultipleAsync(
            IFormFile[] files)
        {
            try
            {
                // Create a 5 digit random number
                // to use a temp directory
                Random rnd = new Random();
                int randomNumber = rnd.Next(10000, 99999);                
                string TempDirectory = randomNumber.ToString();                
                
                string Processingpath =
                    Path.Combine(
                        environment.WebRootPath,
                        "processing\\",
                        TempDirectory);
                Directory.CreateDirectory(Processingpath);
                
                if (HttpContext.Request.Form.Files.Any())
                {
                    foreach (var file in HttpContext.Request.Form.Files)
                    {
                        string FileName = file.FileName;
                        string path =
                            Path.Combine(
                                Processingpath,
                                FileName);
                        using (var stream =
                            new FileStream(path, FileMode.Create))
                        {
                            await file.CopyToAsync(stream);
                        }
                    }
                    
                    // Get all the files
                    var AllFilesInDirectory =
                        Directory.EnumerateFileSystemEntries(Processingpath);
                    // Loop through picture and scale image
                    foreach (var file in AllFilesInDirectory)
                    {
                        using (var image = Image.FromFile(file))
                        {
                            using (var newImage = ScaleImage(image, 500, 500))
                            {
                                try
                                {
                                    var newImageName = Path.Combine(Processingpath, 
                                        Path.GetFileNameWithoutExtension(file) 
                                        + "_resized" + ".png");
                                    newImage.Save(newImageName, ImageFormat.Png);
                                }
                                catch
                                {
                                    continue;
                                }
                            }
                        }
                    }
                    // Get all the non resized files
                    var AllNonResizedFilesInDirectory =
                        Directory.EnumerateFileSystemEntries(Processingpath)
                        .Where(x => !x.Contains("_resized"));
                    // Delete all the non resized files
                    foreach (var file in AllNonResizedFilesInDirectory)
                    {
                        System.IO.File.Delete(file);
                    }
                }
                // Let the index.razor page know the name of the temp directory 
                // so it knows where to look for the images
                return StatusCode(200, TempDirectory);
            }
            catch (Exception ex)
            {
                return StatusCode(500, ex.Message);
            }
        }
        #endregion
        public static Image ScaleImage(Image image, int maxWidth, int maxHeight)
        {
            var newImage = new Bitmap(maxWidth, maxHeight);
            
            using (Graphics graphics = Graphics.FromImage(newImage))
            {
                graphics.CompositingQuality =
               System.Drawing.Drawing2D.CompositingQuality.HighQuality;
                graphics.InterpolationMode =
                System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;
                graphics.SmoothingMode =
                System.Drawing.Drawing2D.SmoothingMode.HighQuality;
                graphics.DrawImage(image, 0, 0, maxWidth, maxHeight);
            }
            return newImage;
        }
    }
}

The user is presented with a box to enter the text and process the video.

When the Create Video File button is clicked, the CreateVideoFile() method is triggered and it calls the CreateWAVFile() method that contacts the Microsoft Speech Service and creates the audio file:

async Task CreateWAVFile()
{
    Processing = true;
    StateHasChanged();
    try
    {
        Status = "";
        Error = "";
        using (var client = new HttpClient())
        {
            // Get a auth token
            client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", SubscriptionKey);
            UriBuilder uriBuilder = new UriBuilder(TokenUri);
            var result = await client.PostAsync(uriBuilder.Uri.AbsoluteUri, null);
            var AuthToken = await result.Content.ReadAsStringAsync();
            // Set Headers
            client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", AuthToken);
            client.DefaultRequestHeaders.Add("User-Agent", "curl");
            client.DefaultRequestHeaders.Add("X-Microsoft-OutputFormat", "audio-16khz-128kbitrate-mono-mp3");
            var ssml = @"<speak version='1.0' xml:lang='en-US' xmlns='http://www.w3.org/2001/10/synthesis' ";
            ssml = ssml + @"xmlns:mstts='http://www.w3.org/2001/mstts'>";
            ssml = ssml + @$"<voice name='en-US-JennyNeural'>{InputText}</voice></speak>";
            // Call the service
            HttpResponseMessage response =
                await client.PostAsync(new Uri(DestinationURL),
                    new StringContent(ssml,
                Encoding.UTF8, "application/ssml+xml"));
            if (response.IsSuccessStatusCode)
            {
                //Read as a byte array
                var bytes = await response.Content.ReadAsByteArrayAsync().ConfigureAwait(false);
                var AudioFilePath = $"{Processingpath}MyWavFile.mp3";
                File.WriteAllBytes(AudioFilePath, bytes);
            }
        }
    }
    catch (Exception ex)
    {
        Error = ex.Message;
    }
    finally
    {
        StateHasChanged();
    }
}

The CreateWAVFile() method was triggered by the CreateVideoFile() method that uses FFMpegCore to create the video:

async Task CreateVideoFile()
{
    Status = "";
    Processing = true;
    StateHasChanged();
    try
    {
        await CreateWAVFile();
        var FileNames = Directory.EnumerateFileSystemEntries(Processingpath)
        .Where(x => x.Contains(".png"));
        // Path to locate the FFMpeg executable
        GlobalFFOptions.Configure(options => options.BinaryFolder = "./bin");
        //Get length of the audio file
        var audioInfo = await FFProbe.AnalyseAsync($"{Processingpath}MyWavFile.mp3");
        var AudioLength = audioInfo.Duration.TotalSeconds;
        var frames = CreateVideoFrames(FileNames, AudioLength);
        var videoFramesSource = new RawVideoPipeSource(frames) { FrameRate = 30 };
        var success = await FFMpegArguments
        .FromPipeInput(videoFramesSource)
        .OutputToFile($"{Processingpath}output.mp4",
            overwrite: true,
            options => options.WithVideoCodec("mpeg4"))
            .ProcessAsynchronously();
        Status = "Video File Created";
        StateHasChanged();
        FFMpeg.ReplaceAudio(
            $"{Processingpath}output.mp4",
            $"{Processingpath}MyWavFile.mp3",
            $"{Processingpath}outputaudio.mp4");
        Status = "Video file with audio Created";
        StateHasChanged();
        //Read as a stream and return a file
        var bytes = File.ReadAllBytes($"{Processingpath}outputaudio.mp4");
        // return file as a stream
        var stream = new MemoryStream(bytes);
        using var streamRef = new DotNetStreamReference(stream: stream);
        await JS.InvokeVoidAsync("downloadFileFromStream", "outputaudio.mp4", streamRef);
        // Delete all the files
        Directory.Delete(Processingpath, true);
    }
    catch (Exception ex)
    {
        Error = ex.Message;
    }
    finally
    {
        Processing = false;
        ProcessingComplete = true;
        StateHasChanged();
    }
}

Download

The project is available on the Downloads page on this site.

You must have Visual Studio 2022 (or higher) installed to run the code.

Links

Azure Cognitive Services Speech
https://speech.microsoft.com/portal

Azure-Samples/cognitive-services-speech-sdk
https://github.com/Azure-Samples/cognitive-services-speech-sdk

FFMpegCore
https://github.com/rosenbjerg/FFMpegCore

Quickstart: Convert text to speech
https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/get-started-text-to-speech

carlfranklin/FFmpegBlazorDemo
https://github.com/carlfranklin/FFmpegBlazorDemo