In my previous journal entries, Denoising a Short Film using AI Part 1 and Part 2, I spoke about how I denoised a short film for a filmmaker I know. In this journal entry, I want to cover how I took the final cleaned (denoised) short film and upscaled it to 4K (using Topaz Video AI).
The original source file for the short film was a .mxf file. I took that into Topaz Video AI and created 2 .dpx 10 bit outputs, one using the Artemis AI model and the other using the new Nyx AI model for denoising the footage (read the journal entries listed above to get all the details).
I then took all 3 sources, the original and the 2 .dpx files and edited them together in DaVinci Resolve to get the best mix between all three versions. I then exported a ProRes 422HQ 1920 x 1080 23.976 FPS version out of DaVinci Resolve and imported that into Topaz Video AI to start the upscale process.
In Topaz AI I dropped that footage in, I verified that the input was read correctly and selected the output I wanted.
As the image above shows, it correctly detected the file as a 1920 x 1080 resolution with an FPS of 23.976. For the output, I selected 2x (3840 x 2160, twice what the original resolution is) and kept the same frame rate (23.976). I could have created a 60fps file but that was not necessary for this project. Do note that if I would have changed the frame rate, that would have affected my choice of AI model I used, as there are specific models for frame interpolation.
Next up were some filters to choose from, such as the ability to stabilize the footage, deblur motion or add frame interpolation. I chose none of those since I already fixed most issues in the denoising process and like I said above, I was not changing the frame rate. Each filter has more parameters inside of them you can control and specific AI models as well.
Stabilization will reduce camera shake.
Motion Blur will reduce motion blur from camera movement.
Frame Interpolation is used for frame conversion or slow motion.
Because I was upscaling the footage, the next "Enhancement" option is automatically selected with the optimal settings for the imported footage. You can adjust the settings, but you can't turn that option off when upscaling.
After running multiple tests, I settled on the "Theia" AI model, which helps in adding detail to the final image. For the model preference, I chose "Fine Tune Fidelity", as I wanted to have more quality consistency across all frames, versus the other option, "Fine Tune Detail", which is focused on getting the best quality and detail for each frame, but may introduce inconsistent quality across the entire footage.
I then adjusted the Revert Compression and Reduce Noise parameters.
Revert Compression: Reduces compression artifacts from codec encoding, such as blockiness or mosquito noise. Higher values are best for low bitrate videos.
Sharpen: Additional sharpening of the video. Only used if the video looks very soft.
Reduce Noise: Removes ISO noise from the input video. Higher values reduce more noise, but also may remove detail.
Add Noise: Adds a layer of digital noise to videos before processing. Used to reduce the effect of noise reduction and texture smoothing.
The last option was to add "natural grain" to the final output for a more natural organic feel. I didn't do that.
I then selected the output settings to be ProRes 422HQ, with no audio, and a .MOV container. This led to a final .mov file of 43.29GB in size. I then took that into DaVinci Resolve, added the audio from the original source file and exported out a new file that matched the timeline sequence settings. That's what I provided back to the filmmaker.
The "Include Live Preview" option (shown above) was checked so that I was able to see each frame as it's being created inside of Topaz while it's generating the 4k image.
That's it. We ended up with a really nice clean 4K short film that will look great played in a big theater screen.
Until the next entry!