This is a 3 part series. Make sure after you read this one, you also read Part 2 and Part 3 (Upscaling).
A filmmaker I know, Chris Browne, who was also on my podcast, asked me if I could help him denoise his short film, Pleroma, using AI. He's going to showcase it theatrically and wanted to remove most of the visible noise in the shots so it looks its best when blown up for the theater.
I gladly jumped at the chance to help him out and took it as a fantastic opportunity to learn more about denoising and working with real world footage using various AI models.
He sent me the file as a .mxf and he wanted back a DPX 10-Bit image sequence.
The tool I ended up using was Topaz Video AI, which I've used before on a short documentary that needed de-interlacing and upscaling to 1080 (from DV) .
The newer version of Topaz Video AI, since I last used it, introduced the ability to do a 2 pass in one step. For example, in prior versions, if you wanted to denoise and upscale, or de-interlace and upscale the footage, the recommended workflow was to first denoise (or de-interlace) it, then take the output of that footage, bring it back into Topaz and upscale that.
Now, in the newer version, under the "Advanced Preferences" window, if you enable "show second enhancement control", you are allowed to add a second enhancement at the same time.
Below, you can see I have a "Fine-Tune / Enhance" as the first pass and then a "Scale 2x" as the second pass.
That's a real time saver as I don't have to go back after the first pass is done and re-set everything for the second pass. The other thing they introduced in the newest version, 3.5.0, is a new denoising AI model called "Nyx", focused on footage shot with a high quality camera. This was great because the footage I was given was shot with a high quality camera, but the low lighting on some of the scenes made the shot too noisy. That's what this new model specializes in, high quality cameras, with low light noise.
I ran the first model, Artemis and told it that the input video condition was low (due to the high noise level). I also lowered the "recover original detail" as I didn't want to reintroduce some of the original noise back into the new enhanced footage.
Artemis did a great job at cleaning up the footage, as you can see from below.
Here is another example from Artemis:
After carefully reviewing all the footage, I found areas where the original footage was a bit better then the new AI version (specifically, areas that were well lit sometimes looked better in the original). There are some ways around this, which I will cover in tomorrow's journal entry, where I'l also talk about the new Nyx AI Model.
By the way, disclaimer, all screen shots of the short film shown here where done with the permission of its filmmaker, Chris Browne. Thank you Chris!
Other things I worked on
This project required that we transfer large files back and forth since we are remote. After careful research and asking Facebook and Linkedin Groups, we settled on MASV. I ended up receiving his file, 12.5GB, in 2.2 minutes. That's crazy fast! I was super impressed by their service and app. So much so I ended up reaching out to them and will probably have them on my podcast to talk about remote workflows for filmmakers.
Until the next entry!