Adobe Research has just introduced VideoGigaGAN, an AI method that can upscale video resolution up to X8 with incredible sharpness and zero quality loss. It might be implemented on Premiere Pro in the future. A Topaz killer?

VideoGigaGAN: Upscaling Video Resolution X8

Here’s the Topaz killer. Adobe has just introduced its new product called VideoGigaGAN which allows upscaling the video resolution by X8 and without any artifacts. The invention was presented by Adobe Research, which means in the R&D phase (far from private Beta), so Adobe is playing with it to see how it goes. Briefly, VideoGigaGAN is able to retain sharp details, ensure smooth transitions, and upsample a video up to 8x. Again, this is only a research preview, so there’s no guarantee that Adobe will make VideoGigaGAN available to consumers via Creative Cloud software like Premiere Pro. However, our guess is that it will reach PP and might be implemented as a smartphone app, so it can be quite a powerful capability.

Abstract (as stated by Adobe Research)

This is how Adobe Research explains the tech behind VideoGigaGAN: “Video super-resolution (VSR) approaches have shown impressive temporal consistency in upsampled videos. However, these approaches tend to generate blurrier results than their image counterparts as they are limited in their generative capability. This raises a fundamental question: can we extend the success of a generative image upsampler to the VSR task while preserving the temporal consistency?”

VideoGigaGAN as a new generative VSR model

As stated by Adobe: “We introduce VideoGigaGAN, a new generative VSR model that can produce videos with high-frequency details and temporal consistency. VideoGigaGAN builds upon a large-scale image upsampler — GigaGAN. Simply inflating GigaGAN to a video model by adding temporal modules produces severe temporal flickering. We identify several key issues and propose techniques that significantly improve the temporal consistency of upsampled videos. Our experiments show that, unlike previous VSR methods, VideoGigaGAN generates temporally consistent videos with more fine-grained appearance details. We validate the effectiveness of VideoGigaGAN by comparing it with state-of-the-art VSR models on public datasets and showcasing video results with 8× super-resolution”.

Limitations

The model encounters challenges when processing extremely long videos (e.g.,200 frames or more). This difficulty arises from misguided feature propagation caused by inaccurate optical flow in such extended video sequences. Additionally, the model does not perform well in handling small objects, such as text and characters, as the information pertaining to these objects is significantly lost in the LR video input. Read the white paper here. More examples are below:

Initial thoughts

Imagine shooting in Full HD and upscaling it to 8K without any artifacts and side effects of imagery. That could be a game-changer. This tech will be implemented in the future via the Premiere Pro app (our assumption) and it could be one of the most powerful features of the software. High-resolution cameras will not be needed anymore.