Description

Real-Time Intermediate Flow Estimation for Video Frame Interpolation, based on rife-ncnn-vulkan.

This is a port of the VapourSynth plugin RIFE.

Requirements:

• Vulkan device

• AviSynth+ r3688 or later (1 / 2 / 3)

• Microsoft VisualC++ Redistributable Package 2022 (can be downloaded from here)

Usage:

RIFE(clip input, int "model", int "factor_num", int "factor_den", int "fps_num", int "fps_den", string "model_path", int "gpu_id",
 int "gpu_thread", bool "tta", bool "uhd", bool "sc", bool "sc1", float "sc_threshold", bool "skip", float "skip_threshold",
  bool "list_gpu", bool "denoise", int "denoise_tr", int "matrix_in", bool "full_range", bool "cache",
  clip "sc_clip", string "sc_prop", bool "sc_next", clip "skip_clip", string "skip_prop", string "cache_path")

Parameters:

input

A clip to process.
It must be in planar format.
The output format is RGBPS.

model

Model to use.
models must be located in the same folder as RIFE.dll.
Some of the models have two versions: speed oriented (ensemble=False / fast=True) and quality oriented (ensemble=True / fast=False).
0: rife
1: rife-HD
2: rife-UHD
3: rife-anime
4: rife-v2
5: rife-v2.3
6: rife-v2.4
7: rife-v3.0
8: rife-v3.1
9: rife-v3.9_ensembleFalse_fastTrue
10: rife-v3.9_ensembleTrue_fastFalse
11: rife-v4_ensembleFalse_fastTrue
12: rife-v4_ensembleTrue_fastFalse
13: rife-v4.1_ensembleFalse_fastTrue
14: rife-v4.1_ensembleTrue_fastFalse
15: rife-v4.2_ensembleFalse_fastTrue
16: rife-v4.2_ensembleTrue_fastFalse
17: rife-v4.3_ensembleFalse_fastTrue
18: rife-v4.3_ensembleTrue_fastFalse
19: rife-v4.4_ensembleFalse_fastTrue
20: rife-v4.4_ensembleTrue_fastFalse
21: rife-v4.5_ensembleFalse
22: rife-v4.5_ensembleTrue
23: rife-v4.6_ensembleFalse
24: rife-v4.6_ensembleTrue
25: rife-v4.7_ensembleFalse
26: rife-v4.7_ensembleTrue
27: rife-v4.8_ensembleFalse
28: rife-v4.8_ensembleTrue
29: rife-v4.9_ensembleFalse
30: rife-v4.9_ensembleTrue
31: rife-v4.10_ensembleFalse
32: rife-v4.10_ensembleTrue
33: rife-v4.11_ensembleFalse
34: rife-v4.11_ensembleTrue
35: rife-v4.12_ensembleFalse
36: rife-v4.12_ensembleTrue
37: rife-v4.12_lite_ensembleFalse
38: rife-v4.12_lite_ensembleTrue
39: rife-v4.13_ensembleFalse
40: rife-v4.13_ensembleTrue
41: rife-v4.13_lite_ensembleFalse
42: rife-v4.13_lite_ensembleTrue
43: rife-v4.14_ensembleFalse
44: rife-v4.14_ensembleTrue
45: rife-v4.14_lite_ensembleFalse
46: rife-v4.14_lite_ensembleTrue
47: rife-v4.15_ensembleFalse
48: rife-v4.15_ensembleTrue
49: rife-v4.15_lite_ensembleFalse
50: rife-v4.15_lite_ensembleTrue
51: rife-v4.16_lite_ensembleFalse
52: rife-v4.16_lite_ensembleTrue
53: rife-v4.17 (ensemble=False)
54: rife-v4.17 (ensemble=True)
55: rife-v4.17-lite (ensemble=False)
56: rife-v4.17-lite (ensemble=True)
57: rife-v4.18 (ensemble=False)
58: rife-v4.18 (ensemble=True)
59: rife-v4.19-beta (ensemble=False)
60: rife-v4.19-beta (ensemble=True)
61: rife-v4.20 (ensemble=False)
62: rife-v4.20 (ensemble=True)
63: rife-v4.21 (ensemble=False)
64: rife-v4.22 (ensemble=False)
65: rife-v4.22-lite (ensemble=False)
66: rife-v4.23-beta (ensembleFalse)
67: rife-v4.24 (ensembleFalse)
68: rife-v4.24 (ensembleTrue)
69: rife-v4.25 (ensembleFalse)
70: rife-v4.25-lite (ensembleFalse)
71: rife-v4.25-heavy_beta (ensembleFalse)
72: rife-v4.26 (ensembleFalse)
73: rife-v4.26-large (ensembleFalse)
Default: 5.

factor_num, factor_den

Factor of target frame rate.
For example factor_num=5, factor_den=2 will multiply input clip FPS by 2.5.
Only rife-v4 model supports custom frame rate.
Default: 2, 1.

fps_num, fps_den

Target frame rate.
Only rife-v4 model supports custom frame rate.
Supersedes factor_num/factor_den parameter if specified.
Default: Not specified.

model_path

RIFE model path.
Supersedes model parameter if specified.
Default: Not specified.

gpu_id

GPU device to use.
By default the default device is selected.

gpu_thread

Thread count for interpolation.
Using larger values may increase GPU usage and consume more GPU memory.
If you find that your GPU is hungry, try increasing thread count to achieve faster processing.
Must be between 1 and the max compute queue count supported by the GPU.
Default: 2.

tta

Enable TTA(Test-Time Augmentation) mode.
Default: False.

uhd

Enable UHD mode.
Default: False.

sc (deprecated, use sc_clip instead)

Avoid interpolating frames over scene changes using internal SAD-based detection..
This cannot be true when sc1=true.
Ignored if sc_clip is provided.
Default: False.

sc1

Blend frames (average) frames over scene changes using internal SAD-based detection.
This cannot be true when sc=true.
Ignored if sc_clip is provided.
Default: False.

sc_threshold

Threshold to determine whether the current frame and the next one are end/beginning of scene using internal detection.
Must be between 0.0..1.0.
Ignored if sc_clip is provided.
Default: 0.12.

skip (deprecated, use skip_clip instead)

Skip interpolating static frames using internal VMAF-based detection.
Requires VMAF plugin.
Ignored if skip_clip is provided.
Default: False.

skip_threshold

PSNR threshold to determine whether the current frame and the next one are static using internal detection.
Must be between 0.0..60.0.
Default: 60.0.

list_gpu

Simply print a list of available GPU devices on the frame and does no interpolation.
Default: False.

denoise

Whether to return only the interpolated frames.
Default: False.

denoise_tr

Frame radius.
For example, denoise_tr=1 means frames n-1 and n+1 are used.
Must be greater than 0.
Default: 1.

matrix_in

Matrix for YUV->RGB conversion.
Mandatory for YUV input.
0: 601
1: 709
2: 2020
Default: not specified.

full_range

Input pixel_range.
Default: True for 32-bit or RGB input.

cache

Whether to share the RIFE model instance between multiple filter calls.
When enabled, instances using the same model, GPU ID, and video format (bit depth, color space, etc.) will share the same memory, reducing VRAM usage.
If set to False, a private instance of the model will be loaded into VRAM for that specific call.
Default: True.

denoise_bf

Backward frame radius for denoise=true.
Allows asymmetric reference frame selection.
Must be greater than 0.
Default: Value of denoise_tr.

denoise_ff

Forward frame radius for denoise=true.
Allows asymmetric reference frame selection.
Must be greater than 0.
Default: Value of denoise_tr.

sc_clip

External clip for scene change detection.
If provided, it supersedes the internal SAD-based detection (sc/sc1).
Must have the exact same number of frames as the input clip.
Can be a mask clip (where a pixel value > 0 indicates a scene change) or a clip carrying frame properties.
Default: Not specified.

sc_prop

Name of the integer frame property in sc_clip to read for scene change flags (value > 0).
If specified, sc_clip is evaluated in property mode.
If omitted, sc_clip is evaluated in mask mode (reading the first pixel of the default plane).
Default: Not specified.

sc_next

Sets whether scene-change markers in sc_clip refer to the frame before or after the cut.
If True, a flag on frame N means the scene change occurs before frame N (between N-1 and N).
If False, a flag on frame N means the scene change occurs after frame N (between N and N+1).
Default: True.

Example replicating the internal scene change detection:

source
propset("Next", 0)
props = propSet("Next", 1)
ConditionalFilter(last, props, last, "YDifferenceFromPrevious()", ">", "20")
RIFE(gpu_thread=1, matrix_in=1, sc_clip=last, sc_prop="Next")

skip_clip

External clip to detect static frames and skip interpolation.
If provided, it supersedes the internal VMAF-based detection (skip).
Must have the exact same number of frames as the input clip.
Can be a mask clip (where a pixel value > 0 indicates a static frame) or a clip carrying frame properties.
Default: Not specified.

skip_prop

Name of the integer frame property in skip_clip to read for skip flags (value > 0).
If specified, skip_clip is evaluated in property mode.
If omitted, skip_clip is evaluated in mask mode (reading the first pixel of the default plane).
Default: Not specified.

Example replicating the internal static frame detection:

source
skip_clip = VMAF2(last, DuplicateFrame(Trim(1, 0), FrameCount() - 1), feature=0)
RIFE(gpu_thread=1, matrix_in=1, skip_clip=skip_clip, skip_prop="psnr_y")

cache_path

File path to save and load the Vulkan pipeline cache.
When provided, compiled Vulkan shaders and driver pipeline state objects are cached to this file, reducing the cold-start latency on subsequent runs.
The target directory must be writable. The cache file is strictly tied to the specific GPU, driver, and ncnn version used to create it.
If omitted or empty, Vulkan pipeline caching is disabled.
Default: Not specified.

Building:

• Requires Vulkan SDK.

git clone --depth 1 --recurse-submodules --shallow-submodules https://github.com/Asd-g/AviSynthPlus-RIFE
cd AviSynthPlus-RIFE

cmake -B build -G Ninja -DCMAKE_PREFIX_PATH=<path_to_vulkan_installation>