VMAF? VMAF! Target Vmaf!

Target VMAF

The "Target VMAF" feature has a really simple goal, instead of guessing what the CQ value of your encoder will give you in regards to the resulting quality, we set the VMAF score we want to achieve and let the algorithm get the closest CQ value that will result in that score, for each split segment. Which simultaneously achieve 3 things, if compared to usual, single value CQ encode.

1. Ensure that complex scenes receive more bit rate to achieve target quality.
2. Increase quantizer value for simple scenes, and not going lower than target quality, and save bit rate.
3. Yields a lower total bit rate.

From my testing, result size can be 50-80% of compared to usual encode, with great visual quality.

VMAF plotting

Plot contains VMAF score for each individual frame.
Plotting after encode will be performed if flag --vmaf set, or --vmaf_target is used.
The plot legend display Mean Average, Lower 1, 25, 75 percentile, which in combination with plot should be insightful enough for judging the quality of the encode instead of a single vmaf value for the whole video.

Example

This is a plot of the encode that was using Target VMAF 96 (don't mind nan in mean, it's fixed at time of this post, but i don't want to reencode:) )

The 1 percentile you can see on this plot shows the likely VMAF score of complex scenes, which usually involves a lot of movement, changes in perspective, zooming in and out and change in video context on every frame.

Usage

To try simply run your default constant quality encoding with --vmaf_target N, where N is VMAF score you try to achieve, i suggest something in 90-96 area.

--vmaf_target sets the VMAF score that Av1an will aim for.
--min_cq - sets the lower boundary for the VMAF probes and limit the minimum cq value at which the video can be encoded. Default is 25.
--max_cq sets the upper boundary for the VMAF probes and limit the maximum cq value at which the video can be encoded. Default is 50.
--vmaf_steps - sets the number of probes that are used to get the best CQ value for the target VMAF. Default value is 4. If min-cq, max_cq are changed that distance between them increase - make sure to set steps that there is a probe for every ~5 CQ of distance

For more information about previous target VMAF refer to 1.8 release. This is evolution upon that method.

Also need to mention that Target VMAF currently only for reference AV1 encoder

Vmaf Plotting as separate package

VMAF plotting is available as separate package, and only needs VMAF's or ffmpeg's libvmaf result xml file to work.
GITHUB_REPO

Extra splits

-xs, --extra_splits will add cuts for every N frames on splits that are longer than N, and spread cuts evenly, intelligently.
This option help split big scenes or files that are long single scene, like camera recording, into splits that paralleled better.

For example:

--extra_splits 400
For a split that goes from frame 14000 to 14900, the distance between cuts is 900, 900/400= 2.25 which rounds to 2.
Av1an will try to find 2 key frames that are closer to frame 14300 and 14600 and place the splits there.
Another split from 0 to 500, Av1an will find key frame closer to 250 and place cut there.
Splits that have less than 400 frames are not affected.

Dynamic Progress Bar, Batch Encode, Config files

(Click on video to watch all new features together)

This is quality of life update, most changes under the hood and just make Av1an run better :)

Dynamic progress bar

It show progress for all workers and works with AOM and Rav1e, SVT-AV1 will be working with old chunk update frame count.

Config files

On first use of -cfg file current encoder, encoding parameters, audio, and FFmpeg settings will be save, and this file can be reused by -cfg file without need of typing same settings again.

Batch encode

Multiple files can be encoded by passing them separated by space, -i file1 file2 they will be encoded with same settings. Pyscenedetect will run on each one individually, so keep in mind to not set -s scene_file. And keep in mind that using --resume will try to resume first file in queue.

AUR

Av1an now available for installation by Arch Linux AUR

Target Vmaf, Vmaf plotting, VP8/VP9 support, PIP package

This is a big update.
I want to thank all the collaborators that worked on features, or helped with development.
I really appreciate everyone's engagement with this project

Pip package

• PIP package allows super easy installation on all platforms, by single command.
• Usage is simple as av1an -i file all_params..., package automatically available system wide.
• Only extra requirements for work is to have encoder of choice, and FFmpeg installed.

Vmaf plotting

If --vmaf is specified (and vmaf configured correctly on system) at end of encode will be plotted Vmaf for each segment of a video, where Y is vmaf, and X is frames and drawn median line.

Target Vmaf mode

!This feature is experimental. It certainly will be changed and improved in the future.
Desired results are not guaranteed, use with caution;)
Best works with 720/1080 videos.
Feedback and suggestions will be appreciated:+1:

By making couple of few fps encoding probes at fastest cpu-used, it's possible to interpolate cq values to Vmaf relations and with certain error predict what Vmaf score full encode will have.
Example of interpolation based on 4 probes. Orange dot is chosen cq value extracted from plot to get targeted Vmaf. Red crosses are probe results.
In folder of encoding this plot will be generated for each segment.

Result with same video as in previous vmaf plot :

--tg_vmaf N - for using Target Vmaf mode specify , where N is desired vmaf number, most stable results in range 90-95 Vmaf.

--vmaf_steps N - number of evenly spaced probes that is used to interpolate vmaf to cq change.
N bigger than 3. Optimal is 4-6 probes. Default: 4.

--vmaf_error N - decrease initial Vmaf values for interpolation.
Increasing number will result in lower CQ and bigger final vmaf score, use to correct whole vmaf plot. For start If target vmaf undershoot increase value by undershoot amount.
Default: 0.

--min_cq, --max_cq - minimum and maximum CQ values used in interpolation.Use to limit CQ values range.
Default: 20, 63.

Added VP9, VP8 support

Added support for vpx

Features, Refactoring, General Improvements

Vmaf

• !Requires FFmpeg with libvmaf enabled
• Added --vmaf option. Show vmaf for each encoded segment.
• Added --vmaf_path for custom vmaf models path.

Refactoring

• Changed all lookup, now faster and better.

Instant resume

• All data about total frame count and chunks are stored in done.txt file and encode can be resumed instantly.

Specify temp folder

• Added --temp option to specify custom temporally folder to use.

Experimental CQ boosting

Boosting

Decreasing CQ value of encoded scene depending on brightness value of scene. (AOM only)
Intended to improve encoding quality of dark scenes.

How it works

For each scene geometrical mean of brightness is calculated from average brightness of every frame.
If brightness of scene is lower than 128, CQ will be decreased, lower values of brightness will receive bigger CQ decrease. CQ change range relates to 0-128 brightness range, meaning that the lowest value CQ will be at 0 brightness, half of that value at 64, etc..
For example (encoding cq = 40) :

Usage

• Enable with --boost
• Option -br will set maximum CQ change from original. Default: 15
• Option -bl will set hard cap on how low CQ can be set by boosting. Default: 10

Example

If this option --boost -br 30 -bl 15 set with --cq-level=30 will mean that we set boost CQ range to 30, and hard limit for boost at 15 . In this example all scenes with brightness 64 and lower will have minimum value of cq (15)

Better Logging, Requested Features

• Better logging
• Added --keep option. Not deleting temporally folders after encode is finished
• Fix for Windows terminal overflow
• Files with spaces are fine now :)

Threadripper Showcase, Reworked Logging, Testing

This is a small update, preparing for 1.7 ;)

Example of Av1an with Threadripper 3970X

~9-12 Fps 1080p AOM encoding with cpu-used 3

Finished personal encodes:

The End of Evangelion (1997) 1080p

1920x1080 23.98 fps 01:26:47 bitrate: 1880 kb/s
Av1an: -w 32 -v ' --end-usage=vbr --target-bitrate=2000 --aq-mode=2 --threads=4 
--arnr-maxframes=15 --enable-fwd-kf=1 --lag-in-frames=25 --cpu-used=3 ' 
Finished in 15690.8s (4:03:32)
Size: 1.14 GB

link mb in AV1 DISCORD :)

Fractalicious8 4K

3840x1920 30 fps 04:49.28 12665 kb/s
Av1an: -w 16 -v "--end-usage=q --cq-level=30 --cpu-used=3 --threads=16 "
Queue: 80 Workers: 16 Passes: 2
Finished in 8767.3s (2:26 min)

Original Encode

Fractalicious8 1080p

1920x960 30 fps 04:49.28 3720 kb
Av1an: -ff ' -vf scale=1920:-1:flag' -v  ' --end-usage=q --cq-level=30 
--cpu-used=3 --threads=16
Queue: 80 Workers: 24 Passes: 2
Finished in  1538.5s or (0:25 min)

Original Encode

Reworked logging

• New logging file contain all info about encode, started and finished chunks, their frame counters and individual speed
• If no logging set, logging file will be created in .temp/

Testing

Started work on making automated tests for Av1an

Troubleshooting and weird errors

• A lot of fixes for ffmpeg usage
• General code improvements

Frame progress, Less dependency

Major Changes

Progress is shown in frames

• Click on image for video.
  
• Bar updates after every finished chunk by the amount of frames from the source clip
• Total size of bar is source number of frames

FFprobe deprecated

FFprobe not longer needed for Av1an, all FFprobe calls replaced with FFmpeg.

~25 times faster frame count checking

Redone frame count checking.
Speed comparison for frame count check:
FFprobe: 4.99s
FFprobe with autothreads: 1.039
FFmpeg: 0.209s

Minor changes

• General code improvements
• Console look
• Bugs of fixes

Encode Resuming, Frame count checking

Major Changes

Resuming encoding without loss of finished chunks.

• Click on image for video.
  

• Only finished chunks are saved.

• .temp folder must be presented and not changed from when encoding was stopped.

• All stages before encoding must be passed for resuming.(scenes split, audio processed)

• Resuming skips scene detection, video split, audio processing.

Checking encoded clips for errors in frame count.

• Click on image for video. Changed frame rate so encoded clips pop warnings.
  

There has been reports that some files with some encoders at different setting drop frames at time of encoding (works on my machine), so this measure were included to warn about possible problems as soon as possible.
If the amount of frames need to change (changing frame rate, etc..) use --no_check option which disables frame checking completely, also can save you second or two on your 80 hours encodes :)

Improved console look

Best one yet :)

Minor Improvements

• Better audio checking and extraction.
• Fixed bug with ffmpeg concat not working.
• Added option to skip split completely and encode whole file with single encoder -s 0. (boring)
• Faster and better default settings for Aom, Rav1e + enabled multi threading and cut number of workers.
• Overall better error handling.
• Logging now works on Windows.

Great appreciations to 🥇 @nicomem 🥇

For a lot of changes, pull's and improvements

Drop-in Windows Build, PySceneDetect integration

• PySceneDetect integrated, less complexity, faster.
• Autobuilds for Windows Executable at Appveyor.
• Windows builds only require FFmpeg, FFprobe, Encoder in main folder with executable.
• At current moment Working Encoders for Windows are AOM (Full), Rav1e ( 1-pass, 2 pass is bugged), SVT-AV1 don't want to work with pipes at all :(
• Attached files are ready to use and contain latest Rav1e and Aom