Tech tip: Getting the most from the least with Handbrake

It's a simple fact of life that all of us have videos on our home PC.  They could be of cats falling into puddles, instructional videos for heavy machinery, home movies, feature films and just about everything in between.  Some videos we're supposed to have, some we're not, but regardless they all have one thing in common: They all take up space, and chances are they're taking up more space than they need to.

That's where Handbrake and the x264 codec come in.

Update: It has been brought to my attention that these settings introduce what's known as "Ringing" to the output.  Ringing is essentially a visual artefact which causes the visible edges of objects to distort the surrounding detail significantly and unpleasantly.  Though the effect is minimal (to the point I didn't really know what I was looking at) with FHD source material, as the resolution decreases the effect becomes substantially more obvious.  As such, I will be going back to the drawing board and posting an update later based on my findings. 

The new settings are, unfortunately, quite a bit slower - early runs are suggesting no real increase in output size but that is offset by taking between 10% and 100% longer.

Of course, the problem is that Handbrake - as with most video encoding programs - comes with more settings than the average person, or even the somewhat advanced person is going to be familiar with.  And, sometimes, changing those settings opens up even more settings.  What makes this worse is that not all of the settings are immediately obvious in what they do, which is what leads me to make this post.

I just want to preface this, however, by saying I'm not intimately familiar with how Handbrake works.  I'm not even all that familiar with x264.  But what I have done is spend the last few weeks encoding and re-coding both entire feature films and small parts there-of to come up with, at least in my opinion, a very good set of guidelines and settings to get you on your way.

INSTALLATION

For those unfamiliar with it, Handbrake itself is available for Mac, Windows and Linux from handbrake.fr.  I would, however, recommend Windows users also take a look at the very excellent VidCoder which offers, in my opinion, a cleaner UI while still using the Handbrake "engine" to do its job.  All settings and presets are interchangeable between the two with identical results so neither is specifically better than the other.

Both also include experimental releases which adds new features.  Handbrake has its "Nightly" builds available from their forums, while VidCoder has a more traditional beta release.

Once you've decided which program you're going to use, and which version of it, it's simply a case of downloading and installing - then you can get started.

CONFIGURATION

For the purposes of this post, I'm going to assume you're already familiar with the basics.  You can set quality levels, adjust cropping and apply basic filters. If you aren't familiar with these things, the FAQ over on the Handbrake website should have enough to get you started.  What I intend to focus on is the things you'll find in the Advanced configuration tab, as well as a few other general guidelines.

First and foremost, for those who don't feel like reading the "why" of it all, the options string I use within VidCoder (which should work identically in Handbrake) for full HD (1080p) input is as follows:

rc-lookahead=10:bframes=5:b-adapt=2:direct=auto:me=umh:merange=24:analyse=all:aq-strength=1.8:psy-rd=1.8,0.00

Now, I make mention of FHD input because HD and SD input isn't going to have as much detail to start with, so you may want to have it keep more detail afterwards.  You can probably still use this as a good starting point however.

For the most part, what you see there is simply increasing the compression at the cost of speed.  5 optimally adaptive, pyramidal B-frames instead of the default 3, using Uneven Multi-Hexagon motion estimation with a range of 24.  These settings are strictly for compression benefit and you can increase or decrease them as you please (though there are practical limits for both minimums and maximums).

The real magic, as I see it, comes in with the AQ and Psy-RD settings.

EXPLANATION

By default, H.264 (and x264 by extension) seems to use something of a bell curve for deciding what detail is preserved.  Anything extremely bright gets less attention as does anything relatively dark.  I don't know if this is by design but that's certainly how it looks.  For the most part this makes a lot of sense - something that's in the shadow probably won't have as much detail, nor will something that is washed out, so why bother devoting as much to them?

Why bother?  Because a lot of videos do, in fact, include dark portions.  I don't mean pitch black - just dark.  Sure it could be the night sky or Batman, but it could also be someone's black hair, a man in a charcoal grey pinstripe suit, or just someone standing in the shadows.  You'll be hard pressed to find a movie which doesn't have something like this; even Astro Boy has such a scene when Tommy is inside Zog that can be considered as "dark" and will be noticeably void of detail.

So with the default settings, these scenes are all negatively affected.  Because detail is effectively "thrown away" by the default x264 compression settings, these scenes will all show areas of banding, blotching and general unpleasentness.  Some people won't mind this, as it doesn't make the video any less watchable.  Others, such as myself, will find it totally unacceptable as it detracts somewhat from the experience.

Simplistically, the best way to combat this is to just up the quality settings.  Instead of using a CRF of 23 as I have been, you might opt for 20, or even 18.  While this works, it is akin to cutting off your arm to stop your finger from bleeding: hugely excessive and it doesn't really solve the problem.  Ultimately it will make a difference, though, and if you get enough quality in there than it will run out of things to focus on instead of these dark scenes and it will have no choice but to start giving them detail - but by then your filesize will have increased so much that it just isn't worth pursuing.

After much experimentation, I've found that the AQ (Adaptive Quantization) and Psy-RD (Psychovisual Rate Distortion) settings appear to have at least some control over this prioritisation.  Psy-RD appears to determine the focus on texture, while AQ determines the quality of edges.  Increasing each of these settings will gradually restore more and more detail to the dark areas - albeit, naturally, at the cost of increased filesize.

Note that the Psy-RD settings will not be available within Handbrake unless the Subpixel ME & Mode Decision setting is on at least 6.

So just how far do you want to go?  Well, I've been using 1.8 on each with great results over the standard 1.0, while 2.0 is the maximum.  However, my test runs have shown that 1.5 simply isn't enough for a genuinely dark video while 2.0 causes a marked increase to the size of the end result.  1.8 is a compromise value - it gives "enough" quality while not taking up too much extra space.

For less-than-FHD input, though, you'll probably want to increase this.  I've found going all the way to 2.0 is pretty well mandatory for SD content to retain the same sort of quality, in conjunction with dropping the CRF value by at least 2.

The beautiful part about these settings is that increasing them both to 2.0 generally won't increase the filesize as much as moving the CRF down a notch (from 23 to 22, for example) but it will provide a much more noticeable increase in picture quality.  This also translates into a similar increase in processing speed as compared to the higher quality setting, which can be important when you're talking about differences that can be measured in hours for some people.

WHAT ELSE?

So now you've played around with the AQ and Psy-RD settings and found something that you're happy with for both size and results, what else would I recommend you check before you go about the business of hours upon hours (potentially) of transcoding?

First, don't be afraid to run some small tests.  Transcoding a minute of video to see if you're happy takes a lot less time than going through your entire collection only to be disappointed.  It could even be worth running a test for every video just to be sure.  You should ideally rename each output file to something meaningful as you go - if you're playing around with different AQ settings, for example, call them "AQ 1.0", "AQ 1.5" and so forth.

Next thing is to make sure you're not overdoing it when it comes to just how much "quality" you're allowing.  There comes a point where things simply cannot look any better - but that's not going to stop them taking up more space while trying.  To really get the most benefit you want to get the filesize down as small as possible without making it unpleasant to watch, and that means settings that give just enough quality to the output.  If CRF 23 is pretty well indistinguishable from CRF 16 to your eye, there's no point going for CRF 16.

Another thing to pay attention to is the audio.  While debate will rage continually about what settings people should or should not be using, the reality is that most people over-do their audio settings when using compression.  While it's true that some commercial media offers a lossless audio experience, the majority of it is going to be compressed - and that's certainly true of older videos.  And how much do these big-budget productions devote to audio?  DTS tracks tend to be 1,536kbps, but AC3 tracks are more often than not just 640kbps for 5.1 audio.  Just something to consider.  Don't forget to lower it accordingly if you're using less channels.

And lastly - perhaps most importantly - you can only get out what you put in.  If your source material is already pretty shoddy than the results cannot be any less shoddy.  This applies to both the video and audio.

SPEEDING THINGS UP

So now that you know a little more about getting more quality from less size, you doubtless want to know more about how to get the job done faster, right?  Like most things in life, it's all about compromise - in the case of transcoding, it's a three-way compromise where you only really get to pick two options: Fast, good or small.  If you want things to go faster, either it's going to take up a lot more space or it's going to look a lot worse.

If space is not an issue than you can really go all out.  Dropping the number of reference frames (I'd still stick with at least 1), ditching B-frames entirely, using lower quality motion estimation with no subpixel estimation and turning off CABAC will all net you increases in speed at the cost of additional storage space.  I'm honestly not entirely sure how much speed you will get, nor how much extra space it will take up, because I've only ever really used Handbrake to produce small files.

On the other side of the coin, you can lower the quality to improve the speed.  This naturally includes reversing the AD and Psy-RD settings I mentioned earlier - or even setting them both to zero.  This will have a correspondingly negative effect on dark areas on the final product but will still produce watchable results, while taking up 10%-20% less space and processing much faster.  You could also lower the CRF a notch or two - though it's a logarithmic scale, a single notch (in my experience) correlates to about a 10% increase or decrease in size and time taken, but this also comes with an impact on the resulting visual quality.  You need to decide how much is acceptable.

The biggest difference, however, comes from reducing the resolution.  Especially in the case of FHD (1080p) source material, you could net yourself a 50%-100% speed increase by dropping down to HD (720p) while not making a significant difference to the output's watchability.  This also has nice side effect of reducing the size of the output, though you may want to turn up the CRF a notch to compensate for the loss of raw pixel data.

LEGALITY

As with most things these days, there are legal elements to consider when it comes to video transcoding.  Handbrake (and VidCoder) and other programs like it are simply tools to convert unlocked videos you already have on your PC.  They cannot bypass copy protection in order to "rip" commercial media or other DRM-protected content.  As such, these programs and their results are generally considered to be legal pretty much everywhere.

However, when it comes to "ripping" - the act of transferring a movie from a protected medium such as a DVD or BluRay to your PC - you start to enter a veritable minefield.  While there are programs which can do this and most of them are considered to be legal where they originated, the individual laws of your country, state, province or county might say differently.  Some laws are even applied so lopsidedly that it is illegal for you to actually create the rip yourself but it is not necessarily illegal for you to have one provided you own the original media.

By and large the opinion is that people should be allowed to transfer any media they own to a different format, either for portability (watching a BluRay on your tablet, for example) or to simply back it up.  Unfortunately an opinion rarely trumps a law, so please be mindful of any applicable laws in your area regarding format transfer, bypassing copy protection and/or acquiring a suitable alternative copy.