Comments on: x264 presets

By: fumo

fumo — Mon, 08 Aug 2011 21:15:28 +0000

Nice job, I was looking for some sort of comparison, even if it’s not perfect it definitely is much better than nothing… I hope you maybe still will do both PSNR and SSIM and maybe an Opticom analyzer 😉
Thanks for including presets parameters – I couldn’t find them anywhere

By: Niobos

Niobos — Wed, 24 Feb 2010 07:46:29 +0000

Hi jet,

Well, the goal of this test was actually to find out how much slower “slower” is, so I wanted the psy to be activated.
I’m glad to see that you recognize the usability PSNR, even though we both know that it is far from a perfect quality measure, that’s exactly what the IEEE article explains (as can be read in the free abstract). But I’m also curious to see what correlations there are between PSNR, SSIM, PEVQ and subjective quality.
When I redo this test more thoroughly, I’ll post (a link to) them on doom9. I didn’t post the results from this post, because I know that they’re not obtained in a scientific relevant way.

thx for the valuable feedback!

By: jet

jet — Tue, 23 Feb 2010 19:35:09 +0000

Hi Niobos,
First I want to say that graphs are so much nicer than tables of numbers so thanks for this… but back to complaining:)
The goal of this test is to find how much does PSNR change (improve) with ever slower presets. Unfortunately, psy stuff skews the result so my suggestion is to rule the psy out for pure PSNR test. I don’t have access to IEEE so i won’t be able check the paper but looking at your graph one might say that –preset fastest is actually one of the best for PSNR.(i can bet that the best result for this one clip, which is on –preset fastest, would lose badly to –placebo with –tune psnr added to both commandlines).
As a side note – while you know that PSNR is so-so for evaluating subjective quality it is actually quite good (from my limited knowledge) for relative comparisons such as this one (developers of x264 actually get better results with psnr than ssim).

The speed-part of this test is fine so maybe, maybe make a graph with speed from this graph and PSNR (quality) from –tune psnr encodings on the other axis? (because we want RELATIVE improvement)?
Please post your results on doom9 forum (especially if you can get your hands on PEVQ analyzer) as there are quite a few people interested in this and you get nice feedback on your work. Also, by re-doing the test you could measure latest speed/quality improvements in x264 to boot:)
Wainting for the results

By: Niobos

Niobos — Tue, 23 Feb 2010 07:46:39 +0000

Hi jet, You are correct about the psy influencing the PSNR negatively. This was by no means a thorough scientific test, just a quick assessment of how the presets relate to each other in terms of speed and "quality" (and I'll put it inside quotes). However, according to this IEEE paper, the calculated PSNR is still valid. Your alternative (using --tune psnr) will be skewed in another way, since I won't be taking the psy into account anymore in the speed-part of the test. I'm planning on redoing this test with both PSNR and SSIM and maybe an Opticom analyzer, which might give more representative results.

By: jet

jet — Mon, 22 Feb 2010 23:37:35 +0000

The PSNR data you got is likely flawed. As you probably know, x264 uses psychovisual optimizations. Generally the more psy the less PSNR value. Internally x264 uses more psy with ‘slower’ options (especially –subme from my experience) which skews metrics (decreases PSNR). Thus, your test is skewed towards faster settings as they have weaker psy-enhancements (which visually corresponds to less detailed texture, blocking, smear, etc but also might look ‘cleaner’). To properly test it, you should add –tune psnr to your commandlines.