I just discovered that FFmpeg, in the rawvideo format and with a pix_fmt of rgb24, it does not care about whether it's reading interleaved (or packed, as in its documentation) or planar RGB 24-bit data. The fuck? You look at the source code, and it straight-up just says:
AV_PIX_FMT_RGB24, ///< packed RGB 8:8:8, 24bpp, RGBRGB...
Let me demonstrate. I have here a nice picture I got off the Internet, and it's 256x256.
(Attribution:
Photo by
Sharon Pittaway on
Unsplash)
So it should only correctly understand interleaved/packed 24-bit RGB, but if you go ahead and do the following with a raw planar RGB version of that image of size 256x256 (rename it to "planar-image.raw"):
ffmpeg -f rawvideo -pix_fmt rgb24 -s 256x256 -i planar-image.raw -f rawvideo -pix_fmt yuv444p planar-image-in-yuv444p.raw
(Convert an image with RGB24 pixel format, 256x256 size, to a raw image in YUV444P pixel format)
Intermediate view using YUViewer:
Then you reverse this process by running:
ffmpeg -f rawvideo -pix_fmt yuv444p -s 256x256 -i planar-image-in-yuv444p.raw -f rawvideo -pix_fmt rgb24 planar-image-in-yuv444p-now-in-rgb24.raw
(Convert the previous image back to RGB24 pixel format.)
...the end result is still in planar RGB, and it only opens correctly if you tell your image viewer to use planar RGB. The source code told me it was packed
, but it seems it doesn't care at all if it's seeing planar or packed. (Check the edit)
You know what does care, though? The image2 format; if you give it image data in planar RGB, it will read it as if it were interleaved RGB. Run:
ffmpeg -f rawvideo -pix_fmt yuv444p -s 256x256 -i planar-image-in-yuv444p.raw -f image2 -pix_fmt rgb24 planar-image-in-yuv444p-now-in-png.png
(Convert a raw image in YUV444P pixel format of size 256x256 into a .png image with pixel format RGB24.)
And the PNG encoder just steamrolls ahead, just "yeah, this is totally interleaved 24-bit RGB", and it produces an image very characteristic of what happens if you open a planar RGB image as an interleaved RGB image. In fact, if you do just that to the original raw image:
My copy of FFmpeg is 4.3.1, the 01-01-2021 full build from gyan.dev, if anyone's keeping track. Either way, interesting oversight in how FFmpeg is coded; there's no differentiation between interleaved and planar RGB24 formats until you try using the PNG encoder, or I presume any image format FFmpeg supports encoding to. It seems to be something caused by incorrect usage, but I really can't tell what's going on. Add that to the list of "not the intended use", because I just seem to be playing real funny tricks with everything I touch.
Edit: It seems that the YUV thing does actually care, but it just stores it as interleaved. Reconverting it back to RGB24, it just literally reads off the image data as if it was interleaved, but of course it's not. It's still planar.
It also seems that planar-source and interleaved-source images have different properties, even in the YUV format. Paulstretching the image data of a planar-source image produces colorful results, while doing the same to an interleaved-source image just results in a black-and-white variation. I don't think they're identical, necessarily, but it resembles it.