fixing video tags, understanding compatability

i might have put this in the Tags category, but i'm suggesting new serverside code so...

video tagging is a mess, "video" itself has 600 entries and no wiki entry (for all i know it's for diegetic video), yet there's over 1000 webms and ~400 mp4s, are they not videos?

i've filed a implication request for this https://booru.allthefallen.moe/forum_topics/133

but here i wanted to discuss the merits of dropping "webm" and "mp4" entirely, and instead classifying videos based on encoding (codec)

all the videos here are displayed with a HTML5 <video> tag. i cant imagine there's any browsers that support <video> but not both MP4 and WEBM inside them.

what determines if a device is able to play a video is the codec, especially mobile devices which don't have the power to brute force software/gpu compute render codecs they lack hardware support for.

this means that searching for "webm" or "mp4" is meaningless, because it's a matter of whether your device supports the codec, which is currently invisible.

i suggest new meta tags, they will require some data from the file header to generate but nothing any number of command line tools could do (i'm sure ffmpeg will return just the codec with the right arguments)

the tag webm should be removed, all existing webms should be tagged "video" and then by codec, which will be either VP8, VP9 or AV1. this is why some WEBMs won't play for people but others will - because they have different codecs.

the same for the tag mp4, to be replaced by h264 and h265 (aliased to AVC and HEVC respectively). again lots of devices can't play HEVC, but everything can play h264.

bonus points for doing this with the AUDIO TOO - i've seen a lot of video files marked as having audio but apparently being silent, and i wonder if it's because this browser doesn't support newer audio codecs (like opus on webms)

finally i suggest aliasing h264 and VP8 to a tag like "compatible video" (thats a shit name but you get the idea) because these will play on all devices, or the inverse, tagging the newer less supported codecs as "limited support".

SUMMARY: BENEFIT FOR NON-NERDS: find videos by searching "video", and find videos which always work with "compatible video"
BENEFIT FOR NERDS: selectively -ing codecs you know your device doesn't like (i bet when AV1 rolls out more almost no phone will play it) so you can just find every video you can watch.

i know this is a long ramble, but there's a lot of confusion here about why some webm's wont play and if it was tagged "VP9" instead of webm, people would realise what the problem was pretty quickly, rather than blaming webm.

https://ffmpeg.org/ffprobe.html#Stream-specifiers-1

here's the particular tool you want, complete with a "print_format" option to spit out the results in whatever text format best suits danbooru

THE FOLLOWING COULD SAVE ATF A LOT OF MONEY DEPENDING ON YOUR HOSTING FEES:

i don't know how much CPU time costs you guys - if you're paying the power bill or for cycles on a cloud server then probably not but:

you're presumably paying for bandwidth. if performing computations is cheap/free for you, perhaps you could take it a step further and automatically generate AV1 containing WEBMs from videos (or even gifs and APNGs, ffmpeg will take those) with large file sizes (and obsolete codecs)? encoding is computationally expensive but the file size reduction would be massive - i see 30mb APNGs here which would be > 2 MB with almost identical quality if re-encoded. i wouldn't bother doing this with any other video codec because they won't reduce file size that much if they're at a high enough bitrate to not artifact. these files could be made the parents of the original files they're generated from, or be presented in a manner similar to preview size images vs original images.

presenting a 2mb AV1 .webm by default to desktop users and only serving up a 36mb apng/h264 if they click "view original (animated png)" could save you guys a lot of tubes

alternatively you could force ppl who try to upload large files with poor compression (gifs/apngs/h264, vp8 to a lesser extent) to do it themselves...

Hey @extracool

I've asked our resident engineer to answer this question, but I'll answer what I can.

I wouldn't want to drop the webm or mp4 distinction because telegram treats the two differently. I'm all for adding code if it help better distinguish quality of the video. But even so I see no reasons why we have to drop any tags.

Plus anyone can ADD A TAG! If you think a tag would help, please add them. There's no reason to have a discussion about adding useful tags. Just do it!

But the other problem is ... beyond whether it's a mp4, webm or a gif I wouldn't know what else to add. I wouldn't know the useful codec. I find a video from ... somewhere. I look it what type of video I have mp4, webm or gif and I just add that.

I'm not sure how to look up useful codecs and even if I did, I probably wouldn't really check (sorry).

extracool said:

i might have put this in the Tags category, but i'm suggesting new serverside code so...

video tagging is a mess, "video" itself has 600 entries and no wiki entry (for all i know it's for diegetic video), yet there's over 1000 webms and ~400 mp4s, are they not videos?

i've filed a implication request for this https://booru.allthefallen.moe/forum_topics/133

but here i wanted to discuss the merits of dropping "webm" and "mp4" entirely, and instead classifying videos based on encoding (codec)

all the videos here are displayed with a HTML5 <video> tag. i cant imagine there's any browsers that support <video> but not both MP4 and WEBM inside them.

what determines if a device is able to play a video is the codec, especially mobile devices which don't have the power to brute force software/gpu compute render codecs they lack hardware support for.

this means that searching for "webm" or "mp4" is meaningless, because it's a matter of whether your device supports the codec, which is currently invisible.

i suggest new meta tags, they will require some data from the file header to generate but nothing any number of command line tools could do (i'm sure ffmpeg will return just the codec with the right arguments)

the tag webm should be removed, all existing webms should be tagged "video" and then by codec, which will be either VP8, VP9 or AV1. this is why some WEBMs won't play for people but others will - because they have different codecs.

the same for the tag mp4, to be replaced by h264 and h265 (aliased to AVC and HEVC respectively). again lots of devices can't play HEVC, but everything can play h264.

bonus points for doing this with the AUDIO TOO - i've seen a lot of video files marked as having audio but apparently being silent, and i wonder if it's because this browser doesn't support newer audio codecs (like opus on webms)

finally i suggest aliasing h264 and VP8 to a tag like "compatible video" (thats a shit name but you get the idea) because these will play on all devices, or the inverse, tagging the newer less supported codecs as "limited support".

SUMMARY: BENEFIT FOR NON-NERDS: find videos by searching "video", and find videos which always work with "compatible video"
BENEFIT FOR NERDS: selectively -ing codecs you know your device doesn't like (i bet when AV1 rolls out more almost no phone will play it) so you can just find every video you can watch.

i know this is a long ramble, but there's a lot of confusion here about why some webm's wont play and if it was tagged "VP9" instead of webm, people would realise what the problem was pretty quickly, rather than blaming webm.

I'm fine with adding the "video" tag whenever it applies. I thought it was automated at first, but apparently I'm wrong.

Currently this is just too much work changing stuff to really be worth it. Perhaps you're better of suggesting this to the danbooru developers, since that's what this booru is based on.