Generate thumbnails for videos and reencode images #105

New Issue

AsafGartner · 2023-03-29T16:03:00Z

AsafGartner commented

2023-03-29 16:03:00 +00:00

We have a lot of user-submitted images and videos. Most of them from Discord, and some uploaded directly to the website.

We're displaying a large number of those on the front page and in some cases it results in a 100mb download, because the images are full size and the videos are set to preload=metadata in order to display the first frame.

We need to do the following:

Extract an image from our videos (and gifs) to serve as the thumbnail. The videos can be in any format, so we would probably have to use ffmpeg.
We should probably have two sizes for video thumbnails: full-size and small (not sure what the size should be exactly for small, but it's for displaying lots of videos in a list)
Convert our images to webp, and create a small thumbnail version of each image as well.
We need to run this on all of our existing assets, as well as new assets as they come in.

We have a lot of user-submitted images and videos. Most of them from Discord, and some uploaded directly to the website. We're displaying a large number of those on the front page and in some cases it results in a 100mb download, because the images are full size and the videos are set to `preload=metadata` in order to display the first frame. We need to do the following: * Extract an image from our videos (and gifs) to serve as the thumbnail. The videos can be in any format, so we would probably have to use ffmpeg. * We should probably have two sizes for video thumbnails: full-size and small (not sure what the size should be exactly for small, but it's for displaying lots of videos in a list) * Convert our images to webp, and create a small thumbnail version of each image as well. * We need to run this on all of our existing assets, as well as new assets as they come in.

bvisness commented

2023-03-29 16:14:40 +00:00

I think avatars are using the new asset system by now, right? I think we should re-encode basically all the images in our whole asset system to smaller sizes / formats, and then we can start using <picture> elements and similar to serve up multiple formats and 2x/3x versions of images as necessary. I saw an enormous benefit on my personal website from doing similar.

The big question for me is what tools to use for the conversions and what strategies to use for various output formats. On my personal website I control the image formats we use, so it's easy for me to say "image/jpeg gets reencoded as smaller jpegs, and image/png and image/webp both get reencoded as png and webp" (source). Obviously we don't quite have that luxury here, plus we need to handle gifs, which are notorious file size hogs.

(Unless we want to treat gifs as videos and extract the first frame?)

I imagine the assets table will need to be restructured to allow multiple files per asset. Curious to see what that will look like.

I think avatars are using the new asset system by now, right? I think we should re-encode basically all the images in our whole asset system to smaller sizes / formats, and then we can start using `<picture>` elements and similar to serve up multiple formats and 2x/3x versions of images as necessary. I saw an enormous benefit on my personal website from doing similar. The big question for me is what tools to use for the conversions and what strategies to use for various output formats. On my personal website I control the image formats we use, so it's easy for me to say "image/jpeg gets reencoded as smaller jpegs, and image/png and image/webp both get reencoded as png and webp" [(source)](https://github.com/bvisness/bvisness.me/blob/main/pkg/images/images.go#L38-L49). Obviously we don't quite have that luxury here, plus we need to handle gifs, which are notorious file size hogs. (Unless we want to treat gifs as videos and extract the first frame?) I imagine the assets table will need to be restructured to allow multiple files per asset. Curious to see what that will look like.

AsafGartner commented

2023-03-29 16:17:36 +00:00

We might as well convert gifs to videos and treat the asset as video. The user experience would be better that way (seeking).

👍 1

Sign in to join this conversation.