Allow more preview formats #13552
Comments
rullzer
added
enhancement
0. Needs triage
|
👍 for WebP, it is now supported by the major web browsers, and is more interesting than JPEG, PNG and GIF if we look at the quality/file size. I think that would be a nice addition, for instance for people who wants to save some storage space on their servers thanks to WebP, for a galery of pictures, and so on. |
|
ref #9857 |
|
This issue has been automatically marked as stale because it has not had recent activity and it seems to be missing some essential informations. It will be closed if no further activity occurs. Thank you for your contributions. |
|
There is a tiny app for WebP support written by @Flow86: Would you agree to merge this functionality upstream? |
|
yes of course you can merge the functionality to upstream |
|
Here's an interesting demo with blurred webp preloads: https://jmperezperez.com/webp-placeholder-images/ |
|
Can we get a status update on this? |
szaimen
added
1. to develop
This has been written in 2019.. when I see the code one would think this could be added to the main server software? Just a thought, when one takes a look at 'oc_filecache' you see that the preview images are generated in the same format as the original (JPG / PNG). Webp is now (version 24.0.3) not really supported. When I upload a webp image, the preview generated is in PNG! 'preview_format' => 'webp', This because webp is smaller then PNG and JPG, so when one would want previews in webp, oen would want it for all previews.. |
|
same for AVIF btw. |
|
We even regressed on this ticket, heif previews are now disabled |
|
Hope this is still active, adding AVIF support would outstanding, its supported in imageMagick. I did some testing with some of my large (60MP) pictures and its able to reduce ~10MB photos (jpg) to 0.5M-2MB (avif) |
|
Clearly the frontrunners here are WEBP and AVIF. AVIF for "best" format (with caveats) and WEBP for the (still current, yet quickly shrinking) advantage of browser support it has over AVIF. JPG and PNG are ancient. https://jakearchibald.com/2020/avif-has-landed/ I am interested in helping with this if it gets more direction. |
|
I haven't actually used Nextcloud, but from what I can tell after reading the code, previews are generated by simply resizing original images and preserving the format (whether it be JPEG, PNG, WebP, ETC). Some compression is added, at least for JPEGs. So we want to allow all previews to be of a specific format. Note that Nextcloud seems to have WebP support [ 6d4afca ]. It does not seem to have AVIF support. Let's break this down - this issue
at this point WebP previews are available maybe this issue, maybe other issues
I am going to look into the main part first - saving in specific formats. The rest is chores. |
|
.MP4, .MKV, and .MOV files work for me. The crucial thing is that FFmpeg is installed, and the settings are configured in the Config. |
|
Then I am missing just the settings, thank you. Will check it. Edit: It works, even AV1. Amazing. Thank you @JanisPlayer !! Can AVIF also be thumbnailed? |
Yes, the patch enables that as well, at least in the current version. I've also provided instructions on how to use it in the PR under the Experimental section: |
|
I am wondering why Nextcloud isn't creating previews for my |
|
This is because the standard Nextcloud does not yet allow this. You have to install the patch from me under the pool request. All you really have to do is replace all the files.Then check file permissions and restart server. So, as I said, i still working on parts of it being included in the main project. |
|
How about adding an option to allow for 10bit AVIF previews? Compatibility-wise, since the era of H.265 and VP9, devices that supported 8bit usually also started supporting 10bit, not going the route H.264 went where 10bit was unsupported on many devices for a long time. 10bit can save bandwidth over 8bit, even with 8bit sources (at least in video streaming). Of course, encoding time for 10bit would be about 10-20% longer. If at all, there probably should be three options for high-bit-depth AVIF previews: I ran some tests that clearly showed a reduction in file size for my 2048x2048 test image (downsampled to 8bit, as the 16bit TIFF messed with the image quality metrics):
Both WEBP and AVIF absolutely annihilated the natural grain of the test image, leading to very low scores compared to JPEG, which does a better job, but also adds horrible artefacts. This is a sort of worst-case scenario. AVIF definitely should be encoded with some grain synthesis here, which I can't do by simply exporting from GIMP; grain synthesis with AOM requires the images to be encoded in 4:4:4 though. This can only be circumvented by first building the grain profile and exporting it as a text file, then adding it back in during encoding. |
|
@DesertCookie AVIF 8bit is already poorly supported; but 10bit creates more compatibility issues. For example baseline 8bit works on new Samsung devices out of the box, while 10 bit does not. For now, it creates probably much more support requests – especially on systems with a large user base – than the few bytes saved will be worth. Qualitywise it makes no real difference because it can't be spotted and previews should give an impression, but don't need high fidelity graphics. |
|
@DesertCookie, yes, those are interesting results. However, it is not yet clear whether AVIF will make it into the PR as a preview format developer feature; that decision still needs to be made. You used a different encoder in your test than what is currently available to me. For Nextcloud, there are only two options to use AVIF: GD or ImageMagick and Imaginary (eliminated), as anything else is considered too ambitious and labor-intensive for the PR. I opted for GD because it was faster and simpler, requiring less rewriting. The encoder quality of GD is different from what you are currently accustomed to, and the available options are fewer. The documentation doesn't even specify whether 10-bit is supported. Therefore, it is interesting for the future and for my personal use, but unfortunately, it is currently challenging to implement for the PR. |
|
@J0WI I think it's possible to replace ImageMagick with GraphicsMagick. It's easily usable in PHP with almost the same functionality. Vips also sounds interesting, but it's more complex to use. However, the current functionality of the Preview Generator won't automatically switch all formats to GraphicsMagick. This means you'd not only have to rewrite existing ImageMagick Preview Generators as a fallback for GD but also add new ones. Whether the performance boost is worth it is the next question. Since a new solution for fast generation seems necessary soon, especially considering Imaginary is no longer being developed by the original developer. It might be used for modernization over time. Just a speculation. Thanks for the acceleration tips; I can use those elsewhere. |
|
@JanisPlayer I'm facing an issue with your patch. Running the pre-generate I get I am unsure whether it is an issue with your patch at all, or possibly my instance/install: Fresh instance of Nextcloud AIO (less than a week old) with v27.1.7 RC1 (your patch is only for v27.1.2). Preview format is set to WebP with a quality of 30. At least since applying your patch I don't even get previews downloaded (or possibly even generated) to my Desktops via the desktop client for images (usually it does load a preview, even when the file is only present online). |
|
I took a brief look at the file. At this point, it checks if your PHP version supports GD WebP with Edit: |
|
Trying again today, I do not get any log output after running |
|
The errors may have disappeared because images had already been generated. But since the patch is actually intended more for manual installation, this may also be due to differences compared to all in one installation.Since there is no way to install it, can you give me more information about your installation and how you ran the patch? So it can actually only be due to a missing lib, module or incorrect PHP version. Or that the version is actually incompatible with the patch. But since WebP previews are enough for you, in version 28 works without a patch. You can read about how Imaginary does this in the documentation. But if you really want AVIF or want to create the previews via GD, this is currently not possible without a patch. Unfortunately, I don't have any information about when a review is planned. |
|
WebP would be enough for me. However, Nextcloud AIO doesn't allow me to upgrade to 28 yet. I simply installed the I simply downloaded the v27 patch ZIP and replaced the files in my volume with the ones from the patch. Then I restarted the container (Nextcloud and Apache). |
|
@JanisPlayer the |
|
At this point, the default value is defined, which is used if no value is set via |
|
Hi. I just want to double check if this patch series enables using avif or webp as format for the previews, even if the original image is a jpeg? I have just upgraded to NC 29 and thought it would have a go at regenerating my previews. I have not applied your patches yet. This is current status for my system after removing all previews and regenerating them using config.php |
|
WebP for previews is already supported in the current version of Nextcloud via Imaginary. If anyone wants to automate this process, for example, using Bash, they can share the script with me. But I'm not sure if there is enough demand to justify it. Ideally, this PR should have been included in Nextcloud long ago. |
I see..I might look at Imaginary again. I just didn't see a huge benefit, and it also had problems with some files that otherwise worked. I think, for example, dng/raw files did not work with Imaginary. I am surprised that Imaginary is a requirement for WebP as PHP natively supports it in its gd library since a while back. I created a ticket for allowing WebP previews via PHP gd. In PHP 8.1 even AVIF is natively supported. #46611 |
|
Hey! So I'm facing a strange issue. Somehow, even though I enabled
|
|
@jmigual There should be an error in the log when dealing with image formats that Imaginary does not support. As a fallback, the GD or Imagemagick generator is used, both of which do not support WebP. It might also be that your Imaginary has too low a limit on the number of images that can be generated simultaneously. This will also appear as an error message in the log. You can increase the limit in the Docker config. With your log and information about the image properties of the files that do not work and the config of your Imaginary container, we might be able to get to the bottom of the problem. @Forza-tng Yes, that's exactly what my pull request does. The code of Preview Generator in its original version is very limited and only allows a few formats, so it's not PHP's fault. A modification for Imaginary was easier back then than rewriting the entire GD generator. I am currently using WebP with 80 percent quality via Imaginary myself. |
|
Hello. I read above that AVIF is planned for NC 30? Is that still true? I have a rather large collection of images mounted externally but none are really accessible via NC since it ignores them. Has anything changed that now allows NC to read AVIF in the Files section? I'd like to second the request for this capability if there is a way to throw my request in. |
|
Please also think about jxl. Its better in compression and speed Else avif. Apple supports it and hope all others Like chrom and Firefox will follow |
|
While JPEG XL is another royality free modern image format, I am not sure if it is better in all aspects than AVIF. Proper requirements analysis has to be done to decide if it should be prioritized over AVIF. Ideally, the architecture of the preview generator should be modular to allow the easy addition of further image formats in the future. By following this issue I think that there is not too much left until we could have AVIF support. So I think it is best to do that first. |
|
@strech345 JPEG XL is currently supported by Apple, but it has been removed from Chrome and Firefox. It can now only be supported via WebASM or JavaScript libraries. It’s a wonderful image format in terms of versatility, but it also has the same problems as JPEG — 8x8 pixel blocks tend to produce artifacts in small images. However, a major advantage is that JPEG XL generally performs better with details found in the real world, while AVIF tends to blur these or completely remove or misrepresent them. Here's a comparison. Since JPEG XL needs to be supported by both web browsers and operating systems for smartphones, it's highly unlikely that it will have a future as a preview format. The smartphone's OS, like with AVIF, would occasionally need an extra library to display all images. For example, AVIF has issues with images over 1080p on Android. Preview formats are only concerned with displaying an image with good quality, minimal size, acceptable encoding speed for the system, and wide support. Unfortunately, given the current support, it doesn't meet those criteria. JPEG XL might still have a future if tools like ImageMagick or PHP’s GD library support it. Then I could convert JPEG XL images into previews. I think Nextcloud's standard Photos app will support it eventually, though I've noticed that some app developers don’t seem eager to support all formats. I believe they'll catch up if they see it stick around, but I'm not entirely sure. Generally speaking, I had the issue where at some point no one provided any feedback, even though everything was finished and needed it. So I have no idea if these new formats even have a future or if you need to ring a bell on GitHub. However, I recently read under a different pull request that supposedly version 30 would be it. I don't really believe that; as you can see, it was closed and bumped up to version 31. So, because of all these issues, I wouldn’t add any more preview formats to my pull request. Instead, I’d add the option to convert JPEG XL into previews if that’s possible through PHP. As I explained above, adding a new preview format is tricky since it requires broader support or extra libraries, and then there are the apps to consider. I also need testers to spot problems and other developers who are actively working on the project to tell me what's possible, like whether I'm allowed to use certain libraries, which could mean significant changes. I think that’s why the developers probably want to handle it themselves. My pull request and the other one may just serve as templates or ideas. That’s why I didn’t want to make my pull request too complicated, so it could be integrated quickly. So, in conclusion, JPEG XL looks difficult to support, especially since I already had problems with a well-supported format like AVIF. |
|
Well, in IMHO, if the preview and image handling features would be image format agnostic, usersand organisation can choose which format is best for their needs. New formats like JPEG-XL would then be easy to add If we keep hard-coding formats, it will always be difficult to make changes.
One frequent complaint for home users is the large disk space usage of the thumbnails. AVIF and WebP would be very beneficial for those, as they can save 50% disk space compared to JPEG. I did a qualitative (SSIM) test of jpeg, avif and webp so it is possible to see for a specific image size and target quality, the resulting file sizes would be for each format. There are many ways to measure quality, and I chose one I could easily script with imagemagic. Feel free to try other methods.
|
There are two things we have to separate here:
Regarding (1): If the preview generator app uses a library on the server that can decode lots of image formats, it would be nice to generate small preview images for these formats. Additionally, there should be an option to enable/disable individual formats. For each image format this requires some (small) amount of code. Your wish to avoid hard coding formats could be realized this way: Adding an option to enable reading any image format that the photo library supports. This way, as soon as new formats are supported by the library, previews can get generated on the Nextcloud server. However, I don't think that this is an option enterprises would want to use on their Nextcloud instances for stability/security reasons. Regarding (2): The list of possible formats for the generated preview images is limited and can only slowly adopt. The reason here is that all clients (supported web browsers, mobile Apps, etc.) need to support all these formats. That's why it takes so long to support e.g. AVIF as format for the preview images. As explained avove, JPEG XL is not (properly) supported by all common webbrowsers. |
For my use case with family photo archive of 100k photos it would make a rather big difference for us. All our devices would have no problem with either webp or avif. Why shouldn't there be an option to configure the format? There is no need, at least for this use case to implement some kind of auto detection, browser sniffing, and all that. Just a simple configuration option. |
|
@Forza-tng yes, it's just not as simple as they imagine. But they can already use WebP today, as I do. The savings are usually only about 10% in most cases. If the visual quality isn't that important, they can lower the quality even further. The encoders in GD, ImageMagick, or Imaginary are usually set for fast encoding speed, which also affects the quality. So, we’re not using the library directly. Therefore, the comparison of perceived visual quality is a bit different. I think I already have close to 50k photos on my Nextcloud, and with WebP, the storage for previews up to 4096px at 80% quality is only about 16.4 GiB. If they're only interested in displaying JPEG XL, meaning having Nextcloud convert a JPEG XL image into a WebP preview image, that's probably easier to support. The prerequisite is that GD or ImageMagick (Imaginary is unlikely, as it’s no longer being updated, and as a Nextcloud developer told me, they don’t want to take it over) provides support for decoding. @langfingaz also made an interesting point about using modulo and libraries, which would require significant maintenance, and only the developers themselves can decide if they want to make such substantial changes to the preview generator. There may also be security concerns with maintaining the libraries. So, it's not that simple. They can check out my PR to understand the basic workings of the current preview generator and how the apps need to be modified so they can load the images. But it's not as easy as they imagine. I think they can also see that JPEG XL, as I explained above, due to the issue with support requiring many libraries, is currently unlikely to be used as a preview format. Converting it to a WebP preview is much simpler if the decoder is supported, and adapting the apps to recognize it as an image and load the preview is doable. But as a preview format, JPEG XL is highly unlikely to be used in productive systems right now due to the lack of native support. My tip is to enable WebP as a preview format, which will save a few percent and, in some cases, even more. |
and others
Right now we serve previews in png or jpg. Which has the advantage of being support on nearly every platform since the late 90s. But of course it is no longer the best solution out there.
I would propose to allow different image formats. The top candidate in my opinion is webp. This allows for smaller images at comparable quality. Of course the backend would need to distinquis what to server but I'm sure we can solve that with some js magic to set the accept headers ;)
This doesn't mean we fully get rid of the jpg an png as older firefox and IE versions do not support it. But I bet this could be performance boost especially when using a lot of images.
The text was updated successfully, but these errors were encountered: