feat: original-sized previews for non-web-friendly images #14446

eligao · 2024-12-02T12:38:32Z

Server: extract and keep the original-sized preview image
- If image.extractEmbedded in options is set, it will attempt to extract the JPG/Preview from RAW images.
- If the preview is too small or the extractEmbedded is not set, it will fall back to creating a full-sized image from RAW.
- The result will be saved as a separate AssetFileType.converted file.
OpenAPI: a new AssetMediaSize.Original size is available under the viewThumbnail API
- It returns the original-sized preview for RAW files
- Or otherwise the original image itself.
Web: photo-viewer
- When zoomed in, the browser loads the AssetMediaSize.Original preview via the viewThumbnail API
- This applies to both RAW and non-RAW images, instead of previously only loading the original file for non-RAW image.
- This file can be right-clicked and downloaded/copied for sharing.

Some loose ends:

The converted preview of RAW don't have original EXIF
- Especially the color profile is missing, making the rendering a bit off other than sRGB images.
- This would be a nice touch to help easier organize and share the compressed images from RAW.
- And it can open up the possibilities to "trim" the RAW images and replace them with just the JPG with all EXIF, freeing the space on the server.
The way to download the file is implicit
- A button, especially on the mobile app, would be helpful, but I'm not familiar with flutter.
The API endpoint to download the original preview and original file for non-RAW files are different, but the contents are identical, this is not good for caching, maybe better use 302 instead?

mertalev

Thank you for the PR! I mostly looked at the server code, where I think it needs some more polishing.

Re: caching, this also affects the case where the user zooms in and no extracted image exists, right? It will load the preview again.

server/src/repositories/media.repository.ts

server/src/services/media.service.ts

eligao · 2024-12-06T06:15:56Z

any more comments from maintainers to get this merged, so that I can continue with previews of other images?
@danieldietzler @alextran1502

alextran1502 · 2024-12-06T06:17:12Z

@eligao no rush here

mertalev · 2024-12-06T06:20:14Z

I think we should wait until we're done with hot fixes for the current release.

michelheusschen · 2024-12-07T07:50:47Z

For a DNG image with a resolution of 5376x3956 and 41.3MiB in size, the following variants are generated using the default settings:

Variant	Format	Resolution	Size
Thumbnail	webp	5280x3956	887KiB
Preview	jpeg	5280x3956	1794KiB
Original	jpeg	5280x3956	1794KiB

These generated variants seem larger than expected and don't match the configured settings, with the preview and original being identical. It's also strange that the resolution has slightly changed.

eligao · 2024-12-07T08:22:27Z

For a DNG image with a resolution of 5376x3956 and 41.3MiB in size, the following variants are generated using the default settings:

Variant Format Resolution Size
Thumbnail webp 5280x3956 887KiB
Preview jpeg 5280x3956 1794KiB
Original jpeg 5280x3956 1794KiB
These generated variants seem larger than expected and don't match the configured settings, with the preview and original being identical. It's also strange that the resolution has slightly increased.

nice catch, the file sizes are indeed wrong, let me fix that and add checks to tests.
as for the sizes being slightly different, it's likely due to the crop tags in exif:

mertalev · 2024-12-10T23:19:19Z

Hi again @eligao! We've been talking about the PR and think there are a few changes it'll need for it to be merge-able.

It should apply to all images that aren't web compatible (using the same list as web, ignoring that Safari can view HEIF images)
It should be configurable in the image settings, similar to the thumbnail and preview sections
- There should be an "enabled" toggle that is disabled by default
- The target format should be configurable in the case where an embedded image is not used and a new image has to be generated
Rename the "converted" enum to "fullsize"

mertalev · 2024-12-16T22:00:57Z

server/src/repositories/media.repository.ts

    return true;
  }

+  async cloneExif(input: string, output: string): Promise<boolean> {


Copying all of the metadata seems unnecessary. It adds to the size of the output, and there are surely metadata entries that are correct for the RAW, but wrong for the extracted image.

Can you narrow down which field(s) specifically need to be added? I'd like to see if we can avoid that exiftool.read call and just apply the data in exifInfo that we already have from metadata extraction.

The size added is neglible, a full set of EXIF without embedded images are usually <20KB as I tested from CR2 and ARW, adding ~1% size to a 2MB fullsize preview image.

My purpose is to preserve integrity of EXIF fields in the fullsize preview, as if it was taken with JPEG+RAW enabled or extracted from camera vendors' editors.

It is a nice catch there might be some extra RAW-specific fields, I'll stick to what we have in ExifEntity now and come up with a cleaner solution later.

I still don't see what the point is to including all the EXIF fields in the image. This is a generated image with a specific purpose. It doesn't need to have anything beyond what's need to display it correctly. If anything, the presence of rich metadata in the image is a downside that prevents sharing due to metadata leakage.

This fullsize image extracted from RAW can be more than just specifically for preview. I'm thinking of adding features like "download as JPG" and "remove RAW but keep JPG to cleanup space" for RAW files later, which would be very handy for workflows, and it's important to preserve full EXIF fields.

As for EXIF leaking, this is indeed an issue when a shared link disallows downloading and hides EXIF but original JPG files can be acquired via thumbnail?size=fullsize, let me address this issue.

But anyway, this cloneExif() is already removed for now.

I see what you're going for, but I think this should be handled when those features are actually implemented. It can be done as part of the request to download them, etc.

Good point, let me keep only colorspace and orientation EXIF in this PR.

Also I have added Permission.ASSET_DOWNLOAD requirement for thumbnail?size=fullsize to prevent leaking EXIF via original files

server/src/repositories/media.repository.ts

mertalev · 2024-12-16T22:06:30Z

server/src/services/media.service.ts

+      this.mediaRepository.generateThumbhash(data, thumbnailOptions),
+      this.mediaRepository.generateThumbnail(data, { ...image.thumbnail, ...thumbnailOptions }, thumbnailPath),
+      this.mediaRepository.generateThumbnail(data, { ...image.preview, ...thumbnailOptions }, previewPath),
+      shouldConvertFullsize &&


I'd prefer making this a list that you conditionally append to, then waiting for that list with Promise.all.

Good suggestion, but it would break inferred type of outputs[0]

e2e/src/api/specs/asset.e2e-spec.ts

mertalev · 2024-12-16T22:12:31Z

open-api/immich-openapi-specs.json

@@ -11775,6 +11777,9 @@
          "extractEmbedded": {
            "type": "boolean"
          },
+          "fullsizePreview": {


I was thinking of something more like "fullsize": {"enabled": false, "format": "jpeg"}, where format is ignored if using the extracted preview.

to make it simple and stupid, I think it is intuitive for "fullsize" to just follow "preview" settings.

In the past, we've gotten complaints about image settings being shared between different files, and there may be other options related to the fullsize image added down the line. For example, a quality option might be added if users are concerned about file size and want 90 or 95 quality instead of 100, Safari users might complain because they don't need fullsize images for HEIF, etc.

I think the config field should be an object to make it easier to add settings in the future. The format is maybe superfluous in a vacuum, but the fact that one setting (format) is shared between the preview and the fullsize and the other two (quality, resolution) are not can lead to confusion. Having them completely separate will be less ambiguous.

server/src/repositories/media.repository.ts

eligao · 2024-12-17T10:43:33Z

server/src/repositories/media.repository.ts

    return true;
  }

+  async cloneExif(input: string, output: string): Promise<boolean> {


The size added is neglible, a full set of EXIF without embedded images are usually <20KB as I tested from CR2 and ARW, adding ~1% size to a 2MB fullsize preview image.

My purpose is to preserve integrity of EXIF fields in the fullsize preview, as if it was taken with JPEG+RAW enabled or extracted from camera vendors' editors.

It is a nice catch there might be some extra RAW-specific fields, I'll stick to what we have in ExifEntity now and come up with a cleaner solution later.

eligao · 2024-12-17T11:05:18Z

server/src/services/media.service.ts

+      this.mediaRepository.generateThumbhash(data, thumbnailOptions),
+      this.mediaRepository.generateThumbnail(data, { ...image.thumbnail, ...thumbnailOptions }, thumbnailPath),
+      this.mediaRepository.generateThumbnail(data, { ...image.preview, ...thumbnailOptions }, previewPath),
+      shouldConvertFullsize &&


Good suggestion, but it would break inferred type of outputs[0]

eligao · 2024-12-17T14:38:14Z

server/src/repositories/media.repository.ts

+        delete exifTagsToWrite['Orientation'];
+      }
+      const result = await exiftool.write(output, exifTagsToWrite, {
+        ignoreMinorErrors: true,


tedious, inaccurate and incomplete, but works for now.

eligao · 2024-12-17T14:51:48Z

open-api/immich-openapi-specs.json

@@ -11775,6 +11777,9 @@
          "extractEmbedded": {
            "type": "boolean"
          },
+          "fullsizePreview": {


to make it simple and stupid, I think it is intuitive for "fullsize" to just follow "preview" settings.

eligao requested a review from danieldietzler as a code owner December 2, 2024 12:38

github-actions bot added 🖥️web 🗄️server labels Dec 2, 2024

danieldietzler added the changelog:enhancement label Dec 2, 2024

eligao changed the title ~~feat/raw fullsize preview~~ feat: original-sized previews for RAW images Dec 2, 2024

eligao force-pushed the feat/raw-fullsize-preview branch from f2b631a to 54b91b4 Compare December 2, 2024 16:35

mertalev reviewed Dec 2, 2024

View reviewed changes

server/src/services/media.service.ts Outdated Show resolved Hide resolved

mertalev reviewed Dec 2, 2024

View reviewed changes

server/src/services/media.service.ts Outdated Show resolved Hide resolved

server/src/services/media.service.ts Outdated Show resolved Hide resolved

eligao force-pushed the feat/raw-fullsize-preview branch 5 times, most recently from 076d59d to a7a9b4d Compare December 3, 2024 21:24

eligao commented Dec 3, 2024

View reviewed changes

server/src/services/media.service.ts Show resolved Hide resolved

eligao force-pushed the feat/raw-fullsize-preview branch from a7a9b4d to 34592b3 Compare December 3, 2024 21:29

eligao force-pushed the feat/raw-fullsize-preview branch from ebcfe9c to e22a123 Compare December 16, 2024 20:37

eligao changed the title ~~feat: original-sized previews for RAW images~~ [WIP] feat: original-sized previews for non-web-friendly images Dec 16, 2024

mertalev reviewed Dec 16, 2024

View reviewed changes

eligao commented Dec 17, 2024

View reviewed changes

eligao changed the title ~~[WIP] feat: original-sized previews for non-web-friendly images~~ feat: original-sized previews for non-web-friendly images Dec 17, 2024

eligao added 2 commits December 18, 2024 00:44

feat(server): extract full-size previews from RAW images

d3ebc28

feat(web): load fullsize preview for RAW images when zoomed in

57436ae

eligao added 4 commits December 18, 2024 00:44

refactor: tweaks for code review

5b499ef

refactor: rename "converted" preview/assets to "fullsize"

5dbe675

feat(web/server): fullsize preview for non-web-friendly images

069c7fc

feat: tweaks for code review

4cee0c7

eligao force-pushed the feat/raw-fullsize-preview branch from 8d43411 to 4cee0c7 Compare December 17, 2024 16:45

eligao added 4 commits December 18, 2024 01:52

feat(server): require ASSET_DOWNLOAD premission for fullsize previews

e892998

test: fix types and interfaces

4b0fb82

chore: gen open-api

152cbbf

feat(server): keep only essential exif in fullsize preview

75d1647

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: original-sized previews for non-web-friendly images #14446

feat: original-sized previews for non-web-friendly images #14446

eligao commented Dec 2, 2024 •

edited

Loading

mertalev left a comment

eligao commented Dec 6, 2024

alextran1502 commented Dec 6, 2024

mertalev commented Dec 6, 2024

michelheusschen commented Dec 7, 2024 •

edited

Loading

eligao commented Dec 7, 2024

mertalev commented Dec 10, 2024

mertalev Dec 16, 2024

eligao Dec 17, 2024

mertalev Dec 17, 2024 •

edited

Loading

eligao Dec 17, 2024

mertalev Dec 17, 2024

eligao Dec 17, 2024

eligao Dec 17, 2024

mertalev Dec 16, 2024

eligao Dec 17, 2024

mertalev Dec 16, 2024

eligao Dec 17, 2024

mertalev Dec 17, 2024

eligao Dec 17, 2024

eligao Dec 17, 2024

eligao Dec 17, 2024

eligao Dec 17, 2024

feat: original-sized previews for non-web-friendly images #14446

Are you sure you want to change the base?

feat: original-sized previews for non-web-friendly images #14446

Conversation

eligao commented Dec 2, 2024 • edited Loading

mertalev left a comment

Choose a reason for hiding this comment

eligao commented Dec 6, 2024

alextran1502 commented Dec 6, 2024

mertalev commented Dec 6, 2024

michelheusschen commented Dec 7, 2024 • edited Loading

eligao commented Dec 7, 2024

mertalev commented Dec 10, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mertalev Dec 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eligao commented Dec 2, 2024 •

edited

Loading

michelheusschen commented Dec 7, 2024 •

edited

Loading

mertalev Dec 17, 2024 •

edited

Loading