Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No Exif data found in HEIF file #2162

Open
novomesk opened this issue Mar 23, 2022 · 18 comments
Open

No Exif data found in HEIF file #2162

novomesk opened this issue Mar 23, 2022 · 18 comments
Labels
enhancement feature / functionality enhancements

Comments

@novomesk
Copy link

Hello,

I am using exiv2 0.27.5 and I observed that sometimes it doesn't find metadata in HEIF/HEIC files.

For example in the following testfile:
test.zip

exiv2 pr test.heic
File name       : test.heic
File size       : 6998 Bytes
MIME type       : image/heic
Image size      : 2001 x 1984
test.heic: No Exif data found in the file

However using exiftool, the Exif and XMP metadata is printed:

exiftool test.heic
ExifTool Version Number         : 12.30
File Name                       : test.heic
Directory                       : .
File Size                       : 6.8 KiB
File Modification Date/Time     : 2022:03:23 09:17:05+01:00
File Access Date/Time           : 2022:03:23 09:17:05+01:00
File Inode Change Date/Time     : 2022:03:23 09:17:05+01:00
File Permissions                : -rw-r--r--
File Type                       : HEIC
File Type Extension             : heic
MIME Type                       : image/heic
Major Brand                     : High Efficiency Image Format HEVC still image (.HEIC)
Minor Version                   : 0.0.0
Compatible Brands               : mif1, heic
Handler Type                    : Picture
Primary Item Reference          : 2
Meta Image Size                 : 2001x1984
Exif Byte Order                 : Little-endian (Intel, II)
Bits Per Sample                 : 8 8 8
Image Description               : Created with GIMP
X Resolution                    : 300
Y Resolution                    : 300
Resolution Unit                 : inches
Software                        : GIMP 2.99.11
Modify Date                     : 2022:03:23 09:17:05
User Comment                    : Created with GIMP
Color Space                     : sRGB
GPS Altitude                    : 0 m
XMP Toolkit                     : XMP Core 4.4.0-Exiv2
Document ID                     : gimp:docid:gimp:ba27dc33-ec08-46a5-a6df-a2f19c755619
Instance ID                     : xmp.iid:ee9ba390-54f2-4c92-bea9-7b9653926afd
Original Document ID            : xmp.did:ca062ce1-05a1-42a5-833e-9a3bb3e66faf
Format                          : image/heif
Api                             : 3.0
Platform                        : Linux
Time Stamp                      : 1648023425366505
Version                         : 2.99.11
Creator Tool                    : GIMP
History Action                  : saved, saved
History Changed                 : /metadata, /
History Instance ID             : xmp.iid:8df779c3-7098-4ff8-a982-4fc709d2feee, xmp.iid:575573a5-3af2-4ba8-a062-b002392b24e5
History Software Agent          : GIMP 2.99.11 (Linux), GIMP 2.99.11 (Linux)
History When                    : 2022:03:23 09:16:48+01:00, 2022:03:23 09:17:05+01:00
Creator                         : Danko
HEVC Configuration Version      : 1
General Profile Space           : Conforming
General Tier Flag               : Main Tier
General Profile IDC             : Main Still Picture
Gen Profile Compatibility Flags : Main Still Picture, Main 10, Main
Constraint Indicator Flags      : 0 0 0 0 0 0
General Level IDC               : 150 (level 5.0)
Min Spatial Segmentation IDC    : 0
Parallelism Type                : 0
Chroma Format                   : 4:2:0
Bit Depth Luma                  : 8
Bit Depth Chroma                : 8
Average Frame Rate              : 0
Constant Frame Rate             : Unknown
Num Temporal Layers             : 1
Temporal ID Nested              : Yes
Image Width                     : 2001
Image Height                    : 1984
Image Spatial Extent            : 2001x1984
Image Pixel Depth               : 8 8 8
Media Data Size                 : 6401
Media Data Offset               : 597
Image Size                      : 2001x1984
Megapixels                      : 4.0

Please investigate what is the problem with the test.heic that the metadata was not found.

@novomesk novomesk added the bug label Mar 23, 2022
@piponazo
Copy link
Collaborator

Thanks @novomesk for reporting this issue. I'll take a look when I have some spare time.

@clanmills
Copy link
Collaborator

There is a little Exif block in that file which can be read as follows:

1102 rmills@rmillsm1:~/Downloads $ dmpf test.heic | grep II
   0xcc0     3264: II*_.___.__.._.___..__..._.___..  ->  49 49 2a 00 08 00 00 00 0b 00 00 01 04 00 01 00 00 00 d1 07 00 00 01 01 04 00 01 00 00 00 c0 07
1103 rmills@rmillsm1:~/Downloads $ dd bs=1 skip=3264 count=10000 if=test.heic | exiv2 -pa -
Exif.Image.ImageWidth                        Long        1  2001
Exif.Image.ImageLength                       Long        1  1984
Exif.Image.BitsPerSample                     Short       3  8 8 8
Exif.Image.ImageDescription                  Ascii      18  Created with GIMP
Exif.Image.XResolution                       Rational    1  300
Exif.Image.YResolution                       Rational    1  300
Exif.Image.ResolutionUnit                    Short       1  inch
Exif.Image.Software                          Ascii      13  GIMP 2.99.11
Exif.Image.DateTime                          Ascii      20  2022:03:23 09:17:05
Exif.Image.ExifTag                           Long        1  220
Exif.Photo.UserComment                       Undefined  25  Created with GIMP
Exif.Photo.ColorSpace                        Short       1  sRGB
Exif.Image.GPSTag                            Long        1  276
Exif.GPSInfo.GPSAltitude                     Rational    1  0.0 m
1104 rmills@rmillsm1:~/Downloads $ 

Exiv2 bmff parser expects the Exif block to an iloc block as described in this diagram.
heic

Your file does have both infe and iloc records:

1101 rmills@rmillsm1:~/Downloads $ tvisitor -pRU test.heic 
STRUCTURE OF JP2 (heic) FILE (MM): test.heic
 address |   length | box  | uuid | data
       0 |       24 | ftyp |      | heic____mif1heic 104 101 105 99 0 0 0 0 109 105 102 49 104 101 105 99
      24 |      565 | meta |      | _______!hdlr________ 0 0 0 0 0 0 0 33 104 100 108 114 0 0 0 0 0 0 0 0
  STRUCTURE OF JP2 FILE (MM): test.heic:36->553
         0 |       33 | hdlr |      | ________pict________ 0 0 0 0 0 0 0 0 112 105 99 116 0 0 0 0 0 0 0 0
        33 |       14 | pitm |      | _____. 0 0 0 0 0 2
        47 |       16 | idat |      | ____.... 0 0 0 0 7 209 7 192
        63 |       96 | iloc |      | .___D@_._.______.U_. 1 0 0 0 68 64 0 4 0 1 0 0 0 0 0 0 2 85 0 1
        71 |       20 |   ID |    1 |      0,  2663
        91 |       20 |   ID |    2 |      0,     8
       111 |       20 |   ID |    3 |      0,   306
       131 |       20 |   ID |    4 |      0,  3432
       159 |      119 | iinf |      | _____.___.infe.__._. 0 0 0 0 0 4 0 0 0 21 105 110 102 101 2 0 0 1 0 1
    STRUCTURE OF JP2 FILE (MM): test.heic:36->553:173->105
           0 |       21 | infe |    1 | .__._.__hvc1_ 2 0 0 1 0 1 0 0 104 118 99 49 0
          21 |       21 | infe |    2 | .____.__grid_ 2 0 0 0 0 2 0 0 103 114 105 100 0
          42 |       21 | infe |    3 | .__._.__Exif_ 2 0 0 1 0 3 0 0 69 120 105 102 0
          63 |       42 | infe |    4 | .__._.__mime_applica 2 0 0 1 0 4 0 0 109 105 109 101 0 97 112 112 108 105 99 97
    END: test.heic:36->553:173->105
       278 |      221 | iprp |      | ___.ipco___{hvcC..p_ 0 0 0 187 105 112 99 111 0 0 0 123 104 118 99 67 1 3 112 0
    STRUCTURE OF JP2 FILE (MM): test.heic:36->553:286->213
           0 |      187 | ipco |      | ___{hvcC..p_________ 0 0 0 123 104 118 99 67 1 3 112 0 0 0 0 0 0 0 0 0
      STRUCTURE OF JP2 FILE (MM): test.heic:36->553:286->213:8->179
             0 |      123 | hvcC |      | ..p_________.._...._ 1 3 112 0 0 0 0 0 0 0 0 0 150 240 0 252 253 248 248 0
           123 |       20 | ispe |      | ______..__.. 0 0 0 0 0 0 7 209 0 0 7 192
        BMFF.ispe.Version                0
        BMFF.ispe.Flags                  0 0 0
        BMFF.ispe.Width                  2001
        BMFF.ispe.Height                 1984
           143 |       20 | ispe |      | ______..__.. 0 0 0 0 0 0 7 210 0 0 7 192
        BMFF.ispe.Version                0
        BMFF.ispe.Flags                  0 0 0
        BMFF.ispe.Width                  2002
        BMFF.ispe.Height                 1984
           163 |       16 | pixi |      | ____.... 0 0 0 0 3 8 8 8
      END: test.heic:36->553:286->213:8->179
         187 |       26 | ipma |      | _______._...._.... 0 0 0 0 0 0 0 2 0 1 2 129 3 0 2 2 2 132
    END: test.heic:36->553:286->213
       499 |       54 | iref |      | _______.dimg_._._.__ 0 0 0 0 0 0 0 14 100 105 109 103 0 2 0 1 0 1 0 0
  END: test.heic:36->553
     589 |     6409 | mdat |      | __.c(.....5.....N.:. 0 0 10 99 40 1 175 19 15 159 53 170 245 235 213 171 78 157 58 149
END: test.heic

My suspicion is that the parser expects the iloc to occur after the infe record. I'm on vacation at the moment and will investigate further when I return home.

@clanmills
Copy link
Collaborator

I don't think the parser is sensitive to the order of the iloc/infe boxes. The Exif data is ID=3 which is at offset 306. The Exif metadata is at offset 3432 which is ID=4. I'll investigate more when I am home from vacation.

@clanmills
Copy link
Collaborator

clanmills commented Mar 25, 2022

The pairs in the iloc array define the offset, length of the metadata block. Both Exif and XMP length 306 and 3432 are correct and can be confirmed with Exiftool. Both Exif and XMP offset are zero, and Exiftool reports the offset as zero.

The correct offset for Exif is 3260 and XMP is 3566.

exiftool -v5 test.heic reports:

 + [Processing items from ItemInformation directory with 4 entries]
 Item 1) 'hvc1' (2663 bytes)
  |     0255: 00 00 0a 63 28 01 af 13 0f 9f 35 aa f5 eb d5 ab [...c(.....5.....]
  |     0265: 4e 9d 3a 95 2a 54 a7 4e 9d 3a 54 68 d1 a3 4a 8d [N.:.*T.N.:Th..J.]
  |     0275: 1a 34 68 d1 80 e6 c8 68 53 ff ff ba e3 a7 94 61 [.4h....hS......a]
...
Item 2) 'grid' (8 bytes)
  |     [not extracted]  (Can't currently extract grid with construction method 1)
Item 3) 'Exif' (306 bytes)
  |     0cbc: 00 00 00 00 49 49 2a 00 08 00 00 00 0b 00 00 01 [....II*.........]
  |     0ccc: 04 00 01 00 00 00 d1 07 00 00 01 01 04 00 01 00 [................]
...
Item 4) 'application/rdf+xml' (3432 bytes)
... 

The information exiv2 -pS is:

518 rmills@rmillsmm-local:~/gnu/github/exiv2/team.exiv2.org/book/build $ exiv2 -pS ~/Downloads/test.heic 
Exiv2::BmffImage::boxHandler: ftyp        0->24 brand: heic
Exiv2::BmffImage::boxHandler: meta       24->565 
  Exiv2::BmffImage::boxHandler: hdlr       36->33 
  Exiv2::BmffImage::boxHandler: pitm       69->14 
  Exiv2::BmffImage::boxHandler: idat       83->16 
  Exiv2::BmffImage::boxHandler: iloc       99->96 
       107 |       20 |   ID |    1 |      0,  2663
       127 |       20 |   ID |    2 |      0,     8
       147 |       20 |   ID |    3 |      0,   306
       167 |       20 |   ID |    4 |      0,  3432
  Exiv2::BmffImage::boxHandler: iinf      195->119 
    Exiv2::BmffImage::boxHandler: infe      209->21 ID =   1 hvc1 
    Exiv2::BmffImage::boxHandler: infe      230->21 ID =   2 grid 
    Exiv2::BmffImage::boxHandler: infe      251->21 ID =   3 Exif  *** Exif ***
    Exiv2::BmffImage::boxHandler: infe      272->42 ID =   4 mime  *** XMP ***
  Exiv2::BmffImage::boxHandler: iprp      314->221 
    Exiv2::BmffImage::boxHandler: ipco      322->187 
      Exiv2::BmffImage::boxHandler: hvcC      330->123 
      Exiv2::BmffImage::boxHandler: ispe      453->20 pixelWidth_, pixelHeight_ = 2001, 1984
      Exiv2::BmffImage::boxHandler: ispe      473->20 pixelWidth_, pixelHeight_ = 2002, 1984
      Exiv2::BmffImage::boxHandler: pixi      493->16 
    Exiv2::BmffImage::boxHandler: ipma      509->26 
  Exiv2::BmffImage::boxHandler: iref      535->54 
Exiv2::BmffImage::boxHandler: mdat      589->6409 
519 rmills@rmillsmm-local:~/gnu/github/exiv2/team.exiv2.org/book/build $ ... 
  END: test.heic:36->553
     589 |     6409 | mdat |      | __.c(.....5.....N.:. 0 0 10 99 40 1 175 19 15 159 53 170 245 235 213 171 78 157 58 149
END: test.heic

So, offset 0 requires you to perform a fixup calculation into the mdat box as follows:

  589 + 2663 = 3252 
 3252 +    8 = 3260
 3260 +  306 = 3566

I'll read the BMFF specification to see what is said about this. There is a case that writing offset = 0 in the ILOC array is a bug in GIMP. However, as Exiftool supports this, it's reasonable that Exiv2 should also handle this. I also observe that the Exif data has 4 leading null bytes. I believe Exiv2 deals correctly with this by hunting for the Tiff marker II or MM to locate the Exif metadata.

I am confident that a couple of new lines of code will fix this and cause no collateral damage. I'll add your file test.heif (appropriately renamed) to the test suite.

@novomesk
Copy link
Author

The test.heic was constructed by libheif.

in GIMP I am using libheif's heif_context_add_exif_metadata, I am passing buffer starting with II*

HeifContext::add_exif_metadata https://github.com/strukturag/libheif/blob/master/libheif/heif_context.cc#L2442 prepends 4 zero bytes as offset.

Unfortunately, I don't know what is right from specification point of view. Dirk @farindk , perhaps you can take a look too, so everyone understand the things in same way?

@farindk
Copy link

farindk commented Mar 25, 2022

According to the HEIF standard (ISO-23008:12, section A.2), the Exif data block starts with a 4-byte offset into the remaining data. I.e. if the data starts immediately with the TIFF header ("MM" or "II"), the offset will be 0. Otherwise, the given number of bytes should be skipped.

I don't know why they included this offset in the specification. It's not even clear to me when it would be useful to have extra bytes in front of the header...

@clanmills
Copy link
Collaborator

It's amazing what we can get done when we work together. Thank you both for your feedback.

I'm confident that Exiv2 "hunts" for the II/MM marker and ignores leading bytes.

The offset = 0 in the ILOC array is very convenient for somebody writing a file. The fixup to deal with offset = 0 is an easy little change. I'll do the PR this weekend for branch 0.27-maintenance.

@farindk
Copy link

farindk commented Mar 25, 2022

Great. Note that the case offset != 0 is not just theoretical. I have seen such images. libheif searches for the "II"/"MM" header in the passed Exif data and then places an offset to this in the file. It does not truncate the extra data in front of the header.

@clanmills
Copy link
Collaborator

clanmills commented Mar 25, 2022

The ISOBMFF specifications are a tough read. For sure, I'm not smart enough to understand the ISO document numbering scheme. For the bmff project, I worked with w15177_15444 and w15177_14496 which states on p88:

The item location box (‘iloc’) specifies the actual storage location of each item within the container file as well as the file size of each item. File name, content type (MIME type), etc., of each item are provided by version 1 of the item information box (‘iinf’).

Note that it states location ... as well as the file size of each item. For sure, I didn't encounter a file with location == 0 during the bmff project. So, I believe test.heic violates the specification. However, it's easy to add a fixup to calculate the offset into the mdat box and I'm happy to do that.

I don't want to get involved in a negotiation with libheif concerning iloc location == 0, however I do believe that should be reviewed and fixed.

@farindk
Copy link

farindk commented Mar 25, 2022

Let me try to clear up the confusion a bit. The offset to the Exif data I was talking about has nothing to do with the ISOBMFF layer (ISO 14496). It is part of the metadata block stored in the iloc. Thus, get the data as specified by the 'iloc' and then, take the first 4 bytes of this data as the offset.

This is the relevant part of the HEIF standard:
Screenshot from 2022-03-26 00-09-12

@clanmills
Copy link
Collaborator

clanmills commented Mar 26, 2022

Thank You, @farindk. I've always been focused on location == 0 in iloc.

I mentioned the 4 nul bytes and now you've found the specification for that. Thanks for digging into that. I was not worried about that as I believe the Exiv2 code "hunts" for the II/MM marker and ignores leading bytes.

When I view the file test.heic on macOS (Catalina 10.15), Preview.app does not show the Exif metadata in the Info panel. I tried unsuccessfully with HexFiend to modify test.heic to replace the iloc location == 0 with the correct value.

When I view test.heic on macOS (Monterey 11.2), it does display Exif metadata, however it's wrong!

screenshot_76

I will modify exiv2 to do the fixup to handle location == 0 in iloc.

@clanmills clanmills added enhancement feature / functionality enhancements and removed bug labels Mar 26, 2022
@j-m-harris
Copy link

+1 for this problem, I have some HEIC files for which previous exiv2 versions read the EXIF but are now reporting 'data not found'.

@1div0
Copy link
Collaborator

1div0 commented Jul 23, 2022

Hi++ @j-m-harris could you provide us few such files for analysis?

Thanks.

@j-m-harris
Copy link

Sure @1div0 , thanks for looking at this.

This file was generated with heif-enc and has metadata added by exiftool

IMG_20200731_210636067.heif.gz

It was generated in March 2021 when exiv2 didn't read the meta data (reporting 'The file contains data of an unknown image type'). In April 2022 I noted that the current gentoo version of exiv2 did now support the meta data, however the latest version now reports 'No Exif data found in the file'.

@kmilos
Copy link
Collaborator

kmilos commented Jul 25, 2022

@1div0 On 0.27-maintenance, the relevant iloc is being skipped because offset is 0.

@etrigan63
Copy link

Any progress on this? I am getting Exiv2 exception in print action for file IMG-20230906-0527.hif: corrupted image metadata when processing a Fuji X-T5 HEIF file.
IMG-20230906-0527.hif.zip

@kmilos
Copy link
Collaborator

kmilos commented Sep 7, 2023

@etrigan63 Your issue was already fixed by #2612 but it is not yet released from the 0.28.x branch (it is in 0.27.7 though). The root cause for this one is different.

@etrigan63
Copy link

Thanks. Any idea as to release date?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement feature / functionality enhancements
Projects
None yet
Development

No branches or pull requests

8 participants