[MINC-development] hdf attribute size

[Soren Christensen]

Hi,
 Has anyone else had problems with the 64kb limit on attributes in the
minc-header?
I am storing a slice-time matrix of values from the originating DICOM files
in order to not loose information from some key DICOM attributes.
Often I will have thounsands of DICOMs going into one MINC file. For
attributes that are uniform accross all DICOMs I just write the atreibute
once, but for varying attributes I fill in the above slice-time matrix. So
for thousands of files this get pretty big. AcquisitionDateTime for example
will take more bytes to store than 64kb. I use strings only for header
items.
So I get errors like these from hdf (via miset_attr_values):

HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:

#000: /home/s/CODE/ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5Adeprec.c
line 165 in H5Acreate1(): unable to create attribute

major: Attribute

minor: Unable to initialize object

#001: /home/s/CODE/ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5A.c line
496 in H5A_create(): unable to create attribute in object header

major: Attribute

minor: Unable to insert object

#002:
/home/s/CODE/ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5Oattribute.c
line 346 in H5O_attr_create(): unable to create new attribute in header

major: Attribute

minor: Unable to insert object

#003: /home/s/CODE/ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5Omessage.c
line 224 in H5O_msg_append_real(): unable to create new message

major: Object header

minor: No space available for allocation

#004: /home/s/CODE/ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5Omessage.c
line 1925 in H5O_msg_alloc(): unable to allocate space for message

major: Object header

minor: Unable to initialize object

#005: /home/s/CODE/ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5Oalloc.c
line 1136 in H5O_alloc(): object header message is too large

major: Object header

minor: Unable to initialize object

problem with key: "AcquisitionDateTime"

Does anyone know of an easy workaround for this?

It seems one can change the hdf call quite easily to do away with this 64kb
limitation, but I am not sure of any potential penalties. An optimistic
interpretation of the below document says it will be more efficient and is
simply a new feature of hdf 1.8. (inserting new header items I suppose).
http://www.hdfgroup.org/HDF5/doc/UG/13_Attributes.html (see 8.5)
Compatibility with hdf<1.8 would be lost

So I suppose one could make an miset_attr_values_large or somehow
optionally create a dense group? Or perhaps replace the setup with "dense"
groups (see hdf document), provided it provides appropriate backwards file
compatibility?

I'd like to pursue this when I get more time available, but maybe someone
well versed in hdf could chime in?

Thanks
Soren
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.bic.mni.mcgill.ca/pipermail/minc-development/attachments/20131011/e2e7f763/attachment.html>