[MINC-users] minccomplete's completeness

Sat May 14 07:24:04 EDT 2016

By the way, if you run something like 

volume_pol corrupt.mnc /opt/minc/share/icbm152_model_09a/mni_icbm152_t1_tal_nlin_sym_09a.mnc 

it will crash, end the return code is set to 1 , because of this: https://github.com/BIC-MNI/libminc/blob/develop/ezminc/minc_1_rw.cpp#L1297 but this method works only in C++


> On May 14, 2016, at 07:15, vladimir.fonov at gmail.com wrote:
> 
> The reason these errors are not handled properly, perhaps, is because we have calls like that:
> 
> https://github.com/BIC-MNI/libminc/blob/master/libsrc/voxel_loop.c#L1505
> 
> - notice the absence of checking for the return value. 
> 
>> On May 14, 2016, at 07:02, Robert D. Vincent <robert.d.vincent at mcgill.ca> wrote:
>> 
>> Hi all,
>> 
>> In any case, Alex is entirely correct HDF5 errors should be better handled
>> in a few key places - outright read errors should cause obvious things like
>> "exit(-1)" or something. The format is so full of redundancy that it's
>> quite unlikely you would successfully read a broken file, with or without
>> checksumming the data (there are already checksums built into most HDF5
>> data structures, e.g.).
>> 
>> I'll make these changes asap.
>> 
>>   -bert
>> 
>> On Sat, May 14, 2016 at 6:53 AM, Vladimir S. FONOV <vladimir.fonov at gmail.com
>>> wrote:
>> 
>>> Hello,
>>> 
>>> the only thing minccomplete does is read the contents of the image:complete
>>> attribute. Which is not even respected by all minc tools.
>>> One real way to check the consistency of the minc file would be to enable
>>> checksum , integrated into HDF5 file format.
>>> 
>>> Currently it can be enabled by setting the environment variable
>>> MINC_CHECKSUM to 1 , at the time when file is created. Obviously, this
>>> works only with MINC2 file format and it was not tested very well yet.
>>> 
>>> On Fri, May 13, 2016 at 3:35 PM, Alex Zijdenbos <zijdenbos at gmail.com>
>>> wrote:
>>> 
>>>> Hi all,
>>>> 
>>>> I managed to generate a large number of broken MINC files;
>>> possibly/likely
>>>> due to a filesystem problem. The processes that created them (e.g.,
>>>> mincaverage) did not produce any warnings and completed successfully; in
>>>> addition, minccomplete tells me that the files are complete.
>>>> 
>>>> Unfortunately, trying to read these files throws HDF5 and miicv errors
>>> (see
>>>> below) and they are obviously corrupt.
>>>> 
>>>> I am thinking that it would be useful to complete minccomplete by having
>>> it
>>>> actually test-read the data, such that it would report on file integrity?
>>>> This would make it easy to find these kinds of corruptions - and could
>>> even
>>>> tack that end the end of scripts to make sure outputs are intact. I'm
>>>> currently using 'mincstats -quiet -min' to locate them, but it seems the
>>>> natural place for this test would actually be minccomplete.
>>>> 
>>>> -- A
>>>> 
>>>> HDF5-DIAG: Error detected in HDF5 (1.8.9) thread 0:
>>>> #000: H5Dio.c line 174 in H5Dread(): can't read data
>>>>   major: Dataset
>>>>   minor: Read failed
>>>> #001: H5Dio.c line 449 in H5D_read(): can't read data
>>>>   major: Dataset
>>>>   minor: Read failed
>>>> #002: H5Dchunk.c line 1729 in H5D_chunk_read(): unable to read raw data
>>>> chunk
>>>>   major: Low-level I/O
>>>>   minor: Read failed
>>>> #003: H5Dchunk.c line 2760 in H5D_chunk_lock(): data pipeline read
>>> failed
>>>>   major: Data filters
>>>>   minor: Filter operation failed
>>>> #004: H5Z.c line 1120 in H5Z_pipeline(): filter returned failure during
>>>> read
>>>>   major: Data filters
>>>>   minor: Read failed
>>>> #005: H5Zdeflate.c line 125 in H5Z_filter_deflate(): inflate() failed
>>>>   major: Data filters
>>>>   minor: Unable to initialize object
>>>> mincstats (from miicv_get): Can't read dataset /minc-2.0/image/0/image
>>>> _______________________________________________
>>>> MINC-users at bic.mni.mcgill.ca
>>>> http://www.bic.mni.mcgill.ca/mailman/listinfo/minc-users
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> Best regards,
>>> 
>>> Vladimir S. Fonov ~ vladimir <dot> fonov <at> gmail <dot> com
>>> _______________________________________________
>>> MINC-users at bic.mni.mcgill.ca
>>> http://www.bic.mni.mcgill.ca/mailman/listinfo/minc-users
>>> 
>> _______________________________________________
>> MINC-users at bic.mni.mcgill.ca
>> http://www.bic.mni.mcgill.ca/mailman/listinfo/minc-users
> 
> Best regards, Vladimir S. FONOV ~ v.s.fonov <at> ilmarin.info
> 
> 
> 
> 

Best regards, Vladimir S. FONOV ~ v.s.fonov <at> ilmarin.info