[MINC-users] mincaverage error

[Robert D. Vincent]

Hi Andrew,

I wouldn't be averse to making it configurable, but I don't see much
advantage. The constant is only used in voxel_loop (to set the maximum
number of open files) and in image_conversion.c, where it is used just as
an increment to the number of ICV pointers allocated. I don't see a problem
with allocated a group of 1000 pointers at a time there.

The voxel_loop behavior when the number of input files exceeds this limit
is really not something we want to trigger - it constantly opens and closes
file handles, maintaining a single open file handle. It's completely
untenable if you accidentally invoke this with compressed files, for
example!

    -bert


On Thu, Jun 11, 2015 at 10:28 AM, Andrew Janke <a.janke at gmail.com> wrote:

> Hi Bert,
>
> What you are proposing sounds reasonable. Peter and I investigated
> doing this a long time back and did increase it from its original
> value. Perhaps it is time to increase it again, as I recall there were
> problems when we made the figure too high but can't remember details.
> If Peter is still lurking he might join in...
>
> Any chance you can both increase the default and also set an ENV var
> for it so that it can be changed at runtime?
>
> ta
>
>
> a
>
>
> On 12 June 2015 at 00:07, Robert D. Vincent <robert.d.vincent at mcgill.ca>
> wrote:
> > Hi all,
> >
> > I have a partial fix for this issue. The voxel_loop code enforces a
> maximum
> > number of open files based on the constant MI_MAX_NUM_ICV, which is
> > currently set to 32. The way the code is written, if it cannot open all
> of
> > the files, it opens them one at a time. This causes some serious overhead
> > that seems to lead to an out-of-resource condition in the HDF5 library.
> >
> > I will continue to investigate whether we can fix the resource leak, but
> in
> > the short run I will increase the number allowable open files to
> something
> > more realistic, like 1000. This can be done by changing the value of
> > MI_MAX_NUM_ICV in minc.h.
> >
> >     -bert
> _______________________________________________
> MINC-users at bic.mni.mcgill.ca
> http://www.bic.mni.mcgill.ca/mailman/listinfo/minc-users
>