From a.janke at gmail.com Tue Aug 9 08:25:36 2011 From: a.janke at gmail.com (Andrew Janke) Date: Tue, 9 Aug 2011 22:25:36 +1000 Subject: [MINC-development] minctracc and large(ish) input files Message-ID: Hi all, Has anyone else had minctracc fail (segfault) on sizeable but not massive input files? In my case the input file is big but not ginormous: file: b.mnc image: unsigned byte 0 to 255 image dimensions: zspace yspace xspace dimension name length step start dircos -------------- ------ ---- ----- ------ zspace 150 0.5 -124.2 0 0 1 yspace 1311 0.5 -327.15 0 1 0 xspace 679 0.5 -169.2 1 0 0 And this will segfault it: ~/cvs/registration/mni_autoreg/minctracc/Main/minctracc -no_super -transformation ident.xfm -clobber -nonlinear -iterations 1 -step 32 32 32 -debug b.mnc b.mnc /tmp/out.xfm It seems to die while doing the amoeba setup step but this is likely just memory being overstepped from somewhere else. Anyone (Claude and his big brain!) have any insights as to which particular rabbit hole I should go looking down first? ta a From claude at bic.mni.mcgill.ca Tue Aug 9 09:09:44 2011 From: claude at bic.mni.mcgill.ca (Claude LEPAGE) Date: Tue, 9 Aug 2011 09:09:44 -0400 Subject: [MINC-development] minctracc and large(ish) input files In-Reply-To: Your message of Tue, 9 Aug 2011 22:25:36 +1000 References: Message-ID: <201108091309.p79D9iTD006518@cassio.bic.mni.mcgill.ca> Hi Andrew, In such cases, valgrind is my friend. You can also run through gdb. There are lots of instances in our tools that don't verify for properly allocated memory. Claude > Has anyone else had minctracc fail (segfault) on sizeable but not > massive input files? > > In my case the input file is big but not ginormous: > > file: b.mnc > image: unsigned byte 0 to 255 > image dimensions: zspace yspace xspace > dimension name length step start dircos > -------------- ------ ---- ----- ------ > zspace 150 0.5 -124.2 0 0 1 > yspace 1311 0.5 -327.15 0 1 0 > xspace 679 0.5 -169.2 1 0 0 > > And this will segfault it: > > ~/cvs/registration/mni_autoreg/minctracc/Main/minctracc -no_super > -transformation ident.xfm -clobber -nonlinear -iterations 1 -step 32 > 32 32 -debug b.mnc b.mnc /tmp/out.xfm > > It seems to die while doing the amoeba setup step but this is likely > just memory being overstepped from somewhere else. > > Anyone (Claude and his big brain!) have any insights as to which > particular rabbit hole I should go looking down first? > > > ta From a.janke at gmail.com Thu Aug 11 03:03:47 2011 From: a.janke at gmail.com (Andrew Janke) Date: Thu, 11 Aug 2011 17:03:47 +1000 Subject: [MINC-development] max_buffer_size_in_kb Message-ID: We all now have machines with more than 64MB of RAM. Would anyone be against me upping the default limit (4MB in most tools)? In larger files upping this to something much larger (4096000 for example) will result in much faster runtimes for anything that uses voxel_loop (minccalc, mincmath, mincreshape, etc). Perhaps we should take an ENV var approach? suggestions? a From jcupitt at gmail.com Thu Aug 11 04:42:02 2011 From: jcupitt at gmail.com (jcupitt at gmail.com) Date: Thu, 11 Aug 2011 09:42:02 +0100 Subject: [MINC-development] max_buffer_size_in_kb In-Reply-To: References: Message-ID: On 11 August 2011 08:03, Andrew Janke wrote: > Would anyone be against me upping the default limit (4MB in most tools)? > > Perhaps we should take an ENV var approach? Sounds good. For what it's worth, most libraries seem to use both env vars and command-line switches, so you could have: $ minc-process file.mnc will use the default buffer size $ MINC_BUFFER=64m minc-process file.mnc will use a 64mb buffer and $ MINC_BUFFER=64m minc-process --minc-buffer=8m file.mnc will use a 8mb buffer (command-line switches always override env vars). The idea is that the env var sets the default, the switch sets the value for this invocation. John From zijdenbos at gmail.com Thu Aug 11 08:33:39 2011 From: zijdenbos at gmail.com (Alex Zijdenbos) Date: Thu, 11 Aug 2011 08:33:39 -0400 Subject: [MINC-development] max_buffer_size_in_kb In-Reply-To: References: Message-ID: I'm all for this. I recently had a case for which processing time went from around 7 hrs to 10 mins or so by changing the buffer size. An ENV var approach would be good, but regardless I think we should change the default as well, it seems rather outdated. -- A On Thu, Aug 11, 2011 at 4:42 AM, wrote: > On 11 August 2011 08:03, Andrew Janke wrote: >> Would anyone be against me upping the default limit (4MB in most tools)? >> >> Perhaps we should take an ENV var approach? > > Sounds good. > > For what it's worth, most libraries seem to use both env vars and > command-line switches, so you could have: > > $ minc-process file.mnc > > will use the default buffer size > > $ MINC_BUFFER=64m minc-process file.mnc > > will use a 64mb buffer and > > $ MINC_BUFFER=64m minc-process --minc-buffer=8m file.mnc > > will use a 8mb buffer (command-line switches always override env > vars). The idea is that the env var sets the default, the switch sets > the value for this invocation. > > John > _______________________________________________ > MINC-development mailing list > MINC-development at bic.mni.mcgill.ca > http://www.bic.mni.mcgill.ca/mailman/listinfo/minc-development > > From a.janke at gmail.com Thu Aug 11 09:12:51 2011 From: a.janke at gmail.com (Andrew Janke) Date: Thu, 11 Aug 2011 23:12:51 +1000 Subject: [MINC-development] max_buffer_size_in_kb In-Reply-To: References: Message-ID: OK, so the approach will be to modify this: https://github.com/andrewjanke/minc/blob/master/libsrc/voxel_loop.c#L2690 And any of the various programs that use voxel_loop argument defaults? Or if we take the ENV approach then we'll have to set the program defaults to "undef" or something so that an ENV var can take precedence. For now I think it will be simpler (for me) to skip the ENV var and just increase the default. Anyone have any particular preference for how big? In my case I usually just set it to "massive", but I suspect this is not the best idea... My initial thought is to set it to about 200MB (~50x the current). 1024*4*64 = 262144kb Thoughts? a On 11 August 2011 22:33, Alex Zijdenbos wrote: > I'm all for this. I recently had a case for which processing time went > from around 7 hrs to 10 mins or so by changing the buffer size. An ENV > var approach would be good, but regardless I think we should change > the default as well, it seems rather outdated. > > -- A > > On Thu, Aug 11, 2011 at 4:42 AM, ? wrote: >> On 11 August 2011 08:03, Andrew Janke wrote: >>> Would anyone be against me upping the default limit (4MB in most tools)? >>> >>> Perhaps we should take an ENV var approach? >> >> Sounds good. >> >> For what it's worth, most libraries seem to use both env vars and >> command-line switches, so you could have: >> >> $ minc-process file.mnc >> >> will use the default buffer size >> >> $ MINC_BUFFER=64m minc-process file.mnc >> >> will use a 64mb buffer and >> >> $ MINC_BUFFER=64m minc-process --minc-buffer=8m file.mnc >> >> will use a 8mb buffer (command-line switches always override env >> vars). The idea is that the env var sets the default, the switch sets >> the value for this invocation. >> >> John >> _______________________________________________ >> MINC-development mailing list >> MINC-development at bic.mni.mcgill.ca >> http://www.bic.mni.mcgill.ca/mailman/listinfo/minc-development >> >> > _______________________________________________ > MINC-development mailing list > MINC-development at bic.mni.mcgill.ca > http://www.bic.mni.mcgill.ca/mailman/listinfo/minc-development > From claude at bic.mni.mcgill.ca Thu Aug 11 10:39:28 2011 From: claude at bic.mni.mcgill.ca (Claude LEPAGE) Date: Thu, 11 Aug 2011 10:39:28 -0400 Subject: [MINC-development] max_buffer_size_in_kb In-Reply-To: Your message of Thu, 11 Aug 2011 23:12:51 +1000 References: Message-ID: <201108111439.p7BEdSWR007234@cassio.bic.mni.mcgill.ca> Hi Andrew, I was the last one to increase the default buffer size from something like 10K to 1Mb. At the time, I ran many tests to find a suitable value that would be optimal for speed for the most typical application: a MRI in stereotaxic space at 1mm (minccalc, mincresample, mincreshape, etc). The file size is about 10Mb (that's minc2 compressed) for unsigned short. Remember also that there is a buffer for internal compression (which might be the same as the I/O buffer - beware). Operating on large compressed blocks slows down processing. > For now I think it will be simpler (for me) to skip the ENV var and > just increase the default. Anyone have any particular preference for > how big? In my case I usually just set it to "massive", but I suspect > this is not the best idea... > > My initial thought is to set it to about 200MB (~50x the current). > 1024*4*64 =3D 262144kb > > Thoughts? I think 200MB is too large for most typical MRI applications. I would hesitate to go above 4MB as a default. However, I'm all in favour of providing a user-friendly option to change the run-time default. The environment variable sounds like a good work-around. Each user can set it up in his (or her!) environment or in individual job scripts. Also, keep in mind that I am processing hi-res histology slices (250Mb per slice/file) with a 1MB buffer and that the performance is quite good. So make sure you run lots of tests for speed and plot graphs for execution time vs buffer size for a wide range of applications (minccalc, mincresample, minctracc) and inputs. There might be applications for which 200MB is optimal, but these will not be the typical applications by most users. Whoever requires 200MB can use an environment variable or recompile with a default of 200MB. Claude From a.janke at gmail.com Thu Aug 11 11:15:47 2011 From: a.janke at gmail.com (Andrew Janke) Date: Fri, 12 Aug 2011 01:15:47 +1000 Subject: [MINC-development] max_buffer_size_in_kb In-Reply-To: <201108111439.p7BEdSWR007234@cassio.bic.mni.mcgill.ca> References: <201108111439.p7BEdSWR007234@cassio.bic.mni.mcgill.ca> Message-ID: Hi Claude, > I was the last one to increase the default buffer size from > something like 10K to 1Mb. At the time, I ran many tests to > find a suitable value that would be optimal for speed for > the most typical application: a MRI in stereotaxic space at > 1mm (minccalc, mincresample, mincreshape, etc). The file size > is about 10Mb (that's minc2 compressed) for unsigned short. > Remember also that there is a buffer for internal compression True, and no doubt there are pathological cases of this. > environment variable sounds like a good work-around. Each user can > set it up in his (or her!) environment or in individual job scripts. > Also, keep in mind that I am processing hi-res histology slices > (250Mb per slice/file) with a 1MB buffer and that the performance > is quite good. To counter this, I have seen "catastrophic" performance in minccalc when doing weighted averages over 19 800MB mouse brains and also on 600+ 10MB human head MRI's. So I'll take your comments about keeping things stable and leave things are they are for now but add the ENV var so that those of us who like to live on the edge can wind up the wick a bit. Let me know what you think of this commit: https://github.com/andrewjanke/minc/commit/f272c25a588f503c90fea16c972b149ca68d3194 Note that I have only changed the logic of minccalc so far, if others agree with this approach I will go and make the changes to all the other minc tools that use voxel_loop. In some cases this will mean adding the -max_buffer_in_kb argument (minclookup for one). a From claude at bic.mni.mcgill.ca Thu Aug 11 11:32:31 2011 From: claude at bic.mni.mcgill.ca (Claude LEPAGE) Date: Thu, 11 Aug 2011 11:32:31 -0400 Subject: [MINC-development] max_buffer_size_in_kb In-Reply-To: Your message of Fri, 12 Aug 2011 01:15:47 +1000 References: <201108111439.p7BEdSWR007234@cassio.bic.mni.mcgill.ca> Message-ID: <201108111532.p7BFWVmM010288@cassio.bic.mni.mcgill.ca> Andrew, > True, and no doubt there are pathological cases of this. > > > environment variable sounds like a good work-around. Each user can > > set it up in his (or her!) environment or in individual job scripts. > > > Also, keep in mind that I am processing hi-res histology slices > > (250Mb per slice/file) with a 1MB buffer and that the performance > > is quite good. > > To counter this, I have seen "catastrophic" performance in minccalc > when doing weighted averages over 19 800MB mouse brains and also on > 600+ 10MB human head MRI's. So I'll take your comments about keeping > things stable and leave things are they are for now but add the ENV > var so that those of us who like to live on the edge can wind up the > wick a bit. Please remind me: is this buffer per file or in total for all files? 600 MRIs at each 1MB buffer = 600MB. 600 MRIs at each 1MB/600=1.7Kb = 1MB. Now we are in trouble at 1.7Kb per file. I agree. Maybe we have to change the meaning of the buffer to be per file, because 1MB is "optimized" for one file. > Note that I have only changed the logic of minccalc so far, if others > agree with this approach I will go and make the changes to all the > other minc tools that use voxel_loop. In some cases this will mean > adding the -max_buffer_in_kb argument (minclookup for one). minccalc is kind of harmless. Do you ever use it with lots of files? mincaverage and mincconcat are probably more problematic. The design to use a constant (#define) in minc is bad. This value should be in a global variable, thus could be changed at run-time. In the context of object-oriented code (C++), it could be defined in the minc class when opening a minc file. Each file could have its own buffer size. wow! Go ahead with your changes. I'm not too picky about the implementation. It's the concept that's important. Claude From a.janke at gmail.com Thu Aug 11 11:44:22 2011 From: a.janke at gmail.com (Andrew Janke) Date: Fri, 12 Aug 2011 01:44:22 +1000 Subject: [MINC-development] max_buffer_size_in_kb In-Reply-To: <201108111532.p7BFWVmM010288@cassio.bic.mni.mcgill.ca> References: <201108111439.p7BEdSWR007234@cassio.bic.mni.mcgill.ca> <201108111532.p7BFWVmM010288@cassio.bic.mni.mcgill.ca> Message-ID: > Please remind me: is this buffer per file or in total for all files? > 600 MRIs at each 1MB buffer = 600MB. > 600 MRIs at each 1MB/600=1.7Kb = 1MB. > Now we are in trouble at 1.7Kb per file. I agree. Its for all files (see line 1793 of libsrc/voxel_loop.c): https://github.com/andrewjanke/minc/blob/f272c25a588f503c90fea16c972b149ca68d3194/libsrc/voxel_loop.c#L1793 You are setting loop_options->total_copy_space which is then divided by num_input_buffers (ie: the number of input files + output files + extra buffers). Peter wrote this no doubt concerned about total memory usage, not usage per file. > Maybe we have to change the meaning of the buffer to be per file, > because 1MB is "optimized" for one file. This would have implications though, (going to swap). What this means is that you want two env vars: export MINC_MAX_MEMORY 40000000 # all of your RAM export MINC_MAX_FILE_BUFFER 4096 # buffer per file You'd then have to check (in voxel_loop) that MINC_MAX_FILE_BUFFER * # input + output files is not larger than MINC_MAX_MEMORY. > minccalc is kind of harmless. Do you ever use it with lots of files? All the time, use it to do weighted averages (remove outliers) when mincaverage wont do or won't work. eg: in one case I have 9000 input files so break the averaging over multiple runs of minccalc -double. > mincaverage and mincconcat are probably more problematic. correct, for the average punter. > The design to use a constant (#define) in minc is bad. meaning what I have done or what was done in the past? > In the context of object-oriented code (C++), it could be defined in the minc class when > opening a minc file. Each file could have its own buffer size. wow! Or not. ;) a From claude at bic.mni.mcgill.ca Thu Aug 11 12:04:41 2011 From: claude at bic.mni.mcgill.ca (Claude LEPAGE) Date: Thu, 11 Aug 2011 12:04:41 -0400 Subject: [MINC-development] max_buffer_size_in_kb In-Reply-To: Your message of Fri, 12 Aug 2011 01:44:22 +1000 References: <201108111439.p7BEdSWR007234@cassio.bic.mni.mcgill.ca> <201108111532.p7BFWVmM010288@cassio.bic.mni.mcgill.ca> Message-ID: <201108111604.p7BG4fdg029130@cassio.bic.mni.mcgill.ca> Andrew, > Its for all files (see line 1793 of libsrc/voxel_loop.c): > > https://github.com/andrewjanke/minc/blob/f272c25a588f503c90fea16c972b149ca68d3194/libsrc/voxel_loop.c#L1793 > > You are setting loop_options->total_copy_space which is then divided > by num_input_buffers (ie: the number of input files + output files + > extra buffers). Peter wrote this no doubt concerned about total > memory usage, not usage per file. > > > Maybe we have to change the meaning of the buffer to be per file, > > because 1MB is "optimized" for one file. > > This would have implications though, (going to swap). What this means > is that you want two env vars: > > export MINC_MAX_MEMORY 40000000 # all of your RAM > export MINC_MAX_FILE_BUFFER 4096 # buffer per file > > You'd then have to check (in voxel_loop) that MINC_MAX_FILE_BUFFER * # > input + output files is not larger than MINC_MAX_MEMORY. While you're at it, this would be a great change to implement for voxel_loop. MINC_MAX_FILE_BUFFER should be 1MB (like current buffer) and should be maxed out as long as #files * MINC_MAX_FILE_BUFFER is less than MINC_MAX_MEMORY. Safely, MINC_MAX_MEMORY could be 2Gb or 4Gb. io_buffer_per_file = max( MINC_MAX_MEMORY/number_files, MINC_MAX_FILE_BUFFER ); I think this implementation would be dynamic enough to do all what you want without the environment variable and command line switch. A static compilation with 4Gb/1Mb would go a long way without having to modify all the code. Just double-check this i/o buffer size with the chunk buffer for internal compression. I think we are converging towards an excellent solution! Claude From claude at bic.mni.mcgill.ca Thu Aug 11 16:14:26 2011 From: claude at bic.mni.mcgill.ca (Claude LEPAGE) Date: Thu, 11 Aug 2011 16:14:26 -0400 Subject: [MINC-development] max_buffer_size_in_kb In-Reply-To: Your message of Fri, 12 Aug 2011 01:44:22 +1000 References: <201108111439.p7BEdSWR007234@cassio.bic.mni.mcgill.ca> <201108111532.p7BFWVmM010288@cassio.bic.mni.mcgill.ca> Message-ID: <201108112014.p7BKEQxk011149@cassio.bic.mni.mcgill.ca> Andrew, In my previous e-mail, I meant min(a,b), not max(). Claude From a.janke at gmail.com Fri Aug 12 01:25:52 2011 From: a.janke at gmail.com (Andrew Janke) Date: Fri, 12 Aug 2011 15:25:52 +1000 Subject: [MINC-development] max_buffer_size_in_kb In-Reply-To: <201108111604.p7BG4fdg029130@cassio.bic.mni.mcgill.ca> References: <201108111439.p7BEdSWR007234@cassio.bic.mni.mcgill.ca> <201108111532.p7BFWVmM010288@cassio.bic.mni.mcgill.ca> <201108111604.p7BG4fdg029130@cassio.bic.mni.mcgill.ca> Message-ID: >> This would have implications though, (going to swap). What this means >> is that you want two env vars: >> >> ? ?export MINC_MAX_MEMORY 40000000 ? ? # all of your RAM >> ? ?export MINC_MAX_FILE_BUFFER 4096 ? ? # buffer per file >> >> You'd then have to check (in voxel_loop) that MINC_MAX_FILE_BUFFER * # >> input + output files is not larger than MINC_MAX_MEMORY. > > While you're at it, this would be a great change to implement for > voxel_loop. MINC_MAX_FILE_BUFFER should be 1MB (like current buffer) > and should be maxed out as long as #files * MINC_MAX_FILE_BUFFER > is less than MINC_MAX_MEMORY. Safely, MINC_MAX_MEMORY could be > 2Gb or 4Gb. You sure about 1MB there? The current default is 4MB (in both voxel_loop.c and mincprogs). > io_buffer_per_file = min( MINC_MAX_MEMORY/number_files, MINC_MAX_FILE_BUFFER ); Agreed. > I think this implementation would be dynamic enough to do all what > you want without the environment variable and command line switch. I hope so, but will still be adding the ENV vars and adding C/L switches to all appropriate tools. So is there consensus on what we call these two new ENV vars? We currently have this list: MINC_FORCE_V2 = {1, 0} MINC_COMPRESS = {0..9}- MINC_CHUNKING = {0..N} MINC_LOGFILE = [+]|stdout|- MINC_LOGLEVEL = 0-4 So do: MINC_MAX_MEMORY MINC_MAX_FILE_BUFFER fit in well with this? Perhaps MINC_MEMMAX and MINC_FILE_MEMMAX? whatever we choose there should be a nice easy to understand link between -max_buffer_size_in_kb and MINC_MAX_FILE_BUFFER. If I hear nothing else, I will press ahead with MINC_MAX_MEMORY + MINC_MAX_FILE_BUFFER tonight. a From zijdenbos at gmail.com Fri Aug 12 07:37:24 2011 From: zijdenbos at gmail.com (Alex Zijdenbos) Date: Fri, 12 Aug 2011 07:37:24 -0400 Subject: [MINC-development] max_buffer_size_in_kb In-Reply-To: References: <201108111439.p7BEdSWR007234@cassio.bic.mni.mcgill.ca> <201108111532.p7BFWVmM010288@cassio.bic.mni.mcgill.ca> <201108111604.p7BG4fdg029130@cassio.bic.mni.mcgill.ca> Message-ID: > So is there consensus on what we call these two new ENV vars? ?We > currently have this list: > > ? MINC_FORCE_V2 = {1, 0} > ? MINC_COMPRESS = {0..9}- > ? MINC_CHUNKING = {0..N} > ? MINC_LOGFILE = [+]|stdout|- > ? MINC_LOGLEVEL = 0-4 > > So do: > > ? MINC_MAX_MEMORY > ? MINC_MAX_FILE_BUFFER > > fit in well with this? ?Perhaps MINC_MEMMAX and MINC_FILE_MEMMAX? > whatever we choose there should be a nice easy to understand link > between -max_buffer_size_in_kb and MINC_MAX_FILE_BUFFER. > > If I hear nothing else, I will press ahead with MINC_MAX_MEMORY + > MINC_MAX_FILE_BUFFER tonight. To be a bit more consistent, I'd suggest MINC_MAX_FILE_MEMORY (why would one be called 'memory' and the other 'buffer'?) But either way, I think the most important is that these actually be documented at the user level. I for one was only aware of the first two original ones, and have no idea what the other 3 do :-) -- A From a.janke at gmail.com Fri Aug 12 07:44:38 2011 From: a.janke at gmail.com (Andrew Janke) Date: Fri, 12 Aug 2011 21:44:38 +1000 Subject: [MINC-development] max_buffer_size_in_kb In-Reply-To: References: <201108111439.p7BEdSWR007234@cassio.bic.mni.mcgill.ca> <201108111532.p7BFWVmM010288@cassio.bic.mni.mcgill.ca> <201108111604.p7BG4fdg029130@cassio.bic.mni.mcgill.ca> Message-ID: >> If I hear nothing else, I will press ahead with MINC_MAX_MEMORY + >> MINC_MAX_FILE_BUFFER tonight. > > To be a bit more consistent, I'd suggest MINC_MAX_FILE_MEMORY (why > would one be called 'memory' and the other 'buffer'?) Well for no other reason than the -max_buffer_size_in_kb C/L argument. > But either way, I think the most important is that these actually be > documented at the user level. I for one was only aware of the first > two original ones, and have no idea what the other 3 do :-) Well someone has never read the MINC README... :) https://github.com/andrewjanke/minc/blob/master/README Still I don't think the ENV vars are something that a general MINC user would want to mess with without a bit of knowledge first. But then I have been called elitist before. a From zijdenbos at gmail.com Fri Aug 12 09:10:00 2011 From: zijdenbos at gmail.com (Alex Zijdenbos) Date: Fri, 12 Aug 2011 09:10:00 -0400 Subject: [MINC-development] max_buffer_size_in_kb In-Reply-To: References: <201108111439.p7BEdSWR007234@cassio.bic.mni.mcgill.ca> <201108111532.p7BFWVmM010288@cassio.bic.mni.mcgill.ca> <201108111604.p7BG4fdg029130@cassio.bic.mni.mcgill.ca> Message-ID: On Fri, Aug 12, 2011 at 7:44 AM, Andrew Janke wrote: >>> If I hear nothing else, I will press ahead with MINC_MAX_MEMORY + >>> MINC_MAX_FILE_BUFFER tonight. >> >> To be a bit more consistent, I'd suggest MINC_MAX_FILE_MEMORY (why >> would one be called 'memory' and the other 'buffer'?) > > Well for no other reason than the -max_buffer_size_in_kb C/L argument. True - so nevermind >> But either way, I think the most important is that these actually be >> documented at the user level. I for one was only aware of the first >> two original ones, and have no idea what the other 3 do :-) > > Well someone has never read the MINC README... :) > > ? https://github.com/andrewjanke/minc/blob/master/README Guilty as charged :) I had actually googled for it but that only comes back with things like AFNI_MINC_FLOATIZE. > Still I don't think the ENV vars are something that a general MINC > user would want to mess with without a bit of knowledge first. ?But > then I have been called elitist before. By whom?! ;) While on this topic though, should the issue of MINC_COMPRESS (and, shudder, MINC_FORCE_V2) be re-discussed? I for one permanently have MINC_COMPRESS=4 MINC_FORCE_V2=1 in my environment and am very happy with that through a wide range of applications. These are also set by CIVET by default. So if the argument is that a general MINC user should not have to know about these environment vars, I think it would make sense if these settings would be the default. -- A From acv at miniguru.ca Fri Aug 12 11:33:31 2011 From: acv at miniguru.ca (acv) Date: Fri, 12 Aug 2011 11:33:31 -0400 Subject: [MINC-development] max_buffer_size_in_kb In-Reply-To: References: <201108111439.p7BEdSWR007234@cassio.bic.mni.mcgill.ca> <201108111532.p7BFWVmM010288@cassio.bic.mni.mcgill.ca> <201108111604.p7BG4fdg029130@cassio.bic.mni.mcgill.ca> Message-ID: <20110812153331.GB15964@miniguru.ca> On Fri, Aug 12, 2011 at 09:10:00AM -0400, Alex Zijdenbos wrote: > >> > >> To be a bit more consistent, I'd suggest MINC_MAX_FILE_MEMORY (why > >> would one be called 'memory' and the other 'buffer'?) > > > > Well for no other reason than the -max_buffer_size_in_kb C/L argument. > > True - so nevermind It would be best I think if the env variable had the unit in them. Nothing I have more then having to dig around to find out if it's KB, pages, MB or centibits. Alexandre Carmel-Veilleux -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available URL: From a.janke at gmail.com Mon Aug 15 00:18:11 2011 From: a.janke at gmail.com (Andrew Janke) Date: Mon, 15 Aug 2011 14:18:11 +1000 Subject: [MINC-development] max_buffer_size_in_kb In-Reply-To: <20110812153331.GB15964@miniguru.ca> References: <201108111439.p7BEdSWR007234@cassio.bic.mni.mcgill.ca> <201108111532.p7BFWVmM010288@cassio.bic.mni.mcgill.ca> <201108111604.p7BG4fdg029130@cassio.bic.mni.mcgill.ca> <20110812153331.GB15964@miniguru.ca> Message-ID: >> True - so nevermind > > It would be best I think if the env variable had the unit in them. Nothing I have > more then having to dig around to find out if it's KB, pages, MB or centibits. So much for design by committee.. :) This keep everyone happy? https://github.com/andrewjanke/minc/commit/86cb2b2c976873c668f5319e022943fa3f54a4d5 Works for me, I'm not entirely convinced by the _KB extensions but see the point. Now to figure out how to integrate this into voxel_loop.c: https://github.com/andrewjanke/minc/blob/master/libsrc/voxel_loop.c#L1792 That particular line already looks long and confusing enough, time for an extra local variable or two I think. a