From sorench at gmail.com Fri Oct 23 06:10:36 2009 From: sorench at gmail.com (Soren Christensen) Date: Fri, 23 Oct 2009 21:10:36 +1100 Subject: [MINC-users] mincconcat problem Message-ID: Hi, I have a problem with mincconcat where I get an unexpected result when first merging, then splitting across time. I do not get back to the starting point in terms of values altough it is close. I originally encountered this as mnc files created by mincconcat (time concat) in some cases differed by integer values of +/-1 relative to the original DICOMs or individual mincfiles prior to running mincconcat. Example: I have two minc files v1.mnc v2.mnc (mincinfo below for all files) I use mincconcat to merge them across time: mincconcat -clobber -concat_dimension time v1.mnc v2.mnc v_concat.mnc Then splitting: mincreshape -clobber -dimrange time=1,0 v_concat.mnc v2_split.mnc Now compare v2_split.mnc to v2.mnc - although valid range is changed, real value should be identical. (I think) mincmath -clobber -ne v2_split.mnc v2.mnc neq_img.mnc mincstats neq_img.mnc File: neq_img.mnc Mask file: (null) Total voxels: 327680 # voxels: 327680 % of total: 100 Volume (mm3): 8064000 Min: 0 Max: 1 Sum: 26235 Sum^2: 26235 Mincstats show 26235 pixels differ by nummerical value 1 (actual difference not shown here but it was +/-1 in all cases when read into matlab and the short images subtracted) This does not happen systematicallly, indeed for some files the above procedure will not result in any discrepancies at all. Out of a perfusion time series I had concatenated, discrepancies only appeared in 2 out of 50 time points, but at these time point there were +1000 discrepancies. Is this expected behavior of mincconcat? And if so - is there a way to get around this? Thanks, Soren mi v1.mnc v2.mnc v2_split.mnc file: v1.mnc image: signed__ short 0 to 1361 image dimensions: zspace yspace xspace dimension name length step start -------------- ------ ---- ----- zspace 20 7 -32.1618 yspace 128 -1.875 127.876 xspace 128 -1.875 116.527 file: v2.mnc image: signed__ short 0 to 1438 image dimensions: zspace yspace xspace dimension name length step start -------------- ------ ---- ----- zspace 20 7 -32.1618 yspace 128 -1.875 127.876 xspace 128 -1.875 116.527 file: v2_split.mnc image: signed__ short -32768 to 32767 image dimensions: zspace yspace xspace dimension name length step start -------------- ------ ---- ----- zspace 20 7 -32.1618 yspace 128 -1.875 127.876 xspace 128 -1.875 116.527 From parya.momayyezsiahkal at mail.mcgill.ca Fri Oct 23 14:10:16 2009 From: parya.momayyezsiahkal at mail.mcgill.ca (Parya Mamayyez Siahkal) Date: Fri, 23 Oct 2009 14:10:16 -0400 Subject: [MINC-users] Problem in Parallel mode Message-ID: <4AE1F188.5090508@mail.mcgill.ca> Hello all, I am using minc library to read and write 4D volumes. The code works fine when I am not running it in parallel. Even in the parallel mode, I don't have any problem with sufficiently small volumes. However, as soon as the volume gets a bit big(e.g. 30*40*40), and I want to run the code in parallel, I'll get different kind of error messages at different runs, some of them are: ncvarid: ncvarid: ncid 3ncid 6: NetCDF: Variable not found: NetCDF: Variable not found LocalParallelCF: posixio.c:571: px_get: Assertion `pxp->bf_refcount <= 0' failed. LocalParallelCF: posixio.c:248: px_pgin: Assertion `*posp == ((off_t)(-1)) || *posp == lseek(nciop->fd, 0, 1)' failed. I have switched between minc1 and minc2, but the same kind of errors are being generated. Does anyone know what is going on? Thanks, Parya From a.janke at gmail.com Mon Oct 26 01:11:21 2009 From: a.janke at gmail.com (Andrew Janke) Date: Mon, 26 Oct 2009 16:11:21 +1100 Subject: [MINC-users] Problem in Parallel mode In-Reply-To: <4AE1F188.5090508@mail.mcgill.ca> References: <4AE1F188.5090508@mail.mcgill.ca> Message-ID: Hi Parya, > I am using minc library to read and write 4D volumes. > The code works fine when I am not running it in parallel. > Even in the parallel mode, I don't have any problem with sufficiently > small volumes. > > However, as soon as the volume gets a bit ?big(e.g. 30*40*40), and I > want to run the code in parallel, I'll get different kind of error > messages at different runs, some of them are: > > ?ncvarid: ncvarid: ncid 3ncid 6: NetCDF: Variable not found: NetCDF: > Variable not found > LocalParallelCF: posixio.c:571: px_get: Assertion `pxp->bf_refcount <= > 0' failed. > LocalParallelCF: posixio.c:248: px_pgin: Assertion `*posp == > ((off_t)(-1)) || *posp == lseek(nciop->fd, 0, 1)' failed. > > I have switched between minc1 and minc2, but the same kind of errors are > being generated. Having not seen your code I can't comment much except to ask if the file in question is MINC1 or MINC2 (use file 'fred.mnc' to find out, MINC1 is netcdf, MINC2 is HDF). both the MINC1 and MINC2 libraries can read minc1 files and this might be muddying the water. -- Andrew Janke (a.janke at gmail.com || http://a.janke.googlepages.com/) Canberra->Australia +61 (402) 700 883 From a.janke at gmail.com Mon Oct 26 01:14:02 2009 From: a.janke at gmail.com (Andrew Janke) Date: Mon, 26 Oct 2009 16:14:02 +1100 Subject: [MINC-users] mincconcat problem In-Reply-To: References: Message-ID: Hi Soren, On Fri, Oct 23, 2009 at 21:10, Soren Christensen wrote: > ?I have a problem with mincconcat where I get an unexpected result when > first merging, then splitting across time. I do not get back to the starting > point in terms of values altough it is close. > I originally encountered this as mnc files created by mincconcat (time > concat) in some cases differed by integer values of +/-1 relative to the > original DICOMs or individual mincfiles prior to running mincconcat. > > Example: > I have two minc files > v1.mnc > v2.mnc > (mincinfo below for all files) > > I use mincconcat to merge them across time: > mincconcat -clobber -concat_dimension time v1.mnc v2.mnc v_concat.mnc > > ... > > Is this expected behavior of mincconcat? > And if so - is there a way to get around this? This will depend on the datatype as if your input volumes have different ranges there is no way to keep the entire range without a bit or range re-scaling and subsequent accuracy loss. The simple fix is to just do all your processing in float. Once this has happened all range scaling is irrelevant. -- Andrew Janke (a.janke at gmail.com || http://a.janke.googlepages.com/) Canberra->Australia +61 (402) 700 883 From parya.momayyezsiahkal at mail.mcgill.ca Mon Oct 26 10:22:09 2009 From: parya.momayyezsiahkal at mail.mcgill.ca (Parya Mamayyez Siahkal) Date: Mon, 26 Oct 2009 10:22:09 -0400 Subject: [MINC-users] Problem in Parallel mode In-Reply-To: References: <4AE1F188.5090508@mail.mcgill.ca> Message-ID: <4AE5B091.20702@mail.mcgill.ca> Hi Andrew, The files I am using are all MINC1 files. Does this help? In terms of where this error messages are being generated, it is just at the beginning of the program where I am trying to initialize the volume: volume_io::set_volume_real_value(_cf_minc_volume,n,z,y,x,0,value); thanks, Parya Andrew Janke wrote: > Hi Parya, > > > >> I am using minc library to read and write 4D volumes. >> The code works fine when I am not running it in parallel. >> Even in the parallel mode, I don't have any problem with sufficiently >> small volumes. >> >> However, as soon as the volume gets a bit big(e.g. 30*40*40), and I >> want to run the code in parallel, I'll get different kind of error >> messages at different runs, some of them are: >> >> ncvarid: ncvarid: ncid 3ncid 6: NetCDF: Variable not found: NetCDF: >> Variable not found >> LocalParallelCF: posixio.c:571: px_get: Assertion `pxp->bf_refcount <= >> 0' failed. >> LocalParallelCF: posixio.c:248: px_pgin: Assertion `*posp == >> ((off_t)(-1)) || *posp == lseek(nciop->fd, 0, 1)' failed. >> >> I have switched between minc1 and minc2, but the same kind of errors are >> being generated. >> > > Having not seen your code I can't comment much except to ask if the > file in question is MINC1 or MINC2 (use file 'fred.mnc' to find out, > MINC1 is netcdf, MINC2 is HDF). both the MINC1 and MINC2 libraries can > read minc1 files and this might be muddying the water. > > -- > Andrew Janke > (a.janke at gmail.com || http://a.janke.googlepages.com/) > Canberra->Australia +61 (402) 700 883 > > _______________________________________________ > MINC-users at bic.mni.mcgill.ca > http://www2.bic.mni.mcgill.ca/mailman/listinfo/minc-users > > From sorench at gmail.com Mon Oct 26 19:38:28 2009 From: sorench at gmail.com (Soren Christensen) Date: Tue, 27 Oct 2009 07:38:28 +0800 Subject: [MINC-users] mincconcat problem In-Reply-To: References: Message-ID: Thanks Andrew! Cheers Soren On 10/26/09, Andrew Janke wrote: > Hi Soren, > > On Fri, Oct 23, 2009 at 21:10, Soren Christensen wrote: >> ?I have a problem with mincconcat where I get an unexpected result when >> first merging, then splitting across time. I do not get back to the >> starting >> point in terms of values altough it is close. >> I originally encountered this as mnc files created by mincconcat (time >> concat) in some cases differed by integer values of +/-1 relative to the >> original DICOMs or individual mincfiles prior to running mincconcat. >> >> Example: >> I have two minc files >> v1.mnc >> v2.mnc >> (mincinfo below for all files) >> >> I use mincconcat to merge them across time: >> mincconcat -clobber -concat_dimension time v1.mnc v2.mnc v_concat.mnc >> >> ... >> >> Is this expected behavior of mincconcat? >> And if so - is there a way to get around this? > > This will depend on the datatype as if your input volumes have > different ranges there is no way to keep the entire range without a > bit or range re-scaling and subsequent accuracy loss. > > The simple fix is to just do all your processing in float. Once this > has happened all range scaling is irrelevant. > > > -- > Andrew Janke > (a.janke at gmail.com || http://a.janke.googlepages.com/) > Canberra->Australia +61 (402) 700 883 > > _______________________________________________ > MINC-users at bic.mni.mcgill.ca > http://www2.bic.mni.mcgill.ca/mailman/listinfo/minc-users > From a.janke at gmail.com Mon Oct 26 20:30:24 2009 From: a.janke at gmail.com (Andrew Janke) Date: Tue, 27 Oct 2009 11:30:24 +1100 Subject: [MINC-users] Problem in Parallel mode In-Reply-To: <4AE5B091.20702@mail.mcgill.ca> References: <4AE1F188.5090508@mail.mcgill.ca> <4AE5B091.20702@mail.mcgill.ca> Message-ID: > The files I am using are all MINC1 files. > Does this help? Well it helps me but you aren't going to like the answer that MINC1 files (as far as I am aware) don't support parallel access as they are netcdf3 based. Netcdf4 does do parallel access but this is because it is HDF5 underneath.. > In terms of where this error messages are being generated, it is just at > the beginning of the program where I am trying to initialize the volume: > > volume_io::set_volume_real_value(_cf_minc_volume,n,z,y,x,0,value); Erk! where does this C++ wrapper come from? In any case I don't like to be the bearer of bad news but I doubt that anything that uses volume_io will ever allow parallel access. (unless you do something internally and have a single process that reads and writes the resulting volume out). It is possible to do parallel access with HDF5 files so if you are careful with how you use the libminc2 functions if should be possible. Either that or just use raw HDF5 calls on the minc volume itself. This link should help: http://www.physics.ohio-state.edu/~wilkins/computing/HDF/hdf5tutorial/pprog.html Good luck! -- Andrew Janke (a.janke at gmail.com || http://a.janke.googlepages.com/) Canberra->Australia +61 (402) 700 883 From sorench at gmail.com Tue Oct 27 23:18:20 2009 From: sorench at gmail.com (Soren Christensen) Date: Wed, 28 Oct 2009 11:18:20 +0800 Subject: [MINC-users] emma on vista Message-ID: Hi, I have trouble getting the emma mex files working on vista. Has anybody else had problems with this? I get error mesages such as: "No error" (yes a strange error message) from miinquire when called with a valid syntax In case a rebuild is necesarry, what would be the best way to approach this in terms of compiler and other prerequisites? Thanks, Soren From parya.momayyezsiahkal at mail.mcgill.ca Wed Oct 28 09:44:43 2009 From: parya.momayyezsiahkal at mail.mcgill.ca (Parya Mamayyez Siahkal) Date: Wed, 28 Oct 2009 09:44:43 -0400 Subject: [MINC-users] Problem in Parallel mode In-Reply-To: References: <4AE1F188.5090508@mail.mcgill.ca> <4AE5B091.20702@mail.mcgill.ca> Message-ID: <4AE84ACB.8010902@mail.mcgill.ca> Andrew Janke wrote: >> The files I am using are all MINC1 files. >> Does this help? >> > > Well it helps me but you aren't going to like the answer that MINC1 > files (as far as I am aware) don't support parallel access as they are > netcdf3 based. Netcdf4 does do parallel access but this is because it > is HDF5 underneath.. > > >> In terms of where this error messages are being generated, it is just at >> the beginning of the program where I am trying to initialize the volume: >> >> volume_io::set_volume_real_value(_cf_minc_volume,n,z,y,x,0,value); >> > > Erk! where does this C++ wrapper come from? In any case I don't like > to be the bearer of bad news but I doubt that anything that uses > volume_io will ever allow parallel access. (unless you do something > internally and have a single process that reads and writes the > resulting volume out). > > It is possible to do parallel access with HDF5 files so if you are > careful with how you use the libminc2 functions if should be possible. > Either that or just use raw HDF5 calls on the minc volume itself. > This link should help: > > http://www.physics.ohio-state.edu/~wilkins/computing/HDF/hdf5tutorial/pprog.html > > Good luck! > > > -- > Andrew Janke > (a.janke at gmail.com || http://a.janke.googlepages.com/) > Canberra->Australia +61 (402) 700 883 > _______________________________________________ > MINC-users at bic.mni.mcgill.ca > http://www2.bic.mni.mcgill.ca/mailman/listinfo/minc-users > > Hi Andrew, Thanks a lot for your response. But what exactly it means that I cannot run the code in parallel!? I am using Intel threading building blocks. and it works for smaller volumes. I 'll get exactly the same result if I run the program on a single processor. Part of the code which can be run in parallel is as follows: void CompletionField::SetAllVoxels(float value, ImageData *input_mask) { int xsize, ysize, zsize; xsize = this->GetXSize(); ysize = this->GetYSize(); zsize = this->GetZSize(); static affinity_partitioner ap; InitVolume init_vol(this, value, input_mask); if (!_parallel) { init_vol(blocked_range3d(0, xsize, 0, ysize, 0, zsize)); } else if (_parallel) { if (_grnSize) { parallel_for( blocked_range3d(0, xsize, int_max(1, floor(xsize/_grnSize)), 0, ysize, int_max(1, floor(ysize/_grnSize)), 0, zsize, int_max(1, floor(zsize/_grnSize))), init_vol); } else { parallel_for( blocked_range3d(0, xsize, 0, ysize, 0, zsize), init_vol, ap); } } } class InitVolume { CompletionField* InCFVolume; ImageData* input_mask; float value; public: void operator() (const blocked_range3d& r) const { int x,y,z,n; for (x = r.pages().begin(); x != r.pages().end(); ++x) { for (y = r.rows().begin(); y != r.rows().end(); ++y) { for (z = r.cols().begin(); z != r.cols().end(); ++z) { if (input_mask->GetValue(x,y,z) > FLT_EPSILON) { for (n = 0; n < InCFVolume->GetNSize(); n++) { InCFVolume->SetValue(x, y, z, n, value); } } } } } } InitVolume(CompletionField *CFVolume, float val, ImageData *mask) { input_mask = mask; InCFVolume = CFVolume; value = val; } }; void CompletionField::SetValue(int x, int y, int z, int n, float value) { if (this->GetDimensions()!=4) { Error("Illegal use of CompletionField volume!"); } set_volume_real_value(_cf_minc_volume,n,z,y,x,0,value); } regards, Parya From sorench at gmail.com Wed Oct 28 22:56:29 2009 From: sorench at gmail.com (Soren Christensen) Date: Thu, 29 Oct 2009 10:56:29 +0800 Subject: [MINC-users] emma Message-ID: Sorry, wrt. my previous question I realized I was referencing softlinks that were not redirected on windows and there are no problems with EMMA. I noticed Matlab now has both NETCDF and HDF support so I wonder if it would be an option to replace (since this has been debated) EMMA with a Matlab based interface that one could speculate would be easier to maintain over time and OSs. Cheers Soren From a.janke at gmail.com Wed Oct 28 23:03:52 2009 From: a.janke at gmail.com (Andrew Janke) Date: Thu, 29 Oct 2009 14:03:52 +1100 Subject: [MINC-users] emma In-Reply-To: References: Message-ID: Hi Soren, On Thu, Oct 29, 2009 at 13:56, Soren Christensen wrote: > Sorry, wrt. my previous question I realized I was referencing > softlinks that were not redirected on windows and there are no > problems with EMMA Good to hear. I was dreading having to put vista back onto my new (Ubuntu) laptop just to test this... > I noticed Matlab now has both NETCDF and HDF support so I wonder if it > would be an option to replace (since this has been debated) EMMA with > a Matlab based interface that one could speculate would be easier to > maintain over time and OSs. Well There is work on new project called niak now by Pierre Bellec http://code.google.com/p/niak/ I have now dumped emma in favour of this if only because it supports octave as well. I don't know if it works on XP, Vista or 7 but if you can get EMMA going on windows then odds are you will be able to sort out niak. -- Andrew Janke (a.janke at gmail.com || http://a.janke.googlepages.com/) Canberra->Australia +61 (402) 700 883 From nikelski at bic.mni.mcgill.ca Thu Oct 29 13:52:14 2009 From: nikelski at bic.mni.mcgill.ca (EJ Nikelski) Date: Thu, 29 Oct 2009 10:52:14 -0700 Subject: [MINC-users] emma In-Reply-To: References: Message-ID: Hi all, Just a quick FYI related to this post. Work on the new mniMincIO package (version 1.0) is now pretty much complete, and will be made available as soon as I finish the documentation and take it for a test drive on a non-trivial internal project. This package uses the Minc2 API and can replace all EMMA functionality (plus a lot more). It's purpose is to provide an easy to use rapid application development (RAD) environment, which will make working with Minc volumes an attractive alternative to the other volume formats. Unlike EMMA, mniMincIO is built as an R package, and as such, RAD development can be done using the very powerful (functional) R language. More info to follow. Cheers, -Jim On Wed, Oct 28, 2009 at 8:03 PM, Andrew Janke wrote: > Hi Soren, > > On Thu, Oct 29, 2009 at 13:56, Soren Christensen wrote: >> Sorry, wrt. my previous question I realized I was referencing >> softlinks that were not redirected on windows and there are no >> problems with EMMA > > Good to hear. I was dreading having to put vista back onto my new > (Ubuntu) laptop just to test this... > >> I noticed Matlab now has both NETCDF and HDF support so I wonder if it >> would be an option to replace (since this has been debated) EMMA with >> a Matlab based interface that one could speculate would be easier to >> maintain over time and OSs. > > Well There is work on new project called niak now by Pierre Bellec > > ? http://code.google.com/p/niak/ > > I have now dumped emma in favour of this if only because it supports > octave as well. > I don't know if it works on XP, Vista or 7 but if you can get EMMA > going on windows then odds are you will be able to sort out niak. > > -- > Andrew Janke > (a.janke at gmail.com || http://a.janke.googlepages.com/) > Canberra->Australia ? ?+61 (402) 700 883 > _______________________________________________ > MINC-users at bic.mni.mcgill.ca > http://www2.bic.mni.mcgill.ca/mailman/listinfo/minc-users > -- ================================= Jim Nikelski, Ph.D. Postdoctoral Research Fellow Bloomfield Centre for Research in Aging Lady Davis Institute for Medical Research Sir Mortimer B. Davis - Jewish General Hospital McGill University