From contact at hdfql.com Wed Mar 27 18:26:05 2019 From: contact at hdfql.com (HDFql) Date: Wed, 27 Mar 2019 15:26:05 -0700 Subject: [MINC-users] HDFql 2.0.0 Release - HDF5 made easy and fast Message-ID: <73e03932d9ae6ed21494ec849b90ca4263d48e59@atmail.dreamhost.com> We are happy to announce the release of HDFql 2.0.0! This version includes: - ADDED SUPPORT FOR PARALLEL HDF5 (PHDF5) - thanks to Quincey (Berkeley Lab, USA), Rob (Argonne Lab, USA), Konrad (CNRS, France), Sebastian (Juelich Supercomputing Centre, Germany), Guy (IN2P3, France) and Holger (Fermilab, USA) for the feedback. - ADDED WRAPPER FOR THE INTEL FORTRAN COMPILER (IFORT) - thanks to Intel for sponsoring HDFql with a free IFORT license. - IMPROVED PERFORMANCE AND MEMORY FOOTPRINT - ADDED CAPABILITY TO SPECIFY THE STORAGE ALLOCATION WHEN CREATING A DATASET (I.E. EARLY, INCREMENTAL OR LATE) - BUG FIXES - thanks to Erik (M+P, The Netherlands), Ryan (KMLabs, USA), Michael (U.S. Army Corps of Engineers, USA), Romain (CEA, France) and Petr (University of West Bohemia, Czech Republic) for reporting these. - UPDATED REFERENCE MANUAL - thanks to Jeff (Redwood Center for Theoretical Neuroscience, USA) for the feedback. (Please check the release notes for further details) For heads up on releases and the latest on HDFql, we welcome you to connect on twitter.com/hdfql [1] Links: ------ [1] https://twitter.com/hdfql From zijdenbos at gmail.com Thu Mar 28 09:13:30 2019 From: zijdenbos at gmail.com (Alex Zijdenbos) Date: Thu, 28 Mar 2019 09:13:30 -0400 Subject: [MINC-users] Time frame start values Message-ID: Hello all, I've running into an issue I don't know what to do with (yet). How are (or should) start values be interpreted in a MINC time-dimension? For spatial dimensions, the voxel positions are in the center of the voxel; but in converting dynamic PET data from various scanners, we are seeing what looks like variations between scanners in terms of whether the start is center-frame, or start-of-frame. We also sometimes end up with negative time steps out of dcm2mnc, which calls into question whether (a) that should ever happen; and (b) how for example mincreshape behaves when re-ordering that dimension. My assumption is that mincreshape will interpret the start to be center-frame, and that therefore changing the sign of the time-step might end up with a wrong start value. If anybody has thoughts about or experience with these issues, please share :-) Thanks, -- A From bert at phalarope.com Thu Mar 28 09:42:17 2019 From: bert at phalarope.com (Robert D. Vincent) Date: Thu, 28 Mar 2019 09:42:17 -0400 Subject: [MINC-users] Time frame start values In-Reply-To: References: Message-ID: Hi Alex, I've heard of these issues in a number of instances, I am (slowly, when time permits) trying to resolve one particular case in dcm2mnc. PET files often use the DICOM field 0054:1300, the "Frame Reference Time", as the main (or only) indication of PET frame start times. However, as you can see at the link I have provided, the exact meaning of this field is undefined, so it is unusually worthless even for DICOM. Some PET sequences use other fields to indicate the frame times, but consistency here is poor. https://www.dabsoft.ch/dicom/3/C.8.9.4.1.5/ My intended solution is to use the frame widths ("Actual Frame Duration", 0018:1242) to reconstruct the MINC dimension by simply setting the start time of each frame to the sum of the preceding frame widths. This relies on the accuracy of these frame widths, but in my experience this may be more reliable than relying on the frame times. However, if the frame times are somehow meaningful, that information will be lost (or at least not stored in a standard place). So my plan is to make this depend on a command line option. -bert On Thu, Mar 28, 2019 at 9:15 AM Alex Zijdenbos wrote: > Hello all, > > I've running into an issue I don't know what to do with (yet). How are (or > should) start values be interpreted in a MINC time-dimension? > > For spatial dimensions, the voxel positions are in the center of the voxel; > but in converting dynamic PET data from various scanners, we are seeing > what looks like variations between scanners in terms of whether the start > is center-frame, or start-of-frame. > > We also sometimes end up with negative time steps out of dcm2mnc, which > calls into question whether (a) that should ever happen; and (b) how for > example mincreshape behaves when re-ordering that dimension. My assumption > is that mincreshape will interpret the start to be center-frame, and that > therefore changing the sign of the time-step might end up with a wrong > start value. > > If anybody has thoughts about or experience with these issues, please share > :-) > > Thanks, > > -- A > _______________________________________________ > MINC-users at bic.mni.mcgill.ca > https://mailman.bic.mni.mcgill.ca/mailman/listinfo/minc-users > From jp at biospective.com Fri Mar 29 16:37:21 2019 From: jp at biospective.com (Jean-Philippe Coutu) Date: Fri, 29 Mar 2019 16:37:21 -0400 Subject: [MINC-users] Time frame start values In-Reply-To: References: Message-ID: Hi Robert, I like your idea of using the frame widths, and I just wanted to mention one caveat we stumbled upon recently, where this might not work. We have had to convert a dataset where the time frames were not contiguous, and therefore there were effectively missing time frames. I think this is unusual and maybe dcm2mnc should not be expected to be able to convert that, but I just thought I would mention it. I think having this option you suggest would definitely be good for this case though. We have been fixing up 4D cases for a while now, which largely seem to have issues in get_file_indices(), mostly because of what you stated: unusual worthlessness of certain dicom fields for certain situations/scanners. Basically there is no ideal order of prioritisation for which dicom tags are always more reliable than other when it comes to finding time indices (and also all sorts of range-checking issues). We haven't made pull requests for those changes yet because I think we need to reorganise them a bit better. Best, JP On Thu, Mar 28, 2019 at 9:42 AM Robert D. Vincent wrote: > Hi Alex, > > I've heard of these issues in a number of instances, I am (slowly, when > time permits) trying to resolve one particular case in dcm2mnc. > > PET files often use the DICOM field 0054:1300, the "Frame Reference Time", > as the main (or only) indication of PET frame start times. However, as you > can see at the link I have provided, the exact meaning of this field is > undefined, so it is unusually worthless even for DICOM. Some PET sequences > use other fields to indicate the frame times, but consistency here is poor. > > https://www.dabsoft.ch/dicom/3/C.8.9.4.1.5/ > > My intended solution is to use the frame widths ("Actual Frame Duration", > 0018:1242) to reconstruct the MINC dimension by simply setting the start > time of each frame to the sum of the preceding frame widths. This relies on > the accuracy of these frame widths, but in my experience this may be more > reliable than relying on the frame times. However, if the frame times are > somehow meaningful, that information will be lost (or at least not stored > in a standard place). So my plan is to make this depend on a command line > option. > > -bert > > On Thu, Mar 28, 2019 at 9:15 AM Alex Zijdenbos > wrote: > > > Hello all, > > > > I've running into an issue I don't know what to do with (yet). How are > (or > > should) start values be interpreted in a MINC time-dimension? > > > > For spatial dimensions, the voxel positions are in the center of the > voxel; > > but in converting dynamic PET data from various scanners, we are seeing > > what looks like variations between scanners in terms of whether the start > > is center-frame, or start-of-frame. > > > > We also sometimes end up with negative time steps out of dcm2mnc, which > > calls into question whether (a) that should ever happen; and (b) how for > > example mincreshape behaves when re-ordering that dimension. My > assumption > > is that mincreshape will interpret the start to be center-frame, and that > > therefore changing the sign of the time-step might end up with a wrong > > start value. > > > > If anybody has thoughts about or experience with these issues, please > share > > :-) > > > > Thanks, > > > > -- A > > _______________________________________________ > > MINC-users at bic.mni.mcgill.ca > > https://mailman.bic.mni.mcgill.ca/mailman/listinfo/minc-users > > > _______________________________________________ > MINC-users at bic.mni.mcgill.ca > https://mailman.bic.mni.mcgill.ca/mailman/listinfo/minc-users > From bert at phalarope.com Sun Mar 31 12:47:46 2019 From: bert at phalarope.com (Robert D. Vincent) Date: Sun, 31 Mar 2019 12:47:46 -0400 Subject: [MINC-users] Time frame start values In-Reply-To: References: Message-ID: Hi Alex, I may have solved some of these issues - I just pushed a few small updates to github. With at least some of the PET data I have, using the Acquisition time field is useful for the PET timings, although it produces a non-zero start value in most cases. Also I found and fixed a bug where we incorporated Siemens proprietary information into the PET DICOM, but our Siemens-specific code is only correct when used with MR data. Please try these out and let me know if you're still having problems. -bert On Fri, Mar 29, 2019 at 4:39 PM Jean-Philippe Coutu wrote: > Hi Robert, > > I like your idea of using the frame widths, and I just wanted to mention > one caveat we stumbled upon recently, where this might not work. We have > had to convert a dataset where the time frames were not contiguous, and > therefore there were effectively missing time frames. I think this is > unusual and maybe dcm2mnc should not be expected to be able to convert > that, but I just thought I would mention it. I think having this option you > suggest would definitely be good for this case though. > > We have been fixing up 4D cases for a while now, which largely seem to have > issues in get_file_indices(), mostly because of what you stated: unusual > worthlessness of certain dicom fields for certain situations/scanners. > Basically there is no ideal order of prioritisation for which dicom tags > are always more reliable than other when it comes to finding time indices > (and also all sorts of range-checking issues). We haven't made pull > requests for those changes yet because I think we need to reorganise them a > bit better. > > Best, > JP > > On Thu, Mar 28, 2019 at 9:42 AM Robert D. Vincent > wrote: > > > Hi Alex, > > > > I've heard of these issues in a number of instances, I am (slowly, when > > time permits) trying to resolve one particular case in dcm2mnc. > > > > PET files often use the DICOM field 0054:1300, the "Frame Reference > Time", > > as the main (or only) indication of PET frame start times. However, as > you > > can see at the link I have provided, the exact meaning of this field is > > undefined, so it is unusually worthless even for DICOM. Some PET > sequences > > use other fields to indicate the frame times, but consistency here is > poor. > > > > https://www.dabsoft.ch/dicom/3/C.8.9.4.1.5/ > > > > My intended solution is to use the frame widths ("Actual Frame Duration", > > 0018:1242) to reconstruct the MINC dimension by simply setting the start > > time of each frame to the sum of the preceding frame widths. This relies > on > > the accuracy of these frame widths, but in my experience this may be more > > reliable than relying on the frame times. However, if the frame times are > > somehow meaningful, that information will be lost (or at least not stored > > in a standard place). So my plan is to make this depend on a command line > > option. > > > > -bert > > > > On Thu, Mar 28, 2019 at 9:15 AM Alex Zijdenbos > > wrote: > > > > > Hello all, > > > > > > I've running into an issue I don't know what to do with (yet). How are > > (or > > > should) start values be interpreted in a MINC time-dimension? > > > > > > For spatial dimensions, the voxel positions are in the center of the > > voxel; > > > but in converting dynamic PET data from various scanners, we are seeing > > > what looks like variations between scanners in terms of whether the > start > > > is center-frame, or start-of-frame. > > > > > > We also sometimes end up with negative time steps out of dcm2mnc, which > > > calls into question whether (a) that should ever happen; and (b) how > for > > > example mincreshape behaves when re-ordering that dimension. My > > assumption > > > is that mincreshape will interpret the start to be center-frame, and > that > > > therefore changing the sign of the time-step might end up with a wrong > > > start value. > > > > > > If anybody has thoughts about or experience with these issues, please > > share > > > :-) > > > > > > Thanks, > > > > > > -- A > > > _______________________________________________ > > > MINC-users at bic.mni.mcgill.ca > > > https://mailman.bic.mni.mcgill.ca/mailman/listinfo/minc-users > > > > > _______________________________________________ > > MINC-users at bic.mni.mcgill.ca > > https://mailman.bic.mni.mcgill.ca/mailman/listinfo/minc-users > > > _______________________________________________ > MINC-users at bic.mni.mcgill.ca > https://mailman.bic.mni.mcgill.ca/mailman/listinfo/minc-users >