[MINC-users] EMMA replacement?
Jason Lerch
jason at bic.mni.mcgill.ca
Fri Jul 31 17:25:04 EDT 2009
I have modules to read MINC2 volumes in both R (RMINC) and python
(pyminc - I'm very creative with my naming scheme). If anyone wants
either I'm happy to send them to you. RMINC is already on launchpad,
and I keep meaning to put the pyminc stuff there too ...
Little side node about R - while I absolutely love it for the majority
of my stats work, it's not the best development language because it's
too slow for most of what us imaging folks want. So every function
that I use often I've rewritten in C, leaving R mostly as a nice shell
for handling data/formulas and for graphics. But rewriting all core
algorithms in C certainly slows down development.
Jason
On 31-Jul-09, at 5:07 PM, Alexandre CARMEL-VEILLEUX wrote:
> Why not hitch your wagon behind a general purpose language like
> Python? People are doing all thee kinds of things in Python already
> (i.e.: http://www.astro.cornell.edu/staff/loredo/statpy/ for example)
>
> Plus if you make that leap, You can use RSPython, the R/SPlus bridge
> to call any R function from python. There's at least two modules
> that can deal with netCDF so the mechanics of opening MINC1 files
> would not be too hard to do. Same for HDF5 / Minc2.
>
> Why limit this to MATLAB vs. R? A python interface would be much
> more useful to the non-fMRI MINC users then R too...
>
> Alex
>
> On Fri, Jul 31, 2009 at 04:40:41PM -0400, EJ Nikelski wrote:
>> Date: Fri, 31 Jul 2009 16:40:41 -0400
>> From: EJ Nikelski <nikelski at bic.mni.mcgill.ca>
>> To: minc users <MINC-users at bic.mni.mcgill.ca>
>> Cc: "Howard Chertkow, Dr." <howard.chertkow at mcgill.ca>
>> Subject: Re: [MINC-users] EMMA replacement?
>>
>> Hi All,
>>
>> I hope that none of the original posters to this thread mind, but I
>> would really like to bounce this conversion past all minc users, as
>> all minc users may have an interest and/or input, and I believe that
>> the points being made are stimulating and excellent grist for a
>> sleepy
>> Friday afternoon.
>>
>> Thanks Pierre for sharing your views and experiences. Matlab does
>> indeed seem to be the currently reigning champ with regard to
>> high-level multi-platform image manipulation software. Of course,
>> most compelling is the fact that it can certainly do the job, as
>> evidenced by Pierre's work, as well as by that done by the SPM group.
>> So, it would appear as if any suggested move away from Matlab would
>> need to be able to show a clear set of advantages over Matlab.
>>
>> Octave, whilst circumventing the Matlab licensing issues still has
>> a number of problems. Firstly, and quite significantly, IMHO Octave
>> is a second class citizen, which simply looks like an inferior Matlab
>> wanna-be. I've been following Octave for years, and its development
>> has always been stop-and-go, and the developers really don't seem
>> willing to plot their own course. Personally, I would think long and
>> hard before hitching my wagon to Octave. Secondly, Octave suffers
>> from the fact that, like Matlab, it's a development language that is
>> geared towards linear algebra -- not statistics.
>>
>> Now, I know that people can get terribly attached to their
>> development environments, so I shall tread carefully. Whether one
>> chooses Matlab or R seems to be a function of what one wishes to do.
>> Matlab is all about linear algebra, which can indeed come in very
>> handy when working with imaging volumes. R, on the other hand, is
>> all
>> about statistics, so ... if, for example, you are doing 80,000
>> regressions on a cortical surface, R would be an appropriate tool.
>> Not
>> only because R can do linear regression out-of-the-box, but it can
>> also work (just as easily) with nonlinear regression, mixed-effects
>> models, generalized additive models, and just about anything else
>> that
>> you might want to throw at it. In addition, it includes an extremely
>> flexible and powerful programming language, extensible with C call
>> (as
>> needed). But wait, there's more! R can also make use of multiple
>> cores, permitting easy parallelization (sp?) of processing (see the
>> "doMC" and "foreach" packages). Of course, Matlab *can* do stats, as
>> demonstrated by Pierre and Keith Worsley, and the SPM folks, however,
>> I suggest that using Matlab for this purpose is like trying to peel a
>> banana while wearing boxing gloves -- it's messy, and there are
>> better
>> alternatives.
>>
>> Finally, nobody has addressed the gorilla in the room --- has
>> nifti-1 basically already killed minc, making all of this rather
>> academic?
>>
>>
>> Have a spliffy weekend,
>>
>> -Jim
>>
>>
>>
>> On Fri, Jul 31, 2009 at 1:04 PM, Samir Das<samirdas99 at gmail.com>
>> wrote:
>>> Hello all,
>>> Just adding a bit to what Pierre said. I agree that his new
>>> pipeline manager
>>> PSOM and the NIAK toolkit is a great way to go, especially since
>>> it can work
>>> with Octave which is an open source language (therefore no pricey
>>> licenses).
>>> From what I understand, it takes care of almost all of the issues
>>> that one
>>> would run into for fMRI processing and it seems to be working
>>> under quite
>>> well for those that have started using it. That said, Pierre is
>>> but one man
>>> and ongoing development continues. The only potential drawback
>>> (other than
>>> one's choice of programming language) is that Octave sometimes
>>> does not work
>>> exactly the same way as Matlab, so work will be required to
>>> maintain it.
>>> The reason I am speaking up is because from the thread below, I
>>> have seen
>>> some comments about developing this in R. I have spent quite a bit
>>> of time
>>> using and understanding Jason's R libraries, particularly as it
>>> relates to
>>> VBM and cortical thickness, and I think I would be able to enhance
>>> or add to
>>> those libraries to enable things like 4D image importing and
>>> manipulation.
>>> My only problem is while I would be interested in looking into
>>> something
>>> like this, I really can't seem to find the time right now.
>>> Again, that could be a moot point since Pierre has
>>> already developed something quite nice. I just thought I would
>>> pipe in.
>>> Samir Das
>>>
>>>
>>> On Thu, Jul 30, 2009 at 5:19 PM, Alan EVANS
>>> <alan at bic.mni.mcgill.ca> wrote:
>>>>
>>>> Hi Pierre,
>>>> Thanks for the rapid (and complete) response.
>>>> Alan
>>>>
>>>>
>>>> On Thu, 30 Jul 2009, Pierre Bellec wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> When reading Jim's email I recognized many of the questions I
>>>>> ran into in
>>>>> the recent past. Here's an update of the choices I've made at
>>>>> the time.
>>>>>
>>>>> There is first the question of the choice of the programming
>>>>> language for
>>>>> future developments. Coming from an applied maths background,
>>>>> matlab was
>>>>> sort of a natural choice. It lets you code powerful algorithms
>>>>> into
>>>>> concise
>>>>> code, has very decent runtime execution performance and great
>>>>> debugging/profiling capabilities. I don't have enough
>>>>> experiences with
>>>>> other
>>>>> languages (including R) to discuss the pros and cons in details,
>>>>> all I
>>>>> know
>>>>> is that it does the job for me and it is very well established
>>>>> in the
>>>>> biomedical engineering community, among others. There is
>>>>> obviously a
>>>>> concern
>>>>> regarding Matlab licenses. I have been trying over the past year
>>>>> to make
>>>>> my
>>>>> codes compatible with both Matlab and Octave (the GNU clone of
>>>>> Matlab)
>>>>> and
>>>>> it is almost painless. Octave is not as comfortable as Matlab in
>>>>> the
>>>>> development phase (mainly due to poor debugging tools), but to
>>>>> run code
>>>>> developed for Matlab it works. My conclusion is that Matlab is a
>>>>> perfectly
>>>>> viable choice to develop and disseminate image analysis/
>>>>> statistical
>>>>> analysis
>>>>> tools. But if for some reason (coding style or existing
>>>>> libraries for
>>>>> example) one wants to go for R, I think that would work too and it
>>>>> certainly
>>>>> did for Jason. It would only take a couple of motivated
>>>>> developers to set
>>>>> up
>>>>> coding guidelines and core functions to extend the Rminc package
>>>>> into
>>>>> something cleaner.
>>>>>
>>>>> Speaking of which, I started such an initiative about a year
>>>>> ago, using
>>>>> Matlab/Octave as my reference language. The project is hosted on
>>>>> google
>>>>> code
>>>>> and called NIAK <http://code.google.com/p/niak/>. The objective
>>>>> for
>>>>> release
>>>>> 1.0 is to have a full pipeline for fMRI preprocessing and
>>>>> general linear
>>>>> model analysis. I am currently working with Felix Carbonell on
>>>>> breaking
>>>>> up
>>>>> fMRIstat into modules and including them in the pipeline. The fMRI
>>>>> preprocessing pipeline has been working for a couple of months
>>>>> now and
>>>>> was
>>>>> used notably by people from Jean Gotman and Alain Dagher's lab.
>>>>> The
>>>>> feedback
>>>>> is good so far even though NIAK is more oriented towards
>>>>> developers than
>>>>> end-users. I am pushing for the linear model pipeline to be
>>>>> integrated in
>>>>> the Cbrain project which would provide the final nice user-
>>>>> friendly
>>>>> interface. The timeline for first release is next winter, and I am
>>>>> planning
>>>>> on starting to advertise for the project at that point.
>>>>>
>>>>> I have developed some (hopefully) clean coding guidelines, and the
>>>>> pipeline
>>>>> runtime is powered by a generic pipeline manager called
>>>>> PSOM<http://code.google.com/p/psom/>.
>>>>> PSOM is essentially an extension of PMP fully written in Matlab.
>>>>> I am
>>>>> currently using it at CLUMEQ (the McGill supercomputing center)
>>>>> and a
>>>>> pipeline of more than 15000 jobs was handled there smoothly. I
>>>>> believe
>>>>> that
>>>>> this framework is quite efficient at handling compicated
>>>>> pipelines with
>>>>> tons
>>>>> of parameters in a distributed computing environment, especially
>>>>> if one
>>>>> wants to play around with the flowchart and the choice of
>>>>> parameters. At
>>>>> least, it was designed with that purpose in mind. My plan is to
>>>>> use NIAK
>>>>> as
>>>>> the framework for all my future developments. I haven't done much
>>>>> publicity
>>>>> still, but Dr Christophe Grova and a couple of his students have
>>>>> adopted
>>>>> it
>>>>> as their development framework, as well as Gaolang Gang (a
>>>>> postdoc from
>>>>> Alan's lab) and the feedback from them is very positive. I
>>>>> would be more
>>>>> than happy if some developers were interested to develop their own
>>>>> toolbox
>>>>> in the NIAK framework, right now it is essentially a solo effort
>>>>> with
>>>>> input
>>>>> from Felix (and Keith in a sense) for the GLM part, and a couple
>>>>> of
>>>>> functions contributed by Vincent Perlbarg from my former lab in
>>>>> Paris.
>>>>> But
>>>>> of course, as I said before, Rminc is also a very nice package
>>>>> that would
>>>>> be
>>>>> a great basis for future development.
>>>>>
>>>>> I almost forgot... the MINC reader/writer ! So I had to get rid
>>>>> of EMMA.
>>>>> First because it is awfully slow, second because I wanted to avoid
>>>>> compiled
>>>>> code and third because I needed something that worked with
>>>>> Octave. There
>>>>> are
>>>>> functions called NIAK_READ_VOL and NIAK_WRITE_VOL that will do
>>>>> the job,
>>>>> and
>>>>> actually support NIFTI and ANALYZE as well (the output header
>>>>> structure
>>>>> is
>>>>> common to the three format, except for an HDR.DETAILS field that
>>>>> contains
>>>>> the complete list of format-specific fields). The reader/writer
>>>>> is just a
>>>>> wraper arond MINCHEADER, RAWTOMINC and MINCTORAW, so as long as
>>>>> the minc
>>>>> tools are installed it should work. I still have the idea to
>>>>> code a
>>>>> reader
>>>>> based on the internal matlab HDF5 and NETCDF libraries (which
>>>>> would be
>>>>> faster and cleaner) but I am not diving in because that will be
>>>>> a pure
>>>>> waste
>>>>> of time as far as octave is concerned.
>>>>>
>>>>> Let me know what you think,
>>>>>
>>>>> Pierre
>>>>>
>>>>> 2009/7/30 Alan EVANS <alan at bic.mni.mcgill.ca>
>>>>>
>>>>>> Hi Jim,
>>>>>> Somebody, has recently developed tools to by-pass the need
>>>>>> for EMMA
>>>>>> (NIAK, Pierre ?). Can somebody on this cc-list clarify ?.
>>>>>> Alan
>>>>>>
>>>>>>
>>>>>> On Tue, 28 Jul 2009, EJ Nikelski wrote:
>>>>>>
>>>>>> Hi Howard and Alan,
>>>>>>>
>>>>>>> The purpose of this e-mail is to elucidate some of the ideas
>>>>>>> that
>>>>>>> were expressed in a recent meeting between Howard and myself
>>>>>>> and gets
>>>>>>> Alan's input. These ideas sprang from our discussion
>>>>>>> regarding our
>>>>>>> PiB data, and how best to write our analysis functions --
>>>>>>> particularly
>>>>>>> given that the PiB data starts life as a 4-D volume.
>>>>>>>
>>>>>>> Background:
>>>>>>> Many years ago, immediately subsequent to the development of
>>>>>>> minc,
>>>>>>> Sean Marrett and other wrote a Matlab toolbox (EMMA), which
>>>>>>> allowed
>>>>>>> users to write Matlab functions capable of reading/writing minc
>>>>>>> volumes. This toolbox has remained heavily used around BIC,
>>>>>>> as it
>>>>>>> allows us to easily manipulate minc data. Unfortunately,
>>>>>>> there are a
>>>>>>> few problems: (1) EMMA requires a Matlab license, which is often
>>>>>>> expensive -- depending upon the whims of The Mathworks people
>>>>>>> and
>>>>>>> whether McGill is able to wrangle a site license, (2) EMMA is
>>>>>>> not
>>>>>>> really supported any more, and certainly isn't being actively
>>>>>>> developed, Matlab, while well suited for linear algebra, is a
>>>>>>> pretty
>>>>>>> poor excuse for a proper programming language.
>>>>>>>
>>>>>>> The alternative to using EMMA would be to continue along the
>>>>>>> path
>>>>>>> that Jason Lerch set with his RMINC package. Jason's RMINC
>>>>>>> package
>>>>>>> uses the R statistical software (instead of Matlab), and
>>>>>>> extends it,
>>>>>>> permitting it to read/write minc2 volumes. Advantages
>>>>>>> include: (1) R
>>>>>>> is a more sophisticated high-level functional language --
>>>>>>> Matlab is
>>>>>>> tuned for linear algebra, and it shows, (2) R is specialized for
>>>>>>> statistical processing, Matlab is not, (3) R is an open source
>>>>>>> project, thus no license fees, and (4) R is actively being
>>>>>>> developed.
>>>>>>>
>>>>>>> Unfortunately, Jason's RMINC package has a few drawbacks: (1)
>>>>>>> it
>>>>>>> will not read/write 4-D volumes, which we *need* when
>>>>>>> processing PiB
>>>>>>> volumes, (2) Jason incorporated VBM functions into the RMINC
>>>>>>> package,
>>>>>>> making the package a bit messy, and (3) Jason wrote the
>>>>>>> interface
>>>>>>> using R S3 classes, which also makes the objects a bit messy
>>>>>>> and less
>>>>>>> maintainable. If we were to go down this road I would suggest
>>>>>>> taking
>>>>>>> inspiration from RMINC, and : (1) design it using S4 classes
>>>>>>> and the
>>>>>>> minc2 API, (2) remove the VBM code, creating a package
>>>>>>> comprised of
>>>>>>> R/minc accessor functions only (call it, say, mniMincIO), (3)
>>>>>>> the
>>>>>>> accessor functions would be able to deal with 4-D volumes, and
>>>>>>> (4) a
>>>>>>> simplified layer could be written on top of mniMincIO to
>>>>>>> present an
>>>>>>> EMMA-like simple interface. I believe that I could write the
>>>>>>> main
>>>>>>> accessor functions in 1 week.
>>>>>>>
>>>>>>> So the question is: Is this functionality worth the work? I
>>>>>>> believe
>>>>>>> that having solid R-based minc access functions would make
>>>>>>> life a lot
>>>>>>> easier for those of us who are writing our own analysis
>>>>>>> routines using
>>>>>>> minc volumes. Having said that, EMMA, as old and decrepit as
>>>>>>> it is,
>>>>>>> *can* currently do the job -- it just isn't as pretty, and
>>>>>>> convenient,
>>>>>>> and requires Matlab. I suppose we also need to consider the
>>>>>>> future of
>>>>>>> minc ... and whether minc *has* a future, given that the
>>>>>>> "provisional"
>>>>>>> nifti-1 standard appears to be very widely used and accepted.
>>>>>>>
>>>>>>> Ideas and suggestions? Alan?
>>>>>>>
>>>>>>> -Jim
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> =================================
>>>>>>> Jim Nikelski, Ph.D.
>>>>>>> Postdoctoral Research Fellow
>>>>>>> Bloomfield Centre for Research in Aging
>>>>>>> Lady Davis Institute for Medical Research
>>>>>>> Sir Mortimer B. Davis - Jewish General Hospital
>>>>>>> McGill University
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Pierre Bellec
>>>>> Home page: http://wiki.bic.mni.mcgill.ca/index.php/PierreBellec
>>>>> Address: McConnel Brain Imaging Center, Webster 2B
>>>>> Montreal Neurological Institute
>>>>> 3801 University Street
>>>>> Montreal, Quebec, Canada H3A 2B4
>>>>> tel: (001)(514) 398 5220
>>>>> fax: (001)(514) 398 8948
>>>>>
>>>
>>>
>>
>>
>>
>> --
>> =================================
>> Jim Nikelski, Ph.D.
>> Postdoctoral Research Fellow
>> Bloomfield Centre for Research in Aging
>> Lady Davis Institute for Medical Research
>> Sir Mortimer B. Davis - Jewish General Hospital
>> McGill University
>>
>> _______________________________________________
>> MINC-users at bic.mni.mcgill.ca
>> http://www2.bic.mni.mcgill.ca/mailman/listinfo/minc-users
> _______________________________________________
> MINC-users at bic.mni.mcgill.ca
> http://www2.bic.mni.mcgill.ca/mailman/listinfo/minc-users
More information about the MINC-users
mailing list