[Loris-dev] Import mri - scripts
Sotirios Nikoloutsopoulos
sotirisnik at gmail.com
Tue Aug 6 07:53:08 EDT 2019
I am using a VM. What i did was:
1) Download the files from https://github.com/aces/Loris and replace them
to my existing /vaw/www/loris folder.
2) Execute the composer install --no-dev and then composer dump-autoload
as mentioned here https://github.com/aces/Loris/wiki/Updating-your-LORIS.
3) Execute install.sh from /var/www/loris/tools and because LORIS already
existed i moved a .config file, then re-executed the install.sh successfully
4 ) As for the database, i dropped the schema and i executed the
installdb.php
Στις Δευ, 5 Αυγ 2019 στις 5:22 μ.μ., ο/η Christine Rogers, Ms. <
christine.rogers at mcgill.ca> έγραψε:
> Hi Sotorios,
>
> Thanks - when you re-installed did you start on a fresh root path / VM or
> might some of your old system setting still apply?
> (And if you did any steps similar to an upgrade, did you follow the
> Upgrade steps in the Release Notes, from both repos?)
>
> About the TR_range column - it was recently split (MRI pr339
> <https://github.com/aces/Loris-MRI/pull/339>) split into TR_min and _max
> columns. Given the code was trying to query TR_range which does not exist
> in your mri_protocol table, double-check which version of the Loris repo
> (tables) and the Loris-MRI repo (code) you have installed to ensure they
> match.
>
> Christine
>
>
> On Mon, Aug 5, 2019 at 9:06 AM Sotirios Nikoloutsopoulos <
> sotirisnik at gmail.com> wrote:
>
>> I reinstalled LORIS to 21 version and now the content is not visible at
>> all at most pages.
>>
>> [image: image.png]
>>
>> Στις Δευ, 5 Αυγ 2019 στις 3:19 μ.μ., ο/η Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> έγραψε:
>>
>>> Hi,
>>>
>>> Is it easy to upgrade to Loris 21? Also the following fails because the
>>> line 1985 of NeuroDB/MRIProcessingUtility.pm query mri_protocol which
>>> doesn't have TR_range
>>> https://github.com/aces/Loris/blob/21.0-release/SQL/0000-00-00-schema.sql#L556
>>>
>>> [image: image.png]
>>>
>>> lorisadmin at hbp:/data/loris/bin/mri$ ./batch_uploads_imageuploader
>>> -profile prod < ~/Desktop/input.txt > log.txt
>>> Use of uninitialized value $_ in pattern match (m//) at
>>> ./batch_uploads_imageuploader line 147.
>>> base is DCC0000_258024_V1
>>> path is /data/incoming/
>>> type is .tar.gz
>>> fullpath is /data/incoming/DCC0000_258024_V1.tar.gz
>>> DBD::mysql::st execute failed: Unknown column 'TR_range' in 'where
>>> clause' at
>>> /data/loris/bin/mri/uploadNeuroDB/NeuroDB/MRIProcessingUtility.pm line 1985.
>>> DBD::mysql::st fetchrow_array failed: fetch() without execute() at
>>> /data/loris/bin/mri/uploadNeuroDB/NeuroDB/MRIProcessingUtility.pm line 1986.
>>> Use of uninitialized value $count_mri_protocol in numeric gt (>) at
>>> /data/loris/bin/mri/uploadNeuroDB/NeuroDB/MRIProcessingUtility.pm line 1995.
>>> DBD::mysql::st execute failed: Unknown column 'TR_range' in 'field list'
>>> at /data/loris/bin/mri/uploadNeuroDB/NeuroDB/MRI.pm line 562.
>>> Can't exec "mail": No such file or directory at
>>> /data/loris/bin/mri/uploadNeuroDB/tarchiveLoader line 721.
>>> print() on closed filehandle MAIL at
>>> /data/loris/bin/mri/uploadNeuroDB/tarchiveLoader line 722.
>>> print() on closed filehandle MAIL at
>>> /data/loris/bin/mri/uploadNeuroDB/tarchiveLoader line 723.
>>> print() on closed filehandle MAIL at
>>> /data/loris/bin/mri/uploadNeuroDB/tarchiveLoader line 724.
>>> print() on closed filehandle MAIL at
>>> /data/loris/bin/mri/uploadNeuroDB/tarchiveLoader line 725.
>>>
>>> No Mincs inserted
>>>
>>> Thanks,
>>>
>>> Sotirios
>>>
>>> Στις Σάβ, 3 Αυγ 2019 στις 1:08 π.μ., ο/η Christine Rogers, Ms. <
>>> christine.rogers at mcgill.ca> έγραψε:
>>>
>>>> Hi Sotirios,
>>>>
>>>> First, just a note: we've released LORIS 21 this week with some major
>>>> new features and cleanup -- especially if you are early in your setup
>>>> phase, check out the Release notes for both the LORIS
>>>> <https://github.com/aces/Loris/releases/tag/v21.0.0> and Loris-MRI
>>>> <https://github.com/aces/Loris-MRI/releases/tag/v21.0.0> repos.
>>>>
>>>> Thanks for your screenshots -- your empty mri_protocol table should be
>>>> at the root of these issues :
>>>>
>>>> The mri_protocol table must be populated
>>>> <https://github.com/aces/Loris-MRI/blob/21.0-dev/docs/02-Install.md#221-database>
>>>> in order for your scans to be registered in LORIS -- it serves as a
>>>> whitelist for the insertion pipeline which filters out scans that aren't
>>>> identified or matched to scan types in this table.
>>>>
>>>> By default, this table is populated with entries for t1, t2, fMRI and
>>>> DTI (Insert statements for 21 release branch are here
>>>> <https://github.com/aces/Loris/blob/21.0-release/SQL/0000-00-00-schema.sql#L591>
>>>> )
>>>> Feel free to restore our default records and broaden their parameters
>>>> to match all your scan types.
>>>>
>>>> re Defaced, native, selected -- these filter the scans displayed as you
>>>> move to the next page (View Session), instead of viewing "all types" of
>>>> scans from a given session :
>>>>
>>>> - "defaced" means that a defacing script
>>>> <https://github.com/aces/Loris-MRI/blob/master/docs/scripts_md/run_defacing_script.md>
>>>> has masked the subject's face in the volume (for patient privacy)
>>>> - "native" means that no pre-processing has been performed on the
>>>> scan (it's raw data),
>>>> - "selected" will show only volumes that have been QC'd and
>>>> "selected" as the best of their type -- used to display only the best
>>>> quality T1 image instead of all T1s acquired in a session, e.g.
>>>>
>>>> re QC Comments --
>>>> Once you have successfully loaded scans (matched to the mri_protocol
>>>> table), you'll be able to use the Imaging Browser's View Session subpage to
>>>> enter/view QC comments on each volume, or for each visit.
>>>> These are stored directly in the database, and linked to each scan.
>>>> (Table name: feedback_mri_comments)
>>>> (Visit-level QC comments are linked to the session.)
>>>>
>>>> For a walkthrough of the Imaging setup - please see the Imaging Setup
>>>> guide
>>>> <https://github.com/aces/Loris-MRI/blob/21.0-dev/docs/02-Install.md>
>>>> -
>>>>
>>>> An old LORIS walkthrough video
>>>> <https://www.youtube.com/watch?v=HDWC6RUbI6A> can also walk you
>>>> through some of the Imaging front-end features, starting at 8:00
>>>> LORIS' imaging modules have been updated many times since this video,
>>>> but the functionality and workflow is similar.
>>>>
>>>> Let us know how it goes once you have entries in your mri_protocol
>>>> table -
>>>> Best,
>>>> Christine
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Aug 2, 2019 at 7:58 AM Sotirios Nikoloutsopoulos <
>>>> sotirisnik at gmail.com> wrote:
>>>>
>>>>> Hi Christine,
>>>>>
>>>>> In the candidate_list page i can see this patient
>>>>>
>>>>> [image: image.png]
>>>>>
>>>>> If i press the link under PSCID i get to see
>>>>> [image: image.png]
>>>>>
>>>>> and if i press the link under Scan Done it redirects me to
>>>>> imaging_browser
>>>>>
>>>>> [image: image.png]
>>>>>
>>>>> At the mri_violations page i also see No results ( neither on resolved
>>>>> / not resolved ).
>>>>>
>>>>> At the imaging_uploader i can see the record that failed.
>>>>>
>>>>> [image: image.png]
>>>>>
>>>>> If i click on the tarchive info i get to see the Protocol Name which i
>>>>> think is the acquisition we wanted?
>>>>> [image: image.png]
>>>>>
>>>>> Also my mri_protocol table is empty.
>>>>>
>>>>> Furthermore i would like to ask about the defaced, native, selected.
>>>>> What exactly is native and selected? And where are the qc comments stored?
>>>>>
>>>>> [image: image.png]
>>>>>
>>>>> Finally a walkthrough would be helpful.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Sotirios
>>>>>
>>>>> Στις Τρί, 30 Ιουλ 2019 στις 8:29 μ.μ., ο/η Christine Rogers, Ms. <
>>>>> christine.rogers at mcgill.ca> έγραψε:
>>>>>
>>>>>> Hi Sotirios,
>>>>>> Glad to hear you're making progress with the visit labels. To follow
>>>>>> up on your last few emails:
>>>>>>
>>>>>> a- The Wiki's Setup Guide contains Sample insert statements for the
>>>>>> Visit Windows table
>>>>>> <https://github.com/aces/Loris/wiki/Project-Customization#iv-visit-windows>
>>>>>> .
>>>>>>
>>>>>> As Cecile mentioned, the consistency of visit label format is
>>>>>> important: V01, v01, V1, 01 -- whichever you select to use is fine; we use
>>>>>> V01 by default across all documentation.
>>>>>>
>>>>>> b- For Password reset, there is a script in the tools/ directory so
>>>>>> that a back-end administrator can reset any front-end user password,
>>>>>> including Admin. (See info on setting up / resetting User passwords in the Wiki's
>>>>>> Setup Guide
>>>>>> <https://github.com/aces/Loris/wiki/Project-Customization#2-create-front-end-users>
>>>>>> )
>>>>>>
>>>>>> c- The minc file not being registered -- due to AcquisitionProtocol
>>>>>> unknown.
>>>>>> This means the SeriesDescription (Acquisition Protocol) in you Dicom
>>>>>> header was not matched to any scan listed in the mri_protocol database
>>>>>> table.
>>>>>> (In the online Loris-MRI Troubleshooting Guide,
>>>>>> <https://github.com/aces/Loris-MRI/blob/minor/docs/AppendixA-Troubleshooting_guideline.md> this
>>>>>> is Table 3, 2nd-last item "no MINCs inserted").
>>>>>>
>>>>>> The MRI Violations module (Readme here)
>>>>>> <https://github.com/aces/Loris/tree/minor/modules/mri_violations>will show
>>>>>> you the details of this mismatch --
>>>>>> Go to the module (under the Imaging menu), find your scan and click
>>>>>> the link "Could not identify scan type" -- in the next subpage you can
>>>>>> compare the scan's actual parameters against the mri_protocol table.
>>>>>>
>>>>>> To resolve this, adjust your mri_protocol table values or perhaps
>>>>>> ensure your DICOMs are labelled consistently.
>>>>>>
>>>>>> Let us know if this works for you -- we can also walk you through an
>>>>>> example of non-identified scans on Demo.loris.ca (MRI Violations
>>>>>> module).
>>>>>>
>>>>>> Best,
>>>>>> Christine
>>>>>>
>>>>>> The LORIS team
>>>>>>
>>>>>>
>>>>>> On Tue, Jul 30, 2019 at 10:47 AM Sotirios Nikoloutsopoulos <
>>>>>> sotirisnik at gmail.com> wrote:
>>>>>>
>>>>>>> I hope this is sufficient as the error for the visit label
>>>>>>> disappeared.
>>>>>>> [image: image.png]
>>>>>>>
>>>>>>> Please see the attached file. Also something i found.
>>>>>>>
>>>>>>> lorisadmin at hbp:/data/loris/bin/mri$ cat
>>>>>>> /data/loris/data//logs/TarLoad-17-39-xubBdv.log
>>>>>>>
>>>>>>> ==> Loading file from disk
>>>>>>> /tmp/TarLoad-17-39-UyQcG2/dcc0000_258024_v1_20121205_102108_501e1_mri.mnc
>>>>>>>
>>>>>>> --> mapping DICOM parameter for
>>>>>>> /tmp/TarLoad-17-39-UyQcG2/dcc0000_258024_v1_20121205_102108_501e1_mri.mnc
>>>>>>>
>>>>>>> ==> computing md5 hash for MINC body.
>>>>>>>
>>>>>>> --> md5: 6b1202fd63e29de4c9be1cd9925888a5
>>>>>>>
>>>>>>> ==> verifying acquisition protocol
>>>>>>>
>>>>>>> Acquisition protocol is unknown
>>>>>>>
>>>>>>> --> The minc file cannot be registered since the
>>>>>>> AcquisitionProtocol is unknown
>>>>>>>
>>>>>>> Στις Τρί, 30 Ιουλ 2019 στις 4:58 μ.μ., ο/η Sotirios Nikoloutsopoulos
>>>>>>> <sotirisnik at gmail.com> έγραψε:
>>>>>>>
>>>>>>>> Ok I fixed the login problem by using a known hash and by deleting
>>>>>>>> the rows from the login_history i was able to login.
>>>>>>>>
>>>>>>>> Στις Τρί, 30 Ιουλ 2019 στις 4:50 μ.μ., ο/η Sotirios
>>>>>>>> Nikoloutsopoulos <sotirisnik at gmail.com> έγραψε:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> there was no visit label 01 in the Visit_Windows table and i don't
>>>>>>>>> know how to insert it because the record requires lots of attributes. Also
>>>>>>>>> the system for a reason asked me to update my password and now i don't
>>>>>>>>> remember it and my account got suspended. Is there a way to reset it or set
>>>>>>>>> a simple password? I updated the password hash in the users table by using
>>>>>>>>> the hash from another LORIS setup we had and i still couldn't login ( also
>>>>>>>>> the attribute password is always NULL and the expiration date is 1999-01-01
>>>>>>>>> for some reason ).
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Στις Δευ, 29 Ιουλ 2019 στις 5:21 μ.μ., ο/η Cecile Madjar <
>>>>>>>>> cecile.madjar at mcin.ca> έγραψε:
>>>>>>>>>
>>>>>>>>>> Hi Sotirios,
>>>>>>>>>>
>>>>>>>>>> Can you try adding the visit label v01 to the Visit_Windows
>>>>>>>>>> table?
>>>>>>>>>>
>>>>>>>>>> However, I noticed that in a previous dataset, the label was V1.
>>>>>>>>>> Does that refer to the same visit? If so, you might want to harmonize the
>>>>>>>>>> visit labels so they are all the same across datasets for a given
>>>>>>>>>> timepoint, meaning you will need to relabel scans when the visit label is
>>>>>>>>>> different but referring to the same visit label (v01 = V1 etc...).
>>>>>>>>>>
>>>>>>>>>> Hope this helps,
>>>>>>>>>>
>>>>>>>>>> Cécile
>>>>>>>>>>
>>>>>>>>>> On Mon, Jul 29, 2019 at 10:15 AM Sotirios Nikoloutsopoulos <
>>>>>>>>>> sotirisnik at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Could you help me with the "Visit label does not exist"? I had
>>>>>>>>>>> changed $subjectID{'createVisitLabel'} to 1 as you had suggested but it
>>>>>>>>>>> appears that this problem hasn't been resolved yet.
>>>>>>>>>>>
>>>>>>>>>>> lorisadmin at hbp:/data/loris/bin/mri$ cat
>>>>>>>>>>> /data/loris/data//logs/TarLoad-14-58-9Xog7H.log
>>>>>>>>>>>
>>>>>>>>>>> ==> Loading file from disk
>>>>>>>>>>> /tmp/TarLoad-14-58-5Z4hYX/dcc0000_258024_v01_20121205_102108_501e1_mri.mnc
>>>>>>>>>>>
>>>>>>>>>>> --> mapping DICOM parameter for
>>>>>>>>>>> /tmp/TarLoad-14-58-5Z4hYX/dcc0000_258024_v01_20121205_102108_501e1_mri.mnc
>>>>>>>>>>>
>>>>>>>>>>> --> using user-defined filterParameters for
>>>>>>>>>>> /tmp/TarLoad-14-58-5Z4hYX/dcc0000_258024_v01_20121205_102108_501e1_mri.mnc
>>>>>>>>>>>
>>>>>>>>>>> Candidate Mismatch Error is Visit label does not exist
>>>>>>>>>>> -> WARNING: This candidate was invalid. Logging to
>>>>>>>>>>> MRICandidateErrors table with reason Visit label
>>>>>>>>>>> does not exist
>>>>>>>>>>>
>>>>>>>>>>> Thank you,
>>>>>>>>>>>
>>>>>>>>>>> Sotirios
>>>>>>>>>>>
>>>>>>>>>>> Στις Παρ, 26 Ιουλ 2019 στις 3:02 μ.μ., ο/η Sotirios
>>>>>>>>>>> Nikoloutsopoulos <sotirisnik at gmail.com> έγραψε:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Cecile,
>>>>>>>>>>>>
>>>>>>>>>>>> The Center error was fixed. Still the insertion of mincs fails.
>>>>>>>>>>>> Please see the output.txt file i attached. Also is there something else i
>>>>>>>>>>>> need to change at the paths configuration?
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you,
>>>>>>>>>>>>
>>>>>>>>>>>> Sotirios
>>>>>>>>>>>>
>>>>>>>>>>>> Στις Πέμ, 25 Ιουλ 2019 στις 6:55 μ.μ., ο/η Cecile Madjar <
>>>>>>>>>>>> cecile.madjar at mcin.ca> έγραψε:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Sotirios,
>>>>>>>>>>>>>
>>>>>>>>>>>>> We are making progress!
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have a feeling this error comes from the fact that your psc
>>>>>>>>>>>>> table does not have the MRI_alias populated for the DCC site. Running the
>>>>>>>>>>>>> following query in the mysql database should fix this:
>>>>>>>>>>>>> UPDATE psc SET MRI_alias="DCC", Alias="DCC" WHERE Name="Data
>>>>>>>>>>>>> Coordinating Center";
>>>>>>>>>>>>> Repeat the same with all your sites (make sure to change the
>>>>>>>>>>>>> Aliases between sites, they should all be unique).
>>>>>>>>>>>>>
>>>>>>>>>>>>> Let me know how it goes!
>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cécile
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Jul 25, 2019 at 11:43 AM Sotirios Nikoloutsopoulos <
>>>>>>>>>>>>> sotirisnik at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Dear Cecile,
>>>>>>>>>>>>>> You were right about your guesses. I did the modifications
>>>>>>>>>>>>>> you suggested, i still get the "no mincs inserted" error, now i am getting
>>>>>>>>>>>>>> the "No center found this candidate" error ( how do i fix that? from what i
>>>>>>>>>>>>>> see the .dcm files contains a 'Institution Name', is that the center? ), so
>>>>>>>>>>>>>> i guess that's why the validation fails, but i changed the force variable
>>>>>>>>>>>>>> of tarchiveLoader to 1, shouldn't it have inserted the mincs then? Also it
>>>>>>>>>>>>>> is optional to have a mail server, right? ( from what i see the mail error
>>>>>>>>>>>>>> does not affect the $valid_study, so i assume we are okay with that )
>>>>>>>>>>>>>> Thank you a lot,
>>>>>>>>>>>>>> Sotirios
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Στις Πέμ, 25 Ιουλ 2019 στις 5:48 μ.μ., ο/η Cecile Madjar <
>>>>>>>>>>>>>> cecile.madjar at mcin.ca> έγραψε:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Sotirios,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> From what I understand, I believe you just need to update a
>>>>>>>>>>>>>>> few config settings from the frontend in the Config module (under the Admin
>>>>>>>>>>>>>>> menu).
>>>>>>>>>>>>>>> [image: Screen Shot 2019-07-25 at 10.31.35 AM.png]
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Then go to the Paths section of the config module and update
>>>>>>>>>>>>>>> the "LORIS-MRI code" config setting to /data/loris/bin/mri (my guess is
>>>>>>>>>>>>>>> that currently, it is set to /data/LORIS/data) and click on the submit
>>>>>>>>>>>>>>> button.
>>>>>>>>>>>>>>> [image: Screen Shot 2019-07-25 at 10.32.50 AM.png]
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Then, go to Imaging Pipeline section of the Configuration
>>>>>>>>>>>>>>> module and update the "Loris-MRI Data Directory" to /data/loris/data/ (my
>>>>>>>>>>>>>>> guess is that it is empty at the moment in your database which is why
>>>>>>>>>>>>>>> $data_dir is not defined when you run the tarchiveLoader step of the
>>>>>>>>>>>>>>> insertion pipeline).
>>>>>>>>>>>>>>> [image: Screen Shot 2019-07-25 at 10.40.52 AM.png]
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Finally, I took a look at your environment file and it might
>>>>>>>>>>>>>>> be a good idea to update your PERL5LIB variable to the following (it has
>>>>>>>>>>>>>>> been added to the code a few releases ago as we realized this was needed
>>>>>>>>>>>>>>> for the libraries. Not sure which version of LORIS-MRI you are using but it
>>>>>>>>>>>>>>> does not hurt to update this variable to what I suggest below):
>>>>>>>>>>>>>>> export
>>>>>>>>>>>>>>> PERL5LIB=/data/loris/bin/mri/uploadNeuroDB:/data/loris/bin/mri/dicom-archive:$PERL5LIB
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hope this helps. Sorry you are experiencing so much trouble
>>>>>>>>>>>>>>> setting things up. We probably need to update our WIKI.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cécile
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Jul 25, 2019 at 9:48 AM Sotirios Nikoloutsopoulos <
>>>>>>>>>>>>>>> sotirisnik at gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The "Loris-MRI Data Directory" should be in the environment
>>>>>>>>>>>>>>>> file?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Στις Πέμ, 25 Ιουλ 2019 στις 12:59 π.μ., ο/η Sotirios
>>>>>>>>>>>>>>>> Nikoloutsopoulos <sotirisnik at gmail.com> έγραψε:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The scripts are located in /data/loris/bin/mri/.
>>>>>>>>>>>>>>>>> Somewhere I might have messed something up but i remember following the
>>>>>>>>>>>>>>>>> instructions in the wiki.For example when i tried to execute the
>>>>>>>>>>>>>>>>> batch_uploads_imageuploader perl script the line
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> my $command = "$bin_dir/uploadNeuroDB/
>>>>>>>>>>>>>>>>> imaging_upload_file.pl "
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> was pointing to an invalid path, for example: it was
>>>>>>>>>>>>>>>>> repeating a pattern and had LORIS instead of loris ( i think the mistake
>>>>>>>>>>>>>>>>> was that in the wiki it suggested to use $PROJECT for the name of the
>>>>>>>>>>>>>>>>> folder and i choosed 'loris'
>>>>>>>>>>>>>>>>> https://github.com/aces/Loris/wiki/Imaging-Database ).
>>>>>>>>>>>>>>>>> See also the in image for the modification i did. Also i didn't find the
>>>>>>>>>>>>>>>>> section to config the $data_dir variable.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thank you,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Sotirios
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Στις Τετ, 24 Ιουλ 2019 στις 6:05 μ.μ., ο/η Cecile Madjar <
>>>>>>>>>>>>>>>>> cecile.madjar at mcin.ca> έγραψε:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Sotirios,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> It looks like you have one Config setting not set (based
>>>>>>>>>>>>>>>>>> on the first error message you got when running it the first time). You
>>>>>>>>>>>>>>>>>> need to set a value for "Loris-MRI Data Directory" in the Config module
>>>>>>>>>>>>>>>>>> under the Imaging pipeline section (from what I can see, it would be
>>>>>>>>>>>>>>>>>> /data/loris/data/ for your setup).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The reason you got that error message the second time is
>>>>>>>>>>>>>>>>>> that the pipeline checks if the DICOM study was already inserted in the
>>>>>>>>>>>>>>>>>> tarchive table. Since you ran the study once already, the study is inserted
>>>>>>>>>>>>>>>>>> into the tarchive, hence the message. However, the pipeline did not
>>>>>>>>>>>>>>>>>> complete as the MINC files were not created and inserted into the
>>>>>>>>>>>>>>>>>> files/parameter_file table. In order to continue the insertion, you will
>>>>>>>>>>>>>>>>>> then need to run the tarchiveLoader command suggested in the terminal
>>>>>>>>>>>>>>>>>> during the second run:
>>>>>>>>>>>>>>>>>> /data/LORIS/data//uploadNeuroDB/tarchiveLoader
>>>>>>>>>>>>>>>>>> -globLocation -profile prod
>>>>>>>>>>>>>>>>>> /data/loris/data/tarchive//2012/DCM_2012-12-05_ImagingUpload-17-43-ZUEFjb.tar
>>>>>>>>>>>>>>>>>> -verbose
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Finally, I am a bit confused by your setup. Where are the
>>>>>>>>>>>>>>>>>> scripts located? In /data/loris/bin/mri or /data/loris/data? It feels like
>>>>>>>>>>>>>>>>>> the Config setting "LORIS-MRI code" in the Paths section was set to
>>>>>>>>>>>>>>>>>> "/data/LORIS/data" but it should have been set to the directory where the
>>>>>>>>>>>>>>>>>> scripts are located (most probably /data/loris/bin/mri).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hope this helps,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Cécile
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Jul 24, 2019 at 10:55 AM Sotirios
>>>>>>>>>>>>>>>>>> Nikoloutsopoulos <sotirisnik at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Ok first i execute this command:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> "lorisadmin at hbp:/data/loris/bin/mri$
>>>>>>>>>>>>>>>>>>> ./batch_uploads_imageuploader -profile prod < ~/Desktop/input.txt > log.txt
>>>>>>>>>>>>>>>>>>> Use of uninitialized value $data_dir in concatenation
>>>>>>>>>>>>>>>>>>> (.) or string at ./batch_uploads_imageuploader line 143.
>>>>>>>>>>>>>>>>>>> Use of uninitialized value $data_dir in concatenation
>>>>>>>>>>>>>>>>>>> (.) or string at ./batch_uploads_imageuploader line 143.
>>>>>>>>>>>>>>>>>>> Use of uninitialized value $_ in pattern match (m//) at
>>>>>>>>>>>>>>>>>>> ./batch_uploads_imageuploader line 147.
>>>>>>>>>>>>>>>>>>> base is DCC0000_258024_V01
>>>>>>>>>>>>>>>>>>> path is /data/incoming/
>>>>>>>>>>>>>>>>>>> type is .tar.gz
>>>>>>>>>>>>>>>>>>> fullpath is /data/incoming/DCC0000_258024_V01.tar.gz
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> source /tmp/ImagingUpload-17-43-ZUEFjb
>>>>>>>>>>>>>>>>>>> targetlocation /data/loris/data/tarchive
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Use of uninitialized value $data_dir in concatenation
>>>>>>>>>>>>>>>>>>> (.) or string at /data/LORIS/data//uploadNeuroDB/tarchiveLoader line 256.
>>>>>>>>>>>>>>>>>>> Use of uninitialized value $data_dir in concatenation
>>>>>>>>>>>>>>>>>>> (.) or string at /data/LORIS/data//uploadNeuroDB/tarchiveLoader line 284.
>>>>>>>>>>>>>>>>>>> Use of uninitialized value $data_dir in concatenation
>>>>>>>>>>>>>>>>>>> (.) or string at /data/loris/bin/mri/uploadNeuroDB/
>>>>>>>>>>>>>>>>>>> tarchive_validation.pl line 219.
>>>>>>>>>>>>>>>>>>> Use of uninitialized value $data_dir in concatenation
>>>>>>>>>>>>>>>>>>> (.) or string at /data/loris/bin/mri/uploadNeuroDB/
>>>>>>>>>>>>>>>>>>> minc_insertion.pl line 270.
>>>>>>>>>>>>>>>>>>> Use of uninitialized value $data_dir in concatenation
>>>>>>>>>>>>>>>>>>> (.) or string at /data/LORIS/data//uploadNeuroDB/tarchiveLoader line 711.
>>>>>>>>>>>>>>>>>>> Can't exec "mail": No such file or directory at
>>>>>>>>>>>>>>>>>>> /data/LORIS/data//uploadNeuroDB/tarchiveLoader line 721.
>>>>>>>>>>>>>>>>>>> print() on closed filehandle MAIL at
>>>>>>>>>>>>>>>>>>> /data/LORIS/data//uploadNeuroDB/tarchiveLoader line 722.
>>>>>>>>>>>>>>>>>>> print() on closed filehandle MAIL at
>>>>>>>>>>>>>>>>>>> /data/LORIS/data//uploadNeuroDB/tarchiveLoader line 723.
>>>>>>>>>>>>>>>>>>> Use of uninitialized value $data_dir in concatenation
>>>>>>>>>>>>>>>>>>> (.) or string at /data/LORIS/data//uploadNeuroDB/tarchiveLoader line 724.
>>>>>>>>>>>>>>>>>>> print() on closed filehandle MAIL at
>>>>>>>>>>>>>>>>>>> /data/LORIS/data//uploadNeuroDB/tarchiveLoader line 724.
>>>>>>>>>>>>>>>>>>> print() on closed filehandle MAIL at
>>>>>>>>>>>>>>>>>>> /data/LORIS/data//uploadNeuroDB/tarchiveLoader line 725.
>>>>>>>>>>>>>>>>>>> Use of uninitialized value $data_dir in concatenation
>>>>>>>>>>>>>>>>>>> (.) or string at /data/LORIS/data//uploadNeuroDB/tarchiveLoader line 735.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> No Mincs inserted
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The tarchiveLoader insertion script has failed.
>>>>>>>>>>>>>>>>>>> Running now the following command:
>>>>>>>>>>>>>>>>>>> /data/loris/data/uploadNeuroDB/imaging_upload_file.pl
>>>>>>>>>>>>>>>>>>> -profile prod -upload_id 3 /data/incoming/DCC0000_258024_V01.tar.gz
>>>>>>>>>>>>>>>>>>> -verbose"
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I inserted the VisitLabel, but still the Mincs are not
>>>>>>>>>>>>>>>>>>> inserted and i am getting this error
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> "lorisadmin at hbp:/data/loris/bin/mri$
>>>>>>>>>>>>>>>>>>> /data/loris/data/uploadNeuroDB/imaging_upload_file.pl
>>>>>>>>>>>>>>>>>>> -profile prod -upload_id 3 /data/incoming/DCC0000_258024_V01.tar.gz -verbose
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> find -path \/tmp\/ImagingUpload\-17\-45\-5MsWEd -name
>>>>>>>>>>>>>>>>>>> '__MACOSX' -delete
>>>>>>>>>>>>>>>>>>> Spool message is:
>>>>>>>>>>>>>>>>>>> The Scan for the uploadID 3 has already been run with
>>>>>>>>>>>>>>>>>>> tarchiveID: 7.
>>>>>>>>>>>>>>>>>>> To continue with the rest of the insertion pipeline,
>>>>>>>>>>>>>>>>>>> please run tarchiveLoader from a terminal as follows:
>>>>>>>>>>>>>>>>>>> /data/LORIS/data//uploadNeuroDB/tarchiveLoader -globLocation -profile prod
>>>>>>>>>>>>>>>>>>> /data/loris/data/tarchive//2012/DCM_2012-12-05_ImagingUpload-17-43-ZUEFjb.tar
>>>>>>>>>>>>>>>>>>> -verbose"
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Which files should i delete? I thought that deleting the
>>>>>>>>>>>>>>>>>>> files from the tarchive was enough. And i executed the last proposed
>>>>>>>>>>>>>>>>>>> command to continue to rest of the insertion pipeline ( check the image ).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thank you
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Στις Τετ, 24 Ιουλ 2019 στις 5:05 μ.μ., ο/η Cecile Madjar
>>>>>>>>>>>>>>>>>>> <cecile.madjar at mcin.ca> έγραψε:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi Sotirios,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Sorry I was not clear before. Every visits of
>>>>>>>>>>>>>>>>>>>> candidates are created in the session table with one entry per
>>>>>>>>>>>>>>>>>>>> CandID/VisitLabel. So if the imaging pipeline created visits in the session
>>>>>>>>>>>>>>>>>>>> table (that are not attached to instruments) and you want to clean your
>>>>>>>>>>>>>>>>>>>> database completely, you can delete entries in that table as well.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Regarding your point 2, the imaging pipeline can create
>>>>>>>>>>>>>>>>>>>> the visit in the session table as long as the visit label is present in the
>>>>>>>>>>>>>>>>>>>> Visit_Windows table (which means the visit label stored in the PatientName
>>>>>>>>>>>>>>>>>>>> is a valid visit label).
>>>>>>>>>>>>>>>>>>>> However, in your prod file, if the
>>>>>>>>>>>>>>>>>>>> variable $subjectID{'createVisitLabel'} is set to 1 for candidates, the
>>>>>>>>>>>>>>>>>>>> visit label should be created in the Visit_Windows table I believe. Here
>>>>>>>>>>>>>>>>>>>> is the line of the prod file you would need to change
>>>>>>>>>>>>>>>>>>>> <https://github.com/aces/Loris-MRI/blob/81bae73ea6e86c9498519dadf574468ee1d992ca/dicom-archive/profileTemplate.pl#L59>
>>>>>>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>>>>>> In general, we prefer populating the Visit_Windows
>>>>>>>>>>>>>>>>>>>> table and set the $subjectID{'createVisitLabel'} to 0 for candidate data to
>>>>>>>>>>>>>>>>>>>> avoid insertion of badly labelled MRI data that are complex to relabel.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Finally, regarding the candidate creation, if you
>>>>>>>>>>>>>>>>>>>> execute the following steps, the pipeline should be able to create
>>>>>>>>>>>>>>>>>>>> candidates:
>>>>>>>>>>>>>>>>>>>> - set in the Imaging Pipeline section of the config
>>>>>>>>>>>>>>>>>>>> module the config setting "Upload creation of candidates" to Yes
>>>>>>>>>>>>>>>>>>>> - manually transfer the scans to the LORIS server
>>>>>>>>>>>>>>>>>>>> instead of uploading it via the imaging browser
>>>>>>>>>>>>>>>>>>>> - run batch_imaging_upload.pl as you did until now.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hope this helps!
>>>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Cécile
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Wed, Jul 24, 2019 at 8:47 AM Sotirios
>>>>>>>>>>>>>>>>>>>> Nikoloutsopoulos <sotirisnik at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> About my second question i replaced the name of the
>>>>>>>>>>>>>>>>>>>>> patient in the .dcm files and now i am getting an error that the visit
>>>>>>>>>>>>>>>>>>>>> label does not exist.
>>>>>>>>>>>>>>>>>>>>> I thought it was supposed to be automatically created.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> "Done adding archive info into database
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> /data/LORIS/data//uploadNeuroDB/tarchiveLoader
>>>>>>>>>>>>>>>>>>>>> -globLocation -profile prod
>>>>>>>>>>>>>>>>>>>>> /data/loris/data/tarchive//DCM_2012-12-05_ImagingUpload-15-42-yUwQQV.tar
>>>>>>>>>>>>>>>>>>>>> -verbose
>>>>>>>>>>>>>>>>>>>>> md5sum
>>>>>>>>>>>>>>>>>>>>> /data/loris/data/tarchive/DCM_2012-12-05_ImagingUpload-15-42-yUwQQV.tar
>>>>>>>>>>>>>>>>>>>>> PSCID is: DCC0000
>>>>>>>>>>>>>>>>>>>>> CandID id: 258024
>>>>>>>>>>>>>>>>>>>>> visit_label is: V1
>>>>>>>>>>>>>>>>>>>>> PSCID is: DCC0000
>>>>>>>>>>>>>>>>>>>>> CandID id: 258024
>>>>>>>>>>>>>>>>>>>>> visit_label is: V1
>>>>>>>>>>>>>>>>>>>>> candidate id 258024
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> => No Visit labelVisit label does not exist
>>>>>>>>>>>>>>>>>>>>> Set centerID = 1
>>>>>>>>>>>>>>>>>>>>> PSCID is: DCC0000
>>>>>>>>>>>>>>>>>>>>> CandID id: 258024
>>>>>>>>>>>>>>>>>>>>> visit_label is: V1
>>>>>>>>>>>>>>>>>>>>> PSCID is: DCC0000
>>>>>>>>>>>>>>>>>>>>> CandID id: 258024
>>>>>>>>>>>>>>>>>>>>> visit_label is: V1
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Number of MINC files that will be considered for
>>>>>>>>>>>>>>>>>>>>> inserting into the database: 1
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> log dir is /logs and log file is
>>>>>>>>>>>>>>>>>>>>> /logs/TarLoad-15-42-L6pocU.log
>>>>>>>>>>>>>>>>>>>>> PSCID is: DCC0000
>>>>>>>>>>>>>>>>>>>>> CandID id: 258024
>>>>>>>>>>>>>>>>>>>>> visit_label is: V1
>>>>>>>>>>>>>>>>>>>>> PSCID is: DCC0000
>>>>>>>>>>>>>>>>>>>>> CandID id: 258024
>>>>>>>>>>>>>>>>>>>>> visit_label is: V1
>>>>>>>>>>>>>>>>>>>>> candidate id 258024
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> => No Visit label
>>>>>>>>>>>>>>>>>>>>> Cleaning up temp files: rm -rf
>>>>>>>>>>>>>>>>>>>>> /tmp/TarLoad-15-42-J2rQms/ImagingUpload-15-42-yUwQQV*"
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Στις Τετ, 24 Ιουλ 2019 στις 3:38 μ.μ., ο/η Sotirios
>>>>>>>>>>>>>>>>>>>>> Nikoloutsopoulos <sotirisnik at gmail.com> έγραψε:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> 1) What are the records in the session table?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> 2) I tried to upload a non-phantom file like this
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> input_file.txt:
>>>>>>>>>>>>>>>>>>>>>> /data/incoming/DCC0000_258024_V1.tar.gz N
>>>>>>>>>>>>>>>>>>>>>> DCC0000_258024_V1
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Terminal:
>>>>>>>>>>>>>>>>>>>>>> ./batch_uploads_imageuploader -profile prod <
>>>>>>>>>>>>>>>>>>>>>> ~/Desktop/input.txt > log.txt
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> and i get the following error in the log.txt file
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> "Running now the following command:
>>>>>>>>>>>>>>>>>>>>>> /data/loris/data/uploadNeuroDB/imaging_upload_file.pl
>>>>>>>>>>>>>>>>>>>>>> -profile prod -upload_id 7 /data/incoming/DCC0000_258024_V1.tar.gz -verbose
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> find -path \/tmp\/ImagingUpload\-14\-56\-FeLCB4
>>>>>>>>>>>>>>>>>>>>>> -name '__MACOSX' -delete
>>>>>>>>>>>>>>>>>>>>>> Spool message is:
>>>>>>>>>>>>>>>>>>>>>> The PatientName read from the DICOM header does not
>>>>>>>>>>>>>>>>>>>>>> start with DCC0000_258024_V1 from the mri_upload table
>>>>>>>>>>>>>>>>>>>>>> "
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> and the spool message is repeated for each .dcm file.
>>>>>>>>>>>>>>>>>>>>>> Should i preprocess every .dcm file and change the PatientName to
>>>>>>>>>>>>>>>>>>>>>> "DCC0000_258024_V1"?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> 3) As far as it concerns the candidate profile i have
>>>>>>>>>>>>>>>>>>>>>> to create a new one each time before i try to upload a file? Is there
>>>>>>>>>>>>>>>>>>>>>> already a way to create a candidate profile and get the DCCID and PSCID
>>>>>>>>>>>>>>>>>>>>>> without the user interface?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thank you,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Sotirios
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Στις Πέμ, 18 Ιουλ 2019 στις 5:00 μ.μ., ο/η Cecile
>>>>>>>>>>>>>>>>>>>>>> Madjar <cecile.madjar at mcin.ca> έγραψε:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi Sotirios,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> See answers to your questions below.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> About the 1st case we had a dicom with a blank
>>>>>>>>>>>>>>>>>>>>>>>> studyUID, maybe someone accidentally removed it.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> That is strange. Something must have been done to
>>>>>>>>>>>>>>>>>>>>>>> the DICOM files. I have never seen a study with no StudyUID out of the
>>>>>>>>>>>>>>>>>>>>>>> scanner. Usually, you need to manually erase it with some tool. However,
>>>>>>>>>>>>>>>>>>>>>>> DICOMs never cease to surprise me...
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Some other questions:
>>>>>>>>>>>>>>>>>>>>>>>> 1) What are the steps to remove all of my patients
>>>>>>>>>>>>>>>>>>>>>>>> and uploads?: because i want to try to reupload all the testing dicoms we
>>>>>>>>>>>>>>>>>>>>>>>> had with a clean database.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Release 21.0 of LORIS-MRI will offer a script to
>>>>>>>>>>>>>>>>>>>>>>> remove all the entries and files specific to an upload. However, this
>>>>>>>>>>>>>>>>>>>>>>> release is not yet out. Hopefully in the next few weeks.
>>>>>>>>>>>>>>>>>>>>>>> In the meantime, since your case is particular and
>>>>>>>>>>>>>>>>>>>>>>> that you just want to start with a clean database, the following deletes
>>>>>>>>>>>>>>>>>>>>>>> should work (hopefully I won't forget any tables):
>>>>>>>>>>>>>>>>>>>>>>> delete from mri_violations_log;
>>>>>>>>>>>>>>>>>>>>>>> delete from mri_protocol_violated_scans;
>>>>>>>>>>>>>>>>>>>>>>> delete from MRICandidateErrors;
>>>>>>>>>>>>>>>>>>>>>>> delete from parameter_file;
>>>>>>>>>>>>>>>>>>>>>>> delete from files_qcstatus; # (if you played with
>>>>>>>>>>>>>>>>>>>>>>> the QC part of the imaging browser for testing)
>>>>>>>>>>>>>>>>>>>>>>> delete from feedback_mri_comments; # (if you played
>>>>>>>>>>>>>>>>>>>>>>> with the QC part of the imaging browser for testing)
>>>>>>>>>>>>>>>>>>>>>>> delete from tarchive_series;
>>>>>>>>>>>>>>>>>>>>>>> delete from tarchive_files;
>>>>>>>>>>>>>>>>>>>>>>> delete from mri_upload;
>>>>>>>>>>>>>>>>>>>>>>> delete from tarchive;
>>>>>>>>>>>>>>>>>>>>>>> delete from mri_scanner;
>>>>>>>>>>>>>>>>>>>>>>> delete from candidate where PSCID="scanner";
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I don't know if you want to also delete entries in
>>>>>>>>>>>>>>>>>>>>>>> the session and candidate table too?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> 2) We do not need to create the profile of a patient
>>>>>>>>>>>>>>>>>>>>>>>> ( are the patients stored in the candidate table? because i can't see the
>>>>>>>>>>>>>>>>>>>>>>>> ghosts ), because will it automatically be created when importing a dicom?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I am not sure I understand fully this question. All
>>>>>>>>>>>>>>>>>>>>>>> candidates are stored in the candidate table. The phantom scans however are
>>>>>>>>>>>>>>>>>>>>>>> attached to a scanner candidate depending on where the scan happened. It is
>>>>>>>>>>>>>>>>>>>>>>> a bit of a weird concept that we have to redesign eventually but never got
>>>>>>>>>>>>>>>>>>>>>>> a chance to get to it.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hopefully this answered your question, otherwise,
>>>>>>>>>>>>>>>>>>>>>>> don't hesitate to let me know.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> 3) What is the difference between uploading a dicom
>>>>>>>>>>>>>>>>>>>>>>>> as phantom instead of importing as PSCCID_DCCID_VisitLabel?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> When you upload the scan as PSCID_DCCID_VisitLabel,
>>>>>>>>>>>>>>>>>>>>>>> there is a check that makes sure the candidate IDs and visit label are
>>>>>>>>>>>>>>>>>>>>>>> valid at the time of upload.
>>>>>>>>>>>>>>>>>>>>>>> When you upload a scan as phantom, it expects that
>>>>>>>>>>>>>>>>>>>>>>> the DICOM field PatientName and uploaded filename contains the string
>>>>>>>>>>>>>>>>>>>>>>> "phantom". We enforced this behaviour on the imaging uploader side in
>>>>>>>>>>>>>>>>>>>>>>> recent releases but I can't remember which one. Probably the upcoming 21.0
>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> All those verifications might seem cumbersome but
>>>>>>>>>>>>>>>>>>>>>>> they are here to ensure that the files inserted are all valid and labelled
>>>>>>>>>>>>>>>>>>>>>>> properly as it is a bit messy to have to delete files that were wrongly
>>>>>>>>>>>>>>>>>>>>>>> labelled. At least, the delete script present in release 21.0 will make
>>>>>>>>>>>>>>>>>>>>>>> this process easier but it is still good practice to verify those things.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thank you,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> With pleasure. Hopefully the answers to your
>>>>>>>>>>>>>>>>>>>>>>> questions will be helpful.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Cécile
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> PS: loris-dev mailing list: you will find the
>>>>>>>>>>>>>>>>>>>>>>> initial answer I gave Sotirios below. I forgot to cc the loris-dev in my
>>>>>>>>>>>>>>>>>>>>>>> earlier reply...
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Στις Τρί, 16 Ιουλ 2019 στις 12:02 π.μ., ο/η Cecile
>>>>>>>>>>>>>>>>>>>>>>>> Madjar <cecile.madjar at mcin.ca> έγραψε:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Dear Sotirios,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for reaching out. Since your email was
>>>>>>>>>>>>>>>>>>>>>>>>> already organized in points, I will reply directly below your questions
>>>>>>>>>>>>>>>>>>>>>>>>> below.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Cécile
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 1) What do we do when the StudyUID is blank on the
>>>>>>>>>>>>>>>>>>>>>>>>>> header?
>>>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/aces/Loris-MRI/blob/minor/docs/AppendixA-Troubleshooting_guideline.md
>>>>>>>>>>>>>>>>>>>>>>>>>> There solution provided at the table 3 does not guide us how to find the
>>>>>>>>>>>>>>>>>>>>>>>>>> information ( is possible ).
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> The StudyUID is the field used in our pipeline at
>>>>>>>>>>>>>>>>>>>>>>>>> the moment to check whether a DICOM study was already inserted into LORIS
>>>>>>>>>>>>>>>>>>>>>>>>> with that same StudyUID since it is supposed to be unique for every single
>>>>>>>>>>>>>>>>>>>>>>>>> study. Currently, we do not support insertion of DICOM studies if they do
>>>>>>>>>>>>>>>>>>>>>>>>> not have a StudyUID associated with them.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Is there a specific reason why the StudyUID is
>>>>>>>>>>>>>>>>>>>>>>>>> blank in the DICOM headers in your DICOM files? Did they go through some
>>>>>>>>>>>>>>>>>>>>>>>>> processes before upload to LORIS?
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 2) When i am about to upload a non-phanton case i
>>>>>>>>>>>>>>>>>>>>>>>>>> need a 'Visit Label'. Is there already a script that can create a new visit
>>>>>>>>>>>>>>>>>>>>>>>>>> label for a specific Candidate, so i can provide it as input afterwards to
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> batch_uploads_imageuploader ( See option 5.1.3 https://github.com/aces/Loris-MRI/blob/minor/docs/05-PipelineLaunchOptions.md )
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I am not sure if the front-end allows for it but
>>>>>>>>>>>>>>>>>>>>>>>>> if you use batch_uploads_imageuploader, the insertion scripts could create
>>>>>>>>>>>>>>>>>>>>>>>>> the visit label for you. However, it does need to be included in the
>>>>>>>>>>>>>>>>>>>>>>>>> patient name in order for the script to know which label it should use.
>>>>>>>>>>>>>>>>>>>>>>>>> For example: you upload V02 for MTL0123 (DCCID:
>>>>>>>>>>>>>>>>>>>>>>>>> 456789) but V02 was not yet created for MTL0123; the insertion scripts
>>>>>>>>>>>>>>>>>>>>>>>>> will create a V02 visit for MTL0123 if the PatientName field for the
>>>>>>>>>>>>>>>>>>>>>>>>> dataset is MTL0123_456789_V02 with Stage marked as "Not Started".
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> However, *note that for the script to create the
>>>>>>>>>>>>>>>>>>>>>>>>> visit for the candidate, you have to make sure that all the visit label of
>>>>>>>>>>>>>>>>>>>>>>>>> your projects were inserted in the Visit_Windows table*
>>>>>>>>>>>>>>>>>>>>>>>>> (otherwise, the visit will never be created as they were not specified as
>>>>>>>>>>>>>>>>>>>>>>>>> being part of the list of visit label to expect). So in the example above,
>>>>>>>>>>>>>>>>>>>>>>>>> you should have one row in Visit_Windows with Visit_label="V02" set.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> 3) Section 4.3
>>>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/aces/Loris-MRI/blob/minor/docs/04-Scripts.md
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> What do we do when the folder contains multiple mnc files?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> This is a case by case. Most of the time, the
>>>>>>>>>>>>>>>>>>>>>>>>> MINC files that failed insertion into the imaging browser end up in the MRI
>>>>>>>>>>>>>>>>>>>>>>>>> violation module where you can see what went wrong with the acquisitions
>>>>>>>>>>>>>>>>>>>>>>>>> not inserted. If you notice that the MINC file should be inserted as a
>>>>>>>>>>>>>>>>>>>>>>>>> specific protocol, you can force the insertion as explained in section 4.3.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> In your case, I have a feeling that you want to be
>>>>>>>>>>>>>>>>>>>>>>>>> able to insert MINC files that did not go through the whole pipeline
>>>>>>>>>>>>>>>>>>>>>>>>> insertion (DICOM archival, then dcm2mnc, then protocol identification, then
>>>>>>>>>>>>>>>>>>>>>>>>> insertion...). In theory, you could call
>>>>>>>>>>>>>>>>>>>>>>>>> minc_insertion.pl on any MINC file, provided you
>>>>>>>>>>>>>>>>>>>>>>>>> have at least the following information:
>>>>>>>>>>>>>>>>>>>>>>>>> - path to the MINC file
>>>>>>>>>>>>>>>>>>>>>>>>> - uploadID associated with the MINC files or DICOM
>>>>>>>>>>>>>>>>>>>>>>>>> archive path from which those MINC files were created from
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Then the minc_insertion.pl script will do the
>>>>>>>>>>>>>>>>>>>>>>>>> subject information and protocol validation etc...
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> If all the MINC files come from the same uploadID
>>>>>>>>>>>>>>>>>>>>>>>>> or TarchiveID, then you could run a loop in bash calling the
>>>>>>>>>>>>>>>>>>>>>>>>> minc_insertion.pl script like this for files
>>>>>>>>>>>>>>>>>>>>>>>>> deriving from uploadID=1:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> ls /path/to/mnc/folder/* | while read f; do
>>>>>>>>>>>>>>>>>>>>>>>>> minc_insertion.pl -profile prod -uploadID 1
>>>>>>>>>>>>>>>>>>>>>>>>> -mincPath $f; done
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Hope this helped!
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>> Loris-dev mailing list
>>>>>>> Loris-dev at bic.mni.mcgill.ca
>>>>>>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> christine.rogers at mcgill.ca
>>>>>> McGill Centre for Integrative Neuroscience | MCIN.ca
>>>>>> Montreal Neurological Institute
>>>>>> McGill University | Montreal | Canada
>>>>>>
>>>>>
>>>>
>>>> --
>>>>
>>>> christine.rogers at mcgill.ca
>>>> McGill Centre for Integrative Neuroscience | MCIN.ca
>>>> Montreal Neurological Institute
>>>> McGill University | Montreal | Canada
>>>>
>>> _______________________________________________
>> Loris-dev mailing list
>> Loris-dev at bic.mni.mcgill.ca
>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>
>
>
> --
>
> christine.rogers at mcgill.ca
> McGill Centre for Integrative Neuroscience | MCIN.ca
> Montreal Neurological Institute
> McGill University | Montreal | Canada
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20190806/649c5dca/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 78176 bytes
Desc: not available
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20190806/649c5dca/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 20518 bytes
Desc: not available
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20190806/649c5dca/attachment-0003.png>
More information about the Loris-dev
mailing list