[Loris-dev] Import mri - scripts

Wed Jul 24 08:47:02 EDT 2019

About my second question i replaced the name of the patient in the .dcm
files and now i am getting an error that the visit label does not exist.
I thought it was supposed to be automatically created.

"Done adding archive info into database

/data/LORIS/data//uploadNeuroDB/tarchiveLoader -globLocation -profile prod
/data/loris/data/tarchive//DCM_2012-12-05_ImagingUpload-15-42-yUwQQV.tar
-verbose
 md5sum
/data/loris/data/tarchive/DCM_2012-12-05_ImagingUpload-15-42-yUwQQV.tar
PSCID is: DCC0000
 CandID id: 258024
 visit_label is: V1
PSCID is: DCC0000
 CandID id: 258024
 visit_label is: V1
candidate id 258024

=> No Visit labelVisit label does not exist
Set centerID = 1
PSCID is: DCC0000
 CandID id: 258024
 visit_label is: V1
PSCID is: DCC0000
 CandID id: 258024
 visit_label is: V1

Number of MINC files that will be considered for inserting into the
database: 1

log dir is /logs and log file is /logs/TarLoad-15-42-L6pocU.log
PSCID is: DCC0000
 CandID id: 258024
 visit_label is: V1
PSCID is: DCC0000
 CandID id: 258024
 visit_label is: V1
candidate id 258024

=> No Visit label
Cleaning up temp files: rm -rf
/tmp/TarLoad-15-42-J2rQms/ImagingUpload-15-42-yUwQQV*"

Στις Τετ, 24 Ιουλ 2019 στις 3:38 μ.μ., ο/η Sotirios Nikoloutsopoulos <
sotirisnik at gmail.com> έγραψε:

> Hi,
>
> 1) What are the records in the session table?
>
> 2) I tried to upload a non-phantom file like this
>
> input_file.txt:
> /data/incoming/DCC0000_258024_V1.tar.gz N DCC0000_258024_V1
>
> Terminal:
>  ./batch_uploads_imageuploader -profile prod < ~/Desktop/input.txt >
> log.txt
>
> and i get the following error in the log.txt file
>
> "Running now the following command: /data/loris/data/uploadNeuroDB/
> imaging_upload_file.pl -profile prod -upload_id 7
> /data/incoming/DCC0000_258024_V1.tar.gz -verbose
>
>  find -path \/tmp\/ImagingUpload\-14\-56\-FeLCB4 -name '__MACOSX' -delete
> Spool message is:
> The PatientName read from the DICOM header does not start with
> DCC0000_258024_V1 from the mri_upload table
>  "
>
> and the spool message is repeated for each .dcm file. Should i preprocess
> every .dcm file and change the PatientName to "DCC0000_258024_V1"?
>
> 3) As far as it concerns the candidate profile i have to create a new one
> each time before i try to upload a file? Is there already a way to create a
> candidate profile and get the DCCID and PSCID without the user interface?
>
> Thank you,
>
> Sotirios
>
> Στις Πέμ, 18 Ιουλ 2019 στις 5:00 μ.μ., ο/η Cecile Madjar <
> cecile.madjar at mcin.ca> έγραψε:
>
>> Hi Sotirios,
>>
>> See answers to your questions below.
>>
>> About the 1st case we had a dicom with a blank studyUID, maybe someone
>>> accidentally removed it.
>>>
>> That is strange. Something must have been done to the DICOM files. I have
>> never seen a study with no StudyUID out of the scanner. Usually, you need
>> to manually erase it with some tool. However, DICOMs never cease to
>> surprise me...
>>
>>
>>> Some other questions:
>>> 1) What are the steps to remove all of my patients and uploads?: because
>>> i want to try to reupload all the testing dicoms we had with a clean
>>> database.
>>>
>>
>> Release 21.0 of LORIS-MRI will offer a script to remove all the entries
>> and files specific to an upload. However, this release is not yet out.
>> Hopefully in the next few weeks.
>> In the meantime, since your case is particular and that you just want to
>> start with a clean database, the following deletes should work (hopefully I
>> won't forget any tables):
>> delete from mri_violations_log;
>> delete from mri_protocol_violated_scans;
>> delete from MRICandidateErrors;
>> delete from parameter_file;
>> delete from files_qcstatus; # (if you played with the QC part of the
>> imaging browser for testing)
>> delete from feedback_mri_comments; # (if you played with the QC part of
>> the imaging browser for testing)
>> delete from tarchive_series;
>> delete from tarchive_files;
>> delete from mri_upload;
>> delete from tarchive;
>> delete from mri_scanner;
>> delete from candidate where PSCID="scanner";
>>
>> I don't know if you want to also delete entries in the session and
>> candidate table too?
>>
>> 2) We do not need to create the profile of a patient ( are the patients
>>> stored in the candidate table? because i can't see the ghosts ), because
>>> will it automatically be created when importing a dicom?
>>>
>>
>> I am not sure I understand fully this question. All candidates are stored
>> in the candidate table. The phantom scans however are attached to a scanner
>> candidate depending on where the scan happened. It is a bit of a weird
>> concept that we have to redesign eventually but never got a chance to get
>> to it.
>>
>> Hopefully this answered your question, otherwise, don't hesitate to let
>> me know.
>>
>> 3) What is the difference between uploading a dicom as phantom instead of
>>> importing as PSCCID_DCCID_VisitLabel?
>>>
>>
>> When you upload the scan as PSCID_DCCID_VisitLabel, there is a check that
>> makes sure the candidate IDs and visit label are valid at the time of
>> upload.
>> When you upload a scan as phantom, it expects that the DICOM field
>> PatientName and uploaded filename contains the string "phantom". We
>> enforced this behaviour on the imaging uploader side in recent releases but
>> I can't remember which one. Probably the upcoming 21.0 release.
>>
>> All those verifications might seem cumbersome but they are here to ensure
>> that the files inserted are all valid and labelled properly as it is a bit
>> messy to have to delete files that were wrongly labelled. At least, the
>> delete script present in release 21.0 will make this process easier but it
>> is still good practice to verify those things.
>>
>>
>>> Thank you,
>>>
>>
>> With pleasure. Hopefully the answers to your questions will be helpful.
>>
>> Best,
>>
>> Cécile
>>
>> PS: loris-dev mailing list: you will find the initial answer I gave
>> Sotirios below. I forgot to cc the loris-dev in my earlier reply...
>>
>> Στις Τρί, 16 Ιουλ 2019 στις 12:02 π.μ., ο/η Cecile Madjar <
>>> cecile.madjar at mcin.ca> έγραψε:
>>>
>>>> Dear Sotirios,
>>>>
>>>> Thank you for reaching out. Since your email was already organized in
>>>> points, I will reply directly below your questions below.
>>>>
>>>> Best,
>>>>
>>>> Cécile
>>>>
>>>> 1) What do we do when the StudyUID is blank on the header?
>>>>> https://github.com/aces/Loris-MRI/blob/minor/docs/AppendixA-Troubleshooting_guideline.md
>>>>> There solution provided at the table 3 does not guide us how to find the
>>>>> information ( is possible ).
>>>>>
>>>>
>>>> The StudyUID is the field used in our pipeline at the moment to check
>>>> whether a DICOM study was already inserted into LORIS with that same
>>>> StudyUID since it is supposed to be unique for every single study.
>>>> Currently, we do not support insertion of DICOM studies if they do not have
>>>> a StudyUID associated with them.
>>>>
>>>> Is there a specific reason why the StudyUID is blank in the DICOM
>>>> headers in your DICOM files? Did they go through some processes before
>>>> upload to LORIS?
>>>>
>>>>
>>>> 2) When i am about to upload a non-phanton case i need a 'Visit Label'.
>>>>> Is there already a script that can create a new visit label for a specific
>>>>> Candidate, so i can provide it as input afterwards to the
>>>>>
>>>>> batch_uploads_imageuploader ( See option 5.1.3 https://github.com/aces/Loris-MRI/blob/minor/docs/05-PipelineLaunchOptions.md )
>>>>>
>>>>>
>>>> I am not sure if the front-end allows for it but if you use
>>>> batch_uploads_imageuploader, the insertion scripts could create the visit
>>>> label for you. However, it does need to be included in the patient name in
>>>> order for the script to know which label it should use.
>>>> For example: you upload V02 for MTL0123 (DCCID: 456789)  but V02 was
>>>> not yet created for MTL0123; the insertion scripts will create a V02 visit
>>>> for MTL0123 if the PatientName field for the dataset is MTL0123_456789_V02
>>>> with Stage marked as "Not Started".
>>>>
>>>> However, *note that for the script to create the visit for the
>>>> candidate, you have to make sure that all the visit label of your projects
>>>> were inserted in the Visit_Windows table* (otherwise, the visit will
>>>> never be created as they were not specified as being part of the list of
>>>> visit label to expect). So in the example above, you should have one row in
>>>> Visit_Windows with Visit_label="V02" set.
>>>>
>>>>
>>>>> 3) Section 4.3
>>>>> https://github.com/aces/Loris-MRI/blob/minor/docs/04-Scripts.md
>>>>>
>>>>> What do we do when the folder contains multiple mnc files?
>>>>>
>>>>> This is a case by case. Most of the time, the MINC files that failed
>>>> insertion into the imaging browser end up in the MRI violation module where
>>>> you can see what went wrong with the acquisitions not inserted. If you
>>>> notice that the MINC file should be inserted as a specific protocol, you
>>>> can force the insertion as explained in section 4.3.
>>>>
>>>> In your case, I have a feeling that you want to be able to insert MINC
>>>> files that did not go through the whole pipeline insertion (DICOM archival,
>>>> then dcm2mnc, then protocol identification, then insertion...). In theory,
>>>> you could call minc_insertion.pl on any MINC file, provided you have
>>>> at least the following information:
>>>> - path to the MINC  file
>>>> - uploadID associated with the MINC files or DICOM archive path from
>>>> which those MINC files were created from
>>>>
>>>> Then the minc_insertion.pl script will do the subject information and
>>>> protocol validation etc...
>>>>
>>>> If all the MINC files come from the same uploadID or TarchiveID, then
>>>> you could run a loop in bash calling the minc_insertion.pl script like
>>>> this for files deriving from uploadID=1:
>>>>
>>>> ls /path/to/mnc/folder/* | while read f; do minc_insertion.pl -profile
>>>> prod -uploadID 1 -mincPath $f; done
>>>>
>>>>
>>>>> Thank you,
>>>>>
>>>>
>>>> Hope this helped!
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20190724/e8fcd223/attachment-0001.html>