[Loris-dev] Import mri - scripts
Sotirios Nikoloutsopoulos
sotirisnik at gmail.com
Fri Nov 8 17:38:16 EST 2019
Hi Cecile,
We were wondering whether Loris has a tool to check if there are problems
with the dcm files, before trying to upload them. Something that could
provide the same results provided in the warning_output, this is the
summary from the mri_upload at the front-page. Also which attributes of the
dcm header would trigger a warning? is there a dcm file header
specification for Loris?
Thanks
Sotirios
Στις Τετ, 2 Οκτ 2019 στις 12:07 μ.μ., ο/η Sotirios Nikoloutsopoulos <
sotirisnik at gmail.com> έγραψε:
> Hi,
>
> Yes, by parsing now() into unix_timestamp function it worked,but i thought
> that something else was causing the 2nd issue.
> Thanks
>
> On Tue, 1 Oct 2019, 18:36 Xavier Lecours Boucher, Mr, <
> xavier.lecoursboucher at mcgill.ca> wrote:
>
>> Hi Sotirios,
>>
>> The first error occurs because the QCFirstChangeTime and columns are of
>> type `unsigned integer` and not `datatime`. You should be using
>> UNIX_TIMESTAMP() instead of NOW().
>>
>> The second error occurs because the trigger triggers a rollback of the
>> insert statement in the files table.
>> See mysql documentation for trigger error handling.
>>
>> An error during either a BEFORE or AFTER trigger results in failure of
>> the entire statement that caused trigger invocation.
>>
>>
>> Thank you for sharing that. I hope it helps.
>> -- Xavier
>>
>> ------------------------------
>> *From:* loris-dev-bounces at bic.mni.mcgill.ca <
>> loris-dev-bounces at bic.mni.mcgill.ca> on behalf of Sotirios
>> Nikoloutsopoulos <sotirisnik at gmail.com>
>> *Sent:* October 1, 2019 8:33 AM
>> *To:* Cecile Madjar <cecile.madjar at mcin.ca>
>> *Cc:* loris-dev at bic.mni.mcgill.ca <loris-dev at bic.mni.mcgill.ca>;
>> Christine Rogers, Ms. <christine.rogers at mcgill.ca>
>> *Subject:* Re: [Loris-dev] Import mri - scripts
>>
>> When mincs are inserted their corresponding rows at the file table are
>> inserted too
>> [image: image.png]
>>
>> i would like to insert their rows at file_qcstatus as well, because i
>> don't want manually to label them as passed throught the interface
>>
>> [image: image.png]
>>
>>
>> That is the trigger i wrote
>>
>> CREATE DEFINER = CURRENT_USER TRIGGER `LORIS`.`files_AFTER_INSERT` AFTER
>> INSERT ON `files` FOR EACH ROW
>> BEGIN
>> INSERT INTO files_qcstatus
>> SET FileID = NEW.FileID,
>> SeriesUID = NEW.SeriesUID,
>> EchoTime = NEW.EchoTime,
>> QCStatus = "Pass",
>> QCFirstChangeTime = NOW(),
>> QCLastChangeTime = NOW();
>> END
>>
>> although it contains some errors
>>
>> DBD::mysql::db do failed: Out of range value for column
>> 'QCFirstChangeTime' at row 1 at
>> /data/loris/bin/mri/uploadNeuroDB/NeuroDB/MRI.pm line 823.
>> DBD::mysql::db do failed: Cannot add or update a child row: a foreign key
>> constraint fails (`LORIS`.`parameter_file`, CONSTRAINT
>> `FK_parameter_file_1` FOREIGN KEY (`FileID`) REFERENCES `files` (`FileID`))
>> at /data/loris/bin/mri/uploadNeuroDB/NeuroDB/MRI.pm line 848.
>>
>>
>>
>>
>>
>>
>>
>>
>> Στις Τρί, 1 Οκτ 2019 στις 4:21 μ.μ., ο/η Cecile Madjar <
>> cecile.madjar at mcin.ca> έγραψε:
>>
>> Hi Sotirios,
>>
>> In the mri_upload table, there is a TarchiveID column associated with the
>> MRI upload you inserted. When this value is NULL, it means no DICOMs were
>> inserted into the tarchive tables. At the end of the insertion of the
>> DICOMs in the tarchive tables, this value is updated with the correct
>> TarchiveID associated with the upload.
>>
>> Could this correspond to what you want to do?
>>
>> Regarding the files_qcstatus, this table is only linked to the files
>> table (hence, the MINC files). You could always create a new table for
>> dicom_qcstatus and link it to the tarchive table?
>>
>> Cécile
>>
>> On Tue, Oct 1, 2019 at 6:56 AM Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> wrote:
>>
>> Hi,
>>
>> i would like whenever a dicom is imported to mark all their coressponding
>> qc_status to "pass". I was thinking of creating a trigger for that, but
>> which tables do i need to add entries to? So far from what i see i need to
>> add entres at the table "files" whenever an insertion happens to
>> files_qcstatus.
>>
>> Thanks
>>
>> Στις Δευ, 30 Σεπ 2019 στις 4:04 μ.μ., ο/η Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> έγραψε:
>>
>> Oh, sorry i should have asked for StudyID, at the beginning of my email i
>> sent 3 hours ago (not StudyUID).
>>
>> Στις Δευ, 30 Σεπ 2019 στις 3:55 μ.μ., ο/η Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> έγραψε:
>>
>> Hi,
>>
>> that's the Study Instance UID, not the StudyID. So the StudyID i am
>> looking for is not stored in the database?
>>
>> [image: image.png]
>>
>> Thanks.
>>
>> Στις Δευ, 30 Σεπ 2019 στις 3:38 μ.μ., ο/η Cecile Madjar <
>> cecile.madjar at mcin.ca> έγραψε:
>>
>> Hi Sotirios,
>>
>> The StudyUID is stored in the DicomArchiveID field of the tarchive table.
>> It can also be found in the metadata field but it is mixed with many other
>> information.
>>
>> Hope this helps,
>>
>> Cécile
>>
>> On Mon, Sep 30, 2019 at 6:00 AM Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> wrote:
>>
>> Hi,
>>
>> Is StudyUID stored somewhere at the tables? At the tarchive table,
>> specifically at the AcquisitonMetadata column i found something called
>> 'Unique Study ID" in its context, but that must be the Study Instance UID.
>>
>> Thanks
>>
>> Στις Παρ, 27 Σεπ 2019 στις 4:22 μ.μ., ο/η Cecile Madjar <
>> cecile.madjar at mcin.ca> έγραψε:
>>
>> Hi Sotirios,
>>
>> Your observation is correct. A little explanation below.
>>
>> MINC files get inserted into the MRI violation tables if:
>>
>> - the scan type could not be identified (not matching an entry in the
>> mri_protocol table)
>> - one parameter of the scan type is out of the expected range present
>> in the mri_protocol_checks (extra filtering in case you need to be stricker
>> on some parameters not present in the mri_protocol table)
>> - if the CandID and PSCID do not match
>>
>> The following cases do not get in the MRI violation tables as it happens
>> before the conversion of the DICOM to MINC files and only MINC files
>> violations are logged there:
>>
>> - "No single DICOM" (since no valid DICOM could be found to convert
>> to MINC)
>> - "Study already inserted" (duplicate StudyUID) since this error
>> happens at the dicomTar.pl level (way before conversion into MINC files)
>>
>> Glad everything is working out!!
>> Best,
>>
>> Cécile
>>
>> On Fri, Sep 27, 2019 at 6:17 AM Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> wrote:
>>
>> Hi,
>>
>> the mri passed, for some reason it consumed 5.12GB of ram. Now about the
>> mri_violations, dicoms are triggered to be inserted there only if there is
>> a violation for Tr_min, Tr_max, in general for its header parameter?
>> Because i don't see the cases of 'No single dicom" or with the 2 studyiuid
>> to be there.
>>
>> Thanks
>>
>> Στις Πέμ, 26 Σεπ 2019 στις 5:07 μ.μ., ο/η Cecile Madjar <
>> cecile.madjar at mcin.ca> έγραψε:
>>
>> Hi Sotirios,
>>
>> Sorry to hear you are having issues with the memory. I think most of our
>> VMs are set up with 4GB of RAM so with 4GB you should be fine.
>>
>> Hope this helps!
>>
>> Cécile
>>
>> On Thu, Sep 26, 2019 at 9:48 AM Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> wrote:
>>
>> Hi,
>>
>> about that dicom that had that insertion error it's about 30mb and when
>> it is being processed the ram is being drastically increased from 2gb to
>> 3.28 and then the whole virtual machine is frozen. What is the recommended
>> size of ram for using Loris?
>>
>> Thanks
>>
>> [image: image.png]
>>
>>
>> Στις Τρί, 24 Σεπ 2019 στις 11:24 μ.μ., ο/η Cecile Madjar <
>> cecile.madjar at mcin.ca> έγραψε:
>>
>> Hi Sotirios,
>>
>> Glad to see that the StudyUID problem was fixed!
>>
>> For the delete script issue, if you are absolutely certain that no
>> processes are run for that uploadID, you can update the mri_upload table
>> and set the column Inserting to 0 instead of 1 for that uploadID. It
>> looks like for some reason the scripts did not update this field when it
>> stopped the insertion. Not sure why that would be the case though.
>> Note: only do that update if you are certain that there is no processing
>> happening.
>>
>> Hope this helps!
>>
>>
>> On Tue, Sep 24, 2019 at 11:57 AM Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> wrote:
>>
>> Hi,
>>
>> about the Dicoms i reported that didn't have a StudyID they actually do
>>
>> [image: image.png]
>>
>>
>> [image: image.png]
>>
>> I thought that this might be a permission because "w" was missing at the
>> group. After using chmod -R 775 only 102809579 passed, but with no mnic
>> files. Also now i have this error
>>
>> (loris-mri-python) lorisadmin at loris-VirtualBox:/data/loris/bin/mri/tools$
>> ./delete_imaging_upload.pl -uploadID 34 -ignore
>> Cannot delete upload 34: the MRI pipeline is currently processing it.
>>
>> How can solve this?
>>
>> Thanks
>>
>>
>> Στις Τρί, 24 Σεπ 2019 στις 2:30 π.μ., ο/η Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> έγραψε:
>>
>> Hi,
>>
>> Here is our auto dicom uploading python script:
>> https://github.com/aueb-wim/DataQualityControlTool/blob/loris1/mipqctool/dicom_uploader.py
>>
>> To give you a brief idea:
>> Per folder it will:
>> 1) Delete all the .bak files
>> 2) Locate .dcm files and update the Patient header
>> 3) Furthermore we are interesting in finding TR_min, TR_max, TE_min,
>> TE_max parameters of T1 protocol and do an update at the mri_protocol table
>> https://github.com/aueb-wim/DataQualityControlTool/blob/loris1/mipqctool/dicom_uploader.py#L205
>> For some reason some files are missing SeriesDescription/ProtocolName
>>
>> In total i have 7 folders
>>
>> [image: image.png]
>>
>> 3/7 passed
>>
>> [image: image.png]
>>
>> Also about the first patient with DC0000 had a violation "T1 AXIAL SE
>> GADO", but my script didn't output a TE of 17 nowhere and i find that weird
>> since Loris detects the SeriesDescription.
>> [image: image.png]
>>
>>
>> Now about the 4/7 that didn't passed
>>
>> 102327840 and 102809579 outputs "The target directory does not contain a
>> single DICOM file.", therefore they are missing their StudyUID
>>
>> 102506134 has 2 studyuid "You can't use it with data from multiple
>> studies."
>>
>> and 102761034 "No data could be converted into valid MINC files.
>> localizer, scout will not be considered!" What is this?
>>
>> If i wanted to force the insertion of 102327840, 102809579 and 102506134
>> can i just pass a flag parameter to batch_uploads_imageuploader.pl? or
>> modify imaging_upload_file.pl?
>>
>> Thank you,
>>
>> Sotirios
>>
>> PS
>>
>> To diagnose a folder search the name of the folder at dicom_output.txt.
>>
>> Στις Παρ, 20 Σεπ 2019 στις 4:54 μ.μ., ο/η Christine Rogers, Ms. <
>> christine.rogers at mcgill.ca> έγραψε:
>>
>> Hi Sotirios,
>> Glad to hear about your progress.
>>
>> Yes, the script you used to delete imaging data is fully documented here
>> <https://github.com/aces/Loris-MRI/blob/81bae73ea6e86c9498519dadf574468ee1d992ca/docs/scripts_md/delete_imaging_upload.md>
>> -- let us know if you didn't find the answers you were looking for there.
>> (Is it possible the null row in *mri_scanner* seen in your database
>> management software is a visual placeholder for you as the user? I'm not
>> sure why a scanner would ever be registered with ID='0' as your screenshot
>> showed.)
>>
>> Best,
>> Christine
>>
>> On Fri, Sep 20, 2019 at 9:45 AM Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> wrote:
>>
>> I fixed it with this
>>
>> [image: image.png]
>>
>> Στις Παρ, 20 Σεπ 2019 στις 4:37 μ.μ., ο/η Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> έγραψε:
>>
>> Hi,
>>
>> I tried to create a new candidate using the API, is there something wrong
>> with the structure? why did i receive a 500 internal error? In the
>> loris-error log it state that there is something wrong with token, but i
>> verfied that the type of the token is a string.
>>
>> [Fri Sep 20 16:35:26.874732 2019] [php7:error] [pid 4535] [client
>> 127.0.0.1:59674] PHP Fatal error: Uncaught TypeError: Argument 1 passed
>> to SinglePointLogin::JWTAuthenticate() must be of the type string, null
>> given, called in /var/www/loris/php/libraries/SinglePointLogin.class.inc on
>> line 169 and defined in
>> /var/www/loris/php/libraries/SinglePointLogin.class.inc:192\nStack
>> trace:\n#0 /var/www/loris/php/libraries/SinglePointLogin.class.inc(169):
>> SinglePointLogin->JWTAuthenticate(NULL)\n#1
>> /var/www/loris/php/libraries/NDB_Client.class.inc(171):
>> SinglePointLogin->authenticate()\n#2
>> /var/www/loris/htdocs/api/v0.0.2/APIBase.php(73):
>> NDB_Client->initialize('/var/www/loris/...')\n#3
>> /var/www/loris/htdocs/api/v0.0.2/Candidates.php(44):
>> Loris\\API\\APIBase->__construct('POST')\n#4
>> /var/www/loris/htdocs/api/v0.0.2/Candidates.php(244):
>> Loris\\API\\Candidates->__construct('POST', Array)\n#5 {main}\n thrown in
>> /var/www/loris/php/libraries/SinglePointLogin.class.inc on line 192
>>
>> [image: image.png]
>>
>> Στις Παρ, 20 Σεπ 2019 στις 2:39 μ.μ., ο/η Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> έγραψε:
>>
>> Hi,
>>
>> Is there a way also to delete candidates? Just to let you know what i did
>> to clear candidates, because maybe i may have missed dependencies.
>>
>> First i deleted all their uploads
>> ./delete_imaging_upload.pl -uploadID 13 etc ( is there a way to omit the
>> backup file ? )
>>
>> Afterwards i had some scanner candidates, so i deleted all the entries
>> from mri_scanner except that i couldn't delete this entry with the 0 ID.
>>
>> [image: image.png]
>>
>>
>> Lastly i deleted the session and then the candidate table.
>>
>> Thanks
>>
>> Στις Πέμ, 19 Σεπ 2019 στις 7:57 μ.μ., ο/η Christine Rogers, Ms. <
>> christine.rogers at mcgill.ca> έγραψε:
>>
>> Hi Sotirios,
>>
>> We recommend that you use our existing tools for creating new candidates
>> - either the LORIS API or if you are using a PHP script, by calling the Candidate
>> class's createNew()
>> <https://github.com/aces/Loris/blob/master/php/libraries/Candidate.class.inc#L200>function.
>> These will create the necessary records for you. (It's not recommended to
>> make direct modifications to mysql database tables in your script, if I
>> understand you.)
>>
>> The CandID should be a randomized 6 digit randomized ID, and there are
>> multiple reasons for this.
>> You can use the PSCID for project-specific IDs and the External ID field
>> in the candidate table can also be used for any values you like.
>> Additionally, any number of custom IDs can be added in parallel - these
>> are entered/visible in the Candidate Information module and added in the
>> back-end as candidate parameters.
>>
>> Best,
>> Christine
>>
>>
>>
>> On Tue, Sep 17, 2019 at 7:31 PM Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> wrote:
>>
>> Hi,
>>
>> We are building a script that will auto insert the candidates based on
>> their ExternalID. We would like to know whethere when we are creating a new
>> profile in the candidate table, if we also have to insert a new record at
>> another table? Furthermore is there a problem that in our case CandID won't
>> be a 6digit? ( should we start it from 100000?)
>>
>> Thanks
>>
>> [image: image.png]
>>
>>
>> Στις Παρ, 13 Σεπ 2019 στις 5:38 μ.μ., ο/η Melanie Legault, Mrs <
>> melanie.legault2 at mcgill.ca> έγραψε:
>>
>> That would normally be the case but the numeric part of the PSCID get
>> paded with 0 on the left.
>>
>> Ex. for sequential ID made of center ID of `ABC` plus 4 numerical char
>> with a min value of 1, the sequence would be:
>>
>> ABC0001
>> ABC0002
>> ...
>> ABC0010
>> ...
>>
>> thus sorting them ascending will always get the latest value generated as
>> the max value.
>>
>> Mélanie
>>
>> ------------------------------
>> *From:* Sotirios Nikoloutsopoulos <sotirisnik at gmail.com>
>> *Sent:* September 13, 2019 10:15
>> *To:* Melanie Legault, Mrs <melanie.legault2 at mcgill.ca>
>> *Cc:* Christine Rogers, Ms. <christine.rogers at mcgill.ca>;
>> loris-dev at bic.mni.mcgill.ca <loris-dev at bic.mni.mcgill.ca>
>> *Subject:* Re: [Loris-dev] Import mri - scripts
>>
>> I see, although the PSCID value is varchar(255), it must be stored within
>> a specific length of characters. Otherwise if we were to sort strings with
>> different lengths the result would not be sorted correctly ( e.x. 1,2,10,11
>> as strings would result to 1, 10, 11, 2 ).
>>
>> Thanks
>>
>> Στις Παρ, 13 Σεπ 2019 στις 4:52 μ.μ., ο/η Melanie Legault, Mrs <
>> melanie.legault2 at mcgill.ca> έγραψε:
>>
>> Hello Sotirios,
>>
>> There is no `last value` stored anywhere.
>> The code simply look for the PSCID with the max value and increase that
>> value by 1 in order to generate the next PSCID.
>>
>> Hope this info help.
>>
>> Mélanie Legault | Software developer | Faculty of Medicine | McGill
>> University
>> 3801 University, Montréal, QC H3A 2B4
>>
>> ------------------------------
>> *From:* loris-dev-bounces at bic.mni.mcgill.ca <
>> loris-dev-bounces at bic.mni.mcgill.ca> on behalf of Sotirios
>> Nikoloutsopoulos <sotirisnik at gmail.com>
>> *Sent:* September 13, 2019 9:43
>> *To:* Christine Rogers, Ms. <christine.rogers at mcgill.ca>
>> *Cc:* loris-dev at bic.mni.mcgill.ca <loris-dev at bic.mni.mcgill.ca>
>> *Subject:* Re: [Loris-dev] Import mri - scripts
>>
>> Hi,
>>
>> how does Loris determine the next available PSCID value for a new
>> candidate? https://github.com/aces/Loris/wiki/Project-Customization e.x
>> in the default case the PSCID is sequential, but where do we store the last
>> value for the sequential sequence?
>>
>> Thanks
>>
>> Στις Τρί, 10 Σεπ 2019 στις 4:00 π.μ., ο/η Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> έγραψε:
>>
>> Hi,
>>
>> the parameters in the mri_protocol you have assigned are global standard?
>> or were assigned after trial and error?
>>
>> Thanks
>>
>> Στις Κυρ, 8 Σεπ 2019 στις 10:11 μ.μ., ο/η Christine Rogers, Ms. <
>> christine.rogers at mcgill.ca> έγραψε:
>>
>> Hi Sotirios,
>>
>> Given you have done the initial setup of your tables to match your
>> intended protocol/parameters according to instructions --
>> the Troubleshooting guide documentation recommends
>> <https://github.com/aces/Loris-MRI/blob/master/docs/AppendixA-Troubleshooting_guideline.md#a4-insertion-script-troubleshooting-notes>
>> re-running the pipeline (and first deleting prior uploads).
>>
>> Best,
>> Christine
>>
>> On Sun, Sep 8, 2019 at 10:22 AM Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> wrote:
>>
>> *if i were to change their status to resolve what would happen?
>>
>> Στις Κυρ, 8 Σεπ 2019 στις 5:12 μ.μ., ο/η Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> έγραψε:
>>
>> I noticed that i can view the mnics at the brainbrowser from the
>> mri_violations page ( those mnics are stored at /data/loris/data/trashbin/
>> if i were to click their issue to resolve what would happend? would they
>> appear in the dicom_archive view too? ).
>>
>>
>>
>> Στις Κυρ, 8 Σεπ 2019 στις 4:17 μ.μ., ο/η Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> έγραψε:
>>
>> Hi,
>>
>> The defaut values of the schema exists in my local database. If i adjust
>> the default values of TR_min and TE_min the mincs will be uploaded?
>>
>> [image: image.png]
>>
>>
>> [image: image.png]
>>
>> Thanks
>>
>> Στις Σάβ, 7 Σεπ 2019 στις 5:48 μ.μ., ο/η Christine Rogers, Ms. <
>> christine.rogers at mcgill.ca> έγραψε:
>>
>> Hi Sotirios,
>>
>> Is it possible that you haven't set up your mri_protocol table ? (and
>> mri_scan_type table too, for additional types of scans)
>>
>> Like the psc table, this is a pre-requisite for the Imaging insertion
>> setup : See the install/setup documentation :
>> https://github.com/aces/Loris-MRI/blob/master/docs/02-Install.md
>>
>> To add new rows, just use MySQL insert statements. You can adapt the
>> insert statements which load the default table values --> e.g. Here on
>> GitHub :
>> https://github.com/aces/Loris/blob/master/SQL/0000-00-00-schema.sql#L718
>> (see also the mri_scan_type table)
>>
>> Best,
>> Christine
>>
>> On Fri, Sep 6, 2019 at 8:32 PM Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> wrote:
>>
>> I see that i can edit the values but not how to insert new rows.
>>
>> [image: image.png]
>>
>> If i choose Inserted with flag then will the minc be inserted?
>>
>> [image: image.png]
>>
>> Στις Σάβ, 7 Σεπ 2019 στις 3:21 π.μ., ο/η Christine Rogers, Ms. <
>> christine.rogers at mcgill.ca> έγραψε:
>>
>> Hi Sotirios,
>>
>> Glad to hear that *dcmodify* worked correctly.
>>
>> For the MRI Violations module, the screenshot is enough.
>>
>> It says "could not identify scan type", which we knew already. Did you
>> click on the link on those words?
>>
>> It will take you to the next page of the module, showing for each scan
>> what the scan parameters were, and will also show for comparison what's
>> stored in your *mri_protocol* table.
>> Compare these values to find which parameter was not correct according to
>> your *mri_protocol* table scan type definitions.
>>
>> You may end up broadening your mri_protocol value ranges (e.g. TR, TE)
>> for scans.
>> This can be done in the front-end, by editing the database table directly
>> in the same subpage of the MRI Violations module.
>>
>> The MRI Violations module features are explained in more detail in the
>> Help text for this module inside LORIS (click the ["?"] icon in the menu
>> bar.)
>>
>> See also: Loris-MRI troubleshooting guide
>> <https://github.com/aces/Loris-MRI/blob/master/docs/AppendixA-Troubleshooting_guideline.md>
>> : no MINCs inserted- violated scans
>>
>> Best,
>> Christine
>>
>> On Fri, Sep 6, 2019 at 8:11 PM Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> wrote:
>>
>> Hi,
>>
>> looks like my previous email's attachment wasn't delivered due to
>> security reasons, i uploaded my file at google drive
>> https://drive.google.com/file/d/1U_TRbo_qGgfpQfs-SqeG9J3bMMqNfUU4/view?usp=sharing
>>
>> Στις Σάβ, 7 Σεπ 2019 στις 3:02 π.μ., ο/η Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> έγραψε:
>>
>> Hi,
>>
>> From Dicat's view seems that dcmodify worked in both cases
>>
>> [image: image.png]
>>
>> [image: image.png]
>>
>> For the mri_violations i attached the .html output from webbrowser, so
>> that you can check the filelds easier.
>>
>> [image: image.png]
>>
>> Thanks
>>
>>
>>
>> Στις Παρ, 6 Σεπ 2019 στις 7:49 μ.μ., ο/η Christine Rogers, Ms. <
>> christine.rogers at mcgill.ca> έγραψε:
>>
>> Hi Sotirios --
>> Great, sounds like more progress.
>>
>> About the dcmodify command -- I'm not sure why you're getting an Endian
>> warning. (it's a warning not an error, correct?)
>> To clarify -- Were the DICOM headers (PatientName) all successfully
>> relabelled, after the command ran?
>>
>> You can use also our DICAT tool (https://github.com/aces/DICAT) to
>> verify and/or update local DICOM headers -- though your dcmodify command is
>> a great and fast solution for bulk header updates.
>>
>> If you're concerned about fields being changed (e.g. "(2001,105f)" from
>> the warning message) - you can also dcmdump a DICOM slice before and look
>> at these fields specifically.
>> It's also not a bad "sanity check" to backup your DICOMS before/after
>> running dcmodify, and use dcmdump on each version to diff the outputs --
>> this will pinpoint what changed.
>>
>> Re the protocol violation -- AcquisitionProtocol not recognized or
>> unknown : this means your scans did not match what is stored in your
>> mri_protocol table.
>> Check the MRI Violations front-end module -- can you see why they didn't
>> match?
>> Send us an example, in addition to the contents of the mri_protocol
>> table, if you can't find the source of the mismatch.
>>
>> Best,
>> Christine
>>
>>
>> On Fri, Sep 6, 2019 at 12:04 PM Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> wrote:
>>
>> Here is the psc table
>> [image: image.png]
>>
>> I created 2 candidates profiles through the interface
>>
>> [image: image.png]
>>
>> then runned dcmodify command to a dicom file
>>
>> dcmodify -ma PatientName="DCC0000_258024_V1"
>> /home/lorisadmin/DICOMS/000535670/501/*.dcm
>>
>> and got those warning: is this okay?
>> W: Found element (2001,105f) with VR UN and undefined length, reading a
>> sequence with transfer syntax LittleEndianImplicit (CP-246)
>> W: Found element (2005,1083) with VR UN and undefined length, reading a
>> sequence with transfer syntax LittleEndianImplicit (CP-246)
>> W: Found element (2005,1402) with VR UN and undefined length, reading a
>> sequence with transfer syntax LittleEndianImplicit (CP-246)
>> W: Found element (2005,140f) with VR UN and undefined length, reading a
>> sequence with transfer syntax LittleEndianImplicit (CP-246)
>> W: Found element (2001,105f) with VR UN and undefined length, reading a
>> sequence with transfer syntax LittleEndianImplicit (CP-246)
>> W: Found element (2005,1083) with VR UN and undefined length, reading a
>> sequence with transfer syntax LittleEndianImplicit (CP-246)
>>
>> dcmodify at another Dicom didn't show warnings. Below you can see the
>> execution for the second dicom. Mnics could not be inserted due to
>> AcquisitionProtocol being unknown.
>>
>> Running now the following command: /data/loris/bin/mri//uploadNeuroDB/
>> imaging_upload_file.pl -profile prod -upload_id 12
>> /data/incoming/DCC0001_602102_V1.tar.gz -verbose
>>
>> find -path \/tmp\/ImagingUpload\-18\-36\-mTrxXs -name '__MACOSX' -delete
>>
>>
>> dicomTar.pl \/tmp\/ImagingUpload\-18\-36\-mTrxXs
>> \/data\/loris\/data\/tarchive\/ -database -profile prod -verbose
>> Source: /tmp/ImagingUpload-18-36-mTrxXs
>> Target: /data/loris/data/tarchive
>>
>> Testing for database connectivity.
>> Database is available.
>>
>> You will archive the dir : ImagingUpload-18-36-mTrxXs
>>
>> You are creating a tar with the following command:
>>
>> tar -cf /data/loris/data/tarchive/ImagingUpload-18-36-mTrxXs.tar
>> ImagingUpload-18-36-mTrxXs
>>
>>
>> getting md5sums and gzipping!!
>>
>> * Taken from dir : /tmp/ImagingUpload-18-36-mTrxXs
>> * Archive target location :
>> /data/loris/data/tarchive/DCM_2016-08-22_ImagingUpload-18-36-mTrxXs.tar
>> * Name of creating host : 127.0.1.1
>> * Name of host OS : Linux
>> * Created by user : lorisadmin
>> * Archived on : 2019-09-06 18:36:50
>> * dicomSummary version : 1
>> * dicomTar version : 1
>> * md5sum for DICOM tarball : b1dcdc8903dd2d9a5443227db2aa2814
>> ImagingUpload-18-36-mTrxXs.tar
>> * md5sum for DICOM tarball gzipped : aeae87f20155a6805f7e0cfe5212ea5f
>> ImagingUpload-18-36-mTrxXs.tar.gz
>> * md5sum for complete archive : 1d9258d1f077ebc49111ab7ba22a8d6e
>> DCM_2016-08-22_ImagingUpload-18-36-mTrxXs.tar
>>
>> Adding archive info into database
>>
>> Removing temporary files from target location
>>
>>
>> Done adding archive info into database
>>
>> \/data\/loris\/bin\/mri\//uploadNeuroDB/tarchiveLoader.pl -globLocation
>> -profile prod
>> \/data\/loris\/data\/tarchive\/\/DCM_2016\-08\-22_ImagingUpload\-18\-36\-mTrxXs\.tar
>> -uploadID 12 -verbose
>> md5sum
>> /data/loris/data/tarchive/DCM_2016-08-22_ImagingUpload-18-36-mTrxXs.tar
>> PSCID is: DCC0001
>> CandID id: 602102
>> visit_label is: V1
>> PSCID is: DCC0001
>> CandID id: 602102
>> visit_label is: V1
>> candidate id 602102
>> Set centerID = 1
>> PSCID is: DCC0001
>> CandID id: 602102
>> visit_label is: V1
>> PSCID is: DCC0001
>> CandID id: 602102
>> visit_label is: V1
>>
>> Number of MINC files that will be considered for inserting into the
>> database: 2
>>
>> log dir is /data/loris/data//logs and log file is
>> /data/loris/data//logs/TarLoad-18-37-31ajWx.log
>> PSCID is: DCC0001
>> CandID id: 602102
>> visit_label is: V1
>> PSCID is: DCC0001
>> CandID id: 602102
>> visit_label is: V1
>> candidate id 602102
>>
>> log dir is /data/loris/data//logs and log file is
>> /data/loris/data//logs/TarLoad-18-37-fanZFj.log
>> PSCID is: DCC0001
>> CandID id: 602102
>> visit_label is: V1
>> PSCID is: DCC0001
>> CandID id: 602102
>> visit_label is: V1
>> candidate id 602102
>>
>> Cleaning up temp files: rm -rf
>> /tmp/TarLoad-18-37-to5pYZ/ImagingUpload-18-36-mTrxXs*
>>
>> (loris-mri-python) lorisadmin at loris-VirtualBox:/data/loris/bin/mri$ cat
>> /data/loris/data/logs/TarLoad-18-37-fanZFj.log
>>
>> ==> Loading file from disk
>> /tmp/TarLoad-18-37-to5pYZ/dcc0001_602102_v1_20160822_072406_205e1d1_mri.mnc
>>
>> --> mapping DICOM parameter for
>> /tmp/TarLoad-18-37-to5pYZ/dcc0001_602102_v1_20160822_072406_205e1d1_mri.mnc
>>
>> ==> computing md5 hash for MINC body.
>>
>> --> md5: 02022dda60d9de429340fec838f50cfe
>>
>> ==> verifying acquisition protocol
>>
>> Acquisition protocol is unknown
>>
>> --> The minc file cannot be registered since the AcquisitionProtocol is
>> unknown
>>
>> Στις Τετ, 4 Σεπ 2019 στις 10:46 μ.μ., ο/η Christine Rogers, Ms. <
>> christine.rogers at mcgill.ca> έγραψε:
>>
>> Hi Sotirios:
>>
>> Sure -- you can delete imaging datasets with the *delete_imaging_upload*
>> script --
>> details here:
>> https://github.com/aces/Loris-MRI/blob/21.0-dev/docs/scripts_md/delete_imaging_upload.md
>>
>> What's in your *psc* table? (Is it properly populated? This is a
>> pre-requisite to loading imaging data.
>> <https://github.com/aces/Loris-MRI/blob/21.0-dev/docs/02-Install.md#221-database>
>> )
>> The foreign key constraint error on the candidate record is curious.
>>
>> There are also a few options for creating candidates when inserting
>> imaging data:
>>
>> -- Method 1 : in 2 steps with the API then DICOM insertion pipeline
>>
>> a. via the LORIS API -- Create the candidates (and visits, optionally I
>> think)
>> How to:
>> https://github.com/aces/Loris/blob/master/docs/API/LorisRESTAPI.md#30-candidate-api
>> Ensure you get the DCCID/CandID assigned by LORIS.
>>
>> Then as a second step:
>> b. Use the imaging insertion pipeline
>> <https://github.com/aces/Loris-MRI/blob/minor/docs/05-PipelineLaunchOptions.md#51---pipeline-launch-options-for-dicom-datasets>
>> to load your DICOMs.
>> You will want to first ensure that the PatientName header in the DICOMs
>> as well as tar package are correctly labelled with PSCID_DCCID_VisitLabel
>>
>> -- Method 2: for BIDS-format datasets:
>> How to :
>> https://github.com/aces/Loris-MRI/blob/master/docs/05-PipelineLaunchOptions.md#52---pipeline-launch-for-bids-datasets
>>
>> Be sure to use the *-c* and *-s* options when running the bids_import script,
>> to automatically create your candidates and sessions.
>>
>> Best,
>> Christine
>>
>> On Wed, Sep 4, 2019 at 11:43 AM Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> wrote:
>>
>> How do i delete a Study? And everytime do i have to create a new
>> candidate to get DCCID and a PSCID?
>>
>> *First execution:*
>>
>> (loris-mri-python) lorisadmin at loris-VirtualBox:/data/loris/bin/mri$ ./
>> batch_uploads_imageuploader.pl -profile prod < ~/Desktop/input.txt >
>> log.txt
>> Use of uninitialized value $_ in pattern match (m//) at ./
>> batch_uploads_imageuploader.pl line 144.
>> DBD::mysql::db do failed: Cannot add or update a child row: a foreign key
>> constraint fails (`LORIS`.`candidate`, CONSTRAINT `FK_candidate_1` FOREIGN
>> KEY (`RegistrationCenterID`) REFERENCES `psc` (`CenterID`)) at
>> /data/loris/bin/mri/uploadNeuroDB/NeuroDB/MRI.pm line 1060.
>> ERROR: Failed to insert record in table mri_scanner:
>> The following database commands failed:
>> PREPARE s FROM 'INSERT INTO mri_scanner
>> (CandID,Model,Software,Serial_number,Manufacturer) VALUES (?,?,?,?,?)';
>> SET
>> @x1='242126', at x2='Achieva', at x3='3.2.2\3.2.2.0', at x4='34037', at x5='Philips
>> Medical Systems';
>> EXECUTE s USING @x1, at x2, at x3, at x4, at x5;
>> Error obtained:Cannot add or update a child row: a foreign key constraint
>> fails (`LORIS`.`mri_scanner`, CONSTRAINT `FK_mri_scanner_1` FOREIGN KEY
>> (`CandID`) REFERENCES `candidate` (`CandID`)) (error code 1452)
>>
>> ERROR: The validation has failed. Either re-run the validation again and
>> fix the problem. Or re-run tarchiveLoader.pl using -force to force the
>> execution.
>>
>>
>> The tarchiveLoader.pl insertion script has failed.
>> Can't exec "mail": No such file or directory at ./
>> batch_uploads_imageuploader.pl line 249.
>> print() on closed filehandle MAIL at ./batch_uploads_imageuploader.pl
>> line 250.
>> print() on closed filehandle MAIL at ./batch_uploads_imageuploader.pl
>> line 251.
>>
>>
>> *And second execution: *
>>
>> (loris-mri-python) lorisadmin at loris-VirtualBox:/data/loris/bin/mri$ ./
>> batch_uploads_imageuploader.pl -profile prod < ~/Desktop/input.txt >
>> log.txt
>> Use of uninitialized value $_ in pattern match (m//) at ./
>> batch_uploads_imageuploader.pl line 144.
>>
>> PROBLEM:
>> The user 'lorisadmin' has already inserted this study.
>> The unique study ID is '1.3.51.0.1.1.10.49.10.222.1422753.1420953'.
>> This is the information retained from the first time the study was
>> inserted:
>>
>> * Taken from dir : /tmp/ImagingUpload-18-33-Qq7HGy
>> * Archive target location :
>> /data/loris/data/tarchive/DCM_2012-12-05_ImagingUpload-18-33-Qq7HGy.tar
>> * Name of creating host : 127.0.1.1
>> * Name of host OS : Linux
>> * Created by user : lorisadmin
>> * Archived on : 2019-09-04 18:33:05
>> * dicomSummary version : 1
>> * dicomTar version : 1
>> * md5sum for DICOM tarball : 4a301b0318178b09b91e63544282364d
>> ImagingUpload-18-33-Qq7HGy.tar
>> * md5sum for DICOM tarball gzipped : 9d95ea2b9111be236808bfd65d7e65ec
>> ImagingUpload-18-33-Qq7HGy.tar.gz
>> * md5sum for complete archive : ab19a86357f1d4053aa3b81c8a071053
>> DCM_2012-12-05_ImagingUpload-18-33-Qq7HGy.tar
>>
>>
>> Last update of record:
>>
>>
>>
>> The dicomTar.pl execution has failed.
>> Can't exec "mail": No such file or directory at ./
>> batch_uploads_imageuploader.pl line 249.
>> print() on closed filehandle MAIL at ./batch_uploads_imageuploader.pl
>> line 250.
>> print() on closed filehandle MAIL at ./batch_uploads_imageuploader.pl
>> line 251.
>>
>> Στις Τετ, 4 Σεπ 2019 στις 5:35 μ.μ., ο/η Christine Rogers, Ms. <
>> christine.rogers at mcgill.ca> έγραψε:
>>
>> Hi Sotirios,
>>
>> Ok, this sounds like good progress. Let us know when you next encounter
>> issues as you progress through the Imaging Install/Setup docs
>> <https://github.com/aces/Loris-MRI/blob/minor/docs/02-Install.md>.
>> I'll look into how we can better handle the incoming/ directory next time.
>> Best,
>> Christine
>>
>> On Wed, Sep 4, 2019 at 10:23 AM Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> wrote:
>>
>> /data/loris/bin/mri/dicom-archive/.loris_mri/database_config.py is
>> populated correctly except its port is 'port' : ''. Also i have tested
>> that i can connect to MySQL with lorisuser.
>>
>> I executed the script again, because the only error i had previously was
>> that the /data/incoming folder didn't exist and there are no errors
>> reported back except of warnings <<mysql: [Warning] Using a password on the
>> command line interface can be insecure>>.
>>
>>
>> Στις Τετ, 4 Σεπ 2019 στις 4:53 μ.μ., ο/η Christine Rogers, Ms. <
>> christine.rogers at mcgill.ca> έγραψε:
>>
>> Hi Sotirios,
>>
>> It's possible -- if the script was unable to connect to the database
>> during its execution (e.g. typo in the password), that would explain the
>> underpopulated Image path and Loris-MRI code path you saw in the Config
>> module.
>> It's hard to tell without seeing the output from your script run -- Did
>> you see a sign of any such error?
>> The Config fields are populated by the imaging_install.sh script (starting
>> at line 222
>> <https://github.com/aces/Loris-MRI/blob/21.0-dev/imaging_install.sh#L222>
>> )
>>
>> For example, check if the database connection information was populated
>> accurately in $mridir/dicom-archive/.loris_mri/database_config.py
>>
>> Best,
>> Christine
>>
>>
>>
>> On Wed, Sep 4, 2019 at 9:34 AM Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> wrote:
>>
>> Hi Christine,
>>
>> If you are referring to the imaging_install.sh here is an image with the
>> configurations i typed. Maybe the problem is somewhere at the last part
>> which asks to configure as much as possible automatically?
>>
>> Thanks
>>
>>
>> Στις Τετ, 4 Σεπ 2019 στις 4:16 μ.μ., ο/η Christine Rogers, Ms. <
>> christine.rogers at mcgill.ca> έγραψε:
>>
>> Hi Sotirios,
>>
>> Glad to hear your LORIS core install is up and working and all the
>> front-end pages are loading.
>>
>> > lorisadmin at loris-VirtualBox:/var/www/loris$ chmod 775 project
>> > and the web interface worked.
>>
>> Yes, it's important that project/ have 775 permissions and that
>> lorisadmin be part of the sudoers group, per step 1 in the install Readme
>> <https://github.com/aces/Loris#install-steps>.
>>
>> > As for the Paths, LORIS-MRI code and Image should change LORIS to
>> loris, right?
>>
>> These imaging paths will be updated during your imaging installation
>> <https://github.com/aces/Loris/wiki/Setup> by an automated script -- you
>> do not need to set them manually via the Config module.
>> Please continue to follow the Setup Guide
>> <https://github.com/aces/Loris/wiki/Setup> for detailed steps to follow.
>>
>> Best,
>> Christine
>>
>>
>>
>>
>> On Wed, Sep 4, 2019 at 8:05 AM Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> wrote:
>>
>> As for the Paths, LORIS-MRI code and Image should change LORIS to loris,
>> right?
>>
>> [image: image.png]
>>
>>
>> Στις Τετ, 4 Σεπ 2019 στις 2:28 μ.μ., ο/η Sotirios Nikoloutsopoulos <
>> sotirisnik at gmail.com> έγραψε:
>>
>> Hi,
>>
>> i used
>>
>> curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
>> sudo apt-get install -y nodejs
>>
>>
>> you had suggested in the past, make worked and now i can see all the
>> contents in the web-interface, but i don't need make install?
>>
>> Thanks
>>
>> _______________________________________________
>> Loris-dev mailing list
>> Loris-dev at bic.mni.mcgill.ca
>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>
>>
>>
>> --
>>
>> christine.rogers at mcgill.ca
>> McGill Centre for Integrative Neuroscience | MCIN.ca
>> Montreal Neurological Institute
>> McGill University | Montreal | Canada
>>
>> _______________________________________________
>> Loris-dev mailing list
>> Loris-dev at bic.mni.mcgill.ca
>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>
>>
>>
>> --
>>
>> christine.rogers at mcgill.ca
>> McGill Centre for Integrative Neuroscience | MCIN.ca
>> Montreal Neurological Institute
>> McGill University | Montreal | Canada
>>
>> _______________________________________________
>> Loris-dev mailing list
>> Loris-dev at bic.mni.mcgill.ca
>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>
>>
>>
>> --
>>
>> christine.rogers at mcgill.ca
>> McGill Centre for Integrative Neuroscience | MCIN.ca
>> Montreal Neurological Institute
>> McGill University | Montreal | Canada
>>
>>
>>
>> --
>>
>> christine.rogers at mcgill.ca
>> McGill Centre for Integrative Neuroscience | MCIN.ca
>> Montreal Neurological Institute
>> McGill University | Montreal | Canada
>>
>> _______________________________________________
>> Loris-dev mailing list
>> Loris-dev at bic.mni.mcgill.ca
>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>
>>
>>
>> --
>>
>> christine.rogers at mcgill.ca
>> McGill Centre for Integrative Neuroscience | MCIN.ca
>> Montreal Neurological Institute
>> McGill University | Montreal | Canada
>>
>> _______________________________________________
>> Loris-dev mailing list
>> Loris-dev at bic.mni.mcgill.ca
>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>
>>
>>
>> --
>>
>> christine.rogers at mcgill.ca
>> McGill Centre for Integrative Neuroscience | MCIN.ca
>> Montreal Neurological Institute
>> McGill University | Montreal | Canada
>>
>> _______________________________________________
>> Loris-dev mailing list
>> Loris-dev at bic.mni.mcgill.ca
>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>
>>
>>
>> --
>>
>> christine.rogers at mcgill.ca
>> McGill Centre for Integrative Neuroscience | MCIN.ca
>> Montreal Neurological Institute
>> McGill University | Montreal | Canada
>>
>> _______________________________________________
>> Loris-dev mailing list
>> Loris-dev at bic.mni.mcgill.ca
>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>
>>
>>
>> --
>>
>> christine.rogers at mcgill.ca
>> McGill Centre for Integrative Neuroscience | MCIN.ca
>> Montreal Neurological Institute
>> McGill University | Montreal | Canada
>>
>> _______________________________________________
>> Loris-dev mailing list
>> Loris-dev at bic.mni.mcgill.ca
>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>
>>
>>
>> --
>>
>> christine.rogers at mcgill.ca
>> McGill Centre for Integrative Neuroscience | MCIN.ca
>> Montreal Neurological Institute
>> McGill University | Montreal | Canada
>>
>> _______________________________________________
>> Loris-dev mailing list
>> Loris-dev at bic.mni.mcgill.ca
>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>
>>
>>
>> --
>>
>> christine.rogers at mcgill.ca
>> McGill Centre for Integrative Neuroscience | MCIN.ca
>> Montreal Neurological Institute
>> McGill University | Montreal | Canada
>>
>> _______________________________________________
>> Loris-dev mailing list
>> Loris-dev at bic.mni.mcgill.ca
>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20191109/76e689c6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: warning_output
Type: application/octet-stream
Size: 36440 bytes
Desc: not available
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20191109/76e689c6/attachment-0001.obj>
More information about the Loris-dev
mailing list