[Loris-dev] Create DCCID and Organizing CIVET outputs

Thu Jun 1 10:13:09 EDT 2017

Hi Mouna,

Thank you for your help and sorry for the delay in getting back to you. I disabled "ONLY_FULL_GROUP_BY mode" in MySQL but I'm still getting a few errors.

DBD::mysql::db do failed: Cannot add or update a child row: a foreign key constraint fails (`loris`.`candidate`, CONSTRAINT `FK_candidate_1` FOREIGN KEY (`CenterID`) REFERENCES `psc` (`CenterID`)) at /data/loris/bin/mri/uploadNeuroDB/NeuroDB/MRIProcessingUtility.pm line 1041.
print() on unopened filehandle LOG at /data/loris/bin/mri/uploadNeuroDB/NeuroDB/MRIProcessingUtility.pm line 1181.
DBD::mysql::db do failed: Cannot add or update a child row: a foreign key constraint fails (`loris`.`session`, CONSTRAINT `FK_session_1` FOREIGN KEY (`CandID`) REFERENCES `candidate` (`CandID`)) at /data/loris/bin/mri/uploadNeuroDB/NeuroDB/MRI.pm line 266.

The logs also show that the "CandID does not exist" and "No centerID"

In my prod file, I have the following code. The PSCID and visitLabel are retrieved from the dicom header. I've assigned a CandID for testing.

    } elsif ($patientID =~ /(.*)-([^-]+$)/) {

        $subjectID{'PSCID'}         = NeuroDB::MRI::my_trim($1);
        $subjectID{'CandID'}        = 123456;
        $subjectID{'visitLabel'}    = NeuroDB::MRI::my_trim($2);
        $subjectID{'createVisitLabel'} = 1;

        print "PSCID is: "           . $subjectID{'PSCID'}      .
                "\nCandID id: "      . $subjectID{'CandID'}     .
                "\nvisit_label is: " . $subjectID{'visitLabel'} . "\n";

    }

Thanks,

Michael

________________________________
From: Mouna Safi-Harab <mouna.safi-harb at mail.mcgill.ca>
Sent: May 9, 2017 2:02:35 PM
To: Michael Joseph; loris-dev at bic.mni.mcgill.ca
Subject: Re: Create DCCID and Organizing CIVET outputs

Hi Michael,

Sorry for the delays in getting back to you on this.

The LORIS-MRI codebase does the generation of the CandID for you (check the function CreateMRICandidates in tarchive_validation which calls createNewCandID in MRI.pm). The PSCID, however, is not created (LORIS supports multiple ways in which you can create the PSCID, this is usually read from the config.xml file in the LORIS project directory, so it is a bit trickier than creating the CandID).

But since your project would have the PSCID_VisitLabel already known and in the DICOM header, and the prod file is made to parse this, I have a workaround for your project needs.

In the determineSubjectID() routine in MRIProcessingUtility.pm, after line 241 (on version 17.0.0), you can add a few lines that selects the CandID the pipeline created a step earlier to be used for the execution of the rest of the steps, while relying on your parsed PSCID from the prod file; as follows:

    # if the candidate was created from the backend, it won't be in the tarchiveInfo, so add it here
    my $dbh = &NeuroDB::DBI::connect_to_db(@Settings::db);
    if (!defined($subjectIDsref->{'CandID'}) && $Settings::createCandidates) {
        my $query = "SELECT CandID FROM candidate WHERE PSCID=?";
        my $sth = ${$this->{'dbhr'}}->prepare($query);
        $sth->execute($subjectIDsref->{'PSCID'});
        if ( $sth->rows > 0 ) {
           $subjectIDsref->{'CandID'} = $sth->fetchrow_array;
        }
    }

This should allow you to proceed with candidate creation from the backend based on your project assumptions. Please note this is very specific to your project (in particular, due to the fact that you know the PSCID, but not the CandID, while there is a one-to-one correspondence between these two in LORIS). But I believe it addresses your request. Let me know if you have any further questions.

- Mouna

________________________________
From: Michael Joseph <michael.joseph at sickkids.ca>
Sent: Friday, April 28, 2017 12:49:50 PM
To: Mouna Safi-Harab; loris-dev at bic.mni.mcgill.ca
Subject: Re: Create DCCID and Organizing CIVET outputs

Hi Mouna,

Thank you for your reply. Your answers certainly help.

For our images, the PatientName header is labeled as PSCID-VisitLabel. I've modified the regex in the prod file to parse this out. I didn't want to modify the header to also include a DCCID because there are other groups sharing the images. I was wondering if there's another way to create a DCCID when images are uploaded (similar to how the scanner is registered).

The Imaging Uploader GUI isn't as important to us as we'll be relying on the Perl scripts to upload all our images.

Thank you for directing me to the DTIPrep directory. It definitely helps as an example for registering processed data.

Michael

________________________________
From: Mouna Safi-Harab <mouna.safi-harb at mail.mcgill.ca>
Sent: April 27, 2017 2:06:58 PM
To: Michael Joseph; loris-dev at bic.mni.mcgill.ca
Subject: Re: Create DCCID and Organizing CIVET outputs

Hi Michael,

Here are my answers to your questions:

1) Setting createCandidates to 1 in the prod file does indeed allow for creation of the candidate from the MRI pipeline. You will need to make sure that 1) the PatientName header is anonymized properly in the DICOM files, and 2) getSubjectIDs() function in this prod file is a) splitting the PatientName header information in a way that is consistent with the anonymization and your project DCCID/PSCID/VisitLabel convention, and b) has the $subjectID{'createVisitLabel'} option set to 1 as you will also likely need the MRI pipeline to create the visit for that candidate as well.

However, projects using the Imaging Uploader GUI unfortunately does not allow for this option. If you want a way to make it work for now, you can disable the check on Lines 378-395  in NDB_Menu_Filter_imaging_uploader.class.inc (which sole purpose is to check if the CandID is registered in the database). We can look at this from our end, and make sure the option createCandidate is propagated to the Imaging Uploader GUI as well in a seamless manner to the user (will discuss this at our internal meeting).

2) Loris-MRI codebase provides an example of how to insert files from another processing pipeline into the database. Check the DTIPrep/ directory for an example you can follow. Essentially, inserting processed files into LORIS is a two-step process: you will need a 1) wrapper script that "understands" or "knows" how/where the processed files are stored (this step is processing pipeline dependent), and then 2) have that wrapper script call a generic file called uploadNeuroDB/register_processed_data.pl. The register_processed_data.pl file would take in many required arguments, some of which are the full path of the file you are going to register. Other parameters are the acquisition protocol ID. So yes, the"white_matter", "pve", "tal_mask" were initially created to allow for CIVET outputs to be registered with the correct acquisition protocol. Other required parameters are the sourceFileID which would be the ID of the initial file you sent to CIVET for processing. The second step should be processing pipeline independent.

I hope the above answers your questions, or at least, gives you a starting point.

- Mouna

________________________________
From: loris-dev-bounces at bic.mni.mcgill.ca <loris-dev-bounces at bic.mni.mcgill.ca> on behalf of Michael Joseph <michael.joseph at sickkids.ca>
Sent: Wednesday, April 19, 2017 4:03 PM
To: loris-dev at bic.mni.mcgill.ca
Subject: [Loris-dev] Create DCCID and Organizing CIVET outputs

Hi Loris dev community,

I have 2 questions:

1) Is there a way to register new candidates into Loris from the back-end without a DCCID? Essentially, I'd like to add candidates and sessions while running the tarchiveLoader script. I noticed that the prod file has a createCandidates variable. Is there anything else that needs to be modified?

2) How are CIVET outputs typically organized in Loris? I noticed that in the mri_scan_type table, there are labels like "white_matter", "pve", "tal_msk". I'm assuming the most of the files from CIVET would get registered using these labels. Also, is there a good way of linking to the htmls from the CIVET QC pipeline?

Thanks,

Michael

________________________________

This e-mail may contain confidential, personal and/or health information(information which may be subject to legal restrictions on use, retention and/or disclosure) for the sole use of the intended recipient. Any review or distribution by anyone other than the person for whom it was originally intended is strictly prohibited. If you have received this e-mail in error, please contact the sender and delete all copies.

________________________________

This e-mail may contain confidential, personal and/or health information(information which may be subject to legal restrictions on use, retention and/or disclosure) for the sole use of the intended recipient. Any review or distribution by anyone other than the person for whom it was originally intended is strictly prohibited. If you have received this e-mail in error, please contact the sender and delete all copies.

________________________________

This e-mail may contain confidential, personal and/or health information(information which may be subject to legal restrictions on use, retention and/or disclosure) for the sole use of the intended recipient. Any review or distribution by anyone other than the person for whom it was originally intended is strictly prohibited. If you have received this e-mail in error, please contact the sender and delete all copies.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20170601/5b193e77/attachment-0001.html>