<div dir="ltr">Hi Alfredo,<div><br></div><div>Regarding your query of the tarchive tables, instead of querying based on the SessionID of the MINC file, it would be best to query based on the TarchiveID of the MINC file. The TarchiveID in the tarchive table is linked to the field TarchiveSource of the files table. (There could be multiple DICOM archives associated to a SessionID while TarchiveID is unique.)<br></div><div><br></div><div><div>Separately, could you try running the two queries below? One will compare the Study Instance UID between the MINC file and the DICOM archive stored in the database tables and the other will do the same for PatientName. I think that will help spotting the weird case you are seeing above.</div><div></div></div><div><br></div><div>Note: they will take time to run given the amount of data you have. One thing that could maybe make them run faster is if you grep the ParameterType ID for patient_name and study_instance_uid in the parameter_type table and give it directly to the query instead of joining the parameter_file and parameter_type table.</div><div><br></div><div><u>Comparison of the patient_name:</u><br></div><div>SELECT <br> t.TarchiveID, <br> f.File,<br> pf.Value<br>FROM files f<br> JOIN parameter_file pf USING (FileID)<br> JOIN parameter_type pt USING (ParameterTypeID)<br> JOIN tarchive t ON (f.TarchiveSource=t.TarchiveID)<br>WHERE<br> pt.Name='patient_name'<br> AND t.PatientName!=pf.Value;<br> <br><u>Comparison of the Study Instance UID:<br></u>SELECT <br> t.TarchiveID, <br> f.File,<br> pf.Value<br>FROM files f<br> JOIN parameter_file pf USING (FileID)<br> JOIN parameter_type pt USING (ParameterTypeID)<br> JOIN tarchive t ON (f.TarchiveSource=t.TarchiveID)<br>WHERE<br> pt.Name='study_instance_uid'<br> AND t.DicomArchiveID!=pf.Value;<br></div><div><br></div><div>Let me know how it goes. </div><div><br></div><div>Cécile</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Feb 3, 2021 at 1:11 PM Morales Pinzon, Alfredo <<a href="mailto:AMORALESPINZON@bwh.harvard.edu">AMORALESPINZON@bwh.harvard.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div style="overflow-wrap: break-word;">
<div dir="auto" style="overflow-wrap: break-word;">
<div>Dear LorisDev team.</div>
<div><br>
</div>
<div>I have an insertion error in LORIS where a scan of subject was added to a different subject. I reviewed the scripts I am using and the logs from my scripts and from LORIS, and everything looks fine except the minc files that were created in LORIS.</div>
<div><br>
</div>
<div>The error surfaced when I was trying to upload a minc file to a subject. LORIS was replying with the following message:</div>
<div><br>
</div>
<div>===</div>
<div>log dir is /data_/ipmsa/loris_data/IPMSA/data/logs and log file is /data_/ipmsa/loris_data/IPMSA/data/logs/TarLoad-20-14-uvzbWX.log</div>
<div><br>
</div>
<div>--> <font color="#ff1f00">ERROR: there is already a file registered in the files table with SeriesUID='1.2.840.113619.2.25.4.2418272.1367947481.424' and EchoTime='0.081816'.</font></div>
<div><font color="#ff1f00"> The already registered file is 'assembly/240720/105ms302w48/mri/native/IPMSA_240720_105ms302w48_t2w_001.mnc'</font></div>
<div> SELECT ArchiveLocation FROM tarchive WHERE TarchiveID = ( SELECT TarchiveID FROM tarchive_series WHERE SeriesUID=? AND EchoTime=? )
PSCID is: AZH101303</div>
<div> CandID id: 814099</div>
<div> visit_label is: 105ms302w48</div>
<div>PSCID is: AZH101303</div>
<div> CandID id: 814099</div>
<div> visit_label is: 105ms302w48</div>
<div>=== </div>
<div><br>
</div>
<div>Based on that error, I looked at the provenance of the file "assembly/240720/105ms302w48/mri/native/IPMSA_240720_105ms302w48_t2w_001.mnc". I extracted the header for that file and found that indeed it has the same "SeriesInstanceUID"</div>
<div><br>
</div>
<div>===</div>
<div>$ mincheader /data_/ipmsa/loris_data/IPMSA/data/assembly/240720/105ms302w48/mri/native/IPMSA_240720_105ms302w48_t2w_001.mnc | grep 0x0020:el_0x000e</div>
<div><font color="#ff1f00">dicom_0x0020:el_0x000e = "1.2.840.113619.2.25.4.2418272.1367947481.424" ;</font></div>
<div>===</div>
<div><br>
</div>
<div>Then I found for the tarfile associated with that the visit for that minc file:</div>
<div><br>
</div>
<div>===</div>
<div>SELECT DicomArchiveID, PatientID, PatientName, PatientDoB, PatientSex, neurodbCenterName, CenterName, LastUpdate, DateAcquired, DateFirstArchived, DateLastArchived, AcquisitionCount, NonDicomFileCount, DicomFileCount, md5sumDicomOnly, md5sumArchive,
CreatingUser, sumTypeVersion, tarTypeVersion, SourceLocation, ArchiveLocation, ScannerManufacturer, ScannerModel, ScannerSerialNumber, ScannerSoftwareVersion, SessionID, uploadAttempt, CreateInfo, TarchiveID, DateSent, PendingTransfer</div>
<div> -> FROM tarchive</div>
<div> -> WHERE SessionID IN ( SELECT ID FROM session WHERE CandID=240720 AND Visit_label='105ms302w48') \G;</div>
<div>*************************** 1. row ***************************</div>
<div> DicomArchiveID: 1.3.12.2.1107.5.2.30.26523.30000014071913412629600000031</div>
<div> PatientID: 105MS301_272-CAA-1_ccc_272-903_105MS302-w48</div>
<div> PatientName: BAG272302_240720_105ms302w48</div>
<div> PatientDoB: 1980-10-12</div>
<div> PatientSex: Anonymized</div>
<div> neurodbCenterName: NULL</div>
<div> CenterName: Anonymized</div>
<div> LastUpdate: NULL</div>
<div> DateAcquired: 2014-07-19</div>
<div> DateFirstArchived: 2020-11-24 07:56:13</div>
<div> DateLastArchived: 2020-11-24 07:56:13</div>
<div> AcquisitionCount: 5</div>
<div> NonDicomFileCount: 0</div>
<div> DicomFileCount: 300</div>
<div> md5sumDicomOnly: 7d9e2c2e653de5586b30ad6125115441 ImagingUpload-7-24-6mxg7z.tar</div>
<div><br>
</div>
<div> md5sumArchive: 6d50ab9ebd4d2174eb961be7298b536f DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar</div>
<div><br>
</div>
<div> CreatingUser: lorisadmin</div>
<div> sumTypeVersion: 1</div>
<div> tarTypeVersion: 1</div>
<div> SourceLocation: /tmp/ImagingUpload-7-24-6mxg7z</div>
<div> ArchiveLocation: 2014/DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar</div>
<div> ScannerManufacturer: SIEMENS</div>
<div> ScannerModel: Avanto</div>
<div> ScannerSerialNumber: 26523</div>
<div>ScannerSoftwareVersion: syngo MR B15</div>
<div> SessionID: 116004</div>
<div> uploadAttempt: 0</div>
<div> CreateInfo: </div>
<div>* Taken from dir : /tmp/ImagingUpload-7-24-6mxg7z</div>
<div>* Archive target location : /data_/ipmsa/loris_data/IPMSA/data/tarchive/DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar</div>
<div>* Name of creating host : 132.216.133.49</div>
<div>* Name of host OS : Linux</div>
<div>* Created by user : lorisadmin</div>
<div>* Archived on : 2020-11-24 07:54:05</div>
<div>* dicomSummary version : 1</div>
<div>* dicomTar version : 1</div>
<div>* md5sum for DICOM tarball : 7d9e2c2e653de5586b30ad6125115441 ImagingUpload-7-24-6mxg7z.tar</div>
<div>* md5sum for DICOM tarball gzipped : d9b420f7495ab7f5bad5ddcf0f198963 ImagingUpload-7-24-6mxg7z.tar.gz</div>
<div>* md5sum for complete archive : 6d50ab9ebd4d2174eb961be7298b536f DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar</div>
<div><br>
</div>
<div> TarchiveID: 35280</div>
<div> DateSent: NULL</div>
<div> PendingTransfer: 0</div>
<div>1 row in set (0.00 sec)</div>
<div>===</div>
<div><br>
</div>
<div>The tar file is the following: /data_/ipmsa/loris_data/IPMSA/data/tarchive/2014/DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar</div>
<div>The "SeriesInstanceUID" for the t2w dicoms is <font color="#ff1f00">
"1.3.12.2.1107.5.2.30.26523.2014071919365885482743202.0.0.0"</font> which different as the one reported in the minc file!</div>
<div><br>
</div>
<div>What could be happening here? The only thing I think might play a role here is that I was uploading 7.500 visits for one study, running 15 insertions in parallel. Maybe some temporary files were overwritten by processes running in parallel?</div>
<div><br>
</div>
<div>Please advice on how to deal with this issue, and how to validate/verify the integrity of the files that have been uploaded so far (~30.000 visits).</div>
<div><br>
</div>
<div>Best,</div>
<div>Alfredo.</div>
<div><br>
</div>
</div>
<div>The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at <a href="http://www.massgeneralbrigham.org/complianceline" target="_blank">http://www.massgeneralbrigham.org/complianceline</a> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.</div></div>
_______________________________________________<br>
Loris-dev mailing list<br>
<a href="mailto:Loris-dev@bic.mni.mcgill.ca" target="_blank">Loris-dev@bic.mni.mcgill.ca</a><br>
<a href="https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev" rel="noreferrer" target="_blank">https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev</a><br>
</blockquote></div>