<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Hi Cecile,
<div class=""><br class="">
</div>
<div class="">Thank you for your answer. I did the query using the files table, I’m getting the same TarchiveID.</div>
<div class=""><br class="">
</div>
<div class="">I will send the results of the query in a separate email. In summary, we have 335 images with different StudyInstanceID, and 265 with different PatientName. The case we are looking at is in those results.</div>
<div class=""><br class="">
</div>
<div class=""> What would you suggest to fix this and to avoid it for future insertions?</div>
<div class=""><br class="">
</div>
<div class="">Best,</div>
<div class="">Alfredo.</div>
<div class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Feb 4, 2021, at 8:50 AM, Cecile Madjar <<a href="mailto:cecile.madjar@mcin.ca" class="">cecile.madjar@mcin.ca</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<p class=""><span style="padding: 3px 10px; border-radius: 5px; color: #ffffff; font-weight: bold; display: inline-block; background-color: #ff0000;" class=""> External Email - Use Caution </span></p>
<div class=""><br class="webkit-block-placeholder">
</div>
<div dir="ltr" class="">Hi Alfredo,
<div class=""><br class="">
</div>
<div class="">Regarding your query of the tarchive tables, instead of querying based on the SessionID of the MINC file, it would be best to query based on the TarchiveID of the MINC file. The TarchiveID in the tarchive table is linked to the field TarchiveSource of
the files table. (There could be multiple DICOM archives associated to a SessionID while TarchiveID is unique.)<br class="">
</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">Separately, could you try running the two queries below? One will compare the Study Instance UID between the MINC file and the DICOM archive stored in the database tables and the other will do the same for PatientName. I think that will help spotting
the weird case you are seeing above.</div>
<div class=""></div>
</div>
<div class=""><br class="">
</div>
<div class="">Note: they will take time to run given the amount of data you have. One thing that could maybe make them run faster is if you grep the ParameterType ID for patient_name and study_instance_uid in the parameter_type table and give it directly to
the query instead of joining the parameter_file and parameter_type table.</div>
<div class=""><br class="">
</div>
<div class=""><u class="">Comparison of the patient_name:</u><br class="">
</div>
<div class="">SELECT <br class="">
t.TarchiveID, <br class="">
f.File,<br class="">
pf.Value<br class="">
FROM files f<br class="">
JOIN parameter_file pf USING (FileID)<br class="">
JOIN parameter_type pt USING (ParameterTypeID)<br class="">
JOIN tarchive t ON (f.TarchiveSource=t.TarchiveID)<br class="">
WHERE<br class="">
pt.Name='patient_name'<br class="">
AND t.PatientName!=pf.Value;<br class="">
<br class="">
<u class="">Comparison of the Study Instance UID:<br class="">
</u>SELECT <br class="">
t.TarchiveID, <br class="">
f.File,<br class="">
pf.Value<br class="">
FROM files f<br class="">
JOIN parameter_file pf USING (FileID)<br class="">
JOIN parameter_type pt USING (ParameterTypeID)<br class="">
JOIN tarchive t ON (f.TarchiveSource=t.TarchiveID)<br class="">
WHERE<br class="">
pt.Name='study_instance_uid'<br class="">
AND t.DicomArchiveID!=pf.Value;<br class="">
</div>
<div class=""><br class="">
</div>
<div class="">Let me know how it goes. </div>
<div class=""><br class="">
</div>
<div class="">Cécile</div>
</div>
<br class="">
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, Feb 3, 2021 at 1:11 PM Morales Pinzon, Alfredo <<a href="mailto:AMORALESPINZON@bwh.harvard.edu" class="">AMORALESPINZON@bwh.harvard.edu</a>> wrote:<br class="">
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div style="overflow-wrap: break-word;" class="">
<div dir="auto" style="overflow-wrap: break-word;" class="">
<div class="">Dear LorisDev team.</div>
<div class=""><br class="">
</div>
<div class="">I have an insertion error in LORIS where a scan of subject was added to a different subject. I reviewed the scripts I am using and the logs from my scripts and from LORIS, and everything looks fine except the minc files that were created in LORIS.</div>
<div class=""><br class="">
</div>
<div class="">The error surfaced when I was trying to upload a minc file to a subject. LORIS was replying with the following message:</div>
<div class=""><br class="">
</div>
<div class="">===</div>
<div class="">log dir is /data_/ipmsa/loris_data/IPMSA/data/logs and log file is /data_/ipmsa/loris_data/IPMSA/data/logs/TarLoad-20-14-uvzbWX.log</div>
<div class=""><br class="">
</div>
<div class="">--> <font color="#ff1f00" class="">ERROR: there is already a file registered in the files table with SeriesUID='1.2.840.113619.2.25.4.2418272.1367947481.424' and EchoTime='0.081816'.</font></div>
<div class=""><font color="#ff1f00" class=""> The already registered file is 'assembly/240720/105ms302w48/mri/native/IPMSA_240720_105ms302w48_t2w_001.mnc'</font></div>
<div class=""> SELECT ArchiveLocation FROM tarchive WHERE TarchiveID = ( SELECT TarchiveID FROM tarchive_series WHERE SeriesUID=? AND EchoTime=? )
PSCID is: AZH101303</div>
<div class=""> CandID id: 814099</div>
<div class=""> visit_label is: 105ms302w48</div>
<div class="">PSCID is: AZH101303</div>
<div class=""> CandID id: 814099</div>
<div class=""> visit_label is: 105ms302w48</div>
<div class="">=== </div>
<div class=""><br class="">
</div>
<div class="">Based on that error, I looked at the provenance of the file "assembly/240720/105ms302w48/mri/native/IPMSA_240720_105ms302w48_t2w_001.mnc". I extracted the header for that file and found that indeed it has the same "SeriesInstanceUID"</div>
<div class=""><br class="">
</div>
<div class="">===</div>
<div class="">$ mincheader /data_/ipmsa/loris_data/IPMSA/data/assembly/240720/105ms302w48/mri/native/IPMSA_240720_105ms302w48_t2w_001.mnc | grep 0x0020:el_0x000e</div>
<div class=""><font color="#ff1f00" class="">dicom_0x0020:el_0x000e = "1.2.840.113619.2.25.4.2418272.1367947481.424" ;</font></div>
<div class="">===</div>
<div class=""><br class="">
</div>
<div class="">Then I found for the tarfile associated with that the visit for that minc file:</div>
<div class=""><br class="">
</div>
<div class="">===</div>
<div class="">SELECT DicomArchiveID, PatientID, PatientName, PatientDoB, PatientSex, neurodbCenterName, CenterName, LastUpdate, DateAcquired, DateFirstArchived, DateLastArchived, AcquisitionCount, NonDicomFileCount, DicomFileCount, md5sumDicomOnly, md5sumArchive,
CreatingUser, sumTypeVersion, tarTypeVersion, SourceLocation, ArchiveLocation, ScannerManufacturer, ScannerModel, ScannerSerialNumber, ScannerSoftwareVersion, SessionID, uploadAttempt, CreateInfo, TarchiveID, DateSent, PendingTransfer</div>
<div class=""> -> FROM tarchive</div>
<div class=""> -> WHERE SessionID IN ( SELECT ID FROM session WHERE CandID=240720 AND Visit_label='105ms302w48') \G;</div>
<div class="">*************************** 1. row ***************************</div>
<div class=""> DicomArchiveID: 1.3.12.2.1107.5.2.30.26523.30000014071913412629600000031</div>
<div class=""> PatientID: 105MS301_272-CAA-1_ccc_272-903_105MS302-w48</div>
<div class=""> PatientName: BAG272302_240720_105ms302w48</div>
<div class=""> PatientDoB: 1980-10-12</div>
<div class=""> PatientSex: Anonymized</div>
<div class=""> neurodbCenterName: NULL</div>
<div class=""> CenterName: Anonymized</div>
<div class=""> LastUpdate: NULL</div>
<div class=""> DateAcquired: 2014-07-19</div>
<div class=""> DateFirstArchived: 2020-11-24 07:56:13</div>
<div class=""> DateLastArchived: 2020-11-24 07:56:13</div>
<div class=""> AcquisitionCount: 5</div>
<div class=""> NonDicomFileCount: 0</div>
<div class=""> DicomFileCount: 300</div>
<div class=""> md5sumDicomOnly: 7d9e2c2e653de5586b30ad6125115441 ImagingUpload-7-24-6mxg7z.tar</div>
<div class=""><br class="">
</div>
<div class=""> md5sumArchive: 6d50ab9ebd4d2174eb961be7298b536f DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar</div>
<div class=""><br class="">
</div>
<div class=""> CreatingUser: lorisadmin</div>
<div class=""> sumTypeVersion: 1</div>
<div class=""> tarTypeVersion: 1</div>
<div class=""> SourceLocation: /tmp/ImagingUpload-7-24-6mxg7z</div>
<div class=""> ArchiveLocation: 2014/DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar</div>
<div class=""> ScannerManufacturer: SIEMENS</div>
<div class=""> ScannerModel: Avanto</div>
<div class=""> ScannerSerialNumber: 26523</div>
<div class="">ScannerSoftwareVersion: syngo MR B15</div>
<div class=""> SessionID: 116004</div>
<div class=""> uploadAttempt: 0</div>
<div class=""> CreateInfo: </div>
<div class="">* Taken from dir : /tmp/ImagingUpload-7-24-6mxg7z</div>
<div class="">* Archive target location : /data_/ipmsa/loris_data/IPMSA/data/tarchive/DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar</div>
<div class="">* Name of creating host : 132.216.133.49</div>
<div class="">* Name of host OS : Linux</div>
<div class="">* Created by user : lorisadmin</div>
<div class="">* Archived on : 2020-11-24 07:54:05</div>
<div class="">* dicomSummary version : 1</div>
<div class="">* dicomTar version : 1</div>
<div class="">* md5sum for DICOM tarball : 7d9e2c2e653de5586b30ad6125115441 ImagingUpload-7-24-6mxg7z.tar</div>
<div class="">* md5sum for DICOM tarball gzipped : d9b420f7495ab7f5bad5ddcf0f198963 ImagingUpload-7-24-6mxg7z.tar.gz</div>
<div class="">* md5sum for complete archive : 6d50ab9ebd4d2174eb961be7298b536f DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar</div>
<div class=""><br class="">
</div>
<div class=""> TarchiveID: 35280</div>
<div class=""> DateSent: NULL</div>
<div class=""> PendingTransfer: 0</div>
<div class="">1 row in set (0.00 sec)</div>
<div class="">===</div>
<div class=""><br class="">
</div>
<div class="">The tar file is the following: /data_/ipmsa/loris_data/IPMSA/data/tarchive/2014/DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar</div>
<div class="">The "SeriesInstanceUID" for the t2w dicoms is <font color="#ff1f00" class="">
"1.3.12.2.1107.5.2.30.26523.2014071919365885482743202.0.0.0"</font> which different as the one reported in the minc file!</div>
<div class=""><br class="">
</div>
<div class="">What could be happening here? The only thing I think might play a role here is that I was uploading 7.500 visits for one study, running 15 insertions in parallel. Maybe some temporary files were overwritten by processes running in parallel?</div>
<div class=""><br class="">
</div>
<div class="">Please advice on how to deal with this issue, and how to validate/verify the integrity of the files that have been uploaded so far (~30.000 visits).</div>
<div class=""><br class="">
</div>
<div class="">Best,</div>
<div class="">Alfredo.</div>
<div class=""><br class="">
</div>
</div>
<div class="">The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine
at <a href="http://secure-web.cisco.com/10HX7ylEnuGO8h3LY2zt0gJf7a7PPQAVhQpPrXMHsEVtfQ6RhGk95vroSNgMAC2fDsnk7UVEWg_yinslFXq2w8UC-aLG1WXCF6GGeiNhM2x55z0b1mbBTzSJaxEUy739kxX7Fi_E12vlrqEgkINpQ2d9AW0TWQJisoEo426LZ3Yp2O2y_L-5BPR-lMylhd91JnrRUyutGt-6QwhzPGWEERWw3K-4YldwXPswAOOdtsV9bhDCrDgYjVVzoWyeAuuZXjx9Pngxl4uiNEj9tG32gxQ/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline" target="_blank" class="">
http://www.massgeneralbrigham.org/complianceline</a> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.</div>
</div>
_______________________________________________<br class="">
Loris-dev mailing list<br class="">
<a href="mailto:Loris-dev@bic.mni.mcgill.ca" target="_blank" class="">Loris-dev@bic.mni.mcgill.ca</a><br class="">
<a href="https://secure-web.cisco.com/1EqnuJLNh8XQVj7v12HMPLQWJP0laLaunyRQ9N0vJbdjQDfMwtwotWCFvu6wy2Hl4rPXiRnnAGfDWO2zN1qsNxEazx0W9E51Sw5z1YSh94E5cHFm-WOb9MAMRpblaqme9sQT51ggwCO1npgk3xpPBQF2ZXXsfxMg95JfPawOJ9yaua4m0YhLlsgGD9X5BYgwm6SDS1zOlxSjse-D-VUkI-XeA41tl5-eltO3RQc3UU8lVCEwgQJZ6zCglH5qac9wzSiTryiO3gLiMIVpdJl74Fw/https%3A%2F%2Fmailman.bic.mni.mcgill.ca%2Fmailman%2Flistinfo%2Floris-dev" rel="noreferrer" target="_blank" class="">https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev</a><br class="">
</blockquote>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
<div>The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.</div></body>
</html>