[Loris-dev] Insertion error
Morales Pinzon, Alfredo
AMORALESPINZON at BWH.HARVARD.EDU
Thu Feb 4 12:45:21 EST 2021
Hi Cecile,
Thank you for your answer. I did the query using the files table, I’m getting the same TarchiveID.
I will send the results of the query in a separate email. In summary, we have 335 images with different StudyInstanceID, and 265 with different PatientName. The case we are looking at is in those results.
What would you suggest to fix this and to avoid it for future insertions?
Best,
Alfredo.
On Feb 4, 2021, at 8:50 AM, Cecile Madjar <cecile.madjar at mcin.ca<mailto:cecile.madjar at mcin.ca>> wrote:
External Email - Use Caution
Hi Alfredo,
Regarding your query of the tarchive tables, instead of querying based on the SessionID of the MINC file, it would be best to query based on the TarchiveID of the MINC file. The TarchiveID in the tarchive table is linked to the field TarchiveSource of the files table. (There could be multiple DICOM archives associated to a SessionID while TarchiveID is unique.)
Separately, could you try running the two queries below? One will compare the Study Instance UID between the MINC file and the DICOM archive stored in the database tables and the other will do the same for PatientName. I think that will help spotting the weird case you are seeing above.
Note: they will take time to run given the amount of data you have. One thing that could maybe make them run faster is if you grep the ParameterType ID for patient_name and study_instance_uid in the parameter_type table and give it directly to the query instead of joining the parameter_file and parameter_type table.
Comparison of the patient_name:
SELECT
t.TarchiveID,
f.File,
pf.Value
FROM files f
JOIN parameter_file pf USING (FileID)
JOIN parameter_type pt USING (ParameterTypeID)
JOIN tarchive t ON (f.TarchiveSource=t.TarchiveID)
WHERE
pt.Name='patient_name'
AND t.PatientName!=pf.Value;
Comparison of the Study Instance UID:
SELECT
t.TarchiveID,
f.File,
pf.Value
FROM files f
JOIN parameter_file pf USING (FileID)
JOIN parameter_type pt USING (ParameterTypeID)
JOIN tarchive t ON (f.TarchiveSource=t.TarchiveID)
WHERE
pt.Name='study_instance_uid'
AND t.DicomArchiveID!=pf.Value;
Let me know how it goes.
Cécile
On Wed, Feb 3, 2021 at 1:11 PM Morales Pinzon, Alfredo <AMORALESPINZON at bwh.harvard.edu<mailto:AMORALESPINZON at bwh.harvard.edu>> wrote:
Dear LorisDev team.
I have an insertion error in LORIS where a scan of subject was added to a different subject. I reviewed the scripts I am using and the logs from my scripts and from LORIS, and everything looks fine except the minc files that were created in LORIS.
The error surfaced when I was trying to upload a minc file to a subject. LORIS was replying with the following message:
===
log dir is /data_/ipmsa/loris_data/IPMSA/data/logs and log file is /data_/ipmsa/loris_data/IPMSA/data/logs/TarLoad-20-14-uvzbWX.log
--> ERROR: there is already a file registered in the files table with SeriesUID='1.2.840.113619.2.25.4.2418272.1367947481.424' and EchoTime='0.081816'.
The already registered file is 'assembly/240720/105ms302w48/mri/native/IPMSA_240720_105ms302w48_t2w_001.mnc'
SELECT ArchiveLocation FROM tarchive WHERE TarchiveID = ( SELECT TarchiveID FROM tarchive_series WHERE SeriesUID=? AND EchoTime=? ) PSCID is: AZH101303
CandID id: 814099
visit_label is: 105ms302w48
PSCID is: AZH101303
CandID id: 814099
visit_label is: 105ms302w48
===
Based on that error, I looked at the provenance of the file "assembly/240720/105ms302w48/mri/native/IPMSA_240720_105ms302w48_t2w_001.mnc". I extracted the header for that file and found that indeed it has the same "SeriesInstanceUID"
===
$ mincheader /data_/ipmsa/loris_data/IPMSA/data/assembly/240720/105ms302w48/mri/native/IPMSA_240720_105ms302w48_t2w_001.mnc | grep 0x0020:el_0x000e
dicom_0x0020:el_0x000e = "1.2.840.113619.2.25.4.2418272.1367947481.424" ;
===
Then I found for the tarfile associated with that the visit for that minc file:
===
SELECT DicomArchiveID, PatientID, PatientName, PatientDoB, PatientSex, neurodbCenterName, CenterName, LastUpdate, DateAcquired, DateFirstArchived, DateLastArchived, AcquisitionCount, NonDicomFileCount, DicomFileCount, md5sumDicomOnly, md5sumArchive, CreatingUser, sumTypeVersion, tarTypeVersion, SourceLocation, ArchiveLocation, ScannerManufacturer, ScannerModel, ScannerSerialNumber, ScannerSoftwareVersion, SessionID, uploadAttempt, CreateInfo, TarchiveID, DateSent, PendingTransfer
-> FROM tarchive
-> WHERE SessionID IN ( SELECT ID FROM session WHERE CandID=240720 AND Visit_label='105ms302w48') \G;
*************************** 1. row ***************************
DicomArchiveID: 1.3.12.2.1107.5.2.30.26523.30000014071913412629600000031
PatientID: 105MS301_272-CAA-1_ccc_272-903_105MS302-w48
PatientName: BAG272302_240720_105ms302w48
PatientDoB: 1980-10-12
PatientSex: Anonymized
neurodbCenterName: NULL
CenterName: Anonymized
LastUpdate: NULL
DateAcquired: 2014-07-19
DateFirstArchived: 2020-11-24 07:56:13
DateLastArchived: 2020-11-24 07:56:13
AcquisitionCount: 5
NonDicomFileCount: 0
DicomFileCount: 300
md5sumDicomOnly: 7d9e2c2e653de5586b30ad6125115441 ImagingUpload-7-24-6mxg7z.tar
md5sumArchive: 6d50ab9ebd4d2174eb961be7298b536f DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar
CreatingUser: lorisadmin
sumTypeVersion: 1
tarTypeVersion: 1
SourceLocation: /tmp/ImagingUpload-7-24-6mxg7z
ArchiveLocation: 2014/DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar
ScannerManufacturer: SIEMENS
ScannerModel: Avanto
ScannerSerialNumber: 26523
ScannerSoftwareVersion: syngo MR B15
SessionID: 116004
uploadAttempt: 0
CreateInfo:
* Taken from dir : /tmp/ImagingUpload-7-24-6mxg7z
* Archive target location : /data_/ipmsa/loris_data/IPMSA/data/tarchive/DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar
* Name of creating host : 132.216.133.49
* Name of host OS : Linux
* Created by user : lorisadmin
* Archived on : 2020-11-24 07:54:05
* dicomSummary version : 1
* dicomTar version : 1
* md5sum for DICOM tarball : 7d9e2c2e653de5586b30ad6125115441 ImagingUpload-7-24-6mxg7z.tar
* md5sum for DICOM tarball gzipped : d9b420f7495ab7f5bad5ddcf0f198963 ImagingUpload-7-24-6mxg7z.tar.gz
* md5sum for complete archive : 6d50ab9ebd4d2174eb961be7298b536f DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar
TarchiveID: 35280
DateSent: NULL
PendingTransfer: 0
1 row in set (0.00 sec)
===
The tar file is the following: /data_/ipmsa/loris_data/IPMSA/data/tarchive/2014/DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar
The "SeriesInstanceUID" for the t2w dicoms is "1.3.12.2.1107.5.2.30.26523.2014071919365885482743202.0.0.0" which different as the one reported in the minc file!
What could be happening here? The only thing I think might play a role here is that I was uploading 7.500 visits for one study, running 15 insertions in parallel. Maybe some temporary files were overwritten by processes running in parallel?
Please advice on how to deal with this issue, and how to validate/verify the integrity of the files that have been uploaded so far (~30.000 visits).
Best,
Alfredo.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/10HX7ylEnuGO8h3LY2zt0gJf7a7PPQAVhQpPrXMHsEVtfQ6RhGk95vroSNgMAC2fDsnk7UVEWg_yinslFXq2w8UC-aLG1WXCF6GGeiNhM2x55z0b1mbBTzSJaxEUy739kxX7Fi_E12vlrqEgkINpQ2d9AW0TWQJisoEo426LZ3Yp2O2y_L-5BPR-lMylhd91JnrRUyutGt-6QwhzPGWEERWw3K-4YldwXPswAOOdtsV9bhDCrDgYjVVzoWyeAuuZXjx9Pngxl4uiNEj9tG32gxQ/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
_______________________________________________
Loris-dev mailing list
Loris-dev at bic.mni.mcgill.ca<mailto:Loris-dev at bic.mni.mcgill.ca>
https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev<https://secure-web.cisco.com/1EqnuJLNh8XQVj7v12HMPLQWJP0laLaunyRQ9N0vJbdjQDfMwtwotWCFvu6wy2Hl4rPXiRnnAGfDWO2zN1qsNxEazx0W9E51Sw5z1YSh94E5cHFm-WOb9MAMRpblaqme9sQT51ggwCO1npgk3xpPBQF2ZXXsfxMg95JfPawOJ9yaua4m0YhLlsgGD9X5BYgwm6SDS1zOlxSjse-D-VUkI-XeA41tl5-eltO3RQc3UU8lVCEwgQJZ6zCglH5qac9wzSiTryiO3gLiMIVpdJl74Fw/https%3A%2F%2Fmailman.bic.mni.mcgill.ca%2Fmailman%2Flistinfo%2Floris-dev>
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210204/f2496e7b/attachment-0001.html>
More information about the Loris-dev
mailing list