[Loris-dev] Insertion error
Cecile Madjar
cecile.madjar at mcin.ca
Thu Feb 4 08:50:16 EST 2021
Hi Alfredo,
Regarding your query of the tarchive tables, instead of querying based on
the SessionID of the MINC file, it would be best to query based on the
TarchiveID of the MINC file. The TarchiveID in the tarchive table is linked
to the field TarchiveSource of the files table. (There could be multiple
DICOM archives associated to a SessionID while TarchiveID is unique.)
Separately, could you try running the two queries below? One will compare
the Study Instance UID between the MINC file and the DICOM archive stored
in the database tables and the other will do the same for PatientName. I
think that will help spotting the weird case you are seeing above.
Note: they will take time to run given the amount of data you have. One
thing that could maybe make them run faster is if you grep the
ParameterType ID for patient_name and study_instance_uid in the
parameter_type table and give it directly to the query instead of joining
the parameter_file and parameter_type table.
*Comparison of the patient_name:*
SELECT
t.TarchiveID,
f.File,
pf.Value
FROM files f
JOIN parameter_file pf USING (FileID)
JOIN parameter_type pt USING (ParameterTypeID)
JOIN tarchive t ON (f.TarchiveSource=t.TarchiveID)
WHERE
pt.Name='patient_name'
AND t.PatientName!=pf.Value;
*Comparison of the Study Instance UID:*SELECT
t.TarchiveID,
f.File,
pf.Value
FROM files f
JOIN parameter_file pf USING (FileID)
JOIN parameter_type pt USING (ParameterTypeID)
JOIN tarchive t ON (f.TarchiveSource=t.TarchiveID)
WHERE
pt.Name='study_instance_uid'
AND t.DicomArchiveID!=pf.Value;
Let me know how it goes.
Cécile
On Wed, Feb 3, 2021 at 1:11 PM Morales Pinzon, Alfredo <
AMORALESPINZON at bwh.harvard.edu> wrote:
> Dear LorisDev team.
>
> I have an insertion error in LORIS where a scan of subject was added to a
> different subject. I reviewed the scripts I am using and the logs from my
> scripts and from LORIS, and everything looks fine except the minc files
> that were created in LORIS.
>
> The error surfaced when I was trying to upload a minc file to a subject.
> LORIS was replying with the following message:
>
> ===
> log dir is /data_/ipmsa/loris_data/IPMSA/data/logs and log file is
> /data_/ipmsa/loris_data/IPMSA/data/logs/TarLoad-20-14-uvzbWX.log
>
> --> ERROR: there is already a file registered in the files table with
> SeriesUID='1.2.840.113619.2.25.4.2418272.1367947481.424' and
> EchoTime='0.081816'.
> The already registered file is
> 'assembly/240720/105ms302w48/mri/native/IPMSA_240720_105ms302w48_t2w_001.mnc'
> SELECT ArchiveLocation FROM tarchive WHERE
> TarchiveID = ( SELECT TarchiveID
> FROM tarchive_series WHERE SeriesUID=? AND EchoTime=?
> ) PSCID is: AZH101303
> CandID id: 814099
> visit_label is: 105ms302w48
> PSCID is: AZH101303
> CandID id: 814099
> visit_label is: 105ms302w48
> ===
>
> Based on that error, I looked at the provenance of the file
> "assembly/240720/105ms302w48/mri/native/IPMSA_240720_105ms302w48_t2w_001.mnc".
> I extracted the header for that file and found that indeed it has the same
> "SeriesInstanceUID"
>
> ===
> $ mincheader
> /data_/ipmsa/loris_data/IPMSA/data/assembly/240720/105ms302w48/mri/native/IPMSA_240720_105ms302w48_t2w_001.mnc
> | grep 0x0020:el_0x000e
> dicom_0x0020:el_0x000e = "1.2.840.113619.2.25.4.2418272.1367947481.424" ;
> ===
>
> Then I found for the tarfile associated with that the visit for that minc
> file:
>
> ===
> SELECT DicomArchiveID, PatientID, PatientName, PatientDoB, PatientSex,
> neurodbCenterName, CenterName, LastUpdate, DateAcquired, DateFirstArchived,
> DateLastArchived, AcquisitionCount, NonDicomFileCount, DicomFileCount,
> md5sumDicomOnly, md5sumArchive, CreatingUser, sumTypeVersion,
> tarTypeVersion, SourceLocation, ArchiveLocation, ScannerManufacturer,
> ScannerModel, ScannerSerialNumber, ScannerSoftwareVersion, SessionID,
> uploadAttempt, CreateInfo, TarchiveID, DateSent, PendingTransfer
> -> FROM tarchive
> -> WHERE SessionID IN ( SELECT ID FROM session WHERE CandID=240720 AND
> Visit_label='105ms302w48') \G;
> *************************** 1. row ***************************
> DicomArchiveID:
> 1.3.12.2.1107.5.2.30.26523.30000014071913412629600000031
> PatientID: 105MS301_272-CAA-1_ccc_272-903_105MS302-w48
> PatientName: BAG272302_240720_105ms302w48
> PatientDoB: 1980-10-12
> PatientSex: Anonymized
> neurodbCenterName: NULL
> CenterName: Anonymized
> LastUpdate: NULL
> DateAcquired: 2014-07-19
> DateFirstArchived: 2020-11-24 07:56:13
> DateLastArchived: 2020-11-24 07:56:13
> AcquisitionCount: 5
> NonDicomFileCount: 0
> DicomFileCount: 300
> md5sumDicomOnly: 7d9e2c2e653de5586b30ad6125115441
> ImagingUpload-7-24-6mxg7z.tar
>
> md5sumArchive: 6d50ab9ebd4d2174eb961be7298b536f
> DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar
>
> CreatingUser: lorisadmin
> sumTypeVersion: 1
> tarTypeVersion: 1
> SourceLocation: /tmp/ImagingUpload-7-24-6mxg7z
> ArchiveLocation: 2014/DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar
> ScannerManufacturer: SIEMENS
> ScannerModel: Avanto
> ScannerSerialNumber: 26523
> ScannerSoftwareVersion: syngo MR B15
> SessionID: 116004
> uploadAttempt: 0
> CreateInfo:
> * Taken from dir : /tmp/ImagingUpload-7-24-6mxg7z
> * Archive target location :
> /data_/ipmsa/loris_data/IPMSA/data/tarchive/DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar
> * Name of creating host : 132.216.133.49
> * Name of host OS : Linux
> * Created by user : lorisadmin
> * Archived on : 2020-11-24 07:54:05
> * dicomSummary version : 1
> * dicomTar version : 1
> * md5sum for DICOM tarball : 7d9e2c2e653de5586b30ad6125115441
> ImagingUpload-7-24-6mxg7z.tar
> * md5sum for DICOM tarball gzipped : d9b420f7495ab7f5bad5ddcf0f198963
> ImagingUpload-7-24-6mxg7z.tar.gz
> * md5sum for complete archive : 6d50ab9ebd4d2174eb961be7298b536f
> DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar
>
> TarchiveID: 35280
> DateSent: NULL
> PendingTransfer: 0
> 1 row in set (0.00 sec)
> ===
>
> The tar file is the following:
> /data_/ipmsa/loris_data/IPMSA/data/tarchive/2014/DCM_2014-07-19_ImagingUpload-7-24-6mxg7z.tar
> The "SeriesInstanceUID" for the t2w dicoms is
> "1.3.12.2.1107.5.2.30.26523.2014071919365885482743202.0.0.0" which
> different as the one reported in the minc file!
>
> What could be happening here? The only thing I think might play a role
> here is that I was uploading 7.500 visits for one study, running 15
> insertions in parallel. Maybe some temporary files were overwritten by
> processes running in parallel?
>
> Please advice on how to deal with this issue, and how to validate/verify
> the integrity of the files that have been uploaded so far (~30.000 visits).
>
> Best,
> Alfredo.
>
> The information in this e-mail is intended only for the person to whom it
> is addressed. If you believe this e-mail was sent to you in error and the
> e-mail contains patient information, please contact the Mass General
> Brigham Compliance HelpLine at
> http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent
> to you in error but does not contain patient information, please contact
> the sender and properly dispose of the e-mail.
> _______________________________________________
> Loris-dev mailing list
> Loris-dev at bic.mni.mcgill.ca
> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210204/927f0400/attachment-0001.html>
More information about the Loris-dev
mailing list