[Loris-dev] Error "not a unique file" inserting segmentation files in LORIS

Fri Aug 30 11:29:19 EDT 2019

in this particular case and clinical
trial (ORATORIO) the filenames can
be identical but their paths are not
and are obvisouly unique.  is this a
problem for LORIS ?

thanks for helping, bye, pisti

On Fri, 30 Aug 2019, Christine Rogers, Ms. wrote:

> Hi Alfredo, 
>       Could you please help me inserting those files which are different in size and md5 from previous uploaded files?
> 
>  
>       The only similarity between the previous uploaded files and the ones that could not be uploaded is the filename.
> 
> 
> To provide a quick answer (since most of our imaging team is on vacation this week) : 
> 
> Yes the MD5hash seems to require a unique filename (* below).
> Can you use distinct filenames ?  i.e. Are you are trying to load additional data for a participant session (i.e. same IDs and visit label) ?  
> Or, are you trying to load more than one participant/session at a time? 
> 
> (*) This line in the actual MRIProcessingUtility library:  (around Line 617)
>       $md5hash = &NeuroDB::MRI::compute_hash(\$file);
> 
> and
>       my $unique = &NeuroDB::MRI::is_unique_hash(\$file);
> 
> 
> 
> I'll check with other imaging devs to see if we have a workaround while our senior devs are away -- I think there must be some solution...  
> Meanwhile, the MD5hash for imaging files is documented here (per this script documentation)  : 
>
>       computeMd5Hash($file, $upload_id)
>       Computes the MD5 hash of a file and makes sure it is unique.
>       INPUTS:
>        o  $file : file to use to compute the MD5 hash
>        o  $upload_id: upload ID of the study
>       RETURNS: 1 if the file is unique, 0 otherwise
> 
> 
> 
> Best, 
> Christine 
> 
> 
> 
> On Thu, Aug 29, 2019 at 9:46 PM Morales Pinzon, Alfredo <AMORALESPINZON at bwh.harvard.edu> wrote:
> Dear DevLoris Team,
> 
> I started the uploading process of processed files, segmentations and transformations, using the file "register_processed_data.pl" but some files are not being
> uploaded. The error reported in "data/logs/registerProcessed" show the following log for one of the files that could not be inserted:
> 
> -------------------------------------------------------------------------------
> ==> Successfully connected to database
> Log file, 2019-08-29_19:08:17
> 
> 
> ==>Mapped DICOM parameters
>  -> using user-defined filterParameters for /xxx/w024/gvf_ISPC-stx152lsq6.mnc.gz
> 
> ==> Verifying acquisition center
>  - Center Name  : UNKN
>  - CenterID     : 0
>  -> Set ScannerID to 0.
> 
> ==> Data found for candidate   : 123456 - Visit: w024
>  -> Set SessionID to 28269.
>  -> Set SourceFileID to 49598.
>  -> Set AcquisitionProtocolID to 1013.
>  -> Set CoordinateSpace to stx152lsq6.
>  -> Set SourcePipeline to ConsensusGd.
>  -> Set PipelineDate to 2019-08-29.
>  -> Set OutputType to gvf.
>  -> Set md5hash to b877648ed0ef9a7458ad4931f4dbfd11.
> 
> ==> NeuroDB::File=HASH(0x136baa8) is not a unique file and will not be added to database.
> -------------------------------------------------------------------------------
> 
> I checked the md5hash for a previous uploaded file, which is different from the previous, but the same "md5hash" was calculated. See the following log:
> 
> -------------------------------------------------------------------------------
> 
> ==> Successfully connected to database
> Log file, 2019-08-19_10:11:53
> 
> 
> ==>Mapped DICOM parameters
>  -> using user-defined filterParameters for
> 
> ==> Verifying acquisition center
>  - Center Name  : UNKN
>  - CenterID     : 0
>  -> Set ScannerID to 0.
> 
> ==> Data found for candidate   : 123456 - Visit: baseline
>  -> Set SessionID to 28268.
>  -> Set SourceFileID to 49593.
>  -> Set AcquisitionProtocolID to 1013.
>  -> Set CoordinateSpace to stx152lsq6.
>  -> Set SourcePipeline to T2Vol.
>  -> Set PipelineDate to 2019-08-19.
>  -> Set OutputType to gvf.
>  -> Set md5hash to b877648ed0ef9a7458ad4931f4dbfd11.
> File /xxx/baseline/gvf_ISPC-stx152lsq6.mnc.gz
>  moved to:
>  /yyy/data/assembly/123456/baseline/mri/processed/T2Vol/IPMSA_123456_baseline_t1c_001_gvf_001.mnc.gz
> 
> ==> FAILED TO INSERT INTERMEDIARY FILES FOR 112945!
> 
> Making JIV
> 
>  ==> Registered /data_/ipmsa/loris_data/IPMSA/data/assembly/307024/baseline/mri/processed/T2Vol/IPMSA_307024_baseline_t1c_001_gvf_001.mnc.gz in database, given
> FileID: 112945
> -------------------------------------------------------------------------------
> 
> Here are the corresponding md5 for each file calculated using the command md5sum:
> /xxx/w024/gvf_ISPC-stx152lsq6.mnc.gz   7306010346c4614d8e338595b386ca5c
> /xxx/baseline/gvf_ISPC-stx152lsq6.mnc.gz 22107253f8b87c9918ac7f82fdc22c36
> 
> Could you please help me inserting those files which are different in size and md5 from previous uploaded files? The only similarity between the previous uploaded
> files and the ones that could not be uploaded is the filename.
> 
> Let me know if you need more information.
> 
> Regards,
> Alfredo.
> 
> The information in this e-mail is intended only for the person to whom it is
> addressed. If you believe this e-mail was sent to you in error and the e-mail
> contains patient information, please contact the Partners Compliance HelpLine at
> http://www.partners.org/complianceline . If the e-mail was sent to you in error
> but does not contain patient information, please contact the sender and properly
> dispose of the e-mail.
> 
> _______________________________________________
> Loris-dev mailing list
> Loris-dev at bic.mni.mcgill.ca
> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
> 
> 
> 
> --
> 
> christine.rogers at mcgill.ca
> McGill Centre for Integrative Neuroscience | MCIN.ca
> Montreal Neurological Institute
> McGill University | Montreal | Canada
> 
>

   ,-*~'`^`'~*-,._.,-*~'`^`'~*-,._
  '
:     istvan.morocz at mcgill.ca
  `    https://www.pistikem.org
  :      pistikem at gmail.com
  :        +1-514-927-1169
  ,      https://noisis.com
:     istvan.morocz at noisis.com
  .
   `~*-,._.,-*~'`^`'~*-,._.,-*~'`"