[Loris-dev] BIDS Error
Cecile Madjar
cecile.madjar at mcin.ca
Thu May 13 10:08:15 EDT 2021
Hi Alfredo,
The requirement of not having any special characters in the Participant ID
or Session ID comes from BIDS and the libraries used to parse the BIDS
dataset before insertion into LORIS.
On the LORIS front:
- the ID used next to sub-<ID> will be used for the PSCID field of the
candidate table (the CandID field being automatically generated as 6 random
digits by the insertion script).
- the ID used next to ses-<ID> will be used for the visit label field of
the session table
There might be another solution for you: modifying the file
https://github.com/aces/Loris-MRI/blob/main/python/lib/bids.json so that it
matches the string used for your data. You would need to modify the pattern
for subject and session in that file to match what you have. In theory,
once done, pybids should be able to read your dataset.
Hope this helps,
Cécile
On Thu, May 13, 2021 at 9:47 AM Morales Pinzon, Alfredo <
AMORALESPINZON at bwh.harvard.edu> wrote:
> Hi Cécile,
>
> Is it possible to modify your IDs to be alphanumerical?
>
>
> It is possible but puts more burden on our side as every single file now
> has to be modified. What would be best format for LORIS for the user ids
> (e.g., six digits) and the visit labels (e.g., alphanumerical without
> dash)? I just want to make sure we are compliant with LORIS requirements
> before modifying the bids files.
>
> Best,
> Alfredo.
>
> On May 11, 2021, at 6:34 PM, Cecile Madjar <cecile.madjar at mcin.ca> wrote:
>
> External Email - Use Caution
>
> Hi Alfredo,
>
> It looks like the pybids is not able to read your BIDS dataset, which is
> why it returns an empty list of subjects.
>
> I just tried on the python terminal to load your dataset on pybids and it
> does not find any subject or visit. This is probably due to the name of the
> subject that contains special characters. That will also be problematic for
> the session label of your dataset. I guess subjects in BIDS should remain
> alphanumerical only. Is it possible to modify your IDs to be alphanumerical?
>
> Cécile
>
> On Tue, May 11, 2021 at 5:37 PM Morales Pinzon, Alfredo <
> AMORALESPINZON at bwh.harvard.edu> wrote:
>
>> Hi Cécile,
>>
>> I just shared the dataset in the sftp in the following path:
>> 2021_05_11/caseCecile_BIDS.zip
>>
>> Additionally, here is the output of the script after adding the prints in
>> the PR 617:
>>
>> ===
>> /PROJECT/bin/mri/python_virtualenvs/loris-mri-python/lib/python3.6/site-packages/bids/layout/validation.py:46:
>> UserWarning: The ability to pass arguments to BIDSLayout that control
>> indexing is likely to be removed in future; possibly as early as PyBIDS
>> 0.14. This includes the `config_filename`, `ignore`, `force_index`, and
>> `index_metadata` arguments. The recommended usage pattern is to initialize
>> a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout
>> via the `indexer` argument.
>> warnings.warn("The ability to pass arguments to BIDSLayout that control
>> "
>>
>> Connecting to:
>> database: DATABASE
>> username: USERDATABASE
>> hostname: 111.222.333.44
>> port : 3306
>>
>>
>> Executing query:
>> SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM
>> ConfigSettings WHERE Name = %s);
>>
>> With arguments:
>> ('default_bids_vl',)
>>
>>
>> Executing query:
>> SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM
>> ConfigSettings WHERE Name = %s);
>>
>> With arguments:
>> ('dataDirBasepath',)
>>
>> Loading the BIDS dataset with BIDS layout library...
>> => BIDS dataset loaded with BIDS layout
>>
>> Grepping candidates from the BIDS layout...
>> Validating the list of participants...
>>
>> ERROR: Participant ID mismatch between participants.tsv and raw data
>> found in the BIDS directory/n
>> CFTY720D2306.0102.00001is missing from the BIDS Layout
>>
>> List of subjects parsed by the BIDS layout:
>> ===
>>
>> Best,
>> Alfredo.
>>
>> On May 11, 2021, at 12:01 PM, Cécile Madjar <cecile.madjar at mcin.ca>
>> wrote:
>>
>> External Email - Use Caution
>>
>> Sounds like a plan. Let me know once it is uploaded and I will ask
>> Nicolas to give me access.
>>
>> Thanks!
>>
>> PS: you can view the diff on the PR and simply copy the code over to your
>> files if easier
>>
>> On May 11, 2021, at 11:47 AM, Morales Pinzon, Alfredo <
>> AMORALESPINZON at BWH.HARVARD.EDU> wrote:
>>
>> Hi Cecile,
>>
>> In my testing dataset I have only one subject, I can share it with you in
>> the LORIS sftp, does that work for you?
>>
>> In the mean time I will diff and pull your PR, and then execute it. I’ll
>> send you the output asap.
>>
>> Best,
>> Alfredo.
>>
>> On May 11, 2021, at 11:36 AM, Cecile Madjar <cecile.madjar at mcin.ca>
>> wrote:
>>
>> External Email - Use Caution
>>
>> Hi Alfredo,
>>
>> I do not have access to your dataset but is it possible that there is one
>> or more sub-<ID> in the participants.tsv that are different from the
>> sub-<ID> used in the directory structure? (a.k.a., folders sub-<ID> and
>> filenames that should start with sub-<ID>.)
>>
>> This function is specifically cross checking the list provided in the
>> participants.tsv file and the information gathered by the BIDSLayout
>> library that read your BIDS dataset.
>>
>> I created a PR on LORIS-MRI to elaborate the log from the function you
>> just sent: https://github.com/aces/Loris-MRI/pull/617
>> <https://secure-web.cisco.com/17BHyJzFWfexg2et-D1qxjFw7-MsSpwCyZsKHDjuQa1Iruj-GV8fZRnaNOQe7rTfFaljPkHD9E0Kch6nwzSvbbe_48vUcKT3ozGDvxPfpe248W0Nml3EbqwefqTQo6V0gIBseIrXtJ4qwpUnrLVYpwZQyTPaDCbkm_NqrpLrBeXaHi-m5KCVluJqMYsb1DZBrqFL_XKJiIevFuiValC7eTehDRbTek9A5RvYsPN6hDTzEHFPQfCzzRD6W_iQSp80c_PbAKgxDUVLU2cePcP45pg/https%3A%2F%2Fgithub.com%2Faces%2FLoris-MRI%2Fpull%2F617> Maybe
>> try adding the print statement in the diff of the PR to your code and rerun
>> to get a better idea of what is going on?
>>
>> Let me know how it goes,
>>
>> Cécile
>>
>>
>> On May 10, 2021, at 4:15 PM, Morales Pinzon, Alfredo <
>> AMORALESPINZON at bwh.harvard.edu> wrote:
>>
>> Dear LorisDev team.
>>
>> I am running the BIDS uploading functionality of LORIS (from this PR
>> suggested by Cecile: https://github.com/aces/Loris-MRI/pull/558/files
>> <https://secure-web.cisco.com/1eWQtSLH8tHTpC1qsinKLnr4amf7Vdt1MLFNigX4jKdNz2mAoHV9ejLLcHZh1a0CUaUlzZMYTU9qzLaUEYYpyP1cl5UCR8e02WTmdMZUKXBNDeD8-cV2O5yTv8cgci7szP_MIGsvZ7FCjhqVmibPUnMFYbs6uYHGXZHOB5MsouWYLRHh89rJtQaFS8DOg4jfbwjp1bxwnO4FyuD3aKRv17dw9zoczaaXWnkbXXr87i6NQKYmA1NXhjM570IkyzSKuq-dlKj94vTZIlAChD5aFeg/https%3A%2F%2Fgithub.com%2Faces%2FLoris-MRI%2Fpull%2F558%2Ffiles> )
>> but I’m getting the following error (see all the output below):
>>
>> ===
>> ERROR: Participant ID mismatch between participants.tsv and raw data
>> found in the BIDS directory/n
>> ===
>>
>> After some debugging I saw that the array subjects in the following line
>> is empty:
>>
>> https://github.com/laemtl/Loris-MRI/blob/3742522fa46b0ee2b9e384e30650a0c1f425c7af/python/lib/bidsreader.py#L134
>> <https://secure-web.cisco.com/1sAIMc1N1U7sTpJYVEGfljx8iP9AUjxof1GeCQwNPKhd82OAXEZmdHoYu7n3Uq8jLaO-quBK1igoIw9Oo7d5JVNCfb3MNy6u-cvR2KkL-rfKwSOW_ovP_V5YKydvDMoE1ltw4MO29keFq-JiTymIk8Wm8TbXeHKxF2njboK6-6bctzRI-5K2tWwz82zUZyzDxyFdn4zXPqI8AihHeuWQQG3GOGgPSbRP-GOUfwbu2eVmvL_P_ldZetO1qjKowic0Gqnt-a9XN_8GORD5BLNlxTw/https%3A%2F%2Fgithub.com%2Flaemtl%2FLoris-MRI%2Fblob%2F3742522fa46b0ee2b9e384e30650a0c1f425c7af%2Fpython%2Flib%2Fbidsreader.py%23L134>
>>
>> I thought the issue was that we have symbolic links in our folder
>> structure but after setting a testing example I am getting the same exact
>> error.
>>
>> Any ideas why this is happening? The error does not help much. I’m happy
>> to share the testing case, with all the BIDS structure to a LORIS
>> developer, if this is helpful.
>>
>> Looking forward to hearing from you.
>>
>> Best.
>> Alfredo.
>>
>> ===
>> /PROJECT/bin/mri/python_virtualenvs/loris-mri-python/lib/python3.6/site-packages/bids/layout/validation.py:46:
>> UserWarning: The ability to pass arguments to BIDSLayout that control
>> indexing is likely to be removed in future; possibly as early as PyBIDS
>> 0.14. This includes the `config_filename`, `ignore`, `force_index`, and
>> `index_metadata` arguments. The recommended usage pattern is to initialize
>> a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout
>> via the `indexer` argument.
>> warnings.warn("The ability to pass arguments to BIDSLayout that control
>> "
>>
>> Connecting to:
>> database: DATABASE
>> username: USERDATABASE
>> hostname: 111.222.333.444
>> port : 3306
>>
>>
>> Executing query:
>> SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM
>> ConfigSettings WHERE Name = %s);
>>
>> With arguments:
>> ('default_bids_vl',)
>>
>>
>> Executing query:
>> SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM
>> ConfigSettings WHERE Name = %s);
>>
>> With arguments:
>> ('dataDirBasepath',)
>>
>> Loading the BIDS dataset with BIDS layout library...
>>
>> => BIDS dataset loaded with BIDS layout
>>
>> Grepping candidates from the BIDS layout...
>> Validating the list of participants...
>>
>> ERROR: Participant ID mismatch between participants.tsv and raw data
>> found in the BIDS directory/n
>> ===
>>
>> The information in this e-mail is intended only for the person to whom it
>> is addressed. If you believe this e-mail was sent to you in error and the
>> e-mail contains patient information, please contact the Mass General
>> Brigham Compliance HelpLine at
>> http://www.massgeneralbrigham.org/complianceline
>> <http://secure-web.cisco.com/1qM_d-YnL441NX1CgLDc0-4p5P74OBrZe4qz8DQn4NKFJlMAA7HiKjibIOLkeFbmot5sDXcFfxupjvo_jX8vC9aYNvFDnADUwQFAF2ETaANXw3yIZaf8bb7_SvUXAkTxwK31eivXt_HlY2DxnGsZ739ls6VersNRKPQN2plxFNbqAzR65ZCFC8YSBrs5Vrcr1GWNsx7YN7RWdCvcDN1j8l1v9uD8Je4NlxsZBp9G86xq4w2ATmf4ENiL-6RShyNpajtkjZKD6TUD0FKl1yu9EdA/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline>
>> . If the e-mail was sent to you in error but does not contain patient
>> information, please contact the sender and properly dispose of the e-mail.
>>
>> Please note that this e-mail is not secure (encrypted). If you do not
>> wish to continue communication over unencrypted e-mail, please notify the
>> sender of this message immediately. Continuing to send or respond to
>> e-mail after receiving this message means you understand and accept this
>> risk and wish to continue to communicate over unencrypted e-mail.
>> _______________________________________________
>> Loris-dev mailing list
>> Loris-dev at bic.mni.mcgill.ca
>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>> <https://secure-web.cisco.com/1nZGLPRvHAc4QXkbuPkDbTk6X7cLRpZjuKhOBth4NO62r205kfGQbnAX-YD9BjpSJaYX2q-XFXkB_JSF0s5z2ahDn5jCwO30vrTUlIZlEANwK8y3vIbbTS3OKeQl5HWbDL3FviFQTAC2JkKwXTysKFx31cF_fdj3Ldph7SeZSr25P3BOdkNzQM7CwOgGsZ74Icj-bh17DcnTl_TpfPBvtKvWdT9JJgzhO4xgU1diPaWsF1-yCLwXh4xN8u993AJYONV8vaXR3KRoKV8-QxssqZw/https%3A%2F%2Fmailman.bic.mni.mcgill.ca%2Fmailman%2Flistinfo%2Floris-dev>
>>
>>
>> The information in this e-mail is intended only for the person to whom it
>> is addressed. If you believe this e-mail was sent to you in error and the
>> e-mail contains patient information, please contact the Mass General
>> Brigham Compliance HelpLine at
>> http://www.massgeneralbrigham.org/complianceline
>> <http://secure-web.cisco.com/1AhAU1QrkHn5uAqLbnmi1HyXx_kpI9yR6ZBTDEz2E2EH9ZJ8W5lI94E700CC6tH7ISW-FmwwtetLtDvUs671_9WDXfhJBJZUlAUiC75p9k2iniolB83RMn1vpjTql7mdvGqhQFlpZCisj05nFxy4_7JaawBv-2YfBcGJkgmL4k4UTe3OZ3Q6CU-jPPmqEDNyFt7jVXNncRMqUUg9ZiUWmhABUAHol2wVTYpBugXYIZfYMj79LvUy54oxCyOtX5FCW00-m4IAVrvYB_EhkHxI5Qw/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline>
>> . If the e-mail was sent to you in error but does not contain patient
>> information, please contact the sender and properly dispose of the e-mail.
>>
>> Please note that this e-mail is not secure (encrypted). If you do not
>> wish to continue communication over unencrypted e-mail, please notify the
>> sender of this message immediately. Continuing to send or respond to
>> e-mail after receiving this message means you understand and accept this
>> risk and wish to continue to communicate over unencrypted e-mail.
>>
>>
>>
>> The information in this e-mail is intended only for the person to whom it
>> is addressed. If you believe this e-mail was sent to you in error and the
>> e-mail contains patient information, please contact the Mass General
>> Brigham Compliance HelpLine at
>> http://www.massgeneralbrigham.org/complianceline
>> <http://secure-web.cisco.com/1Vs76oaBwBuqLoMP4tDeksrqcCgDkogMEbBScJ4ejcwSa6gekqJpFoL6YNIznACwU31BdOoVLyr1lD6sf6FJ0bhDvv_VDNM4ihqPNDRpk_MBie2PPM8xVjhZzqKoCHhcAwaJVO6JrBb8QVg0k78kxRXH7d0wKdNPAHrs292D3K2KcEskKWx2YgQ-McThsXUDOsdj4Y99wDiDj5fzeT-glwFo2ZGa6-7cdKO2jHZVoiG8_XlvkoA76TtLIZxN04JfUn5UFQZ0p3J-jLf5RR8u_7A/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline>
>> . If the e-mail was sent to you in error but does not contain patient
>> information, please contact the sender and properly dispose of the e-mail.
>>
>> Please note that this e-mail is not secure (encrypted). If you do not
>> wish to continue communication over unencrypted e-mail, please notify the
>> sender of this message immediately. Continuing to send or respond to
>> e-mail after receiving this message means you understand and accept this
>> risk and wish to continue to communicate over unencrypted e-mail.
>>
>
> The information in this e-mail is intended only for the person to whom it
> is addressed. If you believe this e-mail was sent to you in error and the
> e-mail contains patient information, please contact the Mass General
> Brigham Compliance HelpLine at
> http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent
> to you in error but does not contain patient information, please contact
> the sender and properly dispose of the e-mail.
>
> Please note that this e-mail is not secure (encrypted). If you do not
> wish to continue communication over unencrypted e-mail, please notify the
> sender of this message immediately. Continuing to send or respond to
> e-mail after receiving this message means you understand and accept this
> risk and wish to continue to communicate over unencrypted e-mail.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210513/2eb91100/attachment-0001.html>
More information about the Loris-dev
mailing list