From rogers at bic.mni.mcgill.ca  Tue Jun  1 13:35:54 2021
From: rogers at bic.mni.mcgill.ca (Christine Rogers)
Date: Tue, 1 Jun 2021 13:35:54 -0400
Subject: [Loris-dev] Can't assign users to a new project
In-Reply-To: <CAN9cDwFFLCe+Ukxy-kqVoGr0PV+rmu=jpfqm4CB3_-+X_9qxkA@mail.gmail.com>
References: <mailman.2.1622217601.9183.loris-dev@bic.mni.mcgill.ca>
 <CAKcpJh443XPvrCGdfuubhjOq003XFY0hYJkyFgXOpbddC2PzuA@mail.gmail.com>
 <CAN9cDwFFLCe+Ukxy-kqVoGr0PV+rmu=jpfqm4CB3_-+X_9qxkA@mail.gmail.com>
Message-ID: <CALbt7NAAJ5HBwj=Fq25KZusFv+7PHJYDF5R+aKY1bQryxWd2=Q@mail.gmail.com>

Hi Alfredo, Eli, et al -

Thanks for your input and feedback on this --
After discussion in the LORIS meeting this morning, we've added this to
LORIS issues for future roadmap discussion:
https://github.com/aces/Loris/issues/7459
Please don't hesitate to let us know your thoughts in the issue.

Thanks,
Christine

the LORIS team

On Mon, May 31, 2021 at 8:47 AM Dave MacFarlane <dave.macfarlane at mcin.ca>
wrote:

> I don't believe it was ever automatically done. It was likely done as part
> of a one-time upgrade script when the relation was introduced, but since
> it's an SQL table there would need to be a time for the insert to be run.
>
>  - Dave
>
> On Fri, May 28, 2021 at 3:45 PM Eli Johnson <joh17964 at umn.edu> wrote:
>
>> Hi Alfredo,
>>
>> You can manually add the user to a project in the user_project_rel table.
>> Once you add the relationship for the admin account you will see it appear
>> as an option in the project selection box of the user_accounts module.
>>
>> Dev team, can new projects be automatically added to the Superuser admin
>> account? I have been manually adding new ones so they appear in the
>> user_accounts project selection boxes. I believe this was automatically
>> done before the v23 update.
>>
>> Best,
>> Eli
>>
>> Eli Johnson
>> Developer/Data Manager
>> Institute of Child Development
>> University of Minnesota
>> joh17964 at umn.edu
>>
>>
>> On Fri, May 28, 2021 at 11:00 AM <loris-dev-request at bic.mni.mcgill.ca>
>> wrote:
>>
>>> Send Loris-dev mailing list submissions to
>>>         loris-dev at bic.mni.mcgill.ca
>>>
>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>         https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>> or, via email, send a message with subject or body 'help' to
>>>         loris-dev-request at bic.mni.mcgill.ca
>>>
>>> You can reach the person managing the list at
>>>         loris-dev-owner at bic.mni.mcgill.ca
>>>
>>> When replying, please edit your Subject line so it is more specific
>>> than "Re: Contents of Loris-dev digest..."
>>> Today's Topics:
>>>
>>>    1. Can't assign users to a new project (Morales Pinzon, Alfredo)
>>>
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: "Morales Pinzon, Alfredo" <AMORALESPINZON at bwh.harvard.edu>
>>> To: "loris-dev at bic.mni.mcgill.ca" <loris-dev at bic.mni.mcgill.ca>
>>> Cc: Rozie Arnaoutelis <rozie.arnaoutelis at mcgill.ca>, Sridar Narayanan <
>>> sridar.narayanan at mcgill.ca>, "Douglas Arnold, Dr." <
>>> douglas.arnold at mcgill.ca>, "Guttmann, Charles, M.D." <
>>> guttmann at bwh.harvard.edu>
>>> Bcc:
>>> Date: Thu, 27 May 2021 20:59:52 +0000
>>> Subject: [Loris-dev] Can't assign users to a new project
>>> Dear LorisDev team,
>>>
>>> I just created a new project using the web interface of LORIS but I
>>> cannot assign users to that new project because it is not listed in the
>>> project list in the edit user section. I can see all the previous projects
>>> except the new one.
>>>
>>> Any ideas why this might be happening? Is there something in the
>>> database I should double check?
>>>
>>> I am running LORIS v23.0.2.
>>>
>>> Best,
>>> Alfredo.
>>> The information in this e-mail is intended only for the person to whom
>>> it is addressed. If you believe this e-mail was sent to you in error and
>>> the e-mail contains patient information, please contact the Mass General
>>> Brigham Compliance HelpLine at
>>> http://www.massgeneralbrigham.org/complianceline . If the e-mail was
>>> sent to you in error but does not contain patient information, please
>>> contact the sender and properly dispose of the e-mail.
>>> Please note that this e-mail is not secure (encrypted).  If you do not
>>> wish to continue communication over unencrypted e-mail, please notify the
>>> sender of this message immediately.  Continuing to send or respond to
>>> e-mail after receiving this message means you understand and accept this
>>> risk and wish to continue to communicate over unencrypted e-mail.
>>>
>>>
>>>
>>> _______________________________________________
>>> Loris-dev mailing list
>>> Loris-dev at bic.mni.mcgill.ca
>>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>>
>> _______________________________________________
>> Loris-dev mailing list
>> Loris-dev at bic.mni.mcgill.ca
>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>
> _______________________________________________
> Loris-dev mailing list
> Loris-dev at bic.mni.mcgill.ca
> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>


-- 
Research Project Officer
christine.rogers at mcgill.ca
McGill Centre for Integrative Neuroscience | MCIN.ca
Montreal Neurological Institute-Hospital
McGill University | Montreal | Canada
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210601/1576ad2b/attachment.html>

From AMORALESPINZON at BWH.HARVARD.EDU  Sun Jun  6 16:48:47 2021
From: AMORALESPINZON at BWH.HARVARD.EDU (Morales Pinzon, Alfredo)
Date: Sun, 6 Jun 2021 20:48:47 +0000
Subject: [Loris-dev] BIDS Error
In-Reply-To: <CAMkiY3cMR4CRYE1Y=u5TGwpOsPk7Q3UdXMmit4V7RR3q8jqpeg@mail.gmail.com>
References: <8E375877-D896-4B0E-B0F7-D6BD5565CA67@bwh.harvard.edu>
 <ADFA6F92-C7E7-4FD5-85A1-4F20D1565F75@gmail.com>
 <4286E305-87C3-492A-BE31-0B83CEFD365E@bwh.harvard.edu>
 <5F1BBE56-5523-4517-ADCF-2A1DB17C952B@mcin.ca>
 <F1F354D5-8CDF-4089-A4BA-6A3A41903CDA@bwh.harvard.edu>
 <CAMkiY3cXNfprNL_VDnqFzH1og332_bF2u1Zhe+enRA0dGBpWHQ@mail.gmail.com>
 <CE0C1F1D-862F-4030-BA2D-401BE3110632@bwh.harvard.edu>
 <CAMkiY3cMR4CRYE1Y=u5TGwpOsPk7Q3UdXMmit4V7RR3q8jqpeg@mail.gmail.com>
Message-ID: <BCBD10CB-5400-4FF3-949E-5641665587E3@bwh.harvard.edu>

Hi C?cile,

We managed to insert the candidates and the visits, and create the specific file names for BIDS before actually launching the BIDS script. The script finally worked, however I don?t see the images in LORIS. I am adding down below the bids log insertion. Is there something additional I need to do to be able to see the images? In the web site I see the candidate and the visit but the column ?Imaging Scan Done? is marked with a question mark (see attachment).

===
/PROJECT/bin/mri/python_virtualenvs/loris-mri-python/lib/python3.6/site-packages/bids/layout/validation.py:46: UserWarning: The ability to pass arguments to BIDSLayout that control indexing is likely to be removed in future; possibly as early as PyBIDS 0.14. This includes the `config_filename`, `ignore`, `force_index`, and `index_metadata` arguments. The recommended usage pattern is to initialize a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout via the `indexer` argument.
  warnings.warn("The ability to pass arguments to BIDSLayout that control "

Connecting to:
        database: DATABASE
        username: USERNAME
        hostname: 111.222.333440
        port    : 1234


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('default_bids_vl',)


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('dataDirBasepath',)

AMP::bids_dir: /DATA/CAND_A/bids
Loading the BIDS dataset with BIDS layout library...

AMP::bids_config: /PROJECT/bin/mri/python/lib/bids.json
root: /DATA/CAND_A/bids
target:: /DATA/CAND_A/bids/dataset_description.json
        => BIDS dataset loaded with BIDS layout

Grepping candidates from the BIDS layout...
Validating the list of participants...
subjects: ['207019']
        => Passed validation of the list of participants

        => List of participants found:
                207019


Grepping list of sessions from the BIDS layout...
        => List of sessions found:

                207019: screening


Grepping the different modalities from the BIDS layout...
        => Done grepping the different modalities from the BIDS layout


Executing query:
        SELECT * FROM candidate WHERE PSCID = %s

With arguments:
        ('207019',)
Executing query:
        SELECT SubprojectID FROM subproject WHERE title = %s

With arguments:
        ['Experimental']


Executing query:
        SELECT * FROM session WHERE CandID = %s AND Visit_label = %s

With arguments:
        ('622350', 'screening')


Executing query:
        SELECT * FROM candidate WHERE PSCID = %s

With arguments:
        ('207019',)


Executing query:
        SELECT SubprojectID FROM subproject WHERE title = %s

With arguments:
        ['Experimental']


Executing query:
        SELECT * FROM session WHERE CandID = %s AND Visit_label = %s

With arguments:
        ('622350', 'screening')


Executing query:
        SELECT type FROM ImagingFileTypes


Executing query:
        SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s

With arguments:
        ('248248fb303a762548e0be602b0b6bd6c6f33b4c860898c69faa8273be7a80f6d93b6413ec3bbeaadf22005fac04b07e9c08487e7b0a6690f64c2781d77d69b4',)


Executing query:
        SELECT type FROM ImagingFileTypes


Executing query:
        SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s

With arguments:
        ('b16bc029ea1cb8271a0728785d04eddaaa66e4d89c09e990552667b3c5598d17cee045b5b4ee3480dc2b8e08b5b05ca4e53c57f50b13b26f6467b9e4f455d09e',)


Executing query:
        SELECT type FROM ImagingFileTypes


Executing query:
        SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s

With arguments:
        ('44346136db049d0faf867a34d57d5714803a320b070013da0e1cf66e064f4dc9db92533f8aa8caaafcb78ce141c5444c5b8f8dc6b1d1c7dac530a4c2375eedcd',)


Executing query:
        SELECT type FROM ImagingFileTypes


Executing query:
        SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s

With arguments:
        ('ee91658a67d7d96007f6f0c625cceb700c9fa5bbecf34cea3cae8f1445a6efce5f735b2ab68d26717a6aaa418808b3d8fb30b03e03ae1982b0eeed834429d081',)


Executing query:
        SELECT type FROM ImagingFileTypes


Executing query:
        SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s

With arguments:
        ('749c00e5aa64583c91f29672b228b9d89e691cef7d8f3380eefbc1fee466bd3d987fd7178001f0e4f63684a50e139a7bb8ed197981d1588ffc4dfc4eed8278b8',)
===

Thank you for your help.

Best,
Alfredo.

[cid:5D3A3868-20A3-45CC-B1D8-FE1F6FAADE66]

On May 13, 2021, at 10:08 AM, Cecile Madjar <cecile.madjar at mcin.ca<mailto:cecile.madjar at mcin.ca>> wrote:


        External Email - Use Caution

Hi Alfredo,

The requirement of not having any special characters in the Participant ID or Session ID comes from BIDS and the libraries used to parse the BIDS dataset before insertion into LORIS.

On the LORIS front:

  *   the ID used next to sub-<ID> will be used for the PSCID field of the candidate table (the CandID field being automatically generated as 6 random digits by the insertion script).
  *   the ID used next to ses-<ID> will be used for the visit label field of the session table

There might be another solution for you: modifying the file https://github.com/aces/Loris-MRI/blob/main/python/lib/bids.json<https://secure-web.cisco.com/1GvLDW-g8RekA6xfXzTQGS9x-cvm3VJ64Me4gEy4E2Yh3_J_-W9ZJiP9DEv9M9oH_JN3CJGEaRyJUvOCUzrNqMXl0tpX8TJ_Cfrs5JA9p_ICYgDGAU_HcPRpK0aHBkRHCW9mHZTixcIoY9EEim1Tt4JhQWyQlJpaTKY-bd1bc5O1lxwHz6sj4nGQ5bXFiDLYRP4Y8DWJzxYpm8EMaj_QYRiVdUOwHiSGfVGMouhDDMYYwBHrEEdV1Ompf5QRyVkSSAJaGL2Q3tSABSa8USdVkOQ/https%3A%2F%2Fgithub.com%2Faces%2FLoris-MRI%2Fblob%2Fmain%2Fpython%2Flib%2Fbids.json> so that it matches the string used for your data. You would need to modify the pattern for subject and session in that file to match what you have. In theory, once done, pybids should be able to read your dataset.

Hope this helps,

C?cile

On Thu, May 13, 2021 at 9:47 AM Morales Pinzon, Alfredo <AMORALESPINZON at bwh.harvard.edu<mailto:AMORALESPINZON at bwh.harvard.edu>> wrote:
Hi C?cile,

Is it possible to modify your IDs to be alphanumerical?

It is possible but puts more burden on our side as every single file now has to be modified. What would be best format for LORIS for the user ids (e.g., six digits) and the visit labels (e.g., alphanumerical without dash)? I just want to make sure we are compliant with LORIS requirements before modifying the bids files.

Best,
Alfredo.

On May 11, 2021, at 6:34 PM, Cecile Madjar <cecile.madjar at mcin.ca<mailto:cecile.madjar at mcin.ca>> wrote:


        External Email - Use Caution

Hi Alfredo,

It looks like the pybids is not able to read your BIDS dataset, which is why it returns an empty list of subjects.

I just tried on the python terminal to load your dataset on pybids and it does not find any subject or visit. This is probably due to the name of the subject that contains special characters. That will also be problematic for the session label of your dataset. I guess subjects in BIDS should remain alphanumerical only. Is it possible to modify your IDs to be alphanumerical?

C?cile

On Tue, May 11, 2021 at 5:37 PM Morales Pinzon, Alfredo <AMORALESPINZON at bwh.harvard.edu<mailto:AMORALESPINZON at bwh.harvard.edu>> wrote:
Hi C?cile,

I just shared the dataset in the sftp in the following path: 2021_05_11/caseCecile_BIDS.zip

Additionally, here is the output of the script after adding the prints in the PR 617:

===
/PROJECT/bin/mri/python_virtualenvs/loris-mri-python/lib/python3.6/site-packages/bids/layout/validation.py:46: UserWarning: The ability to pass arguments to BIDSLayout that control indexing is likely to be removed in future; possibly as early as PyBIDS 0.14. This includes the `config_filename`, `ignore`, `force_index`, and `index_metadata` arguments. The recommended usage pattern is to initialize a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout via the `indexer` argument.
  warnings.warn("The ability to pass arguments to BIDSLayout that control "

Connecting to:
        database: DATABASE
        username: USERDATABASE
        hostname: 111.222.333.44
        port    : 3306


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('default_bids_vl',)


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('dataDirBasepath',)

Loading the BIDS dataset with BIDS layout library...
        => BIDS dataset loaded with BIDS layout

Grepping candidates from the BIDS layout...
Validating the list of participants...

ERROR: Participant ID mismatch between participants.tsv and raw data found in the BIDS directory/n
CFTY720D2306.0102.00001is missing from the BIDS Layout

List of subjects parsed by the BIDS layout:
===

Best,
Alfredo.

On May 11, 2021, at 12:01 PM, C?cile Madjar <cecile.madjar at mcin.ca<mailto:cecile.madjar at mcin.ca>> wrote:


        External Email - Use Caution

Sounds like a plan. Let me know once it is uploaded and I will ask Nicolas to give me access.

Thanks!

PS: you can view the diff on the PR and simply copy the code over to your files if easier

On May 11, 2021, at 11:47 AM, Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU<mailto:AMORALESPINZON at BWH.HARVARD.EDU>> wrote:

Hi Cecile,

In my testing dataset I have only one subject, I can share it with you in the LORIS sftp, does that work for you?

In the mean time I will diff and pull your PR, and then execute it. I?ll send you the output asap.

Best,
Alfredo.

On May 11, 2021, at 11:36 AM, Cecile Madjar <cecile.madjar at mcin.ca<mailto:cecile.madjar at mcin.ca>> wrote:


        External Email - Use Caution

Hi Alfredo,

I do not have access to your dataset but is it possible that there is one or more sub-<ID> in the participants.tsv that are different from the sub-<ID> used in the directory structure? (a.k.a., folders sub-<ID> and filenames that should start with sub-<ID>.)

This function is specifically cross checking the list provided in the participants.tsv file and the information gathered by the BIDSLayout library that read your BIDS dataset.

I created a PR on LORIS-MRI to elaborate the log from the function you just sent: https://github.com/aces/Loris-MRI/pull/617<https://secure-web.cisco.com/17BHyJzFWfexg2et-D1qxjFw7-MsSpwCyZsKHDjuQa1Iruj-GV8fZRnaNOQe7rTfFaljPkHD9E0Kch6nwzSvbbe_48vUcKT3ozGDvxPfpe248W0Nml3EbqwefqTQo6V0gIBseIrXtJ4qwpUnrLVYpwZQyTPaDCbkm_NqrpLrBeXaHi-m5KCVluJqMYsb1DZBrqFL_XKJiIevFuiValC7eTehDRbTek9A5RvYsPN6hDTzEHFPQfCzzRD6W_iQSp80c_PbAKgxDUVLU2cePcP45pg/https%3A%2F%2Fgithub.com%2Faces%2FLoris-MRI%2Fpull%2F617> Maybe try adding the print statement in the diff of the PR to your code and rerun to get a better idea of what is going on?

Let me know how it goes,

C?cile


On May 10, 2021, at 4:15 PM, Morales Pinzon, Alfredo <AMORALESPINZON at bwh.harvard.edu<mailto:AMORALESPINZON at bwh.harvard.edu>> wrote:

Dear LorisDev team.

I am running the BIDS uploading functionality of LORIS (from this PR suggested by Cecile: https://github.com/aces/Loris-MRI/pull/558/files<https://secure-web.cisco.com/1eWQtSLH8tHTpC1qsinKLnr4amf7Vdt1MLFNigX4jKdNz2mAoHV9ejLLcHZh1a0CUaUlzZMYTU9qzLaUEYYpyP1cl5UCR8e02WTmdMZUKXBNDeD8-cV2O5yTv8cgci7szP_MIGsvZ7FCjhqVmibPUnMFYbs6uYHGXZHOB5MsouWYLRHh89rJtQaFS8DOg4jfbwjp1bxwnO4FyuD3aKRv17dw9zoczaaXWnkbXXr87i6NQKYmA1NXhjM570IkyzSKuq-dlKj94vTZIlAChD5aFeg/https%3A%2F%2Fgithub.com%2Faces%2FLoris-MRI%2Fpull%2F558%2Ffiles> ) but I?m getting the following error (see all the output below):

===
ERROR: Participant ID mismatch between participants.tsv and raw data found in the BIDS directory/n
===

After some debugging I saw that the array subjects in the following line is empty:
https://github.com/laemtl/Loris-MRI/blob/3742522fa46b0ee2b9e384e30650a0c1f425c7af/python/lib/bidsreader.py#L134<https://secure-web.cisco.com/1sAIMc1N1U7sTpJYVEGfljx8iP9AUjxof1GeCQwNPKhd82OAXEZmdHoYu7n3Uq8jLaO-quBK1igoIw9Oo7d5JVNCfb3MNy6u-cvR2KkL-rfKwSOW_ovP_V5YKydvDMoE1ltw4MO29keFq-JiTymIk8Wm8TbXeHKxF2njboK6-6bctzRI-5K2tWwz82zUZyzDxyFdn4zXPqI8AihHeuWQQG3GOGgPSbRP-GOUfwbu2eVmvL_P_ldZetO1qjKowic0Gqnt-a9XN_8GORD5BLNlxTw/https%3A%2F%2Fgithub.com%2Flaemtl%2FLoris-MRI%2Fblob%2F3742522fa46b0ee2b9e384e30650a0c1f425c7af%2Fpython%2Flib%2Fbidsreader.py%23L134>

I thought the issue was that we have symbolic links in our folder structure but after setting a testing example I am getting the same exact error.

Any ideas why this is happening? The error does not help much. I?m happy to share the testing case, with all the BIDS structure to a LORIS developer, if this is helpful.

Looking forward to hearing from you.

Best.
Alfredo.

===
/PROJECT/bin/mri/python_virtualenvs/loris-mri-python/lib/python3.6/site-packages/bids/layout/validation.py:46: UserWarning: The ability to pass arguments to BIDSLayout that control indexing is likely to be removed in future; possibly as early as PyBIDS 0.14. This includes the `config_filename`, `ignore`, `force_index`, and `index_metadata` arguments. The recommended usage pattern is to initialize a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout via the `indexer` argument.
  warnings.warn("The ability to pass arguments to BIDSLayout that control "

Connecting to:
        database: DATABASE
        username: USERDATABASE
        hostname: 111.222.333.444
        port    : 3306


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('default_bids_vl',)


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('dataDirBasepath',)

Loading the BIDS dataset with BIDS layout library...

        => BIDS dataset loaded with BIDS layout

Grepping candidates from the BIDS layout...
Validating the list of participants...

ERROR: Participant ID mismatch between participants.tsv and raw data found in the BIDS directory/n
===

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/1qM_d-YnL441NX1CgLDc0-4p5P74OBrZe4qz8DQn4NKFJlMAA7HiKjibIOLkeFbmot5sDXcFfxupjvo_jX8vC9aYNvFDnADUwQFAF2ETaANXw3yIZaf8bb7_SvUXAkTxwK31eivXt_HlY2DxnGsZ739ls6VersNRKPQN2plxFNbqAzR65ZCFC8YSBrs5Vrcr1GWNsx7YN7RWdCvcDN1j8l1v9uD8Je4NlxsZBp9G86xq4w2ATmf4ENiL-6RShyNpajtkjZKD6TUD0FKl1yu9EdA/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
_______________________________________________
Loris-dev mailing list
Loris-dev at bic.mni.mcgill.ca<mailto:Loris-dev at bic.mni.mcgill.ca>
https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev<https://secure-web.cisco.com/1nZGLPRvHAc4QXkbuPkDbTk6X7cLRpZjuKhOBth4NO62r205kfGQbnAX-YD9BjpSJaYX2q-XFXkB_JSF0s5z2ahDn5jCwO30vrTUlIZlEANwK8y3vIbbTS3OKeQl5HWbDL3FviFQTAC2JkKwXTysKFx31cF_fdj3Ldph7SeZSr25P3BOdkNzQM7CwOgGsZ74Icj-bh17DcnTl_TpfPBvtKvWdT9JJgzhO4xgU1diPaWsF1-yCLwXh4xN8u993AJYONV8vaXR3KRoKV8-QxssqZw/https%3A%2F%2Fmailman.bic.mni.mcgill.ca%2Fmailman%2Flistinfo%2Floris-dev>

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/1AhAU1QrkHn5uAqLbnmi1HyXx_kpI9yR6ZBTDEz2E2EH9ZJ8W5lI94E700CC6tH7ISW-FmwwtetLtDvUs671_9WDXfhJBJZUlAUiC75p9k2iniolB83RMn1vpjTql7mdvGqhQFlpZCisj05nFxy4_7JaawBv-2YfBcGJkgmL4k4UTe3OZ3Q6CU-jPPmqEDNyFt7jVXNncRMqUUg9ZiUWmhABUAHol2wVTYpBugXYIZfYMj79LvUy54oxCyOtX5FCW00-m4IAVrvYB_EhkHxI5Qw/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.


The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/1Vs76oaBwBuqLoMP4tDeksrqcCgDkogMEbBScJ4ejcwSa6gekqJpFoL6YNIznACwU31BdOoVLyr1lD6sf6FJ0bhDvv_VDNM4ihqPNDRpk_MBie2PPM8xVjhZzqKoCHhcAwaJVO6JrBb8QVg0k78kxRXH7d0wKdNPAHrs292D3K2KcEskKWx2YgQ-McThsXUDOsdj4Y99wDiDj5fzeT-glwFo2ZGa6-7cdKO2jHZVoiG8_XlvkoA76TtLIZxN04JfUn5UFQZ0p3J-jLf5RR8u_7A/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/1n2UbpIP1Hiy1wedtWg9s-6NXBNM5GXQsk4yul5KQtRtD-fpCcwhqxfc9xPmkoXTkvQiEilB7x5cMjoMnEfP7sVBe6N4BZFTL6lnTA5ZoU6_eL-QZrxz4bet8uKLeH8lFWIbsg3Tq1FqZpdjEzQ9aSVMRC-CNrE0-CJn7xfPqJefF9I1-_5OZcKb9QkXl2Rg_xjO8CfyirontcnUgEOHlJ5anCG65dp3TNqKwXi_HsVUusIi9c6KJxpEoiY-xH-7k_UAsbpirJkzRosIoJNyevA/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210606/c9297ab5/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screen Shot 2021-06-06 at 4.44.57 PM.png
Type: image/png
Size: 90041 bytes
Desc: Screen Shot 2021-06-06 at 4.44.57 PM.png
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210606/c9297ab5/attachment-0001.png>

From cecile.madjar at mcin.ca  Mon Jun  7 09:55:02 2021
From: cecile.madjar at mcin.ca (Cecile Madjar)
Date: Mon, 7 Jun 2021 09:55:02 -0400
Subject: [Loris-dev] BIDS Error
In-Reply-To: <BCBD10CB-5400-4FF3-949E-5641665587E3@bwh.harvard.edu>
References: <8E375877-D896-4B0E-B0F7-D6BD5565CA67@bwh.harvard.edu>
 <ADFA6F92-C7E7-4FD5-85A1-4F20D1565F75@gmail.com>
 <4286E305-87C3-492A-BE31-0B83CEFD365E@bwh.harvard.edu>
 <5F1BBE56-5523-4517-ADCF-2A1DB17C952B@mcin.ca>
 <F1F354D5-8CDF-4089-A4BA-6A3A41903CDA@bwh.harvard.edu>
 <CAMkiY3cXNfprNL_VDnqFzH1og332_bF2u1Zhe+enRA0dGBpWHQ@mail.gmail.com>
 <CE0C1F1D-862F-4030-BA2D-401BE3110632@bwh.harvard.edu>
 <CAMkiY3cMR4CRYE1Y=u5TGwpOsPk7Q3UdXMmit4V7RR3q8jqpeg@mail.gmail.com>
 <BCBD10CB-5400-4FF3-949E-5641665587E3@bwh.harvard.edu>
Message-ID: <CAMkiY3fK3q=Ah1ocuAWS8=fh+hFxKR+TuN15WcMTQbU0Xvq6MA@mail.gmail.com>

Hi Alfredo,

it is possible that some fields in the session are not populated and that
is why you do not see the images. Can you run the following query on one
session that was created by the BIDS import script?

SELECT CandID, Visit_label, Active, Scan_done, CenterID, ProjectID,
SubprojectID FROM session WHERE Visit_label = 'screening';   # (I make the
assumption that screening is the MRI visit that was inserted here)

Let me know what you get from that query. My guess is that the Active field
is NULL and should be set to 'Y' for you to be able to see it in the
imaging browser.

C?cile

On Sun, Jun 6, 2021 at 4:48 PM Morales Pinzon, Alfredo <
AMORALESPINZON at bwh.harvard.edu> wrote:

> Hi C?cile,
>
> We managed to insert the candidates and the visits, and create the
> specific file names for BIDS before actually launching the BIDS script. The
> script finally worked, however I don?t see the images in LORIS. I am adding
> down below the bids log insertion. Is there something additional I need to
> do to be able to see the images? In the web site I see the candidate and
> the visit but the column ?Imaging Scan Done? is marked with a question mark
> (see attachment).
>
> ===
> /PROJECT/bin/mri/python_virtualenvs/loris-mri-python/lib/python3.6/site-packages/bids/layout/validation.py:46:
> UserWarning: The ability to pass arguments to BIDSLayout that control
> indexing is likely to be removed in future; possibly as early as PyBIDS
> 0.14. This includes the `config_filename`, `ignore`, `force_index`, and
> `index_metadata` arguments. The recommended usage pattern is to initialize
> a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout
> via the `indexer` argument.
>   warnings.warn("The ability to pass arguments to BIDSLayout that control "
>
> Connecting to:
>         database: DATABASE
>         username: USERNAME
>         hostname: 111.222.333440
>         port    : 1234
>
>
> Executing query:
>         SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM
> ConfigSettings WHERE Name = %s);
>
> With arguments:
>         ('default_bids_vl',)
>
>
> Executing query:
>         SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM
> ConfigSettings WHERE Name = %s);
>
> With arguments:
>         ('dataDirBasepath',)
>
> AMP::bids_dir: /DATA/CAND_A/bids
> Loading the BIDS dataset with BIDS layout library...
>
> AMP::bids_config: /PROJECT/bin/mri/python/lib/bids.json
> root: /DATA/CAND_A/bids
> target:: /DATA/CAND_A/bids/dataset_description.json
>         => BIDS dataset loaded with BIDS layout
>
> Grepping candidates from the BIDS layout...
> Validating the list of participants...
> subjects: ['207019']
>         => Passed validation of the list of participants
>
>         => List of participants found:
>                 207019
>
>
> Grepping list of sessions from the BIDS layout...
>         => List of sessions found:
>
>                 207019: screening
>
>
> Grepping the different modalities from the BIDS layout...
>         => Done grepping the different modalities from the BIDS layout
>
>
> Executing query:
>         SELECT * FROM candidate WHERE PSCID = %s
>
> With arguments:
>         ('207019',)
> Executing query:
>         SELECT SubprojectID FROM subproject WHERE title = %s
>
> With arguments:
>         ['Experimental']
>
>
> Executing query:
>         SELECT * FROM session WHERE CandID = %s AND Visit_label = %s
>
> With arguments:
>         ('622350', 'screening')
>
>
> Executing query:
>         SELECT * FROM candidate WHERE PSCID = %s
>
> With arguments:
>         ('207019',)
>
>
> Executing query:
>         SELECT SubprojectID FROM subproject WHERE title = %s
>
> With arguments:
>         ['Experimental']
>
>
> Executing query:
>         SELECT * FROM session WHERE CandID = %s AND Visit_label = %s
>
> With arguments:
>         ('622350', 'screening')
>
>
> Executing query:
>         SELECT type FROM ImagingFileTypes
>
>
> Executing query:
>         SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING
> (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s
>
> With arguments:
>
> ('248248fb303a762548e0be602b0b6bd6c6f33b4c860898c69faa8273be7a80f6d93b6413ec3bbeaadf22005fac04b07e9c08487e7b0a6690f64c2781d77d69b4',)
>
>
> Executing query:
>         SELECT type FROM ImagingFileTypes
>
>
> Executing query:
>         SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING
> (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s
>
> With arguments:
>
> ('b16bc029ea1cb8271a0728785d04eddaaa66e4d89c09e990552667b3c5598d17cee045b5b4ee3480dc2b8e08b5b05ca4e53c57f50b13b26f6467b9e4f455d09e',)
>
>
> Executing query:
>         SELECT type FROM ImagingFileTypes
>
>
> Executing query:
>         SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING
> (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s
>
> With arguments:
>
> ('44346136db049d0faf867a34d57d5714803a320b070013da0e1cf66e064f4dc9db92533f8aa8caaafcb78ce141c5444c5b8f8dc6b1d1c7dac530a4c2375eedcd',)
>
>
> Executing query:
>         SELECT type FROM ImagingFileTypes
>
>
> Executing query:
>         SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING
> (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s
>
> With arguments:
>
> ('ee91658a67d7d96007f6f0c625cceb700c9fa5bbecf34cea3cae8f1445a6efce5f735b2ab68d26717a6aaa418808b3d8fb30b03e03ae1982b0eeed834429d081',)
>
>
> Executing query:
>         SELECT type FROM ImagingFileTypes
>
>
> Executing query:
>         SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING
> (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s
>
> With arguments:
>
> ('749c00e5aa64583c91f29672b228b9d89e691cef7d8f3380eefbc1fee466bd3d987fd7178001f0e4f63684a50e139a7bb8ed197981d1588ffc4dfc4eed8278b8',)
> ===
>
> Thank you for your help.
>
> Best,
> Alfredo.
>
>
>
> On May 13, 2021, at 10:08 AM, Cecile Madjar <cecile.madjar at mcin.ca> wrote:
>
>         External Email - Use Caution
>
> Hi Alfredo,
>
> The requirement of not having any special characters in the Participant ID
> or Session ID comes from BIDS and the libraries used to parse the BIDS
> dataset before insertion into LORIS.
>
> On the LORIS front:
>
>    - the ID used next to sub-<ID> will be used for the PSCID field of the
>    candidate table (the CandID field being automatically generated as 6 random
>    digits by the insertion script).
>    - the ID used next to ses-<ID> will be used for the visit label field
>    of the session table
>
> There might be another solution for you: modifying the file
> https://github.com/aces/Loris-MRI/blob/main/python/lib/bids.json
> <https://secure-web.cisco.com/1GvLDW-g8RekA6xfXzTQGS9x-cvm3VJ64Me4gEy4E2Yh3_J_-W9ZJiP9DEv9M9oH_JN3CJGEaRyJUvOCUzrNqMXl0tpX8TJ_Cfrs5JA9p_ICYgDGAU_HcPRpK0aHBkRHCW9mHZTixcIoY9EEim1Tt4JhQWyQlJpaTKY-bd1bc5O1lxwHz6sj4nGQ5bXFiDLYRP4Y8DWJzxYpm8EMaj_QYRiVdUOwHiSGfVGMouhDDMYYwBHrEEdV1Ompf5QRyVkSSAJaGL2Q3tSABSa8USdVkOQ/https%3A%2F%2Fgithub.com%2Faces%2FLoris-MRI%2Fblob%2Fmain%2Fpython%2Flib%2Fbids.json>
> so that it matches the string used for your data. You would need to modify
> the pattern for subject and session in that file to match what you have. In
> theory, once done, pybids should be able to read your dataset.
>
> Hope this helps,
>
> C?cile
>
> On Thu, May 13, 2021 at 9:47 AM Morales Pinzon, Alfredo <
> AMORALESPINZON at bwh.harvard.edu> wrote:
>
>> Hi C?cile,
>>
>> Is it possible to modify your IDs to be alphanumerical?
>>
>>
>> It is possible but puts more burden on our side as every single file now
>> has to be modified. What would be best format for LORIS for the user ids
>> (e.g., six digits) and the visit labels (e.g., alphanumerical without
>> dash)? I just want to make sure we are compliant with LORIS requirements
>> before modifying the bids files.
>>
>> Best,
>> Alfredo.
>>
>> On May 11, 2021, at 6:34 PM, Cecile Madjar <cecile.madjar at mcin.ca> wrote:
>>
>>         External Email - Use Caution
>>
>> Hi Alfredo,
>>
>> It looks like the pybids is not able to read your BIDS dataset, which is
>> why it returns an empty list of subjects.
>>
>> I just tried on the python terminal to load your dataset on pybids and it
>> does not find any subject or visit. This is probably due to the name of the
>> subject that contains special characters. That will also be problematic for
>> the session label of your dataset. I guess subjects in BIDS should remain
>> alphanumerical only. Is it possible to modify your IDs to be alphanumerical?
>>
>> C?cile
>>
>> On Tue, May 11, 2021 at 5:37 PM Morales Pinzon, Alfredo <
>> AMORALESPINZON at bwh.harvard.edu> wrote:
>>
>>> Hi C?cile,
>>>
>>> I just shared the dataset in the sftp in the following path:
>>> 2021_05_11/caseCecile_BIDS.zip
>>>
>>> Additionally, here is the output of the script after adding the prints
>>> in the PR 617:
>>>
>>> ===
>>> /PROJECT/bin/mri/python_virtualenvs/loris-mri-python/lib/python3.6/site-packages/bids/layout/validation.py:46:
>>> UserWarning: The ability to pass arguments to BIDSLayout that control
>>> indexing is likely to be removed in future; possibly as early as PyBIDS
>>> 0.14. This includes the `config_filename`, `ignore`, `force_index`, and
>>> `index_metadata` arguments. The recommended usage pattern is to initialize
>>> a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout
>>> via the `indexer` argument.
>>>   warnings.warn("The ability to pass arguments to BIDSLayout that
>>> control "
>>>
>>> Connecting to:
>>>         database: DATABASE
>>>         username: USERDATABASE
>>>         hostname: 111.222.333.44
>>>         port    : 3306
>>>
>>>
>>> Executing query:
>>>         SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM
>>> ConfigSettings WHERE Name = %s);
>>>
>>> With arguments:
>>>         ('default_bids_vl',)
>>>
>>>
>>> Executing query:
>>>         SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM
>>> ConfigSettings WHERE Name = %s);
>>>
>>> With arguments:
>>>         ('dataDirBasepath',)
>>>
>>> Loading the BIDS dataset with BIDS layout library...
>>>         => BIDS dataset loaded with BIDS layout
>>>
>>> Grepping candidates from the BIDS layout...
>>> Validating the list of participants...
>>>
>>> ERROR: Participant ID mismatch between participants.tsv and raw data
>>> found in the BIDS directory/n
>>> CFTY720D2306.0102.00001is missing from the BIDS Layout
>>>
>>> List of subjects parsed by the BIDS layout:
>>> ===
>>>
>>> Best,
>>> Alfredo.
>>>
>>> On May 11, 2021, at 12:01 PM, C?cile Madjar <cecile.madjar at mcin.ca>
>>> wrote:
>>>
>>>         External Email - Use Caution
>>>
>>> Sounds like a plan. Let me know once it is uploaded and I will ask
>>> Nicolas to give me access.
>>>
>>> Thanks!
>>>
>>> PS: you can view the diff on the PR and simply copy the code over to
>>> your files if easier
>>>
>>> On May 11, 2021, at 11:47 AM, Morales Pinzon, Alfredo <
>>> AMORALESPINZON at BWH.HARVARD.EDU> wrote:
>>>
>>> Hi Cecile,
>>>
>>> In my testing dataset I have only one subject, I can share it with you
>>> in the LORIS sftp, does that work for you?
>>>
>>> In the mean time I will diff and pull your PR, and then execute it. I?ll
>>> send you the output asap.
>>>
>>> Best,
>>> Alfredo.
>>>
>>> On May 11, 2021, at 11:36 AM, Cecile Madjar <cecile.madjar at mcin.ca>
>>> wrote:
>>>
>>>         External Email - Use Caution
>>>
>>> Hi Alfredo,
>>>
>>> I do not have access to your dataset but is it possible that there is
>>> one or more sub-<ID> in the participants.tsv that are different from the
>>> sub-<ID> used in the directory structure? (a.k.a., folders sub-<ID> and
>>> filenames that should start with sub-<ID>.)
>>>
>>> This function is specifically cross checking the list provided in the
>>> participants.tsv file and the information gathered by the BIDSLayout
>>> library that read your BIDS dataset.
>>>
>>> I created a PR on LORIS-MRI to elaborate the log from the function you
>>> just sent: https://github.com/aces/Loris-MRI/pull/617
>>> <https://secure-web.cisco.com/17BHyJzFWfexg2et-D1qxjFw7-MsSpwCyZsKHDjuQa1Iruj-GV8fZRnaNOQe7rTfFaljPkHD9E0Kch6nwzSvbbe_48vUcKT3ozGDvxPfpe248W0Nml3EbqwefqTQo6V0gIBseIrXtJ4qwpUnrLVYpwZQyTPaDCbkm_NqrpLrBeXaHi-m5KCVluJqMYsb1DZBrqFL_XKJiIevFuiValC7eTehDRbTek9A5RvYsPN6hDTzEHFPQfCzzRD6W_iQSp80c_PbAKgxDUVLU2cePcP45pg/https%3A%2F%2Fgithub.com%2Faces%2FLoris-MRI%2Fpull%2F617> Maybe
>>> try adding the print statement in the diff of the PR to your code and rerun
>>> to get a better idea of what is going on?
>>>
>>> Let me know how it goes,
>>>
>>> C?cile
>>>
>>>
>>> On May 10, 2021, at 4:15 PM, Morales Pinzon, Alfredo <
>>> AMORALESPINZON at bwh.harvard.edu> wrote:
>>>
>>> Dear LorisDev team.
>>>
>>> I am running the BIDS uploading functionality of LORIS (from this PR
>>> suggested by Cecile: https://github.com/aces/Loris-MRI/pull/558/files
>>> <https://secure-web.cisco.com/1eWQtSLH8tHTpC1qsinKLnr4amf7Vdt1MLFNigX4jKdNz2mAoHV9ejLLcHZh1a0CUaUlzZMYTU9qzLaUEYYpyP1cl5UCR8e02WTmdMZUKXBNDeD8-cV2O5yTv8cgci7szP_MIGsvZ7FCjhqVmibPUnMFYbs6uYHGXZHOB5MsouWYLRHh89rJtQaFS8DOg4jfbwjp1bxwnO4FyuD3aKRv17dw9zoczaaXWnkbXXr87i6NQKYmA1NXhjM570IkyzSKuq-dlKj94vTZIlAChD5aFeg/https%3A%2F%2Fgithub.com%2Faces%2FLoris-MRI%2Fpull%2F558%2Ffiles> )
>>> but I?m getting the following error (see all the output below):
>>>
>>> ===
>>> ERROR: Participant ID mismatch between participants.tsv and raw data
>>> found in the BIDS directory/n
>>> ===
>>>
>>> After some debugging I saw that the array subjects in the following line
>>> is empty:
>>>
>>> https://github.com/laemtl/Loris-MRI/blob/3742522fa46b0ee2b9e384e30650a0c1f425c7af/python/lib/bidsreader.py#L134
>>> <https://secure-web.cisco.com/1sAIMc1N1U7sTpJYVEGfljx8iP9AUjxof1GeCQwNPKhd82OAXEZmdHoYu7n3Uq8jLaO-quBK1igoIw9Oo7d5JVNCfb3MNy6u-cvR2KkL-rfKwSOW_ovP_V5YKydvDMoE1ltw4MO29keFq-JiTymIk8Wm8TbXeHKxF2njboK6-6bctzRI-5K2tWwz82zUZyzDxyFdn4zXPqI8AihHeuWQQG3GOGgPSbRP-GOUfwbu2eVmvL_P_ldZetO1qjKowic0Gqnt-a9XN_8GORD5BLNlxTw/https%3A%2F%2Fgithub.com%2Flaemtl%2FLoris-MRI%2Fblob%2F3742522fa46b0ee2b9e384e30650a0c1f425c7af%2Fpython%2Flib%2Fbidsreader.py%23L134>
>>>
>>> I thought the issue was that we have symbolic links in our folder
>>> structure but after setting a testing example I am getting the same exact
>>> error.
>>>
>>> Any ideas why this is happening? The error does not help much. I?m happy
>>> to share the testing case, with all the BIDS structure to a LORIS
>>> developer, if this is helpful.
>>>
>>> Looking forward to hearing from you.
>>>
>>> Best.
>>> Alfredo.
>>>
>>> ===
>>> /PROJECT/bin/mri/python_virtualenvs/loris-mri-python/lib/python3.6/site-packages/bids/layout/validation.py:46:
>>> UserWarning: The ability to pass arguments to BIDSLayout that control
>>> indexing is likely to be removed in future; possibly as early as PyBIDS
>>> 0.14. This includes the `config_filename`, `ignore`, `force_index`, and
>>> `index_metadata` arguments. The recommended usage pattern is to initialize
>>> a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout
>>> via the `indexer` argument.
>>>   warnings.warn("The ability to pass arguments to BIDSLayout that
>>> control "
>>>
>>> Connecting to:
>>>         database: DATABASE
>>>         username: USERDATABASE
>>>         hostname: 111.222.333.444
>>>         port    : 3306
>>>
>>>
>>> Executing query:
>>>         SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM
>>> ConfigSettings WHERE Name = %s);
>>>
>>> With arguments:
>>>         ('default_bids_vl',)
>>>
>>>
>>> Executing query:
>>>         SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM
>>> ConfigSettings WHERE Name = %s);
>>>
>>> With arguments:
>>>         ('dataDirBasepath',)
>>>
>>> Loading the BIDS dataset with BIDS layout library...
>>>
>>>         => BIDS dataset loaded with BIDS layout
>>>
>>> Grepping candidates from the BIDS layout...
>>> Validating the list of participants...
>>>
>>> ERROR: Participant ID mismatch between participants.tsv and raw data
>>> found in the BIDS directory/n
>>> ===
>>>
>>> The information in this e-mail is intended only for the person to whom
>>> it is addressed. If you believe this e-mail was sent to you in error and
>>> the e-mail contains patient information, please contact the Mass General
>>> Brigham Compliance HelpLine at
>>> http://www.massgeneralbrigham.org/complianceline
>>> <http://secure-web.cisco.com/1qM_d-YnL441NX1CgLDc0-4p5P74OBrZe4qz8DQn4NKFJlMAA7HiKjibIOLkeFbmot5sDXcFfxupjvo_jX8vC9aYNvFDnADUwQFAF2ETaANXw3yIZaf8bb7_SvUXAkTxwK31eivXt_HlY2DxnGsZ739ls6VersNRKPQN2plxFNbqAzR65ZCFC8YSBrs5Vrcr1GWNsx7YN7RWdCvcDN1j8l1v9uD8Je4NlxsZBp9G86xq4w2ATmf4ENiL-6RShyNpajtkjZKD6TUD0FKl1yu9EdA/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline>
>>>  . If the e-mail was sent to you in error but does not contain patient
>>> information, please contact the sender and properly dispose of the e-mail.
>>>
>>> Please note that this e-mail is not secure (encrypted).  If you do not
>>> wish to continue communication over unencrypted e-mail, please notify the
>>> sender of this message immediately.  Continuing to send or respond to
>>> e-mail after receiving this message means you understand and accept this
>>> risk and wish to continue to communicate over unencrypted e-mail.
>>> _______________________________________________
>>> Loris-dev mailing list
>>> Loris-dev at bic.mni.mcgill.ca
>>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>> <https://secure-web.cisco.com/1nZGLPRvHAc4QXkbuPkDbTk6X7cLRpZjuKhOBth4NO62r205kfGQbnAX-YD9BjpSJaYX2q-XFXkB_JSF0s5z2ahDn5jCwO30vrTUlIZlEANwK8y3vIbbTS3OKeQl5HWbDL3FviFQTAC2JkKwXTysKFx31cF_fdj3Ldph7SeZSr25P3BOdkNzQM7CwOgGsZ74Icj-bh17DcnTl_TpfPBvtKvWdT9JJgzhO4xgU1diPaWsF1-yCLwXh4xN8u993AJYONV8vaXR3KRoKV8-QxssqZw/https%3A%2F%2Fmailman.bic.mni.mcgill.ca%2Fmailman%2Flistinfo%2Floris-dev>
>>>
>>>
>>> The information in this e-mail is intended only for the person to whom
>>> it is addressed. If you believe this e-mail was sent to you in error and
>>> the e-mail contains patient information, please contact the Mass General
>>> Brigham Compliance HelpLine at
>>> http://www.massgeneralbrigham.org/complianceline
>>> <http://secure-web.cisco.com/1AhAU1QrkHn5uAqLbnmi1HyXx_kpI9yR6ZBTDEz2E2EH9ZJ8W5lI94E700CC6tH7ISW-FmwwtetLtDvUs671_9WDXfhJBJZUlAUiC75p9k2iniolB83RMn1vpjTql7mdvGqhQFlpZCisj05nFxy4_7JaawBv-2YfBcGJkgmL4k4UTe3OZ3Q6CU-jPPmqEDNyFt7jVXNncRMqUUg9ZiUWmhABUAHol2wVTYpBugXYIZfYMj79LvUy54oxCyOtX5FCW00-m4IAVrvYB_EhkHxI5Qw/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline>
>>> . If the e-mail was sent to you in error but does not contain patient
>>> information, please contact the sender and properly dispose of the e-mail.
>>>
>>> Please note that this e-mail is not secure (encrypted).  If you do not
>>> wish to continue communication over unencrypted e-mail, please notify the
>>> sender of this message immediately.  Continuing to send or respond to
>>> e-mail after receiving this message means you understand and accept this
>>> risk and wish to continue to communicate over unencrypted e-mail.
>>>
>>>
>>>
>>> The information in this e-mail is intended only for the person to whom
>>> it is addressed. If you believe this e-mail was sent to you in error and
>>> the e-mail contains patient information, please contact the Mass General
>>> Brigham Compliance HelpLine at
>>> http://www.massgeneralbrigham.org/complianceline
>>> <http://secure-web.cisco.com/1Vs76oaBwBuqLoMP4tDeksrqcCgDkogMEbBScJ4ejcwSa6gekqJpFoL6YNIznACwU31BdOoVLyr1lD6sf6FJ0bhDvv_VDNM4ihqPNDRpk_MBie2PPM8xVjhZzqKoCHhcAwaJVO6JrBb8QVg0k78kxRXH7d0wKdNPAHrs292D3K2KcEskKWx2YgQ-McThsXUDOsdj4Y99wDiDj5fzeT-glwFo2ZGa6-7cdKO2jHZVoiG8_XlvkoA76TtLIZxN04JfUn5UFQZ0p3J-jLf5RR8u_7A/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline>
>>> . If the e-mail was sent to you in error but does not contain patient
>>> information, please contact the sender and properly dispose of the e-mail.
>>>
>>> Please note that this e-mail is not secure (encrypted).  If you do not
>>> wish to continue communication over unencrypted e-mail, please notify the
>>> sender of this message immediately.  Continuing to send or respond to
>>> e-mail after receiving this message means you understand and accept this
>>> risk and wish to continue to communicate over unencrypted e-mail.
>>>
>>
>> The information in this e-mail is intended only for the person to whom it
>> is addressed. If you believe this e-mail was sent to you in error and the
>> e-mail contains patient information, please contact the Mass General
>> Brigham Compliance HelpLine at
>> http://www.massgeneralbrigham.org/complianceline
>> <http://secure-web.cisco.com/1n2UbpIP1Hiy1wedtWg9s-6NXBNM5GXQsk4yul5KQtRtD-fpCcwhqxfc9xPmkoXTkvQiEilB7x5cMjoMnEfP7sVBe6N4BZFTL6lnTA5ZoU6_eL-QZrxz4bet8uKLeH8lFWIbsg3Tq1FqZpdjEzQ9aSVMRC-CNrE0-CJn7xfPqJefF9I1-_5OZcKb9QkXl2Rg_xjO8CfyirontcnUgEOHlJ5anCG65dp3TNqKwXi_HsVUusIi9c6KJxpEoiY-xH-7k_UAsbpirJkzRosIoJNyevA/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline>
>> . If the e-mail was sent to you in error but does not contain patient
>> information, please contact the sender and properly dispose of the e-mail.
>>
>> Please note that this e-mail is not secure (encrypted).  If you do not
>> wish to continue communication over unencrypted e-mail, please notify the
>> sender of this message immediately.  Continuing to send or respond to
>> e-mail after receiving this message means you understand and accept this
>> risk and wish to continue to communicate over unencrypted e-mail.
>>
>
> The information in this e-mail is intended only for the person to whom it
> is addressed. If you believe this e-mail was sent to you in error and the
> e-mail contains patient information, please contact the Mass General
> Brigham Compliance HelpLine at
> http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent
> to you in error but does not contain patient information, please contact
> the sender and properly dispose of the e-mail.
>
> Please note that this e-mail is not secure (encrypted).  If you do not
> wish to continue communication over unencrypted e-mail, please notify the
> sender of this message immediately.  Continuing to send or respond to
> e-mail after receiving this message means you understand and accept this
> risk and wish to continue to communicate over unencrypted e-mail.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210607/01bed71d/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screen Shot 2021-06-06 at 4.44.57 PM.png
Type: image/png
Size: 90041 bytes
Desc: not available
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210607/01bed71d/attachment-0001.png>

From AMORALESPINZON at BWH.HARVARD.EDU  Mon Jun  7 12:27:17 2021
From: AMORALESPINZON at BWH.HARVARD.EDU (Morales Pinzon, Alfredo)
Date: Mon, 7 Jun 2021 16:27:17 +0000
Subject: [Loris-dev] BIDS Error
In-Reply-To: <BCBD10CB-5400-4FF3-949E-5641665587E3@bwh.harvard.edu>
References: <8E375877-D896-4B0E-B0F7-D6BD5565CA67@bwh.harvard.edu>
 <ADFA6F92-C7E7-4FD5-85A1-4F20D1565F75@gmail.com>
 <4286E305-87C3-492A-BE31-0B83CEFD365E@bwh.harvard.edu>
 <5F1BBE56-5523-4517-ADCF-2A1DB17C952B@mcin.ca>
 <F1F354D5-8CDF-4089-A4BA-6A3A41903CDA@bwh.harvard.edu>
 <CAMkiY3cXNfprNL_VDnqFzH1og332_bF2u1Zhe+enRA0dGBpWHQ@mail.gmail.com>
 <CE0C1F1D-862F-4030-BA2D-401BE3110632@bwh.harvard.edu>
 <CAMkiY3cMR4CRYE1Y=u5TGwpOsPk7Q3UdXMmit4V7RR3q8jqpeg@mail.gmail.com>
 <BCBD10CB-5400-4FF3-949E-5641665587E3@bwh.harvard.edu>
Message-ID: <C6850CF7-FC93-46A7-BBBB-E83222C44FE5@bwh.harvard.edu>

Hi C?cile,

Thank you for prompt answer. Here is the output of the query:

===
mysql> SELECT CandID, Visit_label, Active, Scan_done, CenterID, ProjectID, SubprojectID FROM session WHERE Visit_label = 'screening' AND CandID=207019;
+--------+-------------+--------+-----------+----------+-----------+--------------+
| CandID | Visit_label | Active | Scan_done | CenterID | ProjectID | SubprojectID |
+--------+-------------+--------+-----------+----------+-----------+--------------+
| 207019 | screening   | Y      | NULL      |      843 |        12 |            2 |
+--------+-------------+--------+-----------+----------+-----------+--------------+
1 row in set (0.00 sec)
===

I guess the ?Scan_done? should be ?Y? instead of ?NULL?. Is this something that the BIDS importing script should take care of? Is the fix as simple as just setting Scan_done to ?Y??

Best,
Alfredo.

On Jun 6, 2021, at 4:48 PM, Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU<mailto:AMORALESPINZON at BWH.HARVARD.EDU>> wrote:

Hi C?cile,

We managed to insert the candidates and the visits, and create the specific file names for BIDS before actually launching the BIDS script. The script finally worked, however I don?t see the images in LORIS. I am adding down below the bids log insertion. Is there something additional I need to do to be able to see the images? In the web site I see the candidate and the visit but the column ?Imaging Scan Done? is marked with a question mark (see attachment).

===
/PROJECT/bin/mri/python_virtualenvs/loris-mri-python/lib/python3.6/site-packages/bids/layout/validation.py:46: UserWarning: The ability to pass arguments to BIDSLayout that control indexing is likely to be removed in future; possibly as early as PyBIDS 0.14. This includes the `config_filename`, `ignore`, `force_index`, and `index_metadata` arguments. The recommended usage pattern is to initialize a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout via the `indexer` argument.
  warnings.warn("The ability to pass arguments to BIDSLayout that control "

Connecting to:
        database: DATABASE
        username: USERNAME
        hostname: 111.222.333440
        port    : 1234


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('default_bids_vl',)


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('dataDirBasepath',)

AMP::bids_dir: /DATA/CAND_A/bids
Loading the BIDS dataset with BIDS layout library...

AMP::bids_config: /PROJECT/bin/mri/python/lib/bids.json
root: /DATA/CAND_A/bids
target:: /DATA/CAND_A/bids/dataset_description.json
        => BIDS dataset loaded with BIDS layout

Grepping candidates from the BIDS layout...
Validating the list of participants...
subjects: ['207019']
        => Passed validation of the list of participants

        => List of participants found:
                207019


Grepping list of sessions from the BIDS layout...
        => List of sessions found:

                207019: screening


Grepping the different modalities from the BIDS layout...
        => Done grepping the different modalities from the BIDS layout


Executing query:
        SELECT * FROM candidate WHERE PSCID = %s

With arguments:
        ('207019',)
Executing query:
        SELECT SubprojectID FROM subproject WHERE title = %s

With arguments:
        ['Experimental']


Executing query:
        SELECT * FROM session WHERE CandID = %s AND Visit_label = %s

With arguments:
        ('622350', 'screening')


Executing query:
        SELECT * FROM candidate WHERE PSCID = %s

With arguments:
        ('207019',)


Executing query:
        SELECT SubprojectID FROM subproject WHERE title = %s

With arguments:
        ['Experimental']


Executing query:
        SELECT * FROM session WHERE CandID = %s AND Visit_label = %s

With arguments:
        ('622350', 'screening')


Executing query:
        SELECT type FROM ImagingFileTypes


Executing query:
        SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s

With arguments:
        ('248248fb303a762548e0be602b0b6bd6c6f33b4c860898c69faa8273be7a80f6d93b6413ec3bbeaadf22005fac04b07e9c08487e7b0a6690f64c2781d77d69b4',)


Executing query:
        SELECT type FROM ImagingFileTypes


Executing query:
        SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s

With arguments:
        ('b16bc029ea1cb8271a0728785d04eddaaa66e4d89c09e990552667b3c5598d17cee045b5b4ee3480dc2b8e08b5b05ca4e53c57f50b13b26f6467b9e4f455d09e',)


Executing query:
        SELECT type FROM ImagingFileTypes


Executing query:
        SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s

With arguments:
        ('44346136db049d0faf867a34d57d5714803a320b070013da0e1cf66e064f4dc9db92533f8aa8caaafcb78ce141c5444c5b8f8dc6b1d1c7dac530a4c2375eedcd',)


Executing query:
        SELECT type FROM ImagingFileTypes


Executing query:
        SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s

With arguments:
        ('ee91658a67d7d96007f6f0c625cceb700c9fa5bbecf34cea3cae8f1445a6efce5f735b2ab68d26717a6aaa418808b3d8fb30b03e03ae1982b0eeed834429d081',)


Executing query:
        SELECT type FROM ImagingFileTypes


Executing query:
        SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s

With arguments:
        ('749c00e5aa64583c91f29672b228b9d89e691cef7d8f3380eefbc1fee466bd3d987fd7178001f0e4f63684a50e139a7bb8ed197981d1588ffc4dfc4eed8278b8',)
===

Thank you for your help.

Best,
Alfredo.

<Screen Shot 2021-06-06 at 4.44.57 PM.png>

On May 13, 2021, at 10:08 AM, Cecile Madjar <cecile.madjar at mcin.ca<mailto:cecile.madjar at mcin.ca>> wrote:


        External Email - Use Caution

Hi Alfredo,

The requirement of not having any special characters in the Participant ID or Session ID comes from BIDS and the libraries used to parse the BIDS dataset before insertion into LORIS.

On the LORIS front:

  *   the ID used next to sub-<ID> will be used for the PSCID field of the candidate table (the CandID field being automatically generated as 6 random digits by the insertion script).
  *   the ID used next to ses-<ID> will be used for the visit label field of the session table

There might be another solution for you: modifying the file https://github.com/aces/Loris-MRI/blob/main/python/lib/bids.json<https://secure-web.cisco.com/1GvLDW-g8RekA6xfXzTQGS9x-cvm3VJ64Me4gEy4E2Yh3_J_-W9ZJiP9DEv9M9oH_JN3CJGEaRyJUvOCUzrNqMXl0tpX8TJ_Cfrs5JA9p_ICYgDGAU_HcPRpK0aHBkRHCW9mHZTixcIoY9EEim1Tt4JhQWyQlJpaTKY-bd1bc5O1lxwHz6sj4nGQ5bXFiDLYRP4Y8DWJzxYpm8EMaj_QYRiVdUOwHiSGfVGMouhDDMYYwBHrEEdV1Ompf5QRyVkSSAJaGL2Q3tSABSa8USdVkOQ/https%3A%2F%2Fgithub.com%2Faces%2FLoris-MRI%2Fblob%2Fmain%2Fpython%2Flib%2Fbids.json> so that it matches the string used for your data. You would need to modify the pattern for subject and session in that file to match what you have. In theory, once done, pybids should be able to read your dataset.

Hope this helps,

C?cile

On Thu, May 13, 2021 at 9:47 AM Morales Pinzon, Alfredo <AMORALESPINZON at bwh.harvard.edu<mailto:AMORALESPINZON at bwh.harvard.edu>> wrote:
Hi C?cile,

Is it possible to modify your IDs to be alphanumerical?

It is possible but puts more burden on our side as every single file now has to be modified. What would be best format for LORIS for the user ids (e.g., six digits) and the visit labels (e.g., alphanumerical without dash)? I just want to make sure we are compliant with LORIS requirements before modifying the bids files.

Best,
Alfredo.

On May 11, 2021, at 6:34 PM, Cecile Madjar <cecile.madjar at mcin.ca<mailto:cecile.madjar at mcin.ca>> wrote:


        External Email - Use Caution

Hi Alfredo,

It looks like the pybids is not able to read your BIDS dataset, which is why it returns an empty list of subjects.

I just tried on the python terminal to load your dataset on pybids and it does not find any subject or visit. This is probably due to the name of the subject that contains special characters. That will also be problematic for the session label of your dataset. I guess subjects in BIDS should remain alphanumerical only. Is it possible to modify your IDs to be alphanumerical?

C?cile

On Tue, May 11, 2021 at 5:37 PM Morales Pinzon, Alfredo <AMORALESPINZON at bwh.harvard.edu<mailto:AMORALESPINZON at bwh.harvard.edu>> wrote:
Hi C?cile,

I just shared the dataset in the sftp in the following path: 2021_05_11/caseCecile_BIDS.zip

Additionally, here is the output of the script after adding the prints in the PR 617:

===
/PROJECT/bin/mri/python_virtualenvs/loris-mri-python/lib/python3.6/site-packages/bids/layout/validation.py:46: UserWarning: The ability to pass arguments to BIDSLayout that control indexing is likely to be removed in future; possibly as early as PyBIDS 0.14. This includes the `config_filename`, `ignore`, `force_index`, and `index_metadata` arguments. The recommended usage pattern is to initialize a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout via the `indexer` argument.
  warnings.warn("The ability to pass arguments to BIDSLayout that control "

Connecting to:
        database: DATABASE
        username: USERDATABASE
        hostname: 111.222.333.44
        port    : 3306


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('default_bids_vl',)


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('dataDirBasepath',)

Loading the BIDS dataset with BIDS layout library...
        => BIDS dataset loaded with BIDS layout

Grepping candidates from the BIDS layout...
Validating the list of participants...

ERROR: Participant ID mismatch between participants.tsv and raw data found in the BIDS directory/n
CFTY720D2306.0102.00001is missing from the BIDS Layout

List of subjects parsed by the BIDS layout:
===

Best,
Alfredo.

On May 11, 2021, at 12:01 PM, C?cile Madjar <cecile.madjar at mcin.ca<mailto:cecile.madjar at mcin.ca>> wrote:


        External Email - Use Caution

Sounds like a plan. Let me know once it is uploaded and I will ask Nicolas to give me access.

Thanks!

PS: you can view the diff on the PR and simply copy the code over to your files if easier

On May 11, 2021, at 11:47 AM, Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU<mailto:AMORALESPINZON at BWH.HARVARD.EDU>> wrote:

Hi Cecile,

In my testing dataset I have only one subject, I can share it with you in the LORIS sftp, does that work for you?

In the mean time I will diff and pull your PR, and then execute it. I?ll send you the output asap.

Best,
Alfredo.

On May 11, 2021, at 11:36 AM, Cecile Madjar <cecile.madjar at mcin.ca<mailto:cecile.madjar at mcin.ca>> wrote:


        External Email - Use Caution

Hi Alfredo,

I do not have access to your dataset but is it possible that there is one or more sub-<ID> in the participants.tsv that are different from the sub-<ID> used in the directory structure? (a.k.a., folders sub-<ID> and filenames that should start with sub-<ID>.)

This function is specifically cross checking the list provided in the participants.tsv file and the information gathered by the BIDSLayout library that read your BIDS dataset.

I created a PR on LORIS-MRI to elaborate the log from the function you just sent: https://github.com/aces/Loris-MRI/pull/617<https://secure-web.cisco.com/17BHyJzFWfexg2et-D1qxjFw7-MsSpwCyZsKHDjuQa1Iruj-GV8fZRnaNOQe7rTfFaljPkHD9E0Kch6nwzSvbbe_48vUcKT3ozGDvxPfpe248W0Nml3EbqwefqTQo6V0gIBseIrXtJ4qwpUnrLVYpwZQyTPaDCbkm_NqrpLrBeXaHi-m5KCVluJqMYsb1DZBrqFL_XKJiIevFuiValC7eTehDRbTek9A5RvYsPN6hDTzEHFPQfCzzRD6W_iQSp80c_PbAKgxDUVLU2cePcP45pg/https%3A%2F%2Fgithub.com%2Faces%2FLoris-MRI%2Fpull%2F617> Maybe try adding the print statement in the diff of the PR to your code and rerun to get a better idea of what is going on?

Let me know how it goes,

C?cile


On May 10, 2021, at 4:15 PM, Morales Pinzon, Alfredo <AMORALESPINZON at bwh.harvard.edu<mailto:AMORALESPINZON at bwh.harvard.edu>> wrote:

Dear LorisDev team.

I am running the BIDS uploading functionality of LORIS (from this PR suggested by Cecile: https://github.com/aces/Loris-MRI/pull/558/files<https://secure-web.cisco.com/1eWQtSLH8tHTpC1qsinKLnr4amf7Vdt1MLFNigX4jKdNz2mAoHV9ejLLcHZh1a0CUaUlzZMYTU9qzLaUEYYpyP1cl5UCR8e02WTmdMZUKXBNDeD8-cV2O5yTv8cgci7szP_MIGsvZ7FCjhqVmibPUnMFYbs6uYHGXZHOB5MsouWYLRHh89rJtQaFS8DOg4jfbwjp1bxwnO4FyuD3aKRv17dw9zoczaaXWnkbXXr87i6NQKYmA1NXhjM570IkyzSKuq-dlKj94vTZIlAChD5aFeg/https%3A%2F%2Fgithub.com%2Faces%2FLoris-MRI%2Fpull%2F558%2Ffiles> ) but I?m getting the following error (see all the output below):

===
ERROR: Participant ID mismatch between participants.tsv and raw data found in the BIDS directory/n
===

After some debugging I saw that the array subjects in the following line is empty:
https://github.com/laemtl/Loris-MRI/blob/3742522fa46b0ee2b9e384e30650a0c1f425c7af/python/lib/bidsreader.py#L134<https://secure-web.cisco.com/1sAIMc1N1U7sTpJYVEGfljx8iP9AUjxof1GeCQwNPKhd82OAXEZmdHoYu7n3Uq8jLaO-quBK1igoIw9Oo7d5JVNCfb3MNy6u-cvR2KkL-rfKwSOW_ovP_V5YKydvDMoE1ltw4MO29keFq-JiTymIk8Wm8TbXeHKxF2njboK6-6bctzRI-5K2tWwz82zUZyzDxyFdn4zXPqI8AihHeuWQQG3GOGgPSbRP-GOUfwbu2eVmvL_P_ldZetO1qjKowic0Gqnt-a9XN_8GORD5BLNlxTw/https%3A%2F%2Fgithub.com%2Flaemtl%2FLoris-MRI%2Fblob%2F3742522fa46b0ee2b9e384e30650a0c1f425c7af%2Fpython%2Flib%2Fbidsreader.py%23L134>

I thought the issue was that we have symbolic links in our folder structure but after setting a testing example I am getting the same exact error.

Any ideas why this is happening? The error does not help much. I?m happy to share the testing case, with all the BIDS structure to a LORIS developer, if this is helpful.

Looking forward to hearing from you.

Best.
Alfredo.

===
/PROJECT/bin/mri/python_virtualenvs/loris-mri-python/lib/python3.6/site-packages/bids/layout/validation.py:46: UserWarning: The ability to pass arguments to BIDSLayout that control indexing is likely to be removed in future; possibly as early as PyBIDS 0.14. This includes the `config_filename`, `ignore`, `force_index`, and `index_metadata` arguments. The recommended usage pattern is to initialize a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout via the `indexer` argument.
  warnings.warn("The ability to pass arguments to BIDSLayout that control "

Connecting to:
        database: DATABASE
        username: USERDATABASE
        hostname: 111.222.333.444
        port    : 3306


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('default_bids_vl',)


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('dataDirBasepath',)

Loading the BIDS dataset with BIDS layout library...

        => BIDS dataset loaded with BIDS layout

Grepping candidates from the BIDS layout...
Validating the list of participants...

ERROR: Participant ID mismatch between participants.tsv and raw data found in the BIDS directory/n
===

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/1qM_d-YnL441NX1CgLDc0-4p5P74OBrZe4qz8DQn4NKFJlMAA7HiKjibIOLkeFbmot5sDXcFfxupjvo_jX8vC9aYNvFDnADUwQFAF2ETaANXw3yIZaf8bb7_SvUXAkTxwK31eivXt_HlY2DxnGsZ739ls6VersNRKPQN2plxFNbqAzR65ZCFC8YSBrs5Vrcr1GWNsx7YN7RWdCvcDN1j8l1v9uD8Je4NlxsZBp9G86xq4w2ATmf4ENiL-6RShyNpajtkjZKD6TUD0FKl1yu9EdA/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
_______________________________________________
Loris-dev mailing list
Loris-dev at bic.mni.mcgill.ca<mailto:Loris-dev at bic.mni.mcgill.ca>
https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev<https://secure-web.cisco.com/1nZGLPRvHAc4QXkbuPkDbTk6X7cLRpZjuKhOBth4NO62r205kfGQbnAX-YD9BjpSJaYX2q-XFXkB_JSF0s5z2ahDn5jCwO30vrTUlIZlEANwK8y3vIbbTS3OKeQl5HWbDL3FviFQTAC2JkKwXTysKFx31cF_fdj3Ldph7SeZSr25P3BOdkNzQM7CwOgGsZ74Icj-bh17DcnTl_TpfPBvtKvWdT9JJgzhO4xgU1diPaWsF1-yCLwXh4xN8u993AJYONV8vaXR3KRoKV8-QxssqZw/https%3A%2F%2Fmailman.bic.mni.mcgill.ca%2Fmailman%2Flistinfo%2Floris-dev>

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/1AhAU1QrkHn5uAqLbnmi1HyXx_kpI9yR6ZBTDEz2E2EH9ZJ8W5lI94E700CC6tH7ISW-FmwwtetLtDvUs671_9WDXfhJBJZUlAUiC75p9k2iniolB83RMn1vpjTql7mdvGqhQFlpZCisj05nFxy4_7JaawBv-2YfBcGJkgmL4k4UTe3OZ3Q6CU-jPPmqEDNyFt7jVXNncRMqUUg9ZiUWmhABUAHol2wVTYpBugXYIZfYMj79LvUy54oxCyOtX5FCW00-m4IAVrvYB_EhkHxI5Qw/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.


The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/1Vs76oaBwBuqLoMP4tDeksrqcCgDkogMEbBScJ4ejcwSa6gekqJpFoL6YNIznACwU31BdOoVLyr1lD6sf6FJ0bhDvv_VDNM4ihqPNDRpk_MBie2PPM8xVjhZzqKoCHhcAwaJVO6JrBb8QVg0k78kxRXH7d0wKdNPAHrs292D3K2KcEskKWx2YgQ-McThsXUDOsdj4Y99wDiDj5fzeT-glwFo2ZGa6-7cdKO2jHZVoiG8_XlvkoA76TtLIZxN04JfUn5UFQZ0p3J-jLf5RR8u_7A/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/1n2UbpIP1Hiy1wedtWg9s-6NXBNM5GXQsk4yul5KQtRtD-fpCcwhqxfc9xPmkoXTkvQiEilB7x5cMjoMnEfP7sVBe6N4BZFTL6lnTA5ZoU6_eL-QZrxz4bet8uKLeH8lFWIbsg3Tq1FqZpdjEzQ9aSVMRC-CNrE0-CJn7xfPqJefF9I1-_5OZcKb9QkXl2Rg_xjO8CfyirontcnUgEOHlJ5anCG65dp3TNqKwXi_HsVUusIi9c6KJxpEoiY-xH-7k_UAsbpirJkzRosIoJNyevA/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.


The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210607/8581cbdd/attachment-0001.html>

From cecile.madjar at mcin.ca  Mon Jun  7 16:50:06 2021
From: cecile.madjar at mcin.ca (Cecile Madjar)
Date: Mon, 7 Jun 2021 16:50:06 -0400
Subject: [Loris-dev] BIDS Error
In-Reply-To: <C6850CF7-FC93-46A7-BBBB-E83222C44FE5@bwh.harvard.edu>
References: <8E375877-D896-4B0E-B0F7-D6BD5565CA67@bwh.harvard.edu>
 <ADFA6F92-C7E7-4FD5-85A1-4F20D1565F75@gmail.com>
 <4286E305-87C3-492A-BE31-0B83CEFD365E@bwh.harvard.edu>
 <5F1BBE56-5523-4517-ADCF-2A1DB17C952B@mcin.ca>
 <F1F354D5-8CDF-4089-A4BA-6A3A41903CDA@bwh.harvard.edu>
 <CAMkiY3cXNfprNL_VDnqFzH1og332_bF2u1Zhe+enRA0dGBpWHQ@mail.gmail.com>
 <CE0C1F1D-862F-4030-BA2D-401BE3110632@bwh.harvard.edu>
 <CAMkiY3cMR4CRYE1Y=u5TGwpOsPk7Q3UdXMmit4V7RR3q8jqpeg@mail.gmail.com>
 <BCBD10CB-5400-4FF3-949E-5641665587E3@bwh.harvard.edu>
 <C6850CF7-FC93-46A7-BBBB-E83222C44FE5@bwh.harvard.edu>
Message-ID: <CAMkiY3ek7Jhavv-GAaR35CYTBqNa0zE6tkLZA8HFsQr6Zt6H0Q@mail.gmail.com>

Hi Alfredo,

Interesting. I was expecting to see the Active field set to NULL which
would have explained why you do not see anything in the imaging browser. My
next theory is that you do not have anything for the parameter_type name
'acquisition_date' which is very likely to be the case for a BIDS dataset
with no date information.

Does the following query return anything for you? That is the query that is
supposed to return data to the imaging browser (that you can find in
modules/imaging_browser/php/imagingbrowserrowprovisioner.class.inc
<https://github.com/aces/Loris/blob/23.0-release/modules/imaging_browser/php/imagingbrowserrowprovisioner.class.inc>
).

SELECT
p.Name as Site,
c.PSCID as PSCID,
c.CandID as DCCID,
Project.Name as project,
s.Visit_label as visitLabel,
$PendingFailSubquery as Visit_QC_Status,
DATE_FORMAT(MIN(pf.Value), \"%Y-%m-%d\") as First_Acquisition,
FROM_UNIXTIME(MIN(f.InsertTime)) as First_Insertion,
FROM_UNIXTIME(MAX(fqc.QCLastChangeTime)) as Last_QC,
$NewDataSubquery as New_Data,
GROUP_CONCAT(DISTINCT OutputType) as Links,
s.ID as sessionID,
GROUP_CONCAT(DISTINCT modality.Scan_type) as sequenceType,
$PendingNewquery as pending,
$modalities_subquery
s.CenterID as CenterID,
c.Entity_type as entityType,
s.ProjectID
FROM psc AS p
JOIN session s ON (s.CenterID=p.CenterID)
JOIN candidate c ON (c.CandID=s.CandID)
LEFT JOIN Project ON (s.ProjectID=Project.ProjectID)
JOIN files f ON (f.SessionID=s.ID)
LEFT JOIN files_qcstatus fqc ON (fqc.FileID=f.FileID)
JOIN parameter_file pf ON (f.FileID=pf.FileID)
JOIN parameter_type pt USING (ParameterTypeID)
LEFT JOIN mri_scan_type modality ON
(f.AcquisitionProtocolID=modality.ID)
$left_joins
WHERE
s.Active = 'Y' AND
f.FileType IN ('mnc', 'nii') AND
pt.Name='acquisition_date'
GROUP BY s.ID
ORDER BY c.PSCID, s.Visit_label

If you get an empty result, try removing the following from the WHERE
section of the query above:

              .....  AND
             pt.Name='acquisition_date'

If my second theory is right, you should then see rows returned, in which
case you will need to modify the query in the imaging_browser code.
Let me know how those go.

Regarding the Scan_type, I should probably be populated by the insertion
script so this may be a bug on the bids_import scripts specific to the
imaging component. In the meantime, you can indeed set the flag to Y for
the sessions you know contain scan data.

Hope this helps,

C?cile


On Mon, Jun 7, 2021 at 12:27 PM Morales Pinzon, Alfredo <
AMORALESPINZON at bwh.harvard.edu> wrote:

> Hi C?cile,
>
> Thank you for prompt answer. Here is the output of the query:
>
> ===
> mysql> SELECT CandID, Visit_label, Active, Scan_done, CenterID, ProjectID,
> SubprojectID FROM session WHERE Visit_label = 'screening' AND CandID=207019;
>
> +--------+-------------+--------+-----------+----------+-----------+--------------+
> | CandID | Visit_label | Active | Scan_done | CenterID | ProjectID |
> SubprojectID |
>
> +--------+-------------+--------+-----------+----------+-----------+--------------+
> | 207019 | screening   | Y      | NULL      |      843 |        12 |
>       2 |
>
> +--------+-------------+--------+-----------+----------+-----------+--------------+
> 1 row in set (0.00 sec)
> ===
>
> I guess the ?Scan_done? should be ?Y? instead of ?NULL?. Is this something
> that the BIDS importing script should take care of? Is the fix as simple as
> just setting Scan_done to ?Y??
>
> Best,
> Alfredo.
>
> On Jun 6, 2021, at 4:48 PM, Morales Pinzon, Alfredo <
> AMORALESPINZON at BWH.HARVARD.EDU> wrote:
>
> Hi C?cile,
>
> We managed to insert the candidates and the visits, and create the
> specific file names for BIDS before actually launching the BIDS script. The
> script finally worked, however I don?t see the images in LORIS. I am adding
> down below the bids log insertion. Is there something additional I need to
> do to be able to see the images? In the web site I see the candidate and
> the visit but the column ?Imaging Scan Done? is marked with a question mark
> (see attachment).
>
> ===
> /PROJECT/bin/mri/python_virtualenvs/loris-mri-python/lib/python3.6/site-packages/bids/layout/validation.py:46:
> UserWarning: The ability to pass arguments to BIDSLayout that control
> indexing is likely to be removed in future; possibly as early as PyBIDS
> 0.14. This includes the `config_filename`, `ignore`, `force_index`, and
> `index_metadata` arguments. The recommended usage pattern is to initialize
> a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout
> via the `indexer` argument.
>   warnings.warn("The ability to pass arguments to BIDSLayout that control "
>
> Connecting to:
>         database: DATABASE
>         username: USERNAME
>         hostname: 111.222.333440
>         port    : 1234
>
>
> Executing query:
>         SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM
> ConfigSettings WHERE Name = %s);
>
> With arguments:
>         ('default_bids_vl',)
>
>
> Executing query:
>         SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM
> ConfigSettings WHERE Name = %s);
>
> With arguments:
>         ('dataDirBasepath',)
>
> AMP::bids_dir: /DATA/CAND_A/bids
> Loading the BIDS dataset with BIDS layout library...
>
> AMP::bids_config: /PROJECT/bin/mri/python/lib/bids.json
> root: /DATA/CAND_A/bids
> target:: /DATA/CAND_A/bids/dataset_description.json
>         => BIDS dataset loaded with BIDS layout
>
> Grepping candidates from the BIDS layout...
> Validating the list of participants...
> subjects: ['207019']
>         => Passed validation of the list of participants
>
>         => List of participants found:
>                 207019
>
>
> Grepping list of sessions from the BIDS layout...
>         => List of sessions found:
>
>                 207019: screening
>
>
> Grepping the different modalities from the BIDS layout...
>         => Done grepping the different modalities from the BIDS layout
>
>
> Executing query:
>         SELECT * FROM candidate WHERE PSCID = %s
>
> With arguments:
>         ('207019',)
> Executing query:
>         SELECT SubprojectID FROM subproject WHERE title = %s
>
> With arguments:
>         ['Experimental']
>
>
> Executing query:
>         SELECT * FROM session WHERE CandID = %s AND Visit_label = %s
>
> With arguments:
>         ('622350', 'screening')
>
>
> Executing query:
>         SELECT * FROM candidate WHERE PSCID = %s
>
> With arguments:
>         ('207019',)
>
>
> Executing query:
>         SELECT SubprojectID FROM subproject WHERE title = %s
>
> With arguments:
>         ['Experimental']
>
>
> Executing query:
>         SELECT * FROM session WHERE CandID = %s AND Visit_label = %s
>
> With arguments:
>         ('622350', 'screening')
>
>
> Executing query:
>         SELECT type FROM ImagingFileTypes
>
>
> Executing query:
>         SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING
> (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s
>
> With arguments:
>
> ('248248fb303a762548e0be602b0b6bd6c6f33b4c860898c69faa8273be7a80f6d93b6413ec3bbeaadf22005fac04b07e9c08487e7b0a6690f64c2781d77d69b4',)
>
>
> Executing query:
>         SELECT type FROM ImagingFileTypes
>
>
> Executing query:
>         SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING
> (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s
>
> With arguments:
>
> ('b16bc029ea1cb8271a0728785d04eddaaa66e4d89c09e990552667b3c5598d17cee045b5b4ee3480dc2b8e08b5b05ca4e53c57f50b13b26f6467b9e4f455d09e',)
>
>
> Executing query:
>         SELECT type FROM ImagingFileTypes
>
>
> Executing query:
>         SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING
> (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s
>
> With arguments:
>
> ('44346136db049d0faf867a34d57d5714803a320b070013da0e1cf66e064f4dc9db92533f8aa8caaafcb78ce141c5444c5b8f8dc6b1d1c7dac530a4c2375eedcd',)
>
>
> Executing query:
>         SELECT type FROM ImagingFileTypes
>
>
> Executing query:
>         SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING
> (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s
>
> With arguments:
>
> ('ee91658a67d7d96007f6f0c625cceb700c9fa5bbecf34cea3cae8f1445a6efce5f735b2ab68d26717a6aaa418808b3d8fb30b03e03ae1982b0eeed834429d081',)
>
>
> Executing query:
>         SELECT type FROM ImagingFileTypes
>
>
> Executing query:
>         SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING
> (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s
>
> With arguments:
>
> ('749c00e5aa64583c91f29672b228b9d89e691cef7d8f3380eefbc1fee466bd3d987fd7178001f0e4f63684a50e139a7bb8ed197981d1588ffc4dfc4eed8278b8',)
> ===
>
> Thank you for your help.
>
> Best,
> Alfredo.
>
> <Screen Shot 2021-06-06 at 4.44.57 PM.png>
>
> On May 13, 2021, at 10:08 AM, Cecile Madjar <cecile.madjar at mcin.ca> wrote:
>
>         External Email - Use Caution
>
> Hi Alfredo,
>
> The requirement of not having any special characters in the Participant ID
> or Session ID comes from BIDS and the libraries used to parse the BIDS
> dataset before insertion into LORIS.
>
> On the LORIS front:
>
>    - the ID used next to sub-<ID> will be used for the PSCID field of the
>    candidate table (the CandID field being automatically generated as 6 random
>    digits by the insertion script).
>    - the ID used next to ses-<ID> will be used for the visit label field
>    of the session table
>
> There might be another solution for you: modifying the file
> https://github.com/aces/Loris-MRI/blob/main/python/lib/bids.json
> <https://secure-web.cisco.com/1GvLDW-g8RekA6xfXzTQGS9x-cvm3VJ64Me4gEy4E2Yh3_J_-W9ZJiP9DEv9M9oH_JN3CJGEaRyJUvOCUzrNqMXl0tpX8TJ_Cfrs5JA9p_ICYgDGAU_HcPRpK0aHBkRHCW9mHZTixcIoY9EEim1Tt4JhQWyQlJpaTKY-bd1bc5O1lxwHz6sj4nGQ5bXFiDLYRP4Y8DWJzxYpm8EMaj_QYRiVdUOwHiSGfVGMouhDDMYYwBHrEEdV1Ompf5QRyVkSSAJaGL2Q3tSABSa8USdVkOQ/https%3A%2F%2Fgithub.com%2Faces%2FLoris-MRI%2Fblob%2Fmain%2Fpython%2Flib%2Fbids.json>
> so that it matches the string used for your data. You would need to modify
> the pattern for subject and session in that file to match what you have. In
> theory, once done, pybids should be able to read your dataset.
>
> Hope this helps,
>
> C?cile
>
> On Thu, May 13, 2021 at 9:47 AM Morales Pinzon, Alfredo <
> AMORALESPINZON at bwh.harvard.edu> wrote:
>
>> Hi C?cile,
>>
>> Is it possible to modify your IDs to be alphanumerical?
>>
>>
>> It is possible but puts more burden on our side as every single file now
>> has to be modified. What would be best format for LORIS for the user ids
>> (e.g., six digits) and the visit labels (e.g., alphanumerical without
>> dash)? I just want to make sure we are compliant with LORIS requirements
>> before modifying the bids files.
>>
>> Best,
>> Alfredo.
>>
>> On May 11, 2021, at 6:34 PM, Cecile Madjar <cecile.madjar at mcin.ca> wrote:
>>
>>         External Email - Use Caution
>>
>> Hi Alfredo,
>>
>> It looks like the pybids is not able to read your BIDS dataset, which is
>> why it returns an empty list of subjects.
>>
>> I just tried on the python terminal to load your dataset on pybids and it
>> does not find any subject or visit. This is probably due to the name of the
>> subject that contains special characters. That will also be problematic for
>> the session label of your dataset. I guess subjects in BIDS should remain
>> alphanumerical only. Is it possible to modify your IDs to be alphanumerical?
>>
>> C?cile
>>
>> On Tue, May 11, 2021 at 5:37 PM Morales Pinzon, Alfredo <
>> AMORALESPINZON at bwh.harvard.edu> wrote:
>>
>>> Hi C?cile,
>>>
>>> I just shared the dataset in the sftp in the following path:
>>> 2021_05_11/caseCecile_BIDS.zip
>>>
>>> Additionally, here is the output of the script after adding the prints
>>> in the PR 617:
>>>
>>> ===
>>> /PROJECT/bin/mri/python_virtualenvs/loris-mri-python/lib/python3.6/site-packages/bids/layout/validation.py:46:
>>> UserWarning: The ability to pass arguments to BIDSLayout that control
>>> indexing is likely to be removed in future; possibly as early as PyBIDS
>>> 0.14. This includes the `config_filename`, `ignore`, `force_index`, and
>>> `index_metadata` arguments. The recommended usage pattern is to initialize
>>> a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout
>>> via the `indexer` argument.
>>>   warnings.warn("The ability to pass arguments to BIDSLayout that
>>> control "
>>>
>>> Connecting to:
>>>         database: DATABASE
>>>         username: USERDATABASE
>>>         hostname: 111.222.333.44
>>>         port    : 3306
>>>
>>>
>>> Executing query:
>>>         SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM
>>> ConfigSettings WHERE Name = %s);
>>>
>>> With arguments:
>>>         ('default_bids_vl',)
>>>
>>>
>>> Executing query:
>>>         SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM
>>> ConfigSettings WHERE Name = %s);
>>>
>>> With arguments:
>>>         ('dataDirBasepath',)
>>>
>>> Loading the BIDS dataset with BIDS layout library...
>>>         => BIDS dataset loaded with BIDS layout
>>>
>>> Grepping candidates from the BIDS layout...
>>> Validating the list of participants...
>>>
>>> ERROR: Participant ID mismatch between participants.tsv and raw data
>>> found in the BIDS directory/n
>>> CFTY720D2306.0102.00001is missing from the BIDS Layout
>>>
>>> List of subjects parsed by the BIDS layout:
>>> ===
>>>
>>> Best,
>>> Alfredo.
>>>
>>> On May 11, 2021, at 12:01 PM, C?cile Madjar <cecile.madjar at mcin.ca>
>>> wrote:
>>>
>>>         External Email - Use Caution
>>>
>>> Sounds like a plan. Let me know once it is uploaded and I will ask
>>> Nicolas to give me access.
>>>
>>> Thanks!
>>>
>>> PS: you can view the diff on the PR and simply copy the code over to
>>> your files if easier
>>>
>>> On May 11, 2021, at 11:47 AM, Morales Pinzon, Alfredo <
>>> AMORALESPINZON at BWH.HARVARD.EDU> wrote:
>>>
>>> Hi Cecile,
>>>
>>> In my testing dataset I have only one subject, I can share it with you
>>> in the LORIS sftp, does that work for you?
>>>
>>> In the mean time I will diff and pull your PR, and then execute it. I?ll
>>> send you the output asap.
>>>
>>> Best,
>>> Alfredo.
>>>
>>> On May 11, 2021, at 11:36 AM, Cecile Madjar <cecile.madjar at mcin.ca>
>>> wrote:
>>>
>>>         External Email - Use Caution
>>>
>>> Hi Alfredo,
>>>
>>> I do not have access to your dataset but is it possible that there is
>>> one or more sub-<ID> in the participants.tsv that are different from the
>>> sub-<ID> used in the directory structure? (a.k.a., folders sub-<ID> and
>>> filenames that should start with sub-<ID>.)
>>>
>>> This function is specifically cross checking the list provided in the
>>> participants.tsv file and the information gathered by the BIDSLayout
>>> library that read your BIDS dataset.
>>>
>>> I created a PR on LORIS-MRI to elaborate the log from the function you
>>> just sent: https://github.com/aces/Loris-MRI/pull/617
>>> <https://secure-web.cisco.com/17BHyJzFWfexg2et-D1qxjFw7-MsSpwCyZsKHDjuQa1Iruj-GV8fZRnaNOQe7rTfFaljPkHD9E0Kch6nwzSvbbe_48vUcKT3ozGDvxPfpe248W0Nml3EbqwefqTQo6V0gIBseIrXtJ4qwpUnrLVYpwZQyTPaDCbkm_NqrpLrBeXaHi-m5KCVluJqMYsb1DZBrqFL_XKJiIevFuiValC7eTehDRbTek9A5RvYsPN6hDTzEHFPQfCzzRD6W_iQSp80c_PbAKgxDUVLU2cePcP45pg/https%3A%2F%2Fgithub.com%2Faces%2FLoris-MRI%2Fpull%2F617> Maybe
>>> try adding the print statement in the diff of the PR to your code and rerun
>>> to get a better idea of what is going on?
>>>
>>> Let me know how it goes,
>>>
>>> C?cile
>>>
>>>
>>> On May 10, 2021, at 4:15 PM, Morales Pinzon, Alfredo <
>>> AMORALESPINZON at bwh.harvard.edu> wrote:
>>>
>>> Dear LorisDev team.
>>>
>>> I am running the BIDS uploading functionality of LORIS (from this PR
>>> suggested by Cecile: https://github.com/aces/Loris-MRI/pull/558/files
>>> <https://secure-web.cisco.com/1eWQtSLH8tHTpC1qsinKLnr4amf7Vdt1MLFNigX4jKdNz2mAoHV9ejLLcHZh1a0CUaUlzZMYTU9qzLaUEYYpyP1cl5UCR8e02WTmdMZUKXBNDeD8-cV2O5yTv8cgci7szP_MIGsvZ7FCjhqVmibPUnMFYbs6uYHGXZHOB5MsouWYLRHh89rJtQaFS8DOg4jfbwjp1bxwnO4FyuD3aKRv17dw9zoczaaXWnkbXXr87i6NQKYmA1NXhjM570IkyzSKuq-dlKj94vTZIlAChD5aFeg/https%3A%2F%2Fgithub.com%2Faces%2FLoris-MRI%2Fpull%2F558%2Ffiles> )
>>> but I?m getting the following error (see all the output below):
>>>
>>> ===
>>> ERROR: Participant ID mismatch between participants.tsv and raw data
>>> found in the BIDS directory/n
>>> ===
>>>
>>> After some debugging I saw that the array subjects in the following line
>>> is empty:
>>>
>>> https://github.com/laemtl/Loris-MRI/blob/3742522fa46b0ee2b9e384e30650a0c1f425c7af/python/lib/bidsreader.py#L134
>>> <https://secure-web.cisco.com/1sAIMc1N1U7sTpJYVEGfljx8iP9AUjxof1GeCQwNPKhd82OAXEZmdHoYu7n3Uq8jLaO-quBK1igoIw9Oo7d5JVNCfb3MNy6u-cvR2KkL-rfKwSOW_ovP_V5YKydvDMoE1ltw4MO29keFq-JiTymIk8Wm8TbXeHKxF2njboK6-6bctzRI-5K2tWwz82zUZyzDxyFdn4zXPqI8AihHeuWQQG3GOGgPSbRP-GOUfwbu2eVmvL_P_ldZetO1qjKowic0Gqnt-a9XN_8GORD5BLNlxTw/https%3A%2F%2Fgithub.com%2Flaemtl%2FLoris-MRI%2Fblob%2F3742522fa46b0ee2b9e384e30650a0c1f425c7af%2Fpython%2Flib%2Fbidsreader.py%23L134>
>>>
>>> I thought the issue was that we have symbolic links in our folder
>>> structure but after setting a testing example I am getting the same exact
>>> error.
>>>
>>> Any ideas why this is happening? The error does not help much. I?m happy
>>> to share the testing case, with all the BIDS structure to a LORIS
>>> developer, if this is helpful.
>>>
>>> Looking forward to hearing from you.
>>>
>>> Best.
>>> Alfredo.
>>>
>>> ===
>>> /PROJECT/bin/mri/python_virtualenvs/loris-mri-python/lib/python3.6/site-packages/bids/layout/validation.py:46:
>>> UserWarning: The ability to pass arguments to BIDSLayout that control
>>> indexing is likely to be removed in future; possibly as early as PyBIDS
>>> 0.14. This includes the `config_filename`, `ignore`, `force_index`, and
>>> `index_metadata` arguments. The recommended usage pattern is to initialize
>>> a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout
>>> via the `indexer` argument.
>>>   warnings.warn("The ability to pass arguments to BIDSLayout that
>>> control "
>>>
>>> Connecting to:
>>>         database: DATABASE
>>>         username: USERDATABASE
>>>         hostname: 111.222.333.444
>>>         port    : 3306
>>>
>>>
>>> Executing query:
>>>         SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM
>>> ConfigSettings WHERE Name = %s);
>>>
>>> With arguments:
>>>         ('default_bids_vl',)
>>>
>>>
>>> Executing query:
>>>         SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM
>>> ConfigSettings WHERE Name = %s);
>>>
>>> With arguments:
>>>         ('dataDirBasepath',)
>>>
>>> Loading the BIDS dataset with BIDS layout library...
>>>
>>>         => BIDS dataset loaded with BIDS layout
>>>
>>> Grepping candidates from the BIDS layout...
>>> Validating the list of participants...
>>>
>>> ERROR: Participant ID mismatch between participants.tsv and raw data
>>> found in the BIDS directory/n
>>> ===
>>>
>>> The information in this e-mail is intended only for the person to whom
>>> it is addressed. If you believe this e-mail was sent to you in error and
>>> the e-mail contains patient information, please contact the Mass General
>>> Brigham Compliance HelpLine at
>>> http://www.massgeneralbrigham.org/complianceline
>>> <http://secure-web.cisco.com/1qM_d-YnL441NX1CgLDc0-4p5P74OBrZe4qz8DQn4NKFJlMAA7HiKjibIOLkeFbmot5sDXcFfxupjvo_jX8vC9aYNvFDnADUwQFAF2ETaANXw3yIZaf8bb7_SvUXAkTxwK31eivXt_HlY2DxnGsZ739ls6VersNRKPQN2plxFNbqAzR65ZCFC8YSBrs5Vrcr1GWNsx7YN7RWdCvcDN1j8l1v9uD8Je4NlxsZBp9G86xq4w2ATmf4ENiL-6RShyNpajtkjZKD6TUD0FKl1yu9EdA/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline>
>>>  . If the e-mail was sent to you in error but does not contain patient
>>> information, please contact the sender and properly dispose of the e-mail.
>>>
>>> Please note that this e-mail is not secure (encrypted).  If you do not
>>> wish to continue communication over unencrypted e-mail, please notify the
>>> sender of this message immediately.  Continuing to send or respond to
>>> e-mail after receiving this message means you understand and accept this
>>> risk and wish to continue to communicate over unencrypted e-mail.
>>> _______________________________________________
>>> Loris-dev mailing list
>>> Loris-dev at bic.mni.mcgill.ca
>>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>> <https://secure-web.cisco.com/1nZGLPRvHAc4QXkbuPkDbTk6X7cLRpZjuKhOBth4NO62r205kfGQbnAX-YD9BjpSJaYX2q-XFXkB_JSF0s5z2ahDn5jCwO30vrTUlIZlEANwK8y3vIbbTS3OKeQl5HWbDL3FviFQTAC2JkKwXTysKFx31cF_fdj3Ldph7SeZSr25P3BOdkNzQM7CwOgGsZ74Icj-bh17DcnTl_TpfPBvtKvWdT9JJgzhO4xgU1diPaWsF1-yCLwXh4xN8u993AJYONV8vaXR3KRoKV8-QxssqZw/https%3A%2F%2Fmailman.bic.mni.mcgill.ca%2Fmailman%2Flistinfo%2Floris-dev>
>>>
>>>
>>> The information in this e-mail is intended only for the person to whom
>>> it is addressed. If you believe this e-mail was sent to you in error and
>>> the e-mail contains patient information, please contact the Mass General
>>> Brigham Compliance HelpLine at
>>> http://www.massgeneralbrigham.org/complianceline
>>> <http://secure-web.cisco.com/1AhAU1QrkHn5uAqLbnmi1HyXx_kpI9yR6ZBTDEz2E2EH9ZJ8W5lI94E700CC6tH7ISW-FmwwtetLtDvUs671_9WDXfhJBJZUlAUiC75p9k2iniolB83RMn1vpjTql7mdvGqhQFlpZCisj05nFxy4_7JaawBv-2YfBcGJkgmL4k4UTe3OZ3Q6CU-jPPmqEDNyFt7jVXNncRMqUUg9ZiUWmhABUAHol2wVTYpBugXYIZfYMj79LvUy54oxCyOtX5FCW00-m4IAVrvYB_EhkHxI5Qw/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline>
>>> . If the e-mail was sent to you in error but does not contain patient
>>> information, please contact the sender and properly dispose of the e-mail.
>>>
>>> Please note that this e-mail is not secure (encrypted).  If you do not
>>> wish to continue communication over unencrypted e-mail, please notify the
>>> sender of this message immediately.  Continuing to send or respond to
>>> e-mail after receiving this message means you understand and accept this
>>> risk and wish to continue to communicate over unencrypted e-mail.
>>>
>>>
>>>
>>> The information in this e-mail is intended only for the person to whom
>>> it is addressed. If you believe this e-mail was sent to you in error and
>>> the e-mail contains patient information, please contact the Mass General
>>> Brigham Compliance HelpLine at
>>> http://www.massgeneralbrigham.org/complianceline
>>> <http://secure-web.cisco.com/1Vs76oaBwBuqLoMP4tDeksrqcCgDkogMEbBScJ4ejcwSa6gekqJpFoL6YNIznACwU31BdOoVLyr1lD6sf6FJ0bhDvv_VDNM4ihqPNDRpk_MBie2PPM8xVjhZzqKoCHhcAwaJVO6JrBb8QVg0k78kxRXH7d0wKdNPAHrs292D3K2KcEskKWx2YgQ-McThsXUDOsdj4Y99wDiDj5fzeT-glwFo2ZGa6-7cdKO2jHZVoiG8_XlvkoA76TtLIZxN04JfUn5UFQZ0p3J-jLf5RR8u_7A/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline>
>>> . If the e-mail was sent to you in error but does not contain patient
>>> information, please contact the sender and properly dispose of the e-mail.
>>>
>>> Please note that this e-mail is not secure (encrypted).  If you do not
>>> wish to continue communication over unencrypted e-mail, please notify the
>>> sender of this message immediately.  Continuing to send or respond to
>>> e-mail after receiving this message means you understand and accept this
>>> risk and wish to continue to communicate over unencrypted e-mail.
>>>
>>
>> The information in this e-mail is intended only for the person to whom it
>> is addressed. If you believe this e-mail was sent to you in error and the
>> e-mail contains patient information, please contact the Mass General
>> Brigham Compliance HelpLine at
>> http://www.massgeneralbrigham.org/complianceline
>> <http://secure-web.cisco.com/1n2UbpIP1Hiy1wedtWg9s-6NXBNM5GXQsk4yul5KQtRtD-fpCcwhqxfc9xPmkoXTkvQiEilB7x5cMjoMnEfP7sVBe6N4BZFTL6lnTA5ZoU6_eL-QZrxz4bet8uKLeH8lFWIbsg3Tq1FqZpdjEzQ9aSVMRC-CNrE0-CJn7xfPqJefF9I1-_5OZcKb9QkXl2Rg_xjO8CfyirontcnUgEOHlJ5anCG65dp3TNqKwXi_HsVUusIi9c6KJxpEoiY-xH-7k_UAsbpirJkzRosIoJNyevA/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline>
>> . If the e-mail was sent to you in error but does not contain patient
>> information, please contact the sender and properly dispose of the e-mail.
>>
>> Please note that this e-mail is not secure (encrypted).  If you do not
>> wish to continue communication over unencrypted e-mail, please notify the
>> sender of this message immediately.  Continuing to send or respond to
>> e-mail after receiving this message means you understand and accept this
>> risk and wish to continue to communicate over unencrypted e-mail.
>>
>
>
> The information in this e-mail is intended only for the person to whom it
> is addressed. If you believe this e-mail was sent to you in error and the
> e-mail contains patient information, please contact the Mass General
> Brigham Compliance HelpLine at
> http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent
> to you in error but does not contain patient information, please contact
> the sender and properly dispose of the e-mail.
>
> Please note that this e-mail is not secure (encrypted).  If you do not
> wish to continue communication over unencrypted e-mail, please notify the
> sender of this message immediately.  Continuing to send or respond to
> e-mail after receiving this message means you understand and accept this
> risk and wish to continue to communicate over unencrypted e-mail.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210607/55e70422/attachment-0001.html>

From AMORALESPINZON at BWH.HARVARD.EDU  Thu Jun 10 18:48:52 2021
From: AMORALESPINZON at BWH.HARVARD.EDU (Morales Pinzon, Alfredo)
Date: Thu, 10 Jun 2021 22:48:52 +0000
Subject: [Loris-dev] BIDS Error
In-Reply-To: <CAMkiY3ek7Jhavv-GAaR35CYTBqNa0zE6tkLZA8HFsQr6Zt6H0Q@mail.gmail.com>
References: <8E375877-D896-4B0E-B0F7-D6BD5565CA67@bwh.harvard.edu>
 <ADFA6F92-C7E7-4FD5-85A1-4F20D1565F75@gmail.com>
 <4286E305-87C3-492A-BE31-0B83CEFD365E@bwh.harvard.edu>
 <5F1BBE56-5523-4517-ADCF-2A1DB17C952B@mcin.ca>
 <F1F354D5-8CDF-4089-A4BA-6A3A41903CDA@bwh.harvard.edu>
 <CAMkiY3cXNfprNL_VDnqFzH1og332_bF2u1Zhe+enRA0dGBpWHQ@mail.gmail.com>
 <CE0C1F1D-862F-4030-BA2D-401BE3110632@bwh.harvard.edu>
 <CAMkiY3cMR4CRYE1Y=u5TGwpOsPk7Q3UdXMmit4V7RR3q8jqpeg@mail.gmail.com>
 <BCBD10CB-5400-4FF3-949E-5641665587E3@bwh.harvard.edu>
 <C6850CF7-FC93-46A7-BBBB-E83222C44FE5@bwh.harvard.edu>
 <CAMkiY3ek7Jhavv-GAaR35CYTBqNa0zE6tkLZA8HFsQr6Zt6H0Q@mail.gmail.com>
Message-ID: <A90AE772-28BA-44EB-819C-54A4B5C1D9D9@bwh.harvard.edu>

Hi C?cile,

I tried to execute the query but there are some variables I don?t know what to set to:

$PendingFailSubquery
$NewDataSubquery
$PendingNewquery
$modalities_subquery
$left_joins

So, I executed the following:

===
SELECT
              p.Name as Site,
              c.PSCID as PSCID,
              c.CandID as DCCID,
              Project.Name as project,
              s.Visit_label as visitLabel,
              DATE_FORMAT(MIN(pf.Value), '%Y-%m-%d') as First_Acquisition,
              FROM_UNIXTIME(MIN(f.InsertTime)) as First_Insertion,
              FROM_UNIXTIME(MAX(fqc.QCLastChangeTime)) as Last_QC,
              GROUP_CONCAT(DISTINCT OutputType) as Links,
              s.ID as sessionID,
              GROUP_CONCAT(DISTINCT modality.Scan_type) as sequenceType,
              s.CenterID as CenterID,
              c.Entity_type as entityType,
              s.ProjectID
            FROM psc AS p
              JOIN session s ON (s.CenterID=p.CenterID)
              JOIN candidate c ON (c.CandID=s.CandID)
              LEFT JOIN Project ON (s.ProjectID=Project.ProjectID)
              JOIN files f ON (f.SessionID=s.ID)
              LEFT JOIN files_qcstatus fqc ON (fqc.FileID=f.FileID)
              JOIN parameter_file pf ON (f.FileID=pf.FileID)
              JOIN parameter_type pt USING (ParameterTypeID)
              LEFT JOIN mri_scan_type modality ON
                (f.AcquisitionProtocolID=modality.ID)
            WHERE
              s.Active = 'Y' AND
              f.FileType IN ('mnc', 'nii') AND
              pt.Name='acquisition_date'
            GROUP BY s.ID
            ORDER BY c.PSCID, s.Visit_label;
===

And I got 34415 rows, here is the last part of it:

===
| 723-DKC-1     | BGH723308 | 925320 | ADVANCE_ATTAIN  | w024             | 2012-05-09        | 2021-03-21 01:26:41 | NULL    | ct1unf.0.87,native                |    123453 | ct1unf.0.87,flair,mtOFF,mtON,pdw,t1c,t1p,t2w        |      842 | Human      |        11 |
| 723-DKC-1     | BGH723308 | 925320 | ADVANCE_ATTAIN  | w048             | 2012-10-10        | 2021-03-21 01:33:28 | NULL    | ct1unf.0.87,native                |    127243 | ct1unf.0.87,flair,mtOFF,mtON,pdw,t1c,t1p,t2w        |      842 | Human      |        11 |
| 723-DKC-1     | BGH723308 | 925320 | ADVANCE_ATTAIN  | w096             | 2013-09-17        | 2020-11-25 23:36:42 | NULL    | native                            |    123465 | flair,mtOFF,mtON,pdw,t1c,t1p,t2w                    |      842 | Human      |        11 |
+---------------+-----------+--------+-----------------+------------------+-------------------+---------------------+---------+-----------------------------------+-----------+-----------------------------------------------------+----------+------------+-----------+
34415 rows in set, 12 warnings (12.43 sec)
===

I added the name of project where I am uploading BIDS to in the WHERE:

===
WHERE ...
         AND     Project.Name = 'INFORMS'
===

I got an empty set. Based on this what should we try?

Regarding the Scan_done I manually modified it but I get an error in the web page:

[cid:1AE14BA4-BC51-4453-8988-C8B86823A12D]

I am happy to do a Zoom call if that helps us move faster.

Best,
Alfredo.

On Jun 7, 2021, at 4:50 PM, Cecile Madjar <cecile.madjar at mcin.ca<mailto:cecile.madjar at mcin.ca>> wrote:


        External Email - Use Caution

Hi Alfredo,

Interesting. I was expecting to see the Active field set to NULL which would have explained why you do not see anything in the imaging browser. My next theory is that you do not have anything for the parameter_type name 'acquisition_date' which is very likely to be the case for a BIDS dataset with no date information.

Does the following query return anything for you? That is the query that is supposed to return data to the imaging browser (that you can find in modules/imaging_browser/php/imagingbrowserrowprovisioner.class.inc<https://secure-web.cisco.com/1weK8jADYBDQk4jpPzRNm8rD0PwVDNCMSqr5yG572iHCS4Y0zqqLqBl9mPaA0wu2sekKrklkT0E-yyzFbYhZ1AepajvvGWnxe688Ok1sCPLPdfeAONPWXzCeqgszXpJ9sgBtvWLUICHxS_J9Ffa7ZOjbld8AX9aUbjG_oYI2dMGGtk_ThMB_SQuneDiRqmTctykQ3dAkWL0sA76pxQnBl3ftDrrfNhBPtIWqATw3ihLuLtYmABCIlnSK4c6I14etC2cOWz4rM78zGhuKp5s0-NMhVRG9h3GS_fcxp9Ltd-TBrjcbTsBuTwN-BmuzRKhZv/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fimaging_browser%2Fphp%2Fimagingbrowserrowprovisioner.class.inc>).

SELECT
p.Name as Site,
c.PSCID as PSCID,
c.CandID as DCCID,
Project.Name as project,
s.Visit_label as visitLabel,
$PendingFailSubquery as Visit_QC_Status,
DATE_FORMAT(MIN(pf.Value), \"%Y-%m-%d\") as First_Acquisition,
FROM_UNIXTIME(MIN(f.InsertTime)) as First_Insertion,
FROM_UNIXTIME(MAX(fqc.QCLastChangeTime)) as Last_QC,
$NewDataSubquery as New_Data,
GROUP_CONCAT(DISTINCT OutputType) as Links,
s.ID as sessionID,
GROUP_CONCAT(DISTINCT modality.Scan_type) as sequenceType,
$PendingNewquery as pending,
$modalities_subquery
s.CenterID as CenterID,
c.Entity_type as entityType,
s.ProjectID
FROM psc AS p
JOIN session s ON (s.CenterID=p.CenterID)
JOIN candidate c ON (c.CandID=s.CandID)
LEFT JOIN Project ON (s.ProjectID=Project.ProjectID)
JOIN files f ON (f.SessionID=s.ID)
LEFT JOIN files_qcstatus fqc ON (fqc.FileID=f.FileID)
JOIN parameter_file pf ON (f.FileID=pf.FileID)
JOIN parameter_type pt USING (ParameterTypeID)
LEFT JOIN mri_scan_type modality ON
(f.AcquisitionProtocolID=modality.ID)
$left_joins
WHERE
s.Active = 'Y' AND
f.FileType IN ('mnc', 'nii') AND
pt.Name='acquisition_date'
GROUP BY s.ID
ORDER BY c.PSCID, s.Visit_label

If you get an empty result, try removing the following from the WHERE section of the query above:

              .....  AND
             pt.Name='acquisition_date'

If my second theory is right, you should then see rows returned, in which case you will need to modify the query in the imaging_browser code.
Let me know how those go.

Regarding the Scan_type, I should probably be populated by the insertion script so this may be a bug on the bids_import scripts specific to the imaging component. In the meantime, you can indeed set the flag to Y for the sessions you know contain scan data.

Hope this helps,

C?cile


On Mon, Jun 7, 2021 at 12:27 PM Morales Pinzon, Alfredo <AMORALESPINZON at bwh.harvard.edu<mailto:AMORALESPINZON at bwh.harvard.edu>> wrote:
Hi C?cile,

Thank you for prompt answer. Here is the output of the query:

===
mysql> SELECT CandID, Visit_label, Active, Scan_done, CenterID, ProjectID, SubprojectID FROM session WHERE Visit_label = 'screening' AND CandID=207019;
+--------+-------------+--------+-----------+----------+-----------+--------------+
| CandID | Visit_label | Active | Scan_done | CenterID | ProjectID | SubprojectID |
+--------+-------------+--------+-----------+----------+-----------+--------------+
| 207019 | screening   | Y      | NULL      |      843 |        12 |            2 |
+--------+-------------+--------+-----------+----------+-----------+--------------+
1 row in set (0.00 sec)
===

I guess the ?Scan_done? should be ?Y? instead of ?NULL?. Is this something that the BIDS importing script should take care of? Is the fix as simple as just setting Scan_done to ?Y??

Best,
Alfredo.

On Jun 6, 2021, at 4:48 PM, Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU<mailto:AMORALESPINZON at BWH.HARVARD.EDU>> wrote:

Hi C?cile,

We managed to insert the candidates and the visits, and create the specific file names for BIDS before actually launching the BIDS script. The script finally worked, however I don?t see the images in LORIS. I am adding down below the bids log insertion. Is there something additional I need to do to be able to see the images? In the web site I see the candidate and the visit but the column ?Imaging Scan Done? is marked with a question mark (see attachment).

===
/PROJECT/bin/mri/python_virtualenvs/loris-mri-python/lib/python3.6/site-packages/bids/layout/validation.py:46: UserWarning: The ability to pass arguments to BIDSLayout that control indexing is likely to be removed in future; possibly as early as PyBIDS 0.14. This includes the `config_filename`, `ignore`, `force_index`, and `index_metadata` arguments. The recommended usage pattern is to initialize a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout via the `indexer` argument.
  warnings.warn("The ability to pass arguments to BIDSLayout that control "

Connecting to:
        database: DATABASE
        username: USERNAME
        hostname: 111.222.333440
        port    : 1234


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('default_bids_vl',)


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('dataDirBasepath',)

AMP::bids_dir: /DATA/CAND_A/bids
Loading the BIDS dataset with BIDS layout library...

AMP::bids_config: /PROJECT/bin/mri/python/lib/bids.json
root: /DATA/CAND_A/bids
target:: /DATA/CAND_A/bids/dataset_description.json
        => BIDS dataset loaded with BIDS layout

Grepping candidates from the BIDS layout...
Validating the list of participants...
subjects: ['207019']
        => Passed validation of the list of participants

        => List of participants found:
                207019


Grepping list of sessions from the BIDS layout...
        => List of sessions found:

                207019: screening


Grepping the different modalities from the BIDS layout...
        => Done grepping the different modalities from the BIDS layout


Executing query:
        SELECT * FROM candidate WHERE PSCID = %s

With arguments:
        ('207019',)
Executing query:
        SELECT SubprojectID FROM subproject WHERE title = %s

With arguments:
        ['Experimental']


Executing query:
        SELECT * FROM session WHERE CandID = %s AND Visit_label = %s

With arguments:
        ('622350', 'screening')


Executing query:
        SELECT * FROM candidate WHERE PSCID = %s

With arguments:
        ('207019',)


Executing query:
        SELECT SubprojectID FROM subproject WHERE title = %s

With arguments:
        ['Experimental']


Executing query:
        SELECT * FROM session WHERE CandID = %s AND Visit_label = %s

With arguments:
        ('622350', 'screening')


Executing query:
        SELECT type FROM ImagingFileTypes


Executing query:
        SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s

With arguments:
        ('248248fb303a762548e0be602b0b6bd6c6f33b4c860898c69faa8273be7a80f6d93b6413ec3bbeaadf22005fac04b07e9c08487e7b0a6690f64c2781d77d69b4',)


Executing query:
        SELECT type FROM ImagingFileTypes


Executing query:
        SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s

With arguments:
        ('b16bc029ea1cb8271a0728785d04eddaaa66e4d89c09e990552667b3c5598d17cee045b5b4ee3480dc2b8e08b5b05ca4e53c57f50b13b26f6467b9e4f455d09e',)


Executing query:
        SELECT type FROM ImagingFileTypes


Executing query:
        SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s

With arguments:
        ('44346136db049d0faf867a34d57d5714803a320b070013da0e1cf66e064f4dc9db92533f8aa8caaafcb78ce141c5444c5b8f8dc6b1d1c7dac530a4c2375eedcd',)


Executing query:
        SELECT type FROM ImagingFileTypes


Executing query:
        SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s

With arguments:
        ('ee91658a67d7d96007f6f0c625cceb700c9fa5bbecf34cea3cae8f1445a6efce5f735b2ab68d26717a6aaa418808b3d8fb30b03e03ae1982b0eeed834429d081',)


Executing query:
        SELECT type FROM ImagingFileTypes


Executing query:
        SELECT f.FileID, f.File FROM files AS f JOIN parameter_file USING (FileID) JOIN parameter_type USING (ParameterTypeID) WHERE Value=%s

With arguments:
        ('749c00e5aa64583c91f29672b228b9d89e691cef7d8f3380eefbc1fee466bd3d987fd7178001f0e4f63684a50e139a7bb8ed197981d1588ffc4dfc4eed8278b8',)
===

Thank you for your help.

Best,
Alfredo.

<Screen Shot 2021-06-06 at 4.44.57 PM.png>

On May 13, 2021, at 10:08 AM, Cecile Madjar <cecile.madjar at mcin.ca<mailto:cecile.madjar at mcin.ca>> wrote:


        External Email - Use Caution

Hi Alfredo,

The requirement of not having any special characters in the Participant ID or Session ID comes from BIDS and the libraries used to parse the BIDS dataset before insertion into LORIS.

On the LORIS front:

  *   the ID used next to sub-<ID> will be used for the PSCID field of the candidate table (the CandID field being automatically generated as 6 random digits by the insertion script).
  *   the ID used next to ses-<ID> will be used for the visit label field of the session table

There might be another solution for you: modifying the file https://github.com/aces/Loris-MRI/blob/main/python/lib/bids.json<https://secure-web.cisco.com/1GvLDW-g8RekA6xfXzTQGS9x-cvm3VJ64Me4gEy4E2Yh3_J_-W9ZJiP9DEv9M9oH_JN3CJGEaRyJUvOCUzrNqMXl0tpX8TJ_Cfrs5JA9p_ICYgDGAU_HcPRpK0aHBkRHCW9mHZTixcIoY9EEim1Tt4JhQWyQlJpaTKY-bd1bc5O1lxwHz6sj4nGQ5bXFiDLYRP4Y8DWJzxYpm8EMaj_QYRiVdUOwHiSGfVGMouhDDMYYwBHrEEdV1Ompf5QRyVkSSAJaGL2Q3tSABSa8USdVkOQ/https%3A%2F%2Fgithub.com%2Faces%2FLoris-MRI%2Fblob%2Fmain%2Fpython%2Flib%2Fbids.json> so that it matches the string used for your data. You would need to modify the pattern for subject and session in that file to match what you have. In theory, once done, pybids should be able to read your dataset.

Hope this helps,

C?cile

On Thu, May 13, 2021 at 9:47 AM Morales Pinzon, Alfredo <AMORALESPINZON at bwh.harvard.edu<mailto:AMORALESPINZON at bwh.harvard.edu>> wrote:
Hi C?cile,

Is it possible to modify your IDs to be alphanumerical?

It is possible but puts more burden on our side as every single file now has to be modified. What would be best format for LORIS for the user ids (e.g., six digits) and the visit labels (e.g., alphanumerical without dash)? I just want to make sure we are compliant with LORIS requirements before modifying the bids files.

Best,
Alfredo.

On May 11, 2021, at 6:34 PM, Cecile Madjar <cecile.madjar at mcin.ca<mailto:cecile.madjar at mcin.ca>> wrote:


        External Email - Use Caution

Hi Alfredo,

It looks like the pybids is not able to read your BIDS dataset, which is why it returns an empty list of subjects.

I just tried on the python terminal to load your dataset on pybids and it does not find any subject or visit. This is probably due to the name of the subject that contains special characters. That will also be problematic for the session label of your dataset. I guess subjects in BIDS should remain alphanumerical only. Is it possible to modify your IDs to be alphanumerical?

C?cile

On Tue, May 11, 2021 at 5:37 PM Morales Pinzon, Alfredo <AMORALESPINZON at bwh.harvard.edu<mailto:AMORALESPINZON at bwh.harvard.edu>> wrote:
Hi C?cile,

I just shared the dataset in the sftp in the following path: 2021_05_11/caseCecile_BIDS.zip

Additionally, here is the output of the script after adding the prints in the PR 617:

===
/PROJECT/bin/mri/python_virtualenvs/loris-mri-python/lib/python3.6/site-packages/bids/layout/validation.py:46: UserWarning: The ability to pass arguments to BIDSLayout that control indexing is likely to be removed in future; possibly as early as PyBIDS 0.14. This includes the `config_filename`, `ignore`, `force_index`, and `index_metadata` arguments. The recommended usage pattern is to initialize a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout via the `indexer` argument.
  warnings.warn("The ability to pass arguments to BIDSLayout that control "

Connecting to:
        database: DATABASE
        username: USERDATABASE
        hostname: 111.222.333.44
        port    : 3306


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('default_bids_vl',)


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('dataDirBasepath',)

Loading the BIDS dataset with BIDS layout library...
        => BIDS dataset loaded with BIDS layout

Grepping candidates from the BIDS layout...
Validating the list of participants...

ERROR: Participant ID mismatch between participants.tsv and raw data found in the BIDS directory/n
CFTY720D2306.0102.00001is missing from the BIDS Layout

List of subjects parsed by the BIDS layout:
===

Best,
Alfredo.

On May 11, 2021, at 12:01 PM, C?cile Madjar <cecile.madjar at mcin.ca<mailto:cecile.madjar at mcin.ca>> wrote:


        External Email - Use Caution

Sounds like a plan. Let me know once it is uploaded and I will ask Nicolas to give me access.

Thanks!

PS: you can view the diff on the PR and simply copy the code over to your files if easier

On May 11, 2021, at 11:47 AM, Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU<mailto:AMORALESPINZON at BWH.HARVARD.EDU>> wrote:

Hi Cecile,

In my testing dataset I have only one subject, I can share it with you in the LORIS sftp, does that work for you?

In the mean time I will diff and pull your PR, and then execute it. I?ll send you the output asap.

Best,
Alfredo.

On May 11, 2021, at 11:36 AM, Cecile Madjar <cecile.madjar at mcin.ca<mailto:cecile.madjar at mcin.ca>> wrote:


        External Email - Use Caution

Hi Alfredo,

I do not have access to your dataset but is it possible that there is one or more sub-<ID> in the participants.tsv that are different from the sub-<ID> used in the directory structure? (a.k.a., folders sub-<ID> and filenames that should start with sub-<ID>.)

This function is specifically cross checking the list provided in the participants.tsv file and the information gathered by the BIDSLayout library that read your BIDS dataset.

I created a PR on LORIS-MRI to elaborate the log from the function you just sent: https://github.com/aces/Loris-MRI/pull/617<https://secure-web.cisco.com/17BHyJzFWfexg2et-D1qxjFw7-MsSpwCyZsKHDjuQa1Iruj-GV8fZRnaNOQe7rTfFaljPkHD9E0Kch6nwzSvbbe_48vUcKT3ozGDvxPfpe248W0Nml3EbqwefqTQo6V0gIBseIrXtJ4qwpUnrLVYpwZQyTPaDCbkm_NqrpLrBeXaHi-m5KCVluJqMYsb1DZBrqFL_XKJiIevFuiValC7eTehDRbTek9A5RvYsPN6hDTzEHFPQfCzzRD6W_iQSp80c_PbAKgxDUVLU2cePcP45pg/https%3A%2F%2Fgithub.com%2Faces%2FLoris-MRI%2Fpull%2F617> Maybe try adding the print statement in the diff of the PR to your code and rerun to get a better idea of what is going on?

Let me know how it goes,

C?cile


On May 10, 2021, at 4:15 PM, Morales Pinzon, Alfredo <AMORALESPINZON at bwh.harvard.edu<mailto:AMORALESPINZON at bwh.harvard.edu>> wrote:

Dear LorisDev team.

I am running the BIDS uploading functionality of LORIS (from this PR suggested by Cecile: https://github.com/aces/Loris-MRI/pull/558/files<https://secure-web.cisco.com/1eWQtSLH8tHTpC1qsinKLnr4amf7Vdt1MLFNigX4jKdNz2mAoHV9ejLLcHZh1a0CUaUlzZMYTU9qzLaUEYYpyP1cl5UCR8e02WTmdMZUKXBNDeD8-cV2O5yTv8cgci7szP_MIGsvZ7FCjhqVmibPUnMFYbs6uYHGXZHOB5MsouWYLRHh89rJtQaFS8DOg4jfbwjp1bxwnO4FyuD3aKRv17dw9zoczaaXWnkbXXr87i6NQKYmA1NXhjM570IkyzSKuq-dlKj94vTZIlAChD5aFeg/https%3A%2F%2Fgithub.com%2Faces%2FLoris-MRI%2Fpull%2F558%2Ffiles> ) but I?m getting the following error (see all the output below):

===
ERROR: Participant ID mismatch between participants.tsv and raw data found in the BIDS directory/n
===

After some debugging I saw that the array subjects in the following line is empty:
https://github.com/laemtl/Loris-MRI/blob/3742522fa46b0ee2b9e384e30650a0c1f425c7af/python/lib/bidsreader.py#L134<https://secure-web.cisco.com/1sAIMc1N1U7sTpJYVEGfljx8iP9AUjxof1GeCQwNPKhd82OAXEZmdHoYu7n3Uq8jLaO-quBK1igoIw9Oo7d5JVNCfb3MNy6u-cvR2KkL-rfKwSOW_ovP_V5YKydvDMoE1ltw4MO29keFq-JiTymIk8Wm8TbXeHKxF2njboK6-6bctzRI-5K2tWwz82zUZyzDxyFdn4zXPqI8AihHeuWQQG3GOGgPSbRP-GOUfwbu2eVmvL_P_ldZetO1qjKowic0Gqnt-a9XN_8GORD5BLNlxTw/https%3A%2F%2Fgithub.com%2Flaemtl%2FLoris-MRI%2Fblob%2F3742522fa46b0ee2b9e384e30650a0c1f425c7af%2Fpython%2Flib%2Fbidsreader.py%23L134>

I thought the issue was that we have symbolic links in our folder structure but after setting a testing example I am getting the same exact error.

Any ideas why this is happening? The error does not help much. I?m happy to share the testing case, with all the BIDS structure to a LORIS developer, if this is helpful.

Looking forward to hearing from you.

Best.
Alfredo.

===
/PROJECT/bin/mri/python_virtualenvs/loris-mri-python/lib/python3.6/site-packages/bids/layout/validation.py:46: UserWarning: The ability to pass arguments to BIDSLayout that control indexing is likely to be removed in future; possibly as early as PyBIDS 0.14. This includes the `config_filename`, `ignore`, `force_index`, and `index_metadata` arguments. The recommended usage pattern is to initialize a new BIDSLayoutIndexer with these arguments, and pass it to the BIDSLayout via the `indexer` argument.
  warnings.warn("The ability to pass arguments to BIDSLayout that control "

Connecting to:
        database: DATABASE
        username: USERDATABASE
        hostname: 111.222.333.444
        port    : 3306


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('default_bids_vl',)


Executing query:
        SELECT Value FROM Config WHERE ConfigID = (SELECT ID FROM ConfigSettings WHERE Name = %s);

With arguments:
        ('dataDirBasepath',)

Loading the BIDS dataset with BIDS layout library...

        => BIDS dataset loaded with BIDS layout

Grepping candidates from the BIDS layout...
Validating the list of participants...

ERROR: Participant ID mismatch between participants.tsv and raw data found in the BIDS directory/n
===

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/1qM_d-YnL441NX1CgLDc0-4p5P74OBrZe4qz8DQn4NKFJlMAA7HiKjibIOLkeFbmot5sDXcFfxupjvo_jX8vC9aYNvFDnADUwQFAF2ETaANXw3yIZaf8bb7_SvUXAkTxwK31eivXt_HlY2DxnGsZ739ls6VersNRKPQN2plxFNbqAzR65ZCFC8YSBrs5Vrcr1GWNsx7YN7RWdCvcDN1j8l1v9uD8Je4NlxsZBp9G86xq4w2ATmf4ENiL-6RShyNpajtkjZKD6TUD0FKl1yu9EdA/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
_______________________________________________
Loris-dev mailing list
Loris-dev at bic.mni.mcgill.ca<mailto:Loris-dev at bic.mni.mcgill.ca>
https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev<https://secure-web.cisco.com/1nZGLPRvHAc4QXkbuPkDbTk6X7cLRpZjuKhOBth4NO62r205kfGQbnAX-YD9BjpSJaYX2q-XFXkB_JSF0s5z2ahDn5jCwO30vrTUlIZlEANwK8y3vIbbTS3OKeQl5HWbDL3FviFQTAC2JkKwXTysKFx31cF_fdj3Ldph7SeZSr25P3BOdkNzQM7CwOgGsZ74Icj-bh17DcnTl_TpfPBvtKvWdT9JJgzhO4xgU1diPaWsF1-yCLwXh4xN8u993AJYONV8vaXR3KRoKV8-QxssqZw/https%3A%2F%2Fmailman.bic.mni.mcgill.ca%2Fmailman%2Flistinfo%2Floris-dev>

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/1AhAU1QrkHn5uAqLbnmi1HyXx_kpI9yR6ZBTDEz2E2EH9ZJ8W5lI94E700CC6tH7ISW-FmwwtetLtDvUs671_9WDXfhJBJZUlAUiC75p9k2iniolB83RMn1vpjTql7mdvGqhQFlpZCisj05nFxy4_7JaawBv-2YfBcGJkgmL4k4UTe3OZ3Q6CU-jPPmqEDNyFt7jVXNncRMqUUg9ZiUWmhABUAHol2wVTYpBugXYIZfYMj79LvUy54oxCyOtX5FCW00-m4IAVrvYB_EhkHxI5Qw/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.


The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/1Vs76oaBwBuqLoMP4tDeksrqcCgDkogMEbBScJ4ejcwSa6gekqJpFoL6YNIznACwU31BdOoVLyr1lD6sf6FJ0bhDvv_VDNM4ihqPNDRpk_MBie2PPM8xVjhZzqKoCHhcAwaJVO6JrBb8QVg0k78kxRXH7d0wKdNPAHrs292D3K2KcEskKWx2YgQ-McThsXUDOsdj4Y99wDiDj5fzeT-glwFo2ZGa6-7cdKO2jHZVoiG8_XlvkoA76TtLIZxN04JfUn5UFQZ0p3J-jLf5RR8u_7A/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/1n2UbpIP1Hiy1wedtWg9s-6NXBNM5GXQsk4yul5KQtRtD-fpCcwhqxfc9xPmkoXTkvQiEilB7x5cMjoMnEfP7sVBe6N4BZFTL6lnTA5ZoU6_eL-QZrxz4bet8uKLeH8lFWIbsg3Tq1FqZpdjEzQ9aSVMRC-CNrE0-CJn7xfPqJefF9I1-_5OZcKb9QkXl2Rg_xjO8CfyirontcnUgEOHlJ5anCG65dp3TNqKwXi_HsVUusIi9c6KJxpEoiY-xH-7k_UAsbpirJkzRosIoJNyevA/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.


The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/1xBLYFoRGc0cg3uWpekzzX-1F8kLpS9mSem5wdFFsyx6-5tIvMJ-VosfMt73qzxu4gF-h7iTXCxdj68nSGS01Y3dm9oswyBOaKE0Nuqw35rLqVCnghdmdzeSlGcVXX067fdYDHZ8q1IpOQjoR8F4o7jFUZcCvFjoOWZECnE0q1lmyszUSQqp6kHg9tPAhrpAqLIJTSYWcpwhdODpU8V4NLcBap4ASgj5h_QIwYyTULsfqjX2kcrzozR5XLSVn-omUM1ODjw4ZirDXaacmDVv7uHBXeeb35rbinIu-8aRspg5XCu1_NzBkQEa9Tc3mOmaN/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210610/a0738352/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screen Shot 2021-06-10 at 6.45.51 PM.png
Type: image/png
Size: 35626 bytes
Desc: Screen Shot 2021-06-10 at 6.45.51 PM.png
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210610/a0738352/attachment-0001.png>

From AMORALESPINZON at BWH.HARVARD.EDU  Thu Jun 10 19:43:23 2021
From: AMORALESPINZON at BWH.HARVARD.EDU (Morales Pinzon, Alfredo)
Date: Thu, 10 Jun 2021 23:43:23 +0000
Subject: [Loris-dev] Error insert candidates api/v0.0.3/candidates
Message-ID: <53511186-E9E5-4DDC-B1D4-DC3323113D2E@bwh.harvard.edu>

Dear LorisDev team,

I can create Candidates using the web interface using an admin account that is linked to all the project and all the sites in the system. However when I try to create a Candidate using the api api/v0.0.3/candidates I am getting the following error:

===
{
"error": "You are not affiliated with the candidate`s site"
}
===

I?m happy to run some queries in the database to figure out what?s happening. Any ideas?

Best,
Alfredo.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210610/bb349531/attachment.html>

From xavier.lecoursboucher at mcgill.ca  Mon Jun 14 11:04:02 2021
From: xavier.lecoursboucher at mcgill.ca (Xavier Lecours Boucher, Mr)
Date: Mon, 14 Jun 2021 15:04:02 +0000
Subject: [Loris-dev] Error insert candidates api/v0.0.3/candidates
In-Reply-To: <53511186-E9E5-4DDC-B1D4-DC3323113D2E@bwh.harvard.edu>
References: <53511186-E9E5-4DDC-B1D4-DC3323113D2E@bwh.harvard.edu>
Message-ID: <QB1PR01MB235636E92D8094FEFF99BB7AFC319@QB1PR01MB2356.CANPRD01.PROD.OUTLOOK.COM>

Hi Alfredo,

There is a discrepancy between the frontend and the API way to determine which site the user have, which site to give the new candidate and if a user is allowed to create a candidate at a given site. Looking at the frontend code, I can't see any validation on the site other than the content of the html dropdown.

The API checks if the user has the given site.

Here is the API code<https://github.com/aces/Loris/blob/23.0-release/modules/api/php/endpoints/candidates.class.inc#L185>:
$usersites = $user->getSiteNames();
if (!in_array($data['Candidate']['Site'], $usersites)) {
    return new \LORIS\Http\Response\JSON\Forbidden(
        'You are not affiliated with the candidate`s site'
    );
}
So, for debugging, can you add the following lines at https://github.com/aces/Loris/blob/23.0-release/modules/api/php/endpoints/candidates.class.inc#L184 , then tell me what it the response to your POST request ?

var_dump($user->getSiteNames(), $data['Candidate']['Site']);
exit;
Thank you
Xavier


<https://github.com/aces/Loris/blob/23.0-release/modules/api/php/endpoints/candidates.class.inc#L185>




________________________________
From: loris-dev-bounces at bic.mni.mcgill.ca <loris-dev-bounces at bic.mni.mcgill.ca> on behalf of Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU>
Sent: June 10, 2021 6:43 PM
To: loris-dev at bic.mni.mcgill.ca <loris-dev at bic.mni.mcgill.ca>
Cc: Rozie Arnaoutelis, Ms. <rozie.arnaoutelis at mcgill.ca>; Sridar Narayanan, Dr. <sridar.narayanan at mcgill.ca>; Douglas Arnold, Dr. <douglas.arnold at mcgill.ca>; Guttmann, Charles, M.D. <guttmann at bwh.harvard.edu>
Subject: [Loris-dev] Error insert candidates api/v0.0.3/candidates

Dear LorisDev team,

I can create Candidates using the web interface using an admin account that is linked to all the project and all the sites in the system. However when I try to create a Candidate using the api api/v0.0.3/candidates I am getting the following error:

===
{
"error": "You are not affiliated with the candidate`s site"
}
===

I?m happy to run some queries in the database to figure out what?s happening. Any ideas?

Best,
Alfredo.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.


Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210614/934b98db/attachment.html>

From AMORALESPINZON at BWH.HARVARD.EDU  Mon Jun 14 18:34:04 2021
From: AMORALESPINZON at BWH.HARVARD.EDU (Morales Pinzon, Alfredo)
Date: Mon, 14 Jun 2021 22:34:04 +0000
Subject: [Loris-dev] Error insert candidates api/v0.0.3/candidates
In-Reply-To: <QB1PR01MB235636E92D8094FEFF99BB7AFC319@QB1PR01MB2356.CANPRD01.PROD.OUTLOOK.COM>
References: <53511186-E9E5-4DDC-B1D4-DC3323113D2E@bwh.harvard.edu>
 <QB1PR01MB235636E92D8094FEFF99BB7AFC319@QB1PR01MB2356.CANPRD01.PROD.OUTLOOK.COM>
Message-ID: <5BD99457-E745-486F-BAE2-C4BB41543CBF@bwh.harvard.edu>

Hi Xavier,

Thank you for reaching out.

There is a first bug int the following line:

https://github.com/aces/Loris/blob/fc574c06f5f6c96483f22788ed446f9aa36a4783/php/libraries/User.class.inc#L71

If a user affiliated with multiple center such that the concatenation of the names of the center has more than 1024 characters, then the response is cut to 1024 and if the center name I want to use is not in there, then I get the same error. This is the case for my admin user and the more than 500 affiliation centers. We saw this with C?cile on Friday.

Here is the response of the API after the changes you suggested using a user that has few center affiliations, including the one I want to add a Candidate to:

===
array(56) {
[0]=>
string(8) "inf_0102"
[1]=>
string(8) "inf_0103"
...
[55]=>
string(8) "inf_0313"
}
string(3) "BGI"
===

This is the same output when using my admin user:

===
array(114) {
[0]=>
string(9) "101-KGH-1"
[1]=>
string(9) "101-LHS-1"
...
[111]=>
string(9) "272-CAA-1"
[112]=>
string(9) "273-DDM-1"
[113]=>
string(1) "2"
}
string(3) "BGI"
===

Let me know what is the next step? I?m happy to debug over Zoom if that helps.

Best,
Alfredo.

On Jun 14, 2021, at 11:04 AM, Xavier Lecours Boucher, Mr <xavier.lecoursboucher at mcgill.ca<mailto:xavier.lecoursboucher at mcgill.ca>> wrote:

        External Email - Use Caution

Hi Alfredo,

There is a discrepancy between the frontend and the API way to determine which site the user have, which site to give the new candidate and if a user is allowed to create a candidate at a given site. Looking at the frontend code, I can't see any validation on the site other than the content of the html dropdown.

The API checks if the user has the given site.

Here is the API code<https://secure-web.cisco.com/1if5Eh5lps70ewShywnybjkoSptbZugNrF3KFp3Ikj31HdtP_kKvS_EXtIV8bo4sQ1XhY1mFp5CQ97Ta_5VBA3uwcObZQDvyGWwqHmVY0RTDJczBF5eOvEjZp7XGPCCyrnC4a97TlYo56eat54xNRJvdoVSn2fB149oaDE6j2hyCWmy1zrBbbWahlzbiuR4ydd0k7ZkfEvAvqWjjFfssiSvOxhGd280OG8sBE1fHVx3TqRc84NdaTQzYkM7BTv0DfNMKxpaQDEwZhM8ymX88Ctg/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L185>:
$usersites = $user->getSiteNames();
if (!in_array($data['Candidate']['Site'], $usersites)) {
    return new \LORIS\Http\Response\JSON\Forbidden(
        'You are not affiliated with the candidate`s site'
    );
}
So, for debugging, can you add the following lines at https://github.com/aces/Loris/blob/23.0-release/modules/api/php/endpoints/candidates.class.inc#L184<https://secure-web.cisco.com/1fNyuDjwP45YKJ02APn6ItF2MKoXIN6PCZlwgpbccCqoLUyWWzBLPJbE9XvOxBwjXow-KCKAzoVz8qwXZiF1zDuQ5f6yXeKAzrO1vsrM5j3wIBR2iM29VBUkAdxLP_5M47korX_RF2RXn6XOisI2aTB6xZnehZf6R1VVjTIBiLrLcDC5Jvcauo6T97HesnSkByNr0iYbJYpIgZlaHnGZkPWOBjVGGezJr6Jh8gsh04CJgRl6JZjW5wAwoqqecRvhHRMnKhW7tmcggfWieTL112Q/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L184> , then tell me what it the response to your POST request ?

var_dump($user->getSiteNames(), $data['Candidate']['Site']);
exit;
Thank you
Xavier


<https://secure-web.cisco.com/1if5Eh5lps70ewShywnybjkoSptbZugNrF3KFp3Ikj31HdtP_kKvS_EXtIV8bo4sQ1XhY1mFp5CQ97Ta_5VBA3uwcObZQDvyGWwqHmVY0RTDJczBF5eOvEjZp7XGPCCyrnC4a97TlYo56eat54xNRJvdoVSn2fB149oaDE6j2hyCWmy1zrBbbWahlzbiuR4ydd0k7ZkfEvAvqWjjFfssiSvOxhGd280OG8sBE1fHVx3TqRc84NdaTQzYkM7BTv0DfNMKxpaQDEwZhM8ymX88Ctg/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L185>




________________________________
From: loris-dev-bounces at bic.mni.mcgill.ca<mailto:loris-dev-bounces at bic.mni.mcgill.ca> <loris-dev-bounces at bic.mni.mcgill.ca<mailto:loris-dev-bounces at bic.mni.mcgill.ca>> on behalf of Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU<mailto:AMORALESPINZON at BWH.HARVARD.EDU>>
Sent: June 10, 2021 6:43 PM
To: loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca> <loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca>>
Cc: Rozie Arnaoutelis, Ms. <rozie.arnaoutelis at mcgill.ca<mailto:rozie.arnaoutelis at mcgill.ca>>; Sridar Narayanan, Dr. <sridar.narayanan at mcgill.ca<mailto:sridar.narayanan at mcgill.ca>>; Douglas Arnold, Dr. <douglas.arnold at mcgill.ca<mailto:douglas.arnold at mcgill.ca>>; Guttmann, Charles, M.D. <guttmann at bwh.harvard.edu<mailto:guttmann at bwh.harvard.edu>>
Subject: [Loris-dev] Error insert candidates api/v0.0.3/candidates

Dear LorisDev team,

I can create Candidates using the web interface using an admin account that is linked to all the project and all the sites in the system. However when I try to create a Candidate using the api api/v0.0.3/candidates I am getting the following error:

===
{
"error": "You are not affiliated with the candidate`s site"
}
===

I?m happy to run some queries in the database to figure out what?s happening. Any ideas?

Best,
Alfredo.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210614/c9fdbe79/attachment-0001.html>

From AMORALESPINZON at BWH.HARVARD.EDU  Mon Jun 14 20:19:49 2021
From: AMORALESPINZON at BWH.HARVARD.EDU (Morales Pinzon, Alfredo)
Date: Tue, 15 Jun 2021 00:19:49 +0000
Subject: [Loris-dev] Error insert candidates api/v0.0.3/candidates
In-Reply-To: <QB1PR01MB235636E92D8094FEFF99BB7AFC319@QB1PR01MB2356.CANPRD01.PROD.OUTLOOK.COM>
References: <53511186-E9E5-4DDC-B1D4-DC3323113D2E@bwh.harvard.edu>
 <QB1PR01MB235636E92D8094FEFF99BB7AFC319@QB1PR01MB2356.CANPRD01.PROD.OUTLOOK.COM>
Message-ID: <D7FD9800-C0FC-460C-8F6D-45921EBEDDE3@bwh.harvard.edu>

Hi Xavier,

Thank you for reaching out.

There is a first bug int the following line:

https://github.com/aces/Loris/blob/fc574c06f5f6c96483f22788ed446f9aa36a4783/php/libraries/User.class.inc#L71

If a user affiliated with multiple center such that the concatenation of the names of the center has more than 1024 characters, then the response is cut to 1024 and if the center name I want to use is not in there, then I get the same error. This is the case for my admin user and the more than 500 affiliation centers. We saw this with C?cile on Friday.

Here is the response of the API after the changes you suggested using a user that has few center affiliations, including the one I want to add a Candidate to:

===
array(56) {
[0]=>
string(8) "inf_0102"
[1]=>
string(8) "inf_0103"
...
[55]=>
string(8) "inf_0313"
}
string(3) "BGI"
===

This is the same output when using my admin user:

===
array(114) {
[0]=>
string(9) "101-KGH-1"
[1]=>
string(9) "101-LHS-1"
...
[111]=>
string(9) "272-CAA-1"
[112]=>
string(9) "273-DDM-1"
[113]=>
string(1) "2"
}
string(3) "BGI"
===

Let me know what is the next step? I?m happy to debug over Zoom if that helps.

Best,
Alfredo.

On Jun 14, 2021, at 11:04 AM, Xavier Lecours Boucher, Mr <xavier.lecoursboucher at mcgill.ca<mailto:xavier.lecoursboucher at mcgill.ca>> wrote:

        External Email - Use Caution

Hi Alfredo,

There is a discrepancy between the frontend and the API way to determine which site the user have, which site to give the new candidate and if a user is allowed to create a candidate at a given site. Looking at the frontend code, I can't see any validation on the site other than the content of the html dropdown.

The API checks if the user has the given site.

Here is the API code<https://secure-web.cisco.com/1if5Eh5lps70ewShywnybjkoSptbZugNrF3KFp3Ikj31HdtP_kKvS_EXtIV8bo4sQ1XhY1mFp5CQ97Ta_5VBA3uwcObZQDvyGWwqHmVY0RTDJczBF5eOvEjZp7XGPCCyrnC4a97TlYo56eat54xNRJvdoVSn2fB149oaDE6j2hyCWmy1zrBbbWahlzbiuR4ydd0k7ZkfEvAvqWjjFfssiSvOxhGd280OG8sBE1fHVx3TqRc84NdaTQzYkM7BTv0DfNMKxpaQDEwZhM8ymX88Ctg/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L185>:
$usersites = $user->getSiteNames();
if (!in_array($data['Candidate']['Site'], $usersites)) {
    return new \LORIS\Http\Response\JSON\Forbidden(
        'You are not affiliated with the candidate`s site'
    );
}
So, for debugging, can you add the following lines at https://github.com/aces/Loris/blob/23.0-release/modules/api/php/endpoints/candidates.class.inc#L184<https://secure-web.cisco.com/1fNyuDjwP45YKJ02APn6ItF2MKoXIN6PCZlwgpbccCqoLUyWWzBLPJbE9XvOxBwjXow-KCKAzoVz8qwXZiF1zDuQ5f6yXeKAzrO1vsrM5j3wIBR2iM29VBUkAdxLP_5M47korX_RF2RXn6XOisI2aTB6xZnehZf6R1VVjTIBiLrLcDC5Jvcauo6T97HesnSkByNr0iYbJYpIgZlaHnGZkPWOBjVGGezJr6Jh8gsh04CJgRl6JZjW5wAwoqqecRvhHRMnKhW7tmcggfWieTL112Q/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L184> , then tell me what it the response to your POST request ?

var_dump($user->getSiteNames(), $data['Candidate']['Site']);
exit;
Thank you
Xavier


<https://secure-web.cisco.com/1if5Eh5lps70ewShywnybjkoSptbZugNrF3KFp3Ikj31HdtP_kKvS_EXtIV8bo4sQ1XhY1mFp5CQ97Ta_5VBA3uwcObZQDvyGWwqHmVY0RTDJczBF5eOvEjZp7XGPCCyrnC4a97TlYo56eat54xNRJvdoVSn2fB149oaDE6j2hyCWmy1zrBbbWahlzbiuR4ydd0k7ZkfEvAvqWjjFfssiSvOxhGd280OG8sBE1fHVx3TqRc84NdaTQzYkM7BTv0DfNMKxpaQDEwZhM8ymX88Ctg/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L185>




________________________________
From: loris-dev-bounces at bic.mni.mcgill.ca<mailto:loris-dev-bounces at bic.mni.mcgill.ca> <loris-dev-bounces at bic.mni.mcgill.ca<mailto:loris-dev-bounces at bic.mni.mcgill.ca>> on behalf of Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU<mailto:AMORALESPINZON at BWH.HARVARD.EDU>>
Sent: June 10, 2021 6:43 PM
To: loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca> <loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca>>
Cc: Rozie Arnaoutelis, Ms. <rozie.arnaoutelis at mcgill.ca<mailto:rozie.arnaoutelis at mcgill.ca>>; Sridar Narayanan, Dr. <sridar.narayanan at mcgill.ca<mailto:sridar.narayanan at mcgill.ca>>; Douglas Arnold, Dr. <douglas.arnold at mcgill.ca<mailto:douglas.arnold at mcgill.ca>>; Guttmann, Charles, M.D. <guttmann at bwh.harvard.edu<mailto:guttmann at bwh.harvard.edu>>
Subject: [Loris-dev] Error insert candidates api/v0.0.3/candidates

Dear LorisDev team,

I can create Candidates using the web interface using an admin account that is linked to all the project and all the sites in the system. However when I try to create a Candidate using the api api/v0.0.3/candidates I am getting the following error:

===
{
"error": "You are not affiliated with the candidate`s site"
}
===

I?m happy to run some queries in the database to figure out what?s happening. Any ideas?

Best,
Alfredo.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210615/68354414/attachment-0001.html>

From xavier.lecoursboucher at mcgill.ca  Tue Jun 15 10:48:41 2021
From: xavier.lecoursboucher at mcgill.ca (Xavier Lecours Boucher, Mr)
Date: Tue, 15 Jun 2021 14:48:41 +0000
Subject: [Loris-dev] Error insert candidates api/v0.0.3/candidates
In-Reply-To: <D7FD9800-C0FC-460C-8F6D-45921EBEDDE3@bwh.harvard.edu>
References: <53511186-E9E5-4DDC-B1D4-DC3323113D2E@bwh.harvard.edu>
 <QB1PR01MB235636E92D8094FEFF99BB7AFC319@QB1PR01MB2356.CANPRD01.PROD.OUTLOOK.COM>,
 <D7FD9800-C0FC-460C-8F6D-45921EBEDDE3@bwh.harvard.edu>
Message-ID: <QB1PR01MB2356503E43B4BEA5D183079CFC309@QB1PR01MB2356.CANPRD01.PROD.OUTLOOK.COM>

Hi Alfredo,

Indeed, this is a bug.
As a quick fix, you could add the following line just before the problematic statement:


$DB->run('SET SESSION group_concat_max_len = 100000');

I have created a github issue so we can find a better solution : https://github.com/aces/Loris/issues/7480

Thank you for the accurate reporting ?
- Xavier

________________________________
From: Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU>
Sent: June 14, 2021 7:19 PM
To: Xavier Lecours Boucher, Mr <xavier.lecoursboucher at mcgill.ca>
Cc: loris-dev at bic.mni.mcgill.ca <loris-dev at bic.mni.mcgill.ca>; Sridar Narayanan, Dr. <sridar.narayanan at mcgill.ca>; Douglas Arnold, Dr. <douglas.arnold at mcgill.ca>; Guttmann, Charles,M.D. <guttmann at bwh.harvard.edu>; Istvan Akos Imre Morocz, Dr <istvan.morocz at mcgill.ca>
Subject: Re: Error insert candidates api/v0.0.3/candidates

Hi Xavier,

Thank you for reaching out.

There is a first bug int the following line:

https://github.com/aces/Loris/blob/fc574c06f5f6c96483f22788ed446f9aa36a4783/php/libraries/User.class.inc#L71

If a user affiliated with multiple center such that the concatenation of the names of the center has more than 1024 characters, then the response is cut to 1024 and if the center name I want to use is not in there, then I get the same error. This is the case for my admin user and the more than 500 affiliation centers. We saw this with C?cile on Friday.

Here is the response of the API after the changes you suggested using a user that has few center affiliations, including the one I want to add a Candidate to:

===
array(56) {
[0]=>
string(8) "inf_0102"
[1]=>
string(8) "inf_0103"
...
[55]=>
string(8) "inf_0313"
}
string(3) "BGI"
===

This is the same output when using my admin user:

===
array(114) {
[0]=>
string(9) "101-KGH-1"
[1]=>
string(9) "101-LHS-1"
...
[111]=>
string(9) "272-CAA-1"
[112]=>
string(9) "273-DDM-1"
[113]=>
string(1) "2"
}
string(3) "BGI"
===

Let me know what is the next step? I?m happy to debug over Zoom if that helps.

Best,
Alfredo.

On Jun 14, 2021, at 11:04 AM, Xavier Lecours Boucher, Mr <xavier.lecoursboucher at mcgill.ca<mailto:xavier.lecoursboucher at mcgill.ca>> wrote:

        External Email - Use Caution

Hi Alfredo,

There is a discrepancy between the frontend and the API way to determine which site the user have, which site to give the new candidate and if a user is allowed to create a candidate at a given site. Looking at the frontend code, I can't see any validation on the site other than the content of the html dropdown.

The API checks if the user has the given site.

Here is the API code<https://secure-web.cisco.com/1if5Eh5lps70ewShywnybjkoSptbZugNrF3KFp3Ikj31HdtP_kKvS_EXtIV8bo4sQ1XhY1mFp5CQ97Ta_5VBA3uwcObZQDvyGWwqHmVY0RTDJczBF5eOvEjZp7XGPCCyrnC4a97TlYo56eat54xNRJvdoVSn2fB149oaDE6j2hyCWmy1zrBbbWahlzbiuR4ydd0k7ZkfEvAvqWjjFfssiSvOxhGd280OG8sBE1fHVx3TqRc84NdaTQzYkM7BTv0DfNMKxpaQDEwZhM8ymX88Ctg/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L185>:
$usersites = $user->getSiteNames();
if (!in_array($data['Candidate']['Site'], $usersites)) {
    return new \LORIS\Http\Response\JSON\Forbidden(
        'You are not affiliated with the candidate`s site'
    );
}
So, for debugging, can you add the following lines at https://github.com/aces/Loris/blob/23.0-release/modules/api/php/endpoints/candidates.class.inc#L184<https://secure-web.cisco.com/1fNyuDjwP45YKJ02APn6ItF2MKoXIN6PCZlwgpbccCqoLUyWWzBLPJbE9XvOxBwjXow-KCKAzoVz8qwXZiF1zDuQ5f6yXeKAzrO1vsrM5j3wIBR2iM29VBUkAdxLP_5M47korX_RF2RXn6XOisI2aTB6xZnehZf6R1VVjTIBiLrLcDC5Jvcauo6T97HesnSkByNr0iYbJYpIgZlaHnGZkPWOBjVGGezJr6Jh8gsh04CJgRl6JZjW5wAwoqqecRvhHRMnKhW7tmcggfWieTL112Q/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L184> , then tell me what it the response to your POST request ?

var_dump($user->getSiteNames(), $data['Candidate']['Site']);
exit;
Thank you
Xavier


<https://secure-web.cisco.com/1if5Eh5lps70ewShywnybjkoSptbZugNrF3KFp3Ikj31HdtP_kKvS_EXtIV8bo4sQ1XhY1mFp5CQ97Ta_5VBA3uwcObZQDvyGWwqHmVY0RTDJczBF5eOvEjZp7XGPCCyrnC4a97TlYo56eat54xNRJvdoVSn2fB149oaDE6j2hyCWmy1zrBbbWahlzbiuR4ydd0k7ZkfEvAvqWjjFfssiSvOxhGd280OG8sBE1fHVx3TqRc84NdaTQzYkM7BTv0DfNMKxpaQDEwZhM8ymX88Ctg/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L185>




________________________________
From: loris-dev-bounces at bic.mni.mcgill.ca<mailto:loris-dev-bounces at bic.mni.mcgill.ca> <loris-dev-bounces at bic.mni.mcgill.ca<mailto:loris-dev-bounces at bic.mni.mcgill.ca>> on behalf of Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU<mailto:AMORALESPINZON at BWH.HARVARD.EDU>>
Sent: June 10, 2021 6:43 PM
To: loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca> <loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca>>
Cc: Rozie Arnaoutelis, Ms. <rozie.arnaoutelis at mcgill.ca<mailto:rozie.arnaoutelis at mcgill.ca>>; Sridar Narayanan, Dr. <sridar.narayanan at mcgill.ca<mailto:sridar.narayanan at mcgill.ca>>; Douglas Arnold, Dr. <douglas.arnold at mcgill.ca<mailto:douglas.arnold at mcgill.ca>>; Guttmann, Charles, M.D. <guttmann at bwh.harvard.edu<mailto:guttmann at bwh.harvard.edu>>
Subject: [Loris-dev] Error insert candidates api/v0.0.3/candidates

Dear LorisDev team,

I can create Candidates using the web interface using an admin account that is linked to all the project and all the sites in the system. However when I try to create a Candidate using the api api/v0.0.3/candidates I am getting the following error:

===
{
"error": "You are not affiliated with the candidate`s site"
}
===

I?m happy to run some queries in the database to figure out what?s happening. Any ideas?

Best,
Alfredo.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.


Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210615/9f84ff9e/attachment-0001.html>

From AMORALESPINZON at BWH.HARVARD.EDU  Tue Jun 15 15:05:42 2021
From: AMORALESPINZON at BWH.HARVARD.EDU (Morales Pinzon, Alfredo)
Date: Tue, 15 Jun 2021 19:05:42 +0000
Subject: [Loris-dev] Error insert candidates api/v0.0.3/candidates
In-Reply-To: <QB1PR01MB2356503E43B4BEA5D183079CFC309@QB1PR01MB2356.CANPRD01.PROD.OUTLOOK.COM>
References: <53511186-E9E5-4DDC-B1D4-DC3323113D2E@bwh.harvard.edu>
 <QB1PR01MB235636E92D8094FEFF99BB7AFC319@QB1PR01MB2356.CANPRD01.PROD.OUTLOOK.COM>
 <D7FD9800-C0FC-460C-8F6D-45921EBEDDE3@bwh.harvard.edu>
 <QB1PR01MB2356503E43B4BEA5D183079CFC309@QB1PR01MB2356.CANPRD01.PROD.OUTLOOK.COM>
Message-ID: <83CDA8EA-9F41-4136-B45F-5B3EAD6A35F0@bwh.harvard.edu>

Hi Xavier,

Thank you for creating the issue and suggesting a fix. As a mentioned in my previous email, the issue is still present for a user that doesn?t have a lot of centers so that the list of centers is not truncated. In this case I am getting the same message "You are not affiliated with the candidate`s site?. In my previous email I sent the results of the API for this user, is the one that starts with ?array(56)?.

What is you input on this case?

Best,
Alfredo.

On Jun 15, 2021, at 10:48 AM, Xavier Lecours Boucher, Mr <xavier.lecoursboucher at mcgill.ca<mailto:xavier.lecoursboucher at mcgill.ca>> wrote:

        External Email - Use Caution

Hi Alfredo,

Indeed, this is a bug.
As a quick fix, you could add the following line just before the problematic statement:


$DB->run('SET SESSION group_concat_max_len = 100000');

I have created a github issue so we can find a better solution : https://github.com/aces/Loris/issues/7480<https://secure-web.cisco.com/1wQOmIgqkiH7F3yTDPeQF-ONl3EmpIJMgfd50iZp4w25bCUCNXf9gqbwyBCJB3qZF-1KEFteDwpdhZLqPaX2yn_BQLYBrnQynKca06QGu_q7kOjJZV4to959daS2ZbV4Xavoj4d3OomO73Hnw37D_EfcnzgGosvSzRmRnMUZ-nFIL-8iUO_2iAk1WtrVo7ho4vGA-JeMmPt2YihoRD7SXpit95jyyOY9yJTSAL7nAAUc5wGgD_wyfRh5cfD0XJzOukG76nShQBozibDmS_cDuSg/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fissues%2F7480>

Thank you for the accurate reporting ?
- Xavier

________________________________
From: Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU<mailto:AMORALESPINZON at BWH.HARVARD.EDU>>
Sent: June 14, 2021 7:19 PM
To: Xavier Lecours Boucher, Mr <xavier.lecoursboucher at mcgill.ca<mailto:xavier.lecoursboucher at mcgill.ca>>
Cc: loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca> <loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca>>; Sridar Narayanan, Dr. <sridar.narayanan at mcgill.ca<mailto:sridar.narayanan at mcgill.ca>>; Douglas Arnold, Dr. <douglas.arnold at mcgill.ca<mailto:douglas.arnold at mcgill.ca>>; Guttmann, Charles,M.D. <guttmann at bwh.harvard.edu<mailto:guttmann at bwh.harvard.edu>>; Istvan Akos Imre Morocz, Dr <istvan.morocz at mcgill.ca<mailto:istvan.morocz at mcgill.ca>>
Subject: Re: Error insert candidates api/v0.0.3/candidates

Hi Xavier,

Thank you for reaching out.

There is a first bug int the following line:

https://github.com/aces/Loris/blob/fc574c06f5f6c96483f22788ed446f9aa36a4783/php/libraries/User.class.inc#L71<https://secure-web.cisco.com/1is0gCN2r6hBd-51j4mZFeeJjBMdeq7KAF8qWNfBz_zVRYRKx0N5hT29cPQRpGT2awzDS4wUaksDBCo4nLjbN-ffNauQNwH716qcgIxE02xF4vz63_eooqwbhE2Y5aCjL17XZPB-bqwybaWidSYy5JWbgnSG9PcsUB-CqucRPZ2Tw1TJyEB12UQY6IgUNqS5H_zga_rxEGeNQUBrzOPbSumr6l4mHs3jHEIPgv6g4nTQ_29J9BZKmtKkIMwjxdHW1L7AzbnOCPZtRlKlu2LzOeA/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2Ffc574c06f5f6c96483f22788ed446f9aa36a4783%2Fphp%2Flibraries%2FUser.class.inc%23L71>

If a user affiliated with multiple center such that the concatenation of the names of the center has more than 1024 characters, then the response is cut to 1024 and if the center name I want to use is not in there, then I get the same error. This is the case for my admin user and the more than 500 affiliation centers. We saw this with C?cile on Friday.

Here is the response of the API after the changes you suggested using a user that has few center affiliations, including the one I want to add a Candidate to:

===
array(56) {
[0]=>
string(8) "inf_0102"
[1]=>
string(8) "inf_0103"
...
[55]=>
string(8) "inf_0313"
}
string(3) "BGI"
===

This is the same output when using my admin user:

===
array(114) {
[0]=>
string(9) "101-KGH-1"
[1]=>
string(9) "101-LHS-1"
...
[111]=>
string(9) "272-CAA-1"
[112]=>
string(9) "273-DDM-1"
[113]=>
string(1) "2"
}
string(3) "BGI"
===

Let me know what is the next step? I?m happy to debug over Zoom if that helps.

Best,
Alfredo.

On Jun 14, 2021, at 11:04 AM, Xavier Lecours Boucher, Mr <xavier.lecoursboucher at mcgill.ca<mailto:xavier.lecoursboucher at mcgill.ca>> wrote:

        External Email - Use Caution

Hi Alfredo,

There is a discrepancy between the frontend and the API way to determine which site the user have, which site to give the new candidate and if a user is allowed to create a candidate at a given site. Looking at the frontend code, I can't see any validation on the site other than the content of the html dropdown.

The API checks if the user has the given site.

Here is the API code<https://secure-web.cisco.com/1if5Eh5lps70ewShywnybjkoSptbZugNrF3KFp3Ikj31HdtP_kKvS_EXtIV8bo4sQ1XhY1mFp5CQ97Ta_5VBA3uwcObZQDvyGWwqHmVY0RTDJczBF5eOvEjZp7XGPCCyrnC4a97TlYo56eat54xNRJvdoVSn2fB149oaDE6j2hyCWmy1zrBbbWahlzbiuR4ydd0k7ZkfEvAvqWjjFfssiSvOxhGd280OG8sBE1fHVx3TqRc84NdaTQzYkM7BTv0DfNMKxpaQDEwZhM8ymX88Ctg/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L185>:
$usersites = $user->getSiteNames();
if (!in_array($data['Candidate']['Site'], $usersites)) {
    return new \LORIS\Http\Response\JSON\Forbidden(
        'You are not affiliated with the candidate`s site'
    );
}
So, for debugging, can you add the following lines at https://github.com/aces/Loris/blob/23.0-release/modules/api/php/endpoints/candidates.class.inc#L184<https://secure-web.cisco.com/1fNyuDjwP45YKJ02APn6ItF2MKoXIN6PCZlwgpbccCqoLUyWWzBLPJbE9XvOxBwjXow-KCKAzoVz8qwXZiF1zDuQ5f6yXeKAzrO1vsrM5j3wIBR2iM29VBUkAdxLP_5M47korX_RF2RXn6XOisI2aTB6xZnehZf6R1VVjTIBiLrLcDC5Jvcauo6T97HesnSkByNr0iYbJYpIgZlaHnGZkPWOBjVGGezJr6Jh8gsh04CJgRl6JZjW5wAwoqqecRvhHRMnKhW7tmcggfWieTL112Q/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L184> , then tell me what it the response to your POST request ?

var_dump($user->getSiteNames(), $data['Candidate']['Site']);
exit;
Thank you
Xavier


<https://secure-web.cisco.com/1if5Eh5lps70ewShywnybjkoSptbZugNrF3KFp3Ikj31HdtP_kKvS_EXtIV8bo4sQ1XhY1mFp5CQ97Ta_5VBA3uwcObZQDvyGWwqHmVY0RTDJczBF5eOvEjZp7XGPCCyrnC4a97TlYo56eat54xNRJvdoVSn2fB149oaDE6j2hyCWmy1zrBbbWahlzbiuR4ydd0k7ZkfEvAvqWjjFfssiSvOxhGd280OG8sBE1fHVx3TqRc84NdaTQzYkM7BTv0DfNMKxpaQDEwZhM8ymX88Ctg/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L185>




________________________________
From: loris-dev-bounces at bic.mni.mcgill.ca<mailto:loris-dev-bounces at bic.mni.mcgill.ca> <loris-dev-bounces at bic.mni.mcgill.ca<mailto:loris-dev-bounces at bic.mni.mcgill.ca>> on behalf of Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU<mailto:AMORALESPINZON at BWH.HARVARD.EDU>>
Sent: June 10, 2021 6:43 PM
To: loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca> <loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca>>
Cc: Rozie Arnaoutelis, Ms. <rozie.arnaoutelis at mcgill.ca<mailto:rozie.arnaoutelis at mcgill.ca>>; Sridar Narayanan, Dr. <sridar.narayanan at mcgill.ca<mailto:sridar.narayanan at mcgill.ca>>; Douglas Arnold, Dr. <douglas.arnold at mcgill.ca<mailto:douglas.arnold at mcgill.ca>>; Guttmann, Charles, M.D. <guttmann at bwh.harvard.edu<mailto:guttmann at bwh.harvard.edu>>
Subject: [Loris-dev] Error insert candidates api/v0.0.3/candidates

Dear LorisDev team,

I can create Candidates using the web interface using an admin account that is linked to all the project and all the sites in the system. However when I try to create a Candidate using the api api/v0.0.3/candidates I am getting the following error:

===
{
"error": "You are not affiliated with the candidate`s site"
}
===

I?m happy to run some queries in the database to figure out what?s happening. Any ideas?

Best,
Alfredo.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/1RIVOq7LsmWlGiSIpDV1iDxaKZDL4FrGgQqzQtNFdUclqgR60trLfQ1Mrv_3Jb_-Nh4SRJc_iFLFJXdP2NJMN-0HWzFBk2MS4e6vgU15rC0uEPg1nX41lWaBkOPKqdov7TstiKnJaFkbN5V7SktHpEnB3YYGFW1gB6K2lnAdqBdfpA7Aqd2s-V5UtJpeqM2FMPp-6AX661pD9MbwDsRNm-XzJR8bNHmTV73XfqLOtbx3837t9gQYbTub-VXeXIHQbr3uLsfcOkE7rA-nTlDJnnQ/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210615/14e45ff4/attachment-0001.html>

From xavier.lecoursboucher at mcgill.ca  Thu Jun 17 09:21:00 2021
From: xavier.lecoursboucher at mcgill.ca (Xavier Lecours Boucher, Mr)
Date: Thu, 17 Jun 2021 13:21:00 +0000
Subject: [Loris-dev] Error insert candidates api/v0.0.3/candidates
In-Reply-To: <83CDA8EA-9F41-4136-B45F-5B3EAD6A35F0@bwh.harvard.edu>
References: <53511186-E9E5-4DDC-B1D4-DC3323113D2E@bwh.harvard.edu>
 <QB1PR01MB235636E92D8094FEFF99BB7AFC319@QB1PR01MB2356.CANPRD01.PROD.OUTLOOK.COM>
 <D7FD9800-C0FC-460C-8F6D-45921EBEDDE3@bwh.harvard.edu>
 <QB1PR01MB2356503E43B4BEA5D183079CFC309@QB1PR01MB2356.CANPRD01.PROD.OUTLOOK.COM>,
 <83CDA8EA-9F41-4136-B45F-5B3EAD6A35F0@bwh.harvard.edu>
Message-ID: <QB1PR01MB2356E6CD70E4D9DECC264283FC0E9@QB1PR01MB2356.CANPRD01.PROD.OUTLOOK.COM>

Hi Alfredo,

What is the content of $data['Candidate']['Site'] and $usersites ? Can you find the string of the first one in the elements of the second one?

Meanwhile, here is the fix we are likely incorporating in our new patch https://github.com/aces/Loris/pull/7486/files

@bient?t
Xavier


________________________________
From: Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU>
Sent: June 15, 2021 2:05 PM
To: Xavier Lecours Boucher, Mr <xavier.lecoursboucher at mcgill.ca>
Cc: loris-dev at bic.mni.mcgill.ca <loris-dev at bic.mni.mcgill.ca>; Sridar Narayanan, Dr. <sridar.narayanan at mcgill.ca>; Douglas Arnold, Dr. <douglas.arnold at mcgill.ca>; Guttmann, Charles,M.D. <guttmann at bwh.harvard.edu>; Istvan Akos Imre Morocz, Dr <istvan.morocz at mcgill.ca>
Subject: Re: Error insert candidates api/v0.0.3/candidates

Hi Xavier,

Thank you for creating the issue and suggesting a fix. As a mentioned in my previous email, the issue is still present for a user that doesn?t have a lot of centers so that the list of centers is not truncated. In this case I am getting the same message "You are not affiliated with the candidate`s site?. In my previous email I sent the results of the API for this user, is the one that starts with ?array(56)?.

What is you input on this case?

Best,
Alfredo.

On Jun 15, 2021, at 10:48 AM, Xavier Lecours Boucher, Mr <xavier.lecoursboucher at mcgill.ca<mailto:xavier.lecoursboucher at mcgill.ca>> wrote:

        External Email - Use Caution

Hi Alfredo,

Indeed, this is a bug.
As a quick fix, you could add the following line just before the problematic statement:


$DB->run('SET SESSION group_concat_max_len = 100000');

I have created a github issue so we can find a better solution : https://github.com/aces/Loris/issues/7480<https://secure-web.cisco.com/1wQOmIgqkiH7F3yTDPeQF-ONl3EmpIJMgfd50iZp4w25bCUCNXf9gqbwyBCJB3qZF-1KEFteDwpdhZLqPaX2yn_BQLYBrnQynKca06QGu_q7kOjJZV4to959daS2ZbV4Xavoj4d3OomO73Hnw37D_EfcnzgGosvSzRmRnMUZ-nFIL-8iUO_2iAk1WtrVo7ho4vGA-JeMmPt2YihoRD7SXpit95jyyOY9yJTSAL7nAAUc5wGgD_wyfRh5cfD0XJzOukG76nShQBozibDmS_cDuSg/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fissues%2F7480>

Thank you for the accurate reporting ?
- Xavier

________________________________
From: Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU<mailto:AMORALESPINZON at BWH.HARVARD.EDU>>
Sent: June 14, 2021 7:19 PM
To: Xavier Lecours Boucher, Mr <xavier.lecoursboucher at mcgill.ca<mailto:xavier.lecoursboucher at mcgill.ca>>
Cc: loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca> <loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca>>; Sridar Narayanan, Dr. <sridar.narayanan at mcgill.ca<mailto:sridar.narayanan at mcgill.ca>>; Douglas Arnold, Dr. <douglas.arnold at mcgill.ca<mailto:douglas.arnold at mcgill.ca>>; Guttmann, Charles,M.D. <guttmann at bwh.harvard.edu<mailto:guttmann at bwh.harvard.edu>>; Istvan Akos Imre Morocz, Dr <istvan.morocz at mcgill.ca<mailto:istvan.morocz at mcgill.ca>>
Subject: Re: Error insert candidates api/v0.0.3/candidates

Hi Xavier,

Thank you for reaching out.

There is a first bug int the following line:

https://github.com/aces/Loris/blob/fc574c06f5f6c96483f22788ed446f9aa36a4783/php/libraries/User.class.inc#L71<https://secure-web.cisco.com/1is0gCN2r6hBd-51j4mZFeeJjBMdeq7KAF8qWNfBz_zVRYRKx0N5hT29cPQRpGT2awzDS4wUaksDBCo4nLjbN-ffNauQNwH716qcgIxE02xF4vz63_eooqwbhE2Y5aCjL17XZPB-bqwybaWidSYy5JWbgnSG9PcsUB-CqucRPZ2Tw1TJyEB12UQY6IgUNqS5H_zga_rxEGeNQUBrzOPbSumr6l4mHs3jHEIPgv6g4nTQ_29J9BZKmtKkIMwjxdHW1L7AzbnOCPZtRlKlu2LzOeA/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2Ffc574c06f5f6c96483f22788ed446f9aa36a4783%2Fphp%2Flibraries%2FUser.class.inc%23L71>

If a user affiliated with multiple center such that the concatenation of the names of the center has more than 1024 characters, then the response is cut to 1024 and if the center name I want to use is not in there, then I get the same error. This is the case for my admin user and the more than 500 affiliation centers. We saw this with C?cile on Friday.

Here is the response of the API after the changes you suggested using a user that has few center affiliations, including the one I want to add a Candidate to:

===
array(56) {
[0]=>
string(8) "inf_0102"
[1]=>
string(8) "inf_0103"
...
[55]=>
string(8) "inf_0313"
}
string(3) "BGI"
===

This is the same output when using my admin user:

===
array(114) {
[0]=>
string(9) "101-KGH-1"
[1]=>
string(9) "101-LHS-1"
...
[111]=>
string(9) "272-CAA-1"
[112]=>
string(9) "273-DDM-1"
[113]=>
string(1) "2"
}
string(3) "BGI"
===

Let me know what is the next step? I?m happy to debug over Zoom if that helps.

Best,
Alfredo.

On Jun 14, 2021, at 11:04 AM, Xavier Lecours Boucher, Mr <xavier.lecoursboucher at mcgill.ca<mailto:xavier.lecoursboucher at mcgill.ca>> wrote:

        External Email - Use Caution

Hi Alfredo,

There is a discrepancy between the frontend and the API way to determine which site the user have, which site to give the new candidate and if a user is allowed to create a candidate at a given site. Looking at the frontend code, I can't see any validation on the site other than the content of the html dropdown.

The API checks if the user has the given site.

Here is the API code<https://secure-web.cisco.com/1if5Eh5lps70ewShywnybjkoSptbZugNrF3KFp3Ikj31HdtP_kKvS_EXtIV8bo4sQ1XhY1mFp5CQ97Ta_5VBA3uwcObZQDvyGWwqHmVY0RTDJczBF5eOvEjZp7XGPCCyrnC4a97TlYo56eat54xNRJvdoVSn2fB149oaDE6j2hyCWmy1zrBbbWahlzbiuR4ydd0k7ZkfEvAvqWjjFfssiSvOxhGd280OG8sBE1fHVx3TqRc84NdaTQzYkM7BTv0DfNMKxpaQDEwZhM8ymX88Ctg/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L185>:
$usersites = $user->getSiteNames();
if (!in_array($data['Candidate']['Site'], $usersites)) {
    return new \LORIS\Http\Response\JSON\Forbidden(
        'You are not affiliated with the candidate`s site'
    );
}
So, for debugging, can you add the following lines at https://github.com/aces/Loris/blob/23.0-release/modules/api/php/endpoints/candidates.class.inc#L184<https://secure-web.cisco.com/1fNyuDjwP45YKJ02APn6ItF2MKoXIN6PCZlwgpbccCqoLUyWWzBLPJbE9XvOxBwjXow-KCKAzoVz8qwXZiF1zDuQ5f6yXeKAzrO1vsrM5j3wIBR2iM29VBUkAdxLP_5M47korX_RF2RXn6XOisI2aTB6xZnehZf6R1VVjTIBiLrLcDC5Jvcauo6T97HesnSkByNr0iYbJYpIgZlaHnGZkPWOBjVGGezJr6Jh8gsh04CJgRl6JZjW5wAwoqqecRvhHRMnKhW7tmcggfWieTL112Q/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L184> , then tell me what it the response to your POST request ?

var_dump($user->getSiteNames(), $data['Candidate']['Site']);
exit;
Thank you
Xavier


<https://secure-web.cisco.com/1if5Eh5lps70ewShywnybjkoSptbZugNrF3KFp3Ikj31HdtP_kKvS_EXtIV8bo4sQ1XhY1mFp5CQ97Ta_5VBA3uwcObZQDvyGWwqHmVY0RTDJczBF5eOvEjZp7XGPCCyrnC4a97TlYo56eat54xNRJvdoVSn2fB149oaDE6j2hyCWmy1zrBbbWahlzbiuR4ydd0k7ZkfEvAvqWjjFfssiSvOxhGd280OG8sBE1fHVx3TqRc84NdaTQzYkM7BTv0DfNMKxpaQDEwZhM8ymX88Ctg/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L185>




________________________________
From: loris-dev-bounces at bic.mni.mcgill.ca<mailto:loris-dev-bounces at bic.mni.mcgill.ca> <loris-dev-bounces at bic.mni.mcgill.ca<mailto:loris-dev-bounces at bic.mni.mcgill.ca>> on behalf of Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU<mailto:AMORALESPINZON at BWH.HARVARD.EDU>>
Sent: June 10, 2021 6:43 PM
To: loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca> <loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca>>
Cc: Rozie Arnaoutelis, Ms. <rozie.arnaoutelis at mcgill.ca<mailto:rozie.arnaoutelis at mcgill.ca>>; Sridar Narayanan, Dr. <sridar.narayanan at mcgill.ca<mailto:sridar.narayanan at mcgill.ca>>; Douglas Arnold, Dr. <douglas.arnold at mcgill.ca<mailto:douglas.arnold at mcgill.ca>>; Guttmann, Charles, M.D. <guttmann at bwh.harvard.edu<mailto:guttmann at bwh.harvard.edu>>
Subject: [Loris-dev] Error insert candidates api/v0.0.3/candidates

Dear LorisDev team,

I can create Candidates using the web interface using an admin account that is linked to all the project and all the sites in the system. However when I try to create a Candidate using the api api/v0.0.3/candidates I am getting the following error:

===
{
"error": "You are not affiliated with the candidate`s site"
}
===

I?m happy to run some queries in the database to figure out what?s happening. Any ideas?

Best,
Alfredo.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/1RIVOq7LsmWlGiSIpDV1iDxaKZDL4FrGgQqzQtNFdUclqgR60trLfQ1Mrv_3Jb_-Nh4SRJc_iFLFJXdP2NJMN-0HWzFBk2MS4e6vgU15rC0uEPg1nX41lWaBkOPKqdov7TstiKnJaFkbN5V7SktHpEnB3YYGFW1gB6K2lnAdqBdfpA7Aqd2s-V5UtJpeqM2FMPp-6AX661pD9MbwDsRNm-XzJR8bNHmTV73XfqLOtbx3837t9gQYbTub-VXeXIHQbr3uLsfcOkE7rA-nTlDJnnQ/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.


Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210617/96581484/attachment-0001.html>

From AMORALESPINZON at BWH.HARVARD.EDU  Mon Jun 21 16:04:53 2021
From: AMORALESPINZON at BWH.HARVARD.EDU (Morales Pinzon, Alfredo)
Date: Mon, 21 Jun 2021 20:04:53 +0000
Subject: [Loris-dev] IPMSA-SPINE: Computational Resources
Message-ID: <CB05BFE6-6ED9-43F0-8D52-B36F2B1B63BA@bwh.harvard.edu>

Hi Louis,

We would like to know what computational resources you use/have to run your image processing algorithms, for instance in clusters, at McGill. We are planning to run algorithms from SPINE in those computational facilities for the IPMSA project. 

Best regards,
Alfredo.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail. 



From AMORALESPINZON at BWH.HARVARD.EDU  Mon Jun 21 16:23:44 2021
From: AMORALESPINZON at BWH.HARVARD.EDU (Morales Pinzon, Alfredo)
Date: Mon, 21 Jun 2021 20:23:44 +0000
Subject: [Loris-dev] Error insert candidates api/v0.0.3/candidates
In-Reply-To: <QB1PR01MB2356E6CD70E4D9DECC264283FC0E9@QB1PR01MB2356.CANPRD01.PROD.OUTLOOK.COM>
References: <53511186-E9E5-4DDC-B1D4-DC3323113D2E@bwh.harvard.edu>
 <QB1PR01MB235636E92D8094FEFF99BB7AFC319@QB1PR01MB2356.CANPRD01.PROD.OUTLOOK.COM>
 <D7FD9800-C0FC-460C-8F6D-45921EBEDDE3@bwh.harvard.edu>
 <QB1PR01MB2356503E43B4BEA5D183079CFC309@QB1PR01MB2356.CANPRD01.PROD.OUTLOOK.COM>
 <83CDA8EA-9F41-4136-B45F-5B3EAD6A35F0@bwh.harvard.edu>
 <QB1PR01MB2356E6CD70E4D9DECC264283FC0E9@QB1PR01MB2356.CANPRD01.PROD.OUTLOOK.COM>
Message-ID: <05388467-EC2A-48C1-A505-609D5E2B837C@bwh.harvard.edu>

Hi Xavier,

I used the path that you provided and now I can create the Candidates.

Thank you very much.

Best,
Alfredo.

On Jun 17, 2021, at 9:21 AM, Xavier Lecours Boucher, Mr <xavier.lecoursboucher at mcgill.ca<mailto:xavier.lecoursboucher at mcgill.ca>> wrote:

        External Email - Use Caution

Hi Alfredo,

What is the content of $data['Candidate']['Site'] and $usersites ? Can you find the string of the first one in the elements of the second one?

Meanwhile, here is the fix we are likely incorporating in our new patch https://github.com/aces/Loris/pull/7486/files<https://secure-web.cisco.com/1Up4fHe8vBRYFrapfz6lKG58mNhQxqnsS4rlj6IK9pbvVaS1hNwu6ey5eaQqGWX68dFoq-6uRC37GfSM2cfMRWw6to3M2XyZ3TTIgLjkzxn4-sRhpqWtc9Cu4Uglrhh1LDW-I_om5h9vR1DuVn1brqO6-TQVyow1XWDonRlcjF7hhZsIcwhFvW8DJQe9l2bA-_MilV9hDJQMfakQzhuWbS11AY4XERzgrOLzhBdvU1LVmhLl7yJySGxm7s4dL7u-K8F85shHxPDXHZy2KJYT8kg/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fpull%2F7486%2Ffiles>

@bient?t
Xavier


________________________________
From: Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU<mailto:AMORALESPINZON at BWH.HARVARD.EDU>>
Sent: June 15, 2021 2:05 PM
To: Xavier Lecours Boucher, Mr <xavier.lecoursboucher at mcgill.ca<mailto:xavier.lecoursboucher at mcgill.ca>>
Cc: loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca> <loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca>>; Sridar Narayanan, Dr. <sridar.narayanan at mcgill.ca<mailto:sridar.narayanan at mcgill.ca>>; Douglas Arnold, Dr. <douglas.arnold at mcgill.ca<mailto:douglas.arnold at mcgill.ca>>; Guttmann, Charles,M.D. <guttmann at bwh.harvard.edu<mailto:guttmann at bwh.harvard.edu>>; Istvan Akos Imre Morocz, Dr <istvan.morocz at mcgill.ca<mailto:istvan.morocz at mcgill.ca>>
Subject: Re: Error insert candidates api/v0.0.3/candidates

Hi Xavier,

Thank you for creating the issue and suggesting a fix. As a mentioned in my previous email, the issue is still present for a user that doesn?t have a lot of centers so that the list of centers is not truncated. In this case I am getting the same message "You are not affiliated with the candidate`s site?. In my previous email I sent the results of the API for this user, is the one that starts with ?array(56)?.

What is you input on this case?

Best,
Alfredo.

On Jun 15, 2021, at 10:48 AM, Xavier Lecours Boucher, Mr <xavier.lecoursboucher at mcgill.ca<mailto:xavier.lecoursboucher at mcgill.ca>> wrote:

        External Email - Use Caution

Hi Alfredo,

Indeed, this is a bug.
As a quick fix, you could add the following line just before the problematic statement:


$DB->run('SET SESSION group_concat_max_len = 100000');

I have created a github issue so we can find a better solution : https://github.com/aces/Loris/issues/7480<https://secure-web.cisco.com/1wQOmIgqkiH7F3yTDPeQF-ONl3EmpIJMgfd50iZp4w25bCUCNXf9gqbwyBCJB3qZF-1KEFteDwpdhZLqPaX2yn_BQLYBrnQynKca06QGu_q7kOjJZV4to959daS2ZbV4Xavoj4d3OomO73Hnw37D_EfcnzgGosvSzRmRnMUZ-nFIL-8iUO_2iAk1WtrVo7ho4vGA-JeMmPt2YihoRD7SXpit95jyyOY9yJTSAL7nAAUc5wGgD_wyfRh5cfD0XJzOukG76nShQBozibDmS_cDuSg/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fissues%2F7480>

Thank you for the accurate reporting ?
- Xavier

________________________________
From: Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU<mailto:AMORALESPINZON at BWH.HARVARD.EDU>>
Sent: June 14, 2021 7:19 PM
To: Xavier Lecours Boucher, Mr <xavier.lecoursboucher at mcgill.ca<mailto:xavier.lecoursboucher at mcgill.ca>>
Cc: loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca> <loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca>>; Sridar Narayanan, Dr. <sridar.narayanan at mcgill.ca<mailto:sridar.narayanan at mcgill.ca>>; Douglas Arnold, Dr. <douglas.arnold at mcgill.ca<mailto:douglas.arnold at mcgill.ca>>; Guttmann, Charles,M.D. <guttmann at bwh.harvard.edu<mailto:guttmann at bwh.harvard.edu>>; Istvan Akos Imre Morocz, Dr <istvan.morocz at mcgill.ca<mailto:istvan.morocz at mcgill.ca>>
Subject: Re: Error insert candidates api/v0.0.3/candidates

Hi Xavier,

Thank you for reaching out.

There is a first bug int the following line:

https://github.com/aces/Loris/blob/fc574c06f5f6c96483f22788ed446f9aa36a4783/php/libraries/User.class.inc#L71<https://secure-web.cisco.com/1is0gCN2r6hBd-51j4mZFeeJjBMdeq7KAF8qWNfBz_zVRYRKx0N5hT29cPQRpGT2awzDS4wUaksDBCo4nLjbN-ffNauQNwH716qcgIxE02xF4vz63_eooqwbhE2Y5aCjL17XZPB-bqwybaWidSYy5JWbgnSG9PcsUB-CqucRPZ2Tw1TJyEB12UQY6IgUNqS5H_zga_rxEGeNQUBrzOPbSumr6l4mHs3jHEIPgv6g4nTQ_29J9BZKmtKkIMwjxdHW1L7AzbnOCPZtRlKlu2LzOeA/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2Ffc574c06f5f6c96483f22788ed446f9aa36a4783%2Fphp%2Flibraries%2FUser.class.inc%23L71>

If a user affiliated with multiple center such that the concatenation of the names of the center has more than 1024 characters, then the response is cut to 1024 and if the center name I want to use is not in there, then I get the same error. This is the case for my admin user and the more than 500 affiliation centers. We saw this with C?cile on Friday.

Here is the response of the API after the changes you suggested using a user that has few center affiliations, including the one I want to add a Candidate to:

===
array(56) {
[0]=>
string(8) "inf_0102"
[1]=>
string(8) "inf_0103"
...
[55]=>
string(8) "inf_0313"
}
string(3) "BGI"
===

This is the same output when using my admin user:

===
array(114) {
[0]=>
string(9) "101-KGH-1"
[1]=>
string(9) "101-LHS-1"
...
[111]=>
string(9) "272-CAA-1"
[112]=>
string(9) "273-DDM-1"
[113]=>
string(1) "2"
}
string(3) "BGI"
===

Let me know what is the next step? I?m happy to debug over Zoom if that helps.

Best,
Alfredo.

On Jun 14, 2021, at 11:04 AM, Xavier Lecours Boucher, Mr <xavier.lecoursboucher at mcgill.ca<mailto:xavier.lecoursboucher at mcgill.ca>> wrote:

        External Email - Use Caution

Hi Alfredo,

There is a discrepancy between the frontend and the API way to determine which site the user have, which site to give the new candidate and if a user is allowed to create a candidate at a given site. Looking at the frontend code, I can't see any validation on the site other than the content of the html dropdown.

The API checks if the user has the given site.

Here is the API code<https://secure-web.cisco.com/1if5Eh5lps70ewShywnybjkoSptbZugNrF3KFp3Ikj31HdtP_kKvS_EXtIV8bo4sQ1XhY1mFp5CQ97Ta_5VBA3uwcObZQDvyGWwqHmVY0RTDJczBF5eOvEjZp7XGPCCyrnC4a97TlYo56eat54xNRJvdoVSn2fB149oaDE6j2hyCWmy1zrBbbWahlzbiuR4ydd0k7ZkfEvAvqWjjFfssiSvOxhGd280OG8sBE1fHVx3TqRc84NdaTQzYkM7BTv0DfNMKxpaQDEwZhM8ymX88Ctg/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L185>:
$usersites = $user->getSiteNames();
if (!in_array($data['Candidate']['Site'], $usersites)) {
    return new \LORIS\Http\Response\JSON\Forbidden(
        'You are not affiliated with the candidate`s site'
    );
}
So, for debugging, can you add the following lines at https://github.com/aces/Loris/blob/23.0-release/modules/api/php/endpoints/candidates.class.inc#L184<https://secure-web.cisco.com/1fNyuDjwP45YKJ02APn6ItF2MKoXIN6PCZlwgpbccCqoLUyWWzBLPJbE9XvOxBwjXow-KCKAzoVz8qwXZiF1zDuQ5f6yXeKAzrO1vsrM5j3wIBR2iM29VBUkAdxLP_5M47korX_RF2RXn6XOisI2aTB6xZnehZf6R1VVjTIBiLrLcDC5Jvcauo6T97HesnSkByNr0iYbJYpIgZlaHnGZkPWOBjVGGezJr6Jh8gsh04CJgRl6JZjW5wAwoqqecRvhHRMnKhW7tmcggfWieTL112Q/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L184> , then tell me what it the response to your POST request ?

var_dump($user->getSiteNames(), $data['Candidate']['Site']);
exit;
Thank you
Xavier


<https://secure-web.cisco.com/1if5Eh5lps70ewShywnybjkoSptbZugNrF3KFp3Ikj31HdtP_kKvS_EXtIV8bo4sQ1XhY1mFp5CQ97Ta_5VBA3uwcObZQDvyGWwqHmVY0RTDJczBF5eOvEjZp7XGPCCyrnC4a97TlYo56eat54xNRJvdoVSn2fB149oaDE6j2hyCWmy1zrBbbWahlzbiuR4ydd0k7ZkfEvAvqWjjFfssiSvOxhGd280OG8sBE1fHVx3TqRc84NdaTQzYkM7BTv0DfNMKxpaQDEwZhM8ymX88Ctg/https%3A%2F%2Fgithub.com%2Faces%2FLoris%2Fblob%2F23.0-release%2Fmodules%2Fapi%2Fphp%2Fendpoints%2Fcandidates.class.inc%23L185>




________________________________
From: loris-dev-bounces at bic.mni.mcgill.ca<mailto:loris-dev-bounces at bic.mni.mcgill.ca> <loris-dev-bounces at bic.mni.mcgill.ca<mailto:loris-dev-bounces at bic.mni.mcgill.ca>> on behalf of Morales Pinzon, Alfredo <AMORALESPINZON at BWH.HARVARD.EDU<mailto:AMORALESPINZON at BWH.HARVARD.EDU>>
Sent: June 10, 2021 6:43 PM
To: loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca> <loris-dev at bic.mni.mcgill.ca<mailto:loris-dev at bic.mni.mcgill.ca>>
Cc: Rozie Arnaoutelis, Ms. <rozie.arnaoutelis at mcgill.ca<mailto:rozie.arnaoutelis at mcgill.ca>>; Sridar Narayanan, Dr. <sridar.narayanan at mcgill.ca<mailto:sridar.narayanan at mcgill.ca>>; Douglas Arnold, Dr. <douglas.arnold at mcgill.ca<mailto:douglas.arnold at mcgill.ca>>; Guttmann, Charles, M.D. <guttmann at bwh.harvard.edu<mailto:guttmann at bwh.harvard.edu>>
Subject: [Loris-dev] Error insert candidates api/v0.0.3/candidates

Dear LorisDev team,

I can create Candidates using the web interface using an admin account that is linked to all the project and all the sites in the system. However when I try to create a Candidate using the api api/v0.0.3/candidates I am getting the following error:

===
{
"error": "You are not affiliated with the candidate`s site"
}
===

I?m happy to run some queries in the database to figure out what?s happening. Any ideas?

Best,
Alfredo.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/1RIVOq7LsmWlGiSIpDV1iDxaKZDL4FrGgQqzQtNFdUclqgR60trLfQ1Mrv_3Jb_-Nh4SRJc_iFLFJXdP2NJMN-0HWzFBk2MS4e6vgU15rC0uEPg1nX41lWaBkOPKqdov7TstiKnJaFkbN5V7SktHpEnB3YYGFW1gB6K2lnAdqBdfpA7Aqd2s-V5UtJpeqM2FMPp-6AX661pD9MbwDsRNm-XzJR8bNHmTV73XfqLOtbx3837t9gQYbTub-VXeXIHQbr3uLsfcOkE7rA-nTlDJnnQ/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline<http://secure-web.cisco.com/1Vwh3UUC6V4rpqXvU_q2VCQpZWGlmFiazKloKWLfmZwJ4rZfTMZAjadDHqpiL4hdyiZwxyJ8d8PAf6E-F1LJ0q-S_1MWdESz5IOFewIa8Q5tdt3PpW4m6WX1GpW4lJzHiBxvhPZSQx0IEXiwIIh2EkQED5DcL5a_7MbJpo9NwB_hq8PWYX9g3zzxor7IGOpaAsgBVF09dT5wNtAH2HLm4uFh26TWsaLgdpVdeMYW8I6Vp6og-Cz_zxlZga7QW03bZG7Go80C3nIJcwxvnjWOPjw/http%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline> . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210621/e943037f/attachment-0001.html>

From samirdas99 at gmail.com  Tue Jun 22 11:37:51 2021
From: samirdas99 at gmail.com (Samir Das)
Date: Tue, 22 Jun 2021 11:37:51 -0400
Subject: [Loris-dev] IPMSA-SPINE: Computational Resources
In-Reply-To: <CB05BFE6-6ED9-43F0-8D52-B36F2B1B63BA@bwh.harvard.edu>
References: <CB05BFE6-6ED9-43F0-8D52-B36F2B1B63BA@bwh.harvard.edu>
Message-ID: <CAF6xo=NwyvKjRC7UcdxUSnpAsg+7d3j_JH-vJNEt50mx-1qWcg@mail.gmail.com>

Hi Alfredo,

We do have several options. In house, we have our local servers...
sometimes we offer them to some collaborators depending on
the circumstances. There is also the BIC systems, which you might be aware
of already. Another possibility is to make use of the HBHL infrastructure,
called NeuroHub. I've CC-ed Bryan Caron who is the lead for that project.
Otherwise, the only other option I can think of is directly using Compute
Canada if it's processing power you need.

Are you looking for storage as well? Maybe you can tell us a bit more?

Best,

Samir Das

On Mon, Jun 21, 2021 at 4:05 PM Morales Pinzon, Alfredo <
AMORALESPINZON at bwh.harvard.edu> wrote:

> Hi Louis,
>
> We would like to know what computational resources you use/have to run
> your image processing algorithms, for instance in clusters, at McGill. We
> are planning to run algorithms from SPINE in those computational facilities
> for the IPMSA project.
>
> Best regards,
> Alfredo.
> The information in this e-mail is intended only for the person to whom it
> is addressed. If you believe this e-mail was sent to you in error and the
> e-mail contains patient information, please contact the Mass General
> Brigham Compliance HelpLine at
> http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent
> to you in error but does not contain patient information, please contact
> the sender and properly dispose of the e-mail.
> Please note that this e-mail is not secure (encrypted).  If you do not
> wish to continue communication over unencrypted e-mail, please notify the
> sender of this message immediately.  Continuing to send or respond to
> e-mail after receiving this message means you understand and accept this
> risk and wish to continue to communicate over unencrypted e-mail.
>
>
> _______________________________________________
> Loris-dev mailing list
> Loris-dev at bic.mni.mcgill.ca
> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210622/e2db505f/attachment.html>

From samirdas99 at gmail.com  Tue Jun 22 11:48:31 2021
From: samirdas99 at gmail.com (Samir Das)
Date: Tue, 22 Jun 2021 11:48:31 -0400
Subject: [Loris-dev] IPMSA-SPINE: Computational Resources
In-Reply-To: <CAF6xo=NwyvKjRC7UcdxUSnpAsg+7d3j_JH-vJNEt50mx-1qWcg@mail.gmail.com>
References: <CB05BFE6-6ED9-43F0-8D52-B36F2B1B63BA@bwh.harvard.edu>
 <CAF6xo=NwyvKjRC7UcdxUSnpAsg+7d3j_JH-vJNEt50mx-1qWcg@mail.gmail.com>
Message-ID: <CAF6xo=OZUpUbh4xHMK0WgL+LGYGDZbxiwZiQokqFTfz_8JWNvQ@mail.gmail.com>

P.S. I just noticed that this was directed at Louis. Maybe you didn't mean
to send this to the loris-dev mailing list? Either way, my prior response
still outlines the possibilities. The BIC servers are probably the part
that Louis can elaborate on more. :)



On Tue, Jun 22, 2021 at 11:37 AM Samir Das <samirdas99 at gmail.com> wrote:

> Hi Alfredo,
>
> We do have several options. In house, we have our local servers...
> sometimes we offer them to some collaborators depending on
> the circumstances. There is also the BIC systems, which you might be aware
> of already. Another possibility is to make use of the HBHL infrastructure,
> called NeuroHub. I've CC-ed Bryan Caron who is the lead for that project.
> Otherwise, the only other option I can think of is directly using Compute
> Canada if it's processing power you need.
>
> Are you looking for storage as well? Maybe you can tell us a bit more?
>
> Best,
>
> Samir Das
>
> On Mon, Jun 21, 2021 at 4:05 PM Morales Pinzon, Alfredo <
> AMORALESPINZON at bwh.harvard.edu> wrote:
>
>> Hi Louis,
>>
>> We would like to know what computational resources you use/have to run
>> your image processing algorithms, for instance in clusters, at McGill. We
>> are planning to run algorithms from SPINE in those computational facilities
>> for the IPMSA project.
>>
>> Best regards,
>> Alfredo.
>> The information in this e-mail is intended only for the person to whom it
>> is addressed. If you believe this e-mail was sent to you in error and the
>> e-mail contains patient information, please contact the Mass General
>> Brigham Compliance HelpLine at
>> http://www.massgeneralbrigham.org/complianceline . If the e-mail was
>> sent to you in error but does not contain patient information, please
>> contact the sender and properly dispose of the e-mail.
>> Please note that this e-mail is not secure (encrypted).  If you do not
>> wish to continue communication over unencrypted e-mail, please notify the
>> sender of this message immediately.  Continuing to send or respond to
>> e-mail after receiving this message means you understand and accept this
>> risk and wish to continue to communicate over unencrypted e-mail.
>>
>>
>> _______________________________________________
>> Loris-dev mailing list
>> Loris-dev at bic.mni.mcgill.ca
>> https://mailman.bic.mni.mcgill.ca/mailman/listinfo/loris-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.bic.mni.mcgill.ca/pipermail/loris-dev/attachments/20210622/8eaa2e9a/attachment.html>