[instrument_manager] Parse Instrument Data from CSV file

[jeffersoncasimir]

Closes #9341

Brief summary of changes

Contains a new CSVParser class, extended by an InstrumentDataParser class.

A template file with the expected headers for the selected instrument can be downloaded from the new "Upload Instrument Data" panel's form in the Upload tab of the Instrument Manager module.

The uploaded .csv file will be saved and tracked in a new table called instrument_data_files.

The file's headers must match the selected instrument's expected headers.

For each row, an instrument is created using the NDB_BVL_Instrument::factory and is then saved to the database.

If an error is detected, the database transaction will be rolled back, and the list of errors will be returned to the user.

If the uploaded file exceeds a certain size (currently hard-coded to 512 MB), the task will be performed in the background as a new server_process_manager type called parse_instrument_data. This process runs the parse_instrument_data.php script. It is expected that the monitor_instrument_data.php script is run periodically to check the status of running tasks. When a task is done executing, the server_process_manager table is updated and the user who uploaded the file receives an email containing the "computed exit text".

• Documentation updated

Testing instructions

1. Select an instrument from the dropdown
2. Download the headers template
3. Manually add rows to the template and upload it
4. Ensure it works as expected for files exceeding MAX_FILE_BYTES
5. Ensure MriUploadServerProcess still works as expected

Screenshots:

New features since writing:

• Create participants and sessions if they don't exist
  • Two different expected/downloadable templates
• Return StudyID -> PSCID mapping when creating participants
• REDCap functionality (upload DD. Converts to .LINST files w/ redcap{@}true in .meta)
• Ability to bulk upload data for multiple instrument at once (mainly for REDCap, but works for all)

Still unresolved:

• examiner is always uploader

[skarya22]

From Nov 19 LORIS call:

• Move data upload to Browse instead of Upload
• Should store the CSV somewhere that was uploaded
• Change from saveValues to save so that there is validation
• TODO: Discuss further about what the behaviour should be when data is not validated correctly. Don't upload anything, or upload all without errors

[MaximeBICMTL]

Is this a bug fix ?

If I got it correctly, it seems to be handling for cases where we have select / radio buttons with no option. The whole function is rather cryptic and lacking in comments IMO, and could benefit from a rewrite, but if this simple fix works, then that is also fine with me.

Anyway, if this is a bug fix as I said, could you split this into its own PR ?

[MaximeBICMTL]

(commenting code to allow thread discussion)

I only reviewed the REDCap-related parts of the PR, the whole part about parsing REDCap instrument CSV files and parsing them to LINST seems very similar to Regis' redcap2linst.php script, and I am worried this introduces significant duplication which will be hard to maintain and may display inconsistencies.

Also, this PR introduces a RedcapCSVParser.class.inc class in libraries, I would prefer that all the REDCap-related stuff, even libraries, go into the REDCap module, as to not bloat the shared libraries and keep the architecture more predictable and modular.