PARREC: bug in sorting for some scan types #408
Description
The current .PAR/.REC implementation uses a fairly simplistic sorting scheme. This has worked for the majority of the data I have tested it with, but fails for some scan types. The main problem is that sorting is done via a call to np.lexsort, but only a few parameters are used as keys. In particular, the method vol_numbers assigns slices to volumes based on the order they are encountered in the .PAR file. However, there is no guarantee that this corresponds to the actual acquisition order.
Here are the first several frames of a .PAR file corresponding to a T1-weighted structural, but using two echos rather than one:
# === IMAGE INFORMATION ==========================================================
# sl ec dyn ph ty idx pix scan% rec size (re)scale window angulation offcentre thick gap info spacing echo dtime ttime diff avg flip freq RR-int turbo delay b grad cont anis diffusion L.ty
54 1 1 1 0 2 0 16 100 256 256 0.00000 9.97143 3.65651e-003 1070 1860 0.00 -0.00 -0.00 -5.06 12.64 34.10 1.000 0.000 0 2 0 2 1.000 1.000 2.30 0.00 0.00 0.00 1 8.00 0 0 0 448 0.0 1 1 7 0 0.000 0.000 0.000 1
91 1 1 1 0 2 1 16 100 256 256 0.00000 9.97143 3.65651e-003 1070 1860 0.00 -0.00 -0.00 -5.06 12.64 -2.90 1.000 0.000 0 2 0 2 1.000 1.000 2.30 0.00 0.00 0.00 1 8.00 0 0 0 448 0.0 1 1 7 0 0.000 0.000 0.000 1
90 1 1 1 0 2 2 16 100 256 256 0.00000 9.97143 3.65651e-003 1070 1860 0.00 -0.00 -0.00 -5.06 12.64 -1.90 1.000 0.000 0 2 0 2 1.000 1.000 2.30 0.00 0.00 0.00 1 8.00 0 0 0 448 0.0 1 1 7 0 0.000 0.000 0.000 1
89 1 1 1 0 2 3 16 100 256 256 0.00000 9.97143 3.65651e-003 1070 1860 0.00 -0.00 -0.00 -5.06 12.64 -0.90 1.000 0.000 0 2 0 2 1.000 1.000 2.30 0.00 0.00 0.00 1 8.00 0 0 0 448 0.0 1 1 7 0 0.000 0.000 0.000 1
92 1 1 1 0 2 4 16 100 256 256 0.00000 9.97143 3.65651e-003 1070 1860 0.00 -0.00 -0.00 -5.06 12.64 -3.90 1.000 0.000 0 2 0 2 1.000 1.000 2.30 0.00 0.00 0.00 1 8.00 0 0 0 448 0.0 1 1 7 0 0.000 0.000 0.000 1
4 2 1 1 0 2 5 16 100 256 256 0.00000 9.97143 3.65651e-003 1070 1860 0.00 -0.00 -0.00 -5.06 12.64 84.10 1.000 0.000 0 2 0 2 1.000 1.000 5.76 0.00 0.00 0.00 1 8.00 0 0 0 448 0.0 1 1 7 0 0.000 0.000 0.000 1
55 1 1 1 0 2 6 16 100 256 256 0.00000 9.97143 3.65651e-003 1070 1860 0.00 -0.00 -0.00 -5.06 12.64 33.10 1.000 0.000 0 2 0 2 1.000 1.000 2.30 0.00 0.00 0.00 1 8.00 0 0 0 448 0.0 1 1 7 0 0.000 0.000 0.000 1
175 2 1 1 0 2 7 16 100 256 256 0.00000 9.97143 3.65651e-003 1070 1860 0.00 -0.00 -0.00 -5.06 12.64 -86.90 1.000 0.000 0 2 0 2 1.000 1.000 5.76 0.00 0.00 0.00 1 8.00 0 0 0 448 0.0 1 1 7 0 0.000 0.000 0.000 1
174 2 1 1 0 2 8 16 100 256 256 0.00000 9.97143 3.65651e-003 1070 1860 0.00 -0.00 -0.00 -5.06 12.64 -85.90 1.000 0.000 0 2 0 2 1.000 1.000 5.76 0.00 0.00 0.00 1 8.00 0 0 0 448 0.0 1 1 7 0 0.000 0.000 0.000 1
173 2 1 1 0 2 9 16 100 256 256 0.00000 9.97143 3.65651e-003 1070 1860 0.00 -0.00 -0.00 -5.06 12.64 -84.90 1.000 0.000 0 2 0 2 1.000 1.000 5.76 0.00 0.00 0.00 1 8.00 0 0 0 448 0.0 1 1 7 0 0.000 0.000 0.000 1
The first column labelled "sl" is the slice number and "ec" is the echo number. As you can see the slices are in a seemingly randomized order, with some occurring first for echo 1 and some for echo 2.
This file is a 3D scan with 2 echos, so the data ends up 4D with size 2 on the last axis. However the echo times are assigned randomly across the volumes.
This can be fixed by adding by adding additional keys in the call to np.lexsort. I will submit a PR shortly with a potential fix.
Note: There is a "sort" checkbox at the scanner when exporting to .PAR/.REC. If that had been checked upon data export, things may have been sorted in a more logical order to start with. It would be nice to get the sorting right in nibabel regardless of the actual order in the file, though!