Degraded CPU performance ARCHER2

[JPRichings]

QFT CPU benchmark performance

ARCHER2 single node performance

Previous performance v3.5.0

	QFT  time
Qubits	nodes	Standard	High freq	high freq % of standard
33	1	267.69	256.35	95.76339884

Current performance v4.1.0:

Final state:

QFT run time: 819.105s
Total run time: 1021.17

Compare QFT run time.

Function that dominates run in V4:

QuEST/quest/src/cpu/cpu_subroutines.cpp

Line 582 in 4d44ec8

void cpu_statevec_anyCtrlOneTargDiagMatr_sub(Qureg qureg, vector<int> ctrls, vector<int> ctrlStates, int targ, DiagMatr1 matr) {

Function that dominates run in V3.5.0:

QuEST/QuEST/src/CPU/QuEST_cpu.c

Line 3208 in 23a0a08

void statevec_controlledPhaseShift (Qureg qureg, int idQubit1, int idQubit2, qreal angle)

I suspect this is due to the changes in some of the openmp directives between old and new I have profiler results I can share directly but main features of the profiling is that voluntary context switching has increased by a factor of 3 and % of time in CPU float instructions down by a factor of 3.

[TysonRayJones]

Oh shoot! Beware that the QFT function in v4 does not include the algorithmic optimisation leveraged in v3 (combining adjacent controlled phase shifts), since a further improvement is possible I didn't yet have time to implement (it narrowly escaped inclusion in this preprint). I lament this in #604. So the QFT functions themselves shouldn't be directly compared, instead just the same constituent circuits. Is that what was done here?

[JPRichings]

We are not calling the built in qft functions we are building our own circuit for these test. The only change in the code we are executing between V3 and V4 is swapping functions names as per the deprecated header and changing the environment initialisation and finalisation.

[JPRichings]

On further experimentation the performance of QuEST V4 is now sensitive to numa effects single node ARCHER2 runs should now be run in a 8 processes per node 16 threads per process decomposition. This has an effect on the number of qubits that can be simulate single node but does not seem to effect scaling more broadly.

[TysonRayJones]

I'll still never sleep soundly until understanding why/how single-core performance degraded - I'll make another issue to remind myself to investigate when I get a sec!