UCX: Propagate MPI serialized for all worker creations

[tvegas1]

What

Continuation of #12577 by addressing comment from backport #12579.

Move MPI to UCX thread mode function to common source. Also use serialized mode for all oshmem initializations.

Test

Use:

shmem_ctx_t ctx;
shmem_ctx_create(0, &ctx);
shmem_ctx_create(SHMEM_CTX_SERIALIZED, &ctx);
shmem_ctx_create(SHMEM_CTX_PRIVATE, &ctx);

[brminich]

we also need to update spml_ucx_init in oshmem/mca/spml/ucx/spml_ucx_component.c

[tvegas1]

we also need to update spml_ucx_init in oshmem/mca/spml/ucx/spml_ucx_component.c

It's already fixed as part of #12577, please let me know otherwise

[brminich]

we also need to update spml_ucx_init in oshmem/mca/spml/ucx/spml_ucx_component.c

It's already fixed as part of #12577, please let me know otherwise

then why would we need to move it to common_ucx, if it used in pml only?

[tvegas1]

then why would we need to move it to common_ucx, if it used in pml only?

Unfortunately, oshmem and ompi thread mode enums are different, so function cannot be reused. Also, the two oshmem thread mode checking functions are slightly different and cannot be merged.

But moving to common file makes sense anyway I think, and I guess I'd reuse it for osc/ucx to be done in v4.1.x.

[tvegas1]

@bosilca / @yosefe please check?

[yosefe]

do we need it for osc/ucx?

[tvegas1]

do we need it for osc/ucx?

osc/ucx change is needed in v4.1.x only