[from mpich] darray (romio) needs a resize in the block()/cyclic() subtype creation

Merging a romio commit that was accepted by mpich:

> darray (romio) needs a resize in the block()/cyclic() subtype creation
>
> When Type_create_darray() uses the block()/cyclic() function to
> construct a subtype for whatever dimension it's processing, it
> needs to resize the resulting type before it goes into the type
> construction as the input for processing the next dimension.
>
> The same problem is in the main path's type creation, and in romio's
> mirroring of it.
>
> Gist for a testcase:
> https://gist.github.com/markalle/940de93d64fd779e304ee124855b8e6a
>
> The darray_bug_romio.c testcase creates a darray using
> * 4 ranks in a 2x2 grid
> * looking at the type made for rank 0
> * inner dimension: 4 ints distributed block over 2 ranks with 2 items each
> * outer dimension: 6 of the above distributed cyclic over 2 ranks with 2 items each
>
> The type created by processing the first dimension should look like this
> [ x x . . ]
>
> And then distributing those for the second dimension becomes
>
> [ x x x x ]
> [ . . . . ]
> [ . . . . ]
> [ . . . . ]
> [ x x x x ]
> [ . . . . ]
>
> Going to the MPI standard to justify why the first layout is right,
> it's where the definiton of the cyclic() function has a ub_marker
> of gsize*ex, eg 4 ints for that first type.

Signed-off-by: Mark Allen <[email protected]>

Author: markalle

ompi/mca/io/romio321/romio/adio/common/ad_darray.c

@@ -199,6 +199,13 @@ static int MPIOI_Type_block(int *array_of_gsizes, int dim, int ndims, int nprocs
      /* in terms of no. of elements of type oldtype in this dimension */
     if (mysize == 0) *st_offset = 0;
 
+    MPI_Aint ex;
+    MPI_Type_extent(type_old, &ex);
+    MPI_Datatype type_tmp;
+    MPI_Type_create_resized(*type_new, 0, array_of_gsizes[dim] * ex, &type_tmp);
+    MPI_Type_free(type_new);
+    *type_new = type_tmp;
+
     return MPI_SUCCESS;
 }
 
@@ -287,5 +294,12 @@ static int MPIOI_Type_cyclic(int *array_of_gsizes, int dim, int ndims, int nproc
 
     if (local_size == 0) *st_offset = 0;
 
+    MPI_Aint ex;
+    MPI_Type_extent(type_old, &ex);
+    MPI_Datatype type_tmp2;
+    MPI_Type_create_resized(*type_new, 0, array_of_gsizes[dim] * ex, &type_tmp2);
+    MPI_Type_free(type_new);
+    *type_new = type_tmp2;
+
     return MPI_SUCCESS;
 }