c++ MPI_Get,目标和源上的跨步访问:在规范/便携范围内?

yi0zb3m4  于 2023-05-24  发布在  其他
关注(0)|答案(1)|浏览(116)

通过MPI_Get和MPI_Type_vector,可以使用跨式远程内存访问。然而,我只看到过vector类型用于访问源内存的例子,而不是目标内存:

MPI_Datatype type_origin;
MPI_Type_vector(..., &type_origin);
MPI_Get(myBuf, size, MPI_DOUBLE, rank_origin, disp_origin, 1, type_origin, win);

我想知道使用vector类型(=strided)访问源内存和目标内存是否符合规范:

MPI_Datatype type_origin, type_target;
MPI_Type_vector(..., &type_origin);
MPI_Type_vector(..., &type_target);
MPI_Get(myBuf + disp_target, 1, type_target, rank_origin, disp_origin, 1, type_origin, win);

我试用了它,在我的机器上运行良好,使用Fedora 36、g++ v.12.2.1、OpenMPI v.3.1和Eigen v.3.4
但我不知道这是否符合规范和可移植性。请指示。
以下是我的完整示例:

#define OMPI_SKIP_MPICXX 1
#include <mpi.h>
#include <Eigen/Dense>
#include <iostream>

using namespace Eigen;

MPI_Datatype getVectorType(
    const Ref<const Array2i>& totalBlockSize,
    const Ref<const Array2i>& subBlockSize,
    Index nComponents
){
    MPI_Datatype vec;
    MPI_Type_vector(
        subBlockSize.y(),
        subBlockSize.x() * nComponents,
        totalBlockSize.x() * nComponents,
        MPI_DOUBLE,
        &vec
    );
    return vec;
}

int getDisp(
    const Ref<const Array2i>& start,
    const Ref<const Array2i>& size,
    Index nComponents
){
    return ( start.y() * size.x() + start.x() ) * nComponents;
}

int main(int argc, char* argv[]){
    MPI_Init(&argc,&argv);
    MPI_Comm comm {MPI_COMM_WORLD};

    int nRanks, rank;
    MPI_Comm_size(comm, &nRanks);
    MPI_Comm_rank(comm, &rank);

    /* let's just say it's ranks 1 and 2 that have to communicate */
    int
        rank_origin {1},
        rank_target {2};
    /* and what they have to communicate is a block of data,
     * which is not contiguous on either rank */
    Array2i
        size_origin { 8,12},
        size_target { size_origin + 1 },
        start_block { 3, 4},
        size_block  { 4, 6};

    ArrayXXd arr_origin, arr_target;

    /* number of components per cell, equals number of rows in arrays */
    /* to make it simple, it's set to 1 here, so it can be ignored below */
    Index nComp {1};

    auto reshaped = [&](ArrayXXd& arr, const Array2i& size){
        return arr.reshaped( nComp * size.x(), size.y() );
    };
    auto reshapedBlock = [&](auto& resh,
        const Array2i& start_block,
        const Array2i& size_block
    ){
        return resh.block(
            nComp * start_block.x(), start_block.y(),
            nComp * size_block .x(), size_block .y()
        );
    };
    auto print = [&](const auto& resh){
        std::cout
            << "On rank " << rank
            << ", array content (reshaped):\n" << resh
            << "\n";
    };

    if ( rank == rank_origin ){
        arr_origin.resize( nComp, size_origin.prod() );
        /* set here as a default value so that we know where it's from */
        arr_origin = -rank_origin;
        auto resh { reshaped(arr_origin, size_origin) };
        auto reshBlock { reshapedBlock(resh, start_block, size_block) };
        reshBlock = rank_origin;
        print(resh);
    }

    MPI_Datatype type_origin, type_target;
    if ( rank == rank_target ){
        arr_target.resize( nComp, size_target.prod() );
        arr_target= -rank_target;
        type_origin = getVectorType(size_origin, size_block, nComp);
        type_target = getVectorType(size_target, size_block, nComp);
        MPI_Type_commit(&type_origin);
        MPI_Type_commit(&type_target);
    }

    MPI_Win win;
    constexpr int disp { sizeof(double) };
    MPI_Win_create(
        arr_origin.data(), arr_origin.size() * disp, disp,
        MPI_INFO_NULL, comm, &win
    );
    MPI_Win_fence(0, win);
    if ( rank == rank_target ){
        int
            disp_origin { getDisp(start_block, size_origin, nComp) },
            disp_target { getDisp(start_block, size_target, nComp) };
        MPI_Get(
            arr_target.data() +
            disp_target, 1, type_target,
            rank_origin,
            disp_origin, 1, type_origin,
            win
        );
        MPI_Type_free(&type_origin);
        MPI_Type_free(&type_target);
    }
    MPI_Win_fence(0, win);

    if ( rank == rank_target ){
        print( reshaped(arr_target, size_target) );
    }

    MPI_Win_free(&win);
    MPI_Finalize();
    return 0;
}
ttvkxqim

ttvkxqim1#

是的,这是法律的的。对于MPI_Get可以使用的数据类型没有限制。尽管非连续的源缓冲区比远程缓冲区更容易实现,但这可能会影响性能。

相关问题