I’m trying to understand how to use MPI RMA in various situations so I’m working through some scenarios that I envision using in the future. In this one, the worker ranks store information (i.e., their rank number) in an array and accumulate that array with the rank==0 process.

I can get this to work when the send and receive “buffers” are allocated on the stack, but not when allocation is done on the heap with either the “new” operator or by using MPI_Alloc_mem.

The code is shown next:

#include <cstdio>
#include <cstddef>
#include <mpi.h>

int main (int argc, char *argv[])
{
  int my_rank, size;
  int count = 5;

  int *snd_buf;
  int *rcv_buf;

  MPI_Init(&argc, &argv);

  MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);
  MPI_Comm_size(MPI_COMM_WORLD, &size);

  int expected_value = 0.0;

  if (my_rank == 0)
  {
    for(int i=0; i<size; ++i)  expected_value += i;
  }

  MPI_Alloc_mem(count * sizeof(int), MPI_INFO_NULL, &rcv_buf);
  MPI_Alloc_mem(count * sizeof(int), MPI_INFO_NULL, &snd_buf);
  memset(rcv_buf, -1, count * sizeof(int));

  MPI_Win window;

  if (my_rank == 0)
  {
    MPI_Win_create(&rcv_buf, count * sizeof(int), 1, MPI_INFO_NULL, MPI_COMM_WORLD, &window);
  }
  else
  {
    MPI_Win_create(MPI_BOTTOM, 0, 1, MPI_INFO_NULL, MPI_COMM_WORLD, &window);
  }

  if (my_rank != 0)
  {
    for(int i=0; i<count; ++i) snd_buf[i] =  my_rank;

    MPI_Win_lock(MPI_LOCK_EXCLUSIVE, 0, 0, window);
    MPI_Accumulate(&snd_buf, count, MPI_INT, 0, 0, count, MPI_INT, MPI_SUM, window);
    MPI_Win_unlock(0, window);
  }

  MPI_Barrier(MPI_COMM_WORLD);

  if (my_rank == 0)
  {
    printf("n Expected result for all locations: %dn", expected_value);
    printf(" Location   Valuen");
    for(int i=0; i<count; ++i)
      printf(" %5d %9dn", i, rcv_buf[i]);
    printf("n");
  }

  MPI_Free_mem(rcv_buf);
  MPI_Free_mem(snd_buf);

  MPI_Win_free(&window);
  MPI_Finalize();

  printf(" PE [%2d] donen", my_rank);
}

I’m using OpenMPI 4.1 and gcc 12.3 to build the code. The executable from the above segfaults when I try to run it.

If I don’t include the MPI_Barrier the code doesn’t segfault, but runs with the output printed from the rcv_buf variable still being -1 instead of whats contained in expected_value.

I would appreciate some help because I know that I’ve overlooked something, but don’t know what it is.