get_snapshot() and set_snapshot() issue with MPI run

[XinyanYang95]

Hi,

I wrote a hoomd.custom.Action and updater. Here is a simpler version of my action code:

class TestAction(hoomd.custom.Action):

def act(self, timestep):
    # get snapshot by all ranks
    snap = self._state.get_snapshot()

     # update particle positions with root rank
    if rank==0:
        snap.particles.position[[10,20]] = np.vstack([snap.particles.position[[10,20]][:,0] + 0.1*timestep
        snap.particles.position[[10,20]][:,1],
        snap.particles.position[[10,20]][:,2]]).T

    communicator.barrier_all()
    # set new snapshot by all ranks after the position update is complete
    self._state.set_snapshot(snap)    
    pass

---

It updates some particle positions every timestep. The particles to be updated are bonded to some center particles of rigid bodies.

It runs well for 1-core simulation. However, when I turned to MPI run with many cores, there is always an error message "RuntimeError: Error while updating constituent particles:Composite particle with body tag xxx incomplete ..." Did I use get_snapshot() and set_snapshot() incorrectly?

I even tried to test a simpler version with only get_snapshot() and set_snapshot(). This still gave that error.

class TestAction(hoomd.custom.Action):

def act(self, timestep):
    snap = self._state.get_snapshot()
    self._state.set_snapshot(snap)    
    pass

---

Any help would be appreciated. Thank you!

Best,
Xinyan

[joaander]

I recommend using components implemented in C++ to modify particle positions on every step. Local snapshots are the next best performing option.

[XinyanYang95]

Thanks! I wonder if bonded particles are always assigned to the same domain when doing domain decomposition? Would the particle go from one domain to another in the simulation?

Also, following my initial post, is there any insight why the simulation gets terminated when I call snap=self._state.get_snapshot() then self._state.set_snapshot(snap) in MPI run?

[joaander]

HOOMD-blue always places particles on the local domain. Bonded and constituent particles in neighboring domains are communicated as ghost particles.

I have no guesses on the cause of the error in the original simulation. I would need a complete and minimal working example that demonstrates the error to find the cause.

[b-butler]

I can't tell for sure, but my guess is the move switches the local rank of a central particle and the ghost particles are not being communicated properly.