When one is combining the shared- and distributed-memory models, some subtle issues arise regarding local and global synchronization. For example, the cluster slaves that are forked off by the cluster master need to wait until it has obtained shared memory, which is under user control, before commencing operations on shared memory. They can't wait in an ordinary barrier because a user barrier is created in user-obtained shared memory, which is exactly what they don't have at the time they are created. Therefore a special built-in barrier is provided for this purpose, accessed by p4_cluster_mem_synch.
One unusual implementation of p4 was on the short-lived Alliant CAMPUS system, which had HiPPI switches connecting shared-memory Alliants. For this machine we and Robert Harrison wrote an interface to the HiPPI switch that would support p4 and TCGMSG types of operations. It is exactly this type of hardware configuration that p4's cluster model is for.