|
Teuchos - Trilinos Tools Package Version of the Day
|
Implementation of Comm that uses MPI for communication. More...
#include <Teuchos_DefaultMpiComm_decl.hpp>
Related Symbols | |
(Note that these are not member symbols.) | |
| template<typename Ordinal > | |
| RCP< MpiComm< Ordinal > > | createMpiComm (const RCP< const OpaqueWrapper< MPI_Comm > > &rawMpiComm) |
Helper function that creates a dynamically allocated MpiComm object or returns Teuchos::null to correctly represent a null communicator. | |
| template<typename Ordinal > | |
| RCP< MpiComm< Ordinal > > | createMpiComm (const RCP< const OpaqueWrapper< MPI_Comm > > &rawMpiComm, const int defaultTag) |
Helper function that creates a dynamically allocated MpiComm object or returns Teuchos::null to correctly represent a null communicator. | |
| template<typename Ordinal > | |
| MPI_Comm | getRawMpiComm (const Comm< Ordinal > &comm) |
Helper function that extracts a raw MPI_Comm object out of a Teuchos::MpiComm wrapper object. | |
Related Symbols inherited from Teuchos::Comm< Ordinal > | |
| enum | EReductionType |
| Predefined reduction operations that Teuchos::Comm understands. More... | |
| template<typename Ordinal > | |
| int | rank (const Comm< Ordinal > &comm) |
| Get the process rank. | |
| template<typename Ordinal > | |
| int | size (const Comm< Ordinal > &comm) |
| Get the number of processes in the communicator. | |
| template<typename Ordinal > | |
| void | barrier (const Comm< Ordinal > &comm) |
| Barrier. | |
| template<typename Ordinal , typename Packet > | |
| void | broadcast (const Comm< Ordinal > &comm, const int rootRank, const Ordinal count, Packet buffer[]) |
| Broadcast array of objects that use value semantics. | |
| template<typename Ordinal , typename Packet > | |
| void | broadcast (const Comm< Ordinal > &comm, const int rootRank, const ArrayView< Packet > &buffer) |
| Broadcast array of objects that use value semantics. | |
| template<typename Ordinal , typename Packet > | |
| void | broadcast (const Comm< Ordinal > &comm, const int rootRank, Packet *object) |
| Broadcast single object that use value semantics. | |
| template<typename Ordinal , typename Packet > | |
| void | broadcast (const Comm< Ordinal > &comm, const int rootRank, const Ptr< Packet > &object) |
| Broadcast single object that use value semantics. | |
| template<typename Ordinal , typename Packet > | |
| void | broadcast (const Comm< Ordinal > &comm, const Serializer< Ordinal, Packet > &serializer, const int rootRank, const Ordinal count, Packet *const buffer[]) |
| Broadcast array of objects that use reference semantics. | |
| template<typename Ordinal , typename Packet > | |
| void | broadcast (const Comm< Ordinal > &comm, const Serializer< Ordinal, Packet > &serializer, const int rootRank, const ArrayView< const Ptr< Packet > > &buffer) |
| Broadcast array of objects that use reference semantics. | |
| template<typename Ordinal , typename Packet , typename Serializer > | |
| void | broadcast (const Comm< Ordinal > &comm, const Serializer &serializer, const int rootRank, const Ordinal count, Packet buffer[]) |
| Broadcast array of objects that use value semantics using customized serializer. | |
| template<typename Ordinal , typename Packet > | |
| void | gather (const Packet sendBuf[], const Ordinal sendCount, Packet recvBuf[], const Ordinal recvCount, const int root, const Comm< Ordinal > &comm) |
| Gather values from each process to the root process. | |
| template<typename Ordinal , typename Packet > | |
| void | gatherv (const Packet sendBuf[], const Ordinal sendCount, Packet recvBuf[], const Ordinal recvCounts[], const Ordinal displs[], const int root, const Comm< Ordinal > &comm) |
| Gather arrays of possibly different lengths from each process to the root process. | |
| template<typename Ordinal , typename Packet > | |
| void | gatherAll (const Comm< Ordinal > &comm, const Ordinal sendCount, const Packet sendBuffer[], const Ordinal recvCount, Packet recvBuffer[]) |
| Gather array of objects that use value semantics from every process to every process. | |
| template<typename Ordinal , typename Packet > | |
| void | gatherAll (const Comm< Ordinal > &comm, const Serializer< Ordinal, Packet > &serializer, const Ordinal sendCount, const Packet *const sendBuffer[], const Ordinal recvCount, Packet *const recvBuffer[]) |
| Gather array of objects that use reference semantics from every process to every process. | |
| template<typename Ordinal , typename Packet , typename Serializer > | |
| void | gatherAll (const Comm< Ordinal > &comm, const Serializer &serializer, const Ordinal sendCount, const Packet sendBuffer[], const Ordinal recvCount, Packet recvBuffer[]) |
| Gather array of objects that use value semantics from every process to every process using customized serializer. | |
| template<typename Ordinal , typename Packet > | |
| void | scatter (const Packet sendBuf[], const Ordinal sendCount, Packet recvBuf[], const Ordinal recvCount, const Ordinal root, const Comm< Ordinal > &comm) |
| Wrapper for MPI_Scatter; scatter collective. | |
| template<typename Ordinal , typename Packet > | |
| void | reduce (const Packet sendBuf[], Packet recvBuf[], const Ordinal count, const EReductionType reductType, const Ordinal root, const Comm< Ordinal > &comm) |
| Wrapper for MPI_Reduce; reduction to one process, using a built-in reduction operator selected by enum. | |
| template<typename Ordinal , typename Packet > | |
| void | reduceAll (const Comm< Ordinal > &comm, const ValueTypeReductionOp< Ordinal, Packet > &reductOp, const Ordinal count, const Packet sendBuffer[], Packet globalReducts[]) |
| Wrapper for MPI_Allreduce that takes a custom reduction operator. | |
| template<typename Ordinal , typename Packet > | |
| void | reduceAll (const Comm< Ordinal > &comm, const EReductionType reductType, const Ordinal count, const Packet sendBuffer[], Packet globalReducts[]) |
| Collective reduce all of array of objects using value semantics using a pre-defined reduction type. | |
| template<typename Ordinal , typename Packet > | |
| void | reduceAll (const Comm< Ordinal > &comm, const EReductionType reductType, const Packet &send, const Ptr< Packet > &globalReduct) |
| Collective reduce all for single object using value semantics using a pre-defined reduction type. | |
| template<typename Ordinal , typename Packet > | |
| void | reduceAll (const Comm< Ordinal > &comm, const Serializer< Ordinal, Packet > &serializer, const ReferenceTypeReductionOp< Ordinal, Packet > &reductOp, const Ordinal count, const Packet *const sendBuffer[], Packet *const globalReducts[]) |
| Collective reduce all for array of objects using reference semantics. | |
| template<typename Ordinal , typename Packet , typename Serializer > | |
| void | reduceAll (const Comm< Ordinal > &comm, const Serializer &serializer, const ValueTypeReductionOp< Ordinal, Packet > &reductOp, const Ordinal count, const Packet sendBuffer[], Packet globalReducts[]) |
| Collective reduce all of array of objects using value semantics using a user-defined reduction operator and customized serializer. | |
| template<typename Ordinal , typename Packet , typename Serializer > | |
| void | reduceAll (const Comm< Ordinal > &comm, const Serializer &serializer, const EReductionType reductType, const Ordinal count, const Packet sendBuffer[], Packet globalReducts[]) |
| Collective reduce all of array of objects using value semantics using a pre-defined reduction type and customized serializer. | |
| template<typename Ordinal , typename Packet > | |
| void | scan (const Comm< Ordinal > &comm, const ValueTypeReductionOp< Ordinal, Packet > &reductOp, const Ordinal count, const Packet sendBuffer[], Packet scanReducts[]) |
| Scan/Reduce array of objects that use value semantics using a user-defined reduction operator. | |
| template<typename Ordinal , typename Packet > | |
| void | scan (const Comm< Ordinal > &comm, const EReductionType reductType, const Ordinal count, const Packet sendBuffer[], Packet scanReducts[]) |
| Scan/Reduce array of objects using value semantics using a predefined reduction type. | |
| template<typename Ordinal , typename Packet > | |
| void | scan (const Comm< Ordinal > &comm, const EReductionType reductType, const Packet &send, const Ptr< Packet > &scanReduct) |
| Scan/Reduce single object using value semantics using a predefined reduction type. | |
| template<typename Ordinal , typename Packet > | |
| void | scan (const Comm< Ordinal > &comm, const Serializer< Ordinal, Packet > &serializer, const ReferenceTypeReductionOp< Ordinal, Packet > &reductOp, const Ordinal count, const Packet *const sendBuffer[], Packet *const scanReducts[]) |
| Scan/Reduce array of objects that use reference semantics using a user-defined reduction operator. | |
| template<typename Ordinal , typename Packet , typename Serializer > | |
| void | scan (const Comm< Ordinal > &comm, const Serializer &serializer, const ValueTypeReductionOp< Ordinal, Packet > &reductOp, const Ordinal count, const Packet sendBuffer[], Packet scanReducts[]) |
| Scan/Reduce array of objects that use value semantics using a user-defined reduction operator and customized serializer. | |
| template<typename Ordinal , typename Packet , typename Serializer > | |
| void | scan (const Comm< Ordinal > &comm, const Serializer &serializer, const EReductionType reductType, const Ordinal count, const Packet sendBuffer[], Packet scanReducts[]) |
| Scan/Reduce array of objects using value semantics using a predefined reduction type and customized serializer. | |
| template<typename Ordinal , typename Packet > | |
| void | send (const Comm< Ordinal > &comm, const Ordinal count, const Packet sendBuffer[], const int destRank) |
| Send objects that use values semantics to another process. | |
| template<typename Ordinal , typename Packet > | |
| void | ssend (const Comm< Ordinal > &comm, const Ordinal count, const Packet sendBuffer[], const int destRank) |
| Synchronously send objects that use values semantics to another process. | |
| template<typename Ordinal , typename Packet > | |
| void | send (const Comm< Ordinal > &comm, const Packet &send, const int destRank) |
| Send a single object that use values semantics to another process. | |
| template<typename Ordinal , typename Packet > | |
| void | ssend (const Comm< Ordinal > &comm, const Packet &send, const int destRank) |
| Synchronously send a single object that use values semantics to another process. | |
| template<typename Ordinal , typename Packet > | |
| void | send (const Comm< Ordinal > &comm, const Serializer< Ordinal, Packet > &serializer, const Ordinal count, const Packet *const sendBuffer[], const int destRank) |
| Send objects that use reference semantics to another process. | |
| template<typename Ordinal , typename Packet , typename Serializer > | |
| void | send (const Comm< Ordinal > &comm, const Serializer &serializer, const Ordinal count, const Packet sendBuffer[], const int destRank) |
| Send objects that use values semantics to another process using customized serializer. | |
| template<typename Ordinal , typename Packet > | |
| int | receive (const Comm< Ordinal > &comm, const int sourceRank, const Ordinal count, Packet recvBuffer[]) |
| Receive objects that use values semantics from another process. | |
| template<typename Ordinal , typename Packet > | |
| int | receive (const Comm< Ordinal > &comm, const int sourceRank, Packet *recv) |
| Receive a single object that use values semantics from another process. | |
| template<typename Ordinal , typename Packet > | |
| int | receive (const Comm< Ordinal > &comm, const Serializer< Ordinal, Packet > &serializer, const int sourceRank, const Ordinal count, Packet *const recvBuffer[]) |
| Receive objects that use reference semantics from another process. | |
| template<typename Ordinal , typename Packet , typename Serializer > | |
| int | receive (const Comm< Ordinal > &comm, const Serializer &serializer, const int sourceRank, const Ordinal count, Packet recvBuffer[]) |
| Receive objects that use values semantics from another process using customized serializer. | |
| template<typename Ordinal , typename Packet > | |
| void | readySend (const Comm< Ordinal > &comm, const ArrayView< const Packet > &sendBuffer, const int destRank) |
| Ready-Send an array of objects that use values semantics to another process. | |
| template<typename Ordinal , typename Packet > | |
| void | readySend (const Comm< Ordinal > &comm, const Packet &send, const int destRank) |
| Ready-Send a single object that use values semantics to another process. | |
| template<typename Ordinal , typename Packet , typename Serializer > | |
| void | readySend (const Comm< Ordinal > &comm, const Serializer &serializer, const ArrayView< const Packet > &sendBuffer, const int destRank) |
| Ready-Send an array of objects that use values semantics to another process using customized serializer. | |
| template<typename Ordinal , typename Packet > | |
| RCP< CommRequest< Ordinal > > | isend (const Comm< Ordinal > &comm, const ArrayRCP< const Packet > &sendBuffer, const int destRank) |
| Send objects that use values semantics to another process. | |
| template<typename Ordinal , typename Packet > | |
| RCP< CommRequest< Ordinal > > | isend (const Comm< Ordinal > &comm, const RCP< const Packet > &send, const int destRank) |
| Send a single object that use values semantics to another process. | |
| template<typename Ordinal , typename Packet , typename Serializer > | |
| RCP< CommRequest< Ordinal > > | isend (const Comm< Ordinal > &comm, const Serializer &serializer, const ArrayRCP< const Packet > &sendBuffer, const int destRank) |
| Send objects that use values semantics to another process using customized serializer. | |
| template<typename Ordinal , typename Packet > | |
| RCP< CommRequest< Ordinal > > | ireceive (const Comm< Ordinal > &comm, const ArrayRCP< Packet > &recvBuffer, const int sourceRank) |
| Receive one or more objects (that use values semantics) from another process. | |
| template<typename Ordinal , typename Packet > | |
| RCP< CommRequest< Ordinal > > | ireceive (const Comm< Ordinal > &comm, const RCP< Packet > &recv, const int sourceRank) |
| Receive one object (that uses values semantics) from another process. | |
| template<typename Ordinal , typename Packet , typename Serializer > | |
| RCP< CommRequest< Ordinal > > | ireceive (const Comm< Ordinal > &comm, const Serializer &serializer, const ArrayRCP< Packet > &recvBuffer, const int sourceRank) |
| Send objects that use values semantics to another process using customized serializer. | |
| template<typename Ordinal > | |
| void | waitAll (const Comm< Ordinal > &comm, const ArrayView< RCP< CommRequest< Ordinal > > > &requests) |
| Wait for an array of Teuchos::CommRequest objects. | |
| template<typename Ordinal > | |
| void | waitAll (const Comm< Ordinal > &comm, const ArrayView< RCP< CommRequest< Ordinal > > > &requests, const ArrayView< RCP< CommStatus< Ordinal > > > &statuses) |
| Wait on one or more communication requests, and return their statuses. | |
| template<typename Ordinal > | |
| RCP< CommStatus< Ordinal > > | wait (const Comm< Ordinal > &comm, const Ptr< RCP< CommRequest< Ordinal > > > &request) |
| Wait on a single communication request, and return its status. | |
Related Symbols inherited from Teuchos::Describable | |
| DescribableStreamManipulatorState | describe (const Describable &describable, const EVerbosityLevel verbLevel=Describable::verbLevel_default) |
| Describable output stream manipulator. | |
| std::ostream & | operator<< (std::ostream &os, const DescribableStreamManipulatorState &d) |
| Output stream operator for Describable manipulator. | |
Implementation of Describable interface | |
| std::string | description () const |
| int | getTag () const |
| The current tag. | |
| int | incrementTag () |
| Increments the tag and then returns it. | |
Additional Inherited Members | |
Static Public Attributes inherited from Teuchos::Describable | |
| static const EVerbosityLevel | verbLevel_default = VERB_DEFAULT |
Default value for the verbLevel argument of describe(). | |
Implementation of Comm that uses MPI for communication.
| Ordinal | The index type for communication; same as the template parameter of Comm. |
This class uses MPI (the Message Passing Interface) to implement the Comm interface. It includes constructors that take an MPI_Comm from the application.
Assertions:
getRawMpiComm().get() != NULL*getRawMpiComm() != MPI_COMM_NULLgetSize() > 00 <= getRank() && getRank() < getSize() Definition at line 189 of file Teuchos_DefaultMpiComm_decl.hpp.
|
explicit |
Construct an MpiComm with an MPI_Comm.
This constructs an MpiComm that uses the given "raw" MPI communicator underneath. The MPI_Comm must be valid for the lifetime of this MpiComm. You are responsible for freeing the MPI_Comm (using MPI_Comm_free) if necessary.
This constructor should be used only in one of two cases:
If you cannot be sure of either of these two conditions, you should use the version of the constructor that takes an RCP<const OpaqueWrapper<MPI_Comm> >.
Precondition:
rawMpiComm != MPI_COMM_NULL | Teuchos::MpiComm< Ordinal >::MpiComm | ( | const RCP< const OpaqueWrapper< MPI_Comm > > & | rawMpiComm | ) |
Construct an MpiComm with the given wrapped MPI_Comm.
This constructs an MpiComm that uses the given "raw" MPI communicator underneath. This version of the constructor accepts the MPI_Comm wrapped in an OpaqueWrapper, which along with the RCP has the option to free the MPI_Comm (via MPI_Comm_free) after use if necessary. You are responsible when creating the OpaqueWrapper for supplying a "free" function if needed. We recommend using details::safeCommFree for the "free" function, if one is needed.
Preconditions:
rawMpiComm.get() != NULL*rawMpiComm != MPI_COMM_NULL | Teuchos::MpiComm< Ordinal >::MpiComm | ( | const RCP< const OpaqueWrapper< MPI_Comm > > & | rawMpiComm, |
| const int | defaultTag | ||
| ) |
Construct an MpiComm with a wrapped MPI_Comm and a default tag.
This constructor has the same meaning as the one-argument constructor that takes RCP<const OpaqueWrapper<MPI_Comm> >, except that it sets the default message tag on all processes to defaultTag. This avoids the MPI_Bcast that the other two constructors do.
This constructor is declared private for now, because it is an implementation detail of duplicate(). We may choose to expose it in the future.
Preconditions:
rawMpiComm.get() != NULL*rawMpiComm != MPI_COMM_NULLdefaultTag is the same on all processes in the given communicator | Teuchos::MpiComm< Ordinal >::MpiComm | ( | const MpiComm< Ordinal > & | other | ) |
Construct a communicator with a new context with the same properties as the original.
The newly constructed communicator will have a duplicate communication space that has the same properties (e.g. processes, attributes, topologies) as the input communicator.
| other | The communicator to copy from. |
Preconditions:
other.getRawMpiComm().get() != NULL && *other.getRawMpiComm() != NULL
|
inline |
Return the embedded wrapped opaque MPI_Comm object.
Definition at line 271 of file Teuchos_DefaultMpiComm_decl.hpp.
| void Teuchos::MpiComm< Ordinal >::setErrorHandler | ( | const RCP< const OpaqueWrapper< MPI_Errhandler > > & | errHandler | ) |
Set the MPI error handler for this communicator.
| errHandler | [in] The error handler to set. If null, do nothing. |
MPI lets you set an error handler function specific to each communicator. (See Section 8.3 of the MPI 3.0 Standard.) MpiComm wraps this functionality using this method. You must first either create an error handler using MPI_Comm_create_errhandler() (or MPI_Errhandler_create() if you are stuck with an MPI 1 implementation), or use one of the default error handlers that the MPI standard or your MPI implementation provides. You will need to wrap the MPI error handler in an OpaqueWrapper. (See the documentation of OpaqueWrapper for the rationale behind not using MPI's opaque objects directly.)
MpiComm will not attempt to call MPI_Errhandler_free() on the error handler you provide. You are responsible for arranging that this be done. Note that MPI_Comm_create_errhandler() (which creates an error handler, given a function pointer) does not attach the error handler to an MPI_Comm, so the lifetime of the error handler is not tied to the MPI_Comm to which it is assigned. An error handler can be assigned to more than one MPI_Comm, in fact. You just need to guarantee that if you create a custom error handler, then that handler gets freed at some point. "The call to <tt>MPI_FINALIZE</tt> does not free
objects created by MPI calls; these objects are freed using
<tt>MPI_xxx_FREE</tt> calls" (Section 8.7, MPI 3.0 Standard). Note that it is legitimate to call MPI_Errhandler_free() right after setting the MPI_Comm's error handler; see Section 8.3.4 of the MPI 3.0 Standard ("The error handler [given to
MPI_Errhandler_free] will be deallocated after all the objects
associated with it (communicator, window, or file) have been
deallocated"). You might instead attach your error handler as a attribute to MPI_COMM_SELF, in such a way that MPI_Errhandler_free() will be called when MPI_COMM_SELF is freed (which MPI_Finalize() does automatically). We do not take responsibility for doing any of these things; you are responsible for freeing the error handler.
Here is an example showing how to change an MpiComm's error handler. The default error handler for any MPI_Comm is MPI_ERRORS_ARE_FATAL. This handler immediately aborts if MPI encounters an error, without returning an error code from the MPI function. Suppose that instead you would like MPI functions to return an error code if MPI should encounter an error. (In that case, Teuchos' MPI wrappers will detect the returned error code and throw an exception with an appropriate error message. If MPI aborts immediately on error, Teuchos won't have the chance to detect and report the error.) If so, you may set the error handler to MPI_ERRORS_RETURN, one of MPI's built-in error handlers. Here is how you may do this for an MpiComm:
|
virtual |
The calling process' rank.
Implements Teuchos::Comm< Ordinal >.
|
virtual |
The number of processes in the communicator.
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Execute a barrier; must be called collectively.
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Gather values from all processes to the root process.
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Variant of ssend() that takes a message tag.
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Variant of readySend() that accepts a message tag.
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Variant of isend() that takes a tag.
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Implements Teuchos::Comm< Ordinal >.
|
virtual |
Reimplemented from Teuchos::Describable.
|
inlinevirtual |
The current tag.
Implements Teuchos::Comm< Ordinal >.
Definition at line 472 of file Teuchos_DefaultMpiComm_decl.hpp.
|
inlinevirtual |
Increments the tag and then returns it.
Implements Teuchos::Comm< Ordinal >.
Definition at line 474 of file Teuchos_DefaultMpiComm_decl.hpp.
|
related |
Helper function that creates a dynamically allocated MpiComm object or returns Teuchos::null to correctly represent a null communicator.
Postconditions: [rawMpiComm.get()!=NULL && *rawMpiComm!=MPI_COMM_NULL] return.get()!=NULL [rawMpiComm.get()==NULL || *rawMpiComm==MPI_COMM_NULL] return.get()==NULL
|
related |
Helper function that creates a dynamically allocated MpiComm object or returns Teuchos::null to correctly represent a null communicator.
Postconditions: [rawMpiComm.get()!=NULL && *rawMpiComm!=MPI_COMM_NULL] return.get()!=NULL [rawMpiComm.get()==NULL || *rawMpiComm==MPI_COMM_NULL] return.get()==NULL
Helper function that extracts a raw MPI_Comm object out of a Teuchos::MpiComm wrapper object.
Preconditions: dynamic_cast<const MpiComm<Ordinal>*>(&comm) != 0
If the underlying type is not an MpiComm<Ordinal> object, then the function with throw an exception which contains the type information as for why it failed.
WARNING: The lifetime of the returned MPI_Comm object is controlled by the owning RCP<OpaqueWrapper<MPI_Comm> > object and is not guaranteed to live the entire life of the program. Therefore, only use this function to grab and use the underlying MPI_Comm object in a vary narrow scope and then forget it. If you need it again, get it off of the comm object each time.
If you want a memory safe RCP<OpaqueWrapper<MPI_Comm> > to the raw MPI_Comm object, then call:
* dyn_cast<const MpiComm<Ordinal> >(comm).getRawMpiComm() *