LifeV
MeshPartitioner< MeshType > Class Template Reference

Class that handles mesh partitioning. More...

#include <MeshPartitioner.hpp>

+ Collaboration diagram for MeshPartitioner< MeshType >:

Data Structures

struct  EntityPIDList
 Store ownership for each entity, subdivided by entity type. More...
 

Private Member Functions

 MeshPartitioner (const MeshPartitioner &)
 
MeshPartitioneroperator= (const MeshPartitioner &)
 
typedef MeshType mesh_Type
 
typedef std::shared_ptr< MeshTypemeshPtr_Type
 
typedef std::vector< IntidList_Type
 
typedef std::vector< idList_Typegraph_Type
 
typedef std::shared_ptr< graph_TypegraphPtr_Type
 
typedef std::vector< meshPtr_TypepartMesh_Type
 
typedef std::shared_ptr< partMesh_TypepartMeshPtr_Type
 

Constructors & Destructors

 MeshPartitioner ()
 Default empty constructor. More...
 
 MeshPartitioner (meshPtr_Type &mesh, std::shared_ptr< Epetra_Comm > &comm, Epetra_Map *interfaceMap=0, Epetra_Map *interfaceMapRep=0)
 Constructor. More...
 
virtual ~MeshPartitioner ()
 Empty destructor. More...
 

Public Methods

void doPartition (meshPtr_Type &mesh, std::shared_ptr< Epetra_Comm > &comm, Epetra_Map *interfaceMap=0, Epetra_Map *interfaceMapRep=0)
 Partition the mesh. More...
 
void setup (UInt numPartitions, std::shared_ptr< Epetra_Comm > comm)
 To be used with the new constructor. More...
 
void update ()
 Call update() method after loading the graph, to rebuild all data structures. More...
 
void attachUnpartitionedMesh (meshPtr_Type &mesh, Epetra_Map *interfaceMap=0, Epetra_Map *interfaceMapRep=0)
 Stores a pointer to the unpartitioned mesh in the M_originalMesh data member. More...
 
void releaseUnpartitionedMesh ()
 Releases the original unpartitioned mesh. More...
 
void doPartitionGraph ()
 Executes the ParMETIS graph partitioning. More...
 
void doPartitionMesh ()
 Builds the partitioned mesh using the partitioned graph. More...
 
void fillEntityPID ()
 Initialize M_entityPID. More...
 
const meshPtr_TypeLIFEV_DEPRECATED (getPartition(Int k) const)
 Return a pointer to the mesh partition with rank k. More...
 
void cleanUp ()
 Clean structures that are not needed after partitioning. More...
 
void showMe (std::ostream &output=std::cout) const
 Prints information about the state (data) of the object. More...
 

Get Methods

const std::vector< Int > & vertexDistribution () const
 Return a reference to M_vertexDistribution. More...
 
const meshPtr_TypemeshPartition () const
 Return a const pointer to M_meshPartitions[0] - for parallel. More...
 
meshPtr_TypemeshPartition ()
 
const partMeshPtr_TypemeshPartitions () const
 Return a pointer to M_meshPartitions. More...
 
const std::vector< Int > & graphVertexLocations () const
 Return a pointer to M_graphVertexLocations. More...
 
const graphPtr_TypeelementDomains () const
 Return a pointer to M_elementDomains. More...
 
graphPtr_TypeelementDomains ()
 
const std::shared_ptr< Epetra_Comm > & comm () const
 Return a pointer to the communicator M_comm. More...
 

Set methos

void setPartitionOverlap (UInt const overlap)
 Set M_buildOverlappingPartitions. More...
 
void init ()
 Private Methods. More...
 
void execute ()
 Execute mesh partitioning using the configured MPI processes (online partitioning) More...
 
void distributeElements (UInt numElements)
 Build the graph vertex distribution vector. More...
 
void findRepeatedFacesFSI ()
 Find faces on the boundaries between domains (FSI) More...
 
void partitionConnectivityGraph (UInt numParts)
 Partition the connectivity graph using ParMETIS. More...
 
void matchFluidPartitionsFSI ()
 Updates the map between elements and processors in FSI. More...
 
void redistributeElements ()
 Redistribute elements among processes. More...
 
void constructLocalMesh ()
 Construct local mesh. More...
 
void constructNodes ()
 Construct vertices. More...
 
void constructElements ()
 Construct volumes. More...
 
void constructRidges ()
 Construct ridges. More...
 
void constructFacets ()
 Construct facets. More...
 
void finalSetup ()
 Final setup of local mesh. More...
 
void markGhostEntities ()
 Mark ghost entities. More...
 
UInt M_numPartitions
 Private Data Members. More...
 
partMeshPtr_Type M_meshPartitions
 
std::vector< IntM_vertexDistribution
 
std::vector< IntM_adjacencyGraphKeys
 
std::vector< IntM_adjacencyGraphValues
 
std::shared_ptr< Epetra_Comm > M_comm
 
Int M_me
 
std::vector< std::vector< Int > > M_localNodes
 
std::vector< std::set< Int > > M_localRidges
 
std::vector< std::set< Int > > M_localFacets
 
std::vector< std::vector< Int > > M_localElements
 
std::vector< std::map< Int, Int > > M_globalToLocalNode
 
std::vector< std::map< Int, Int > > M_globalToLocalElement
 
std::vector< UIntM_nBoundaryPoints
 
std::vector< UIntM_nBoundaryRidges
 
std::vector< UIntM_nBoundaryFacets
 
meshPtr_Type M_originalMesh
 
Epetra_Map * M_interfaceMap
 
Epetra_Map * M_interfaceMapRep
 
UInt M_elementVertices
 Number of partitions handled. 1 for parallel (old way), != 1 for serial. More...
 
UInt M_elementFacets
 
UInt M_elementRidges
 
UInt M_facetVertices
 
std::shared_ptr< std::vector< Int > > M_repeatedFacet
 
std::shared_ptr< std::vector< Int > > M_isOnProc
 
std::vector< IntM_graphVertexLocations
 
graphPtr_Type M_elementDomains
 
bool M_serialMode
 
UInt M_partitionOverlap
 
struct LifeV::MeshPartitioner::EntityPIDList M_entityPID
 

Detailed Description

template<typename MeshType>
class LifeV::MeshPartitioner< MeshType >

Class that handles mesh partitioning.

Author
Gilles Fourestey gille.nosp@m.s.fo.nosp@m.urest.nosp@m.ey@e.nosp@m.pfl.c.nosp@m.h
Contributor:
Radu Popescu radu..nosp@m.pope.nosp@m.scu@e.nosp@m.pfl..nosp@m.ch

This class implements the partitioning of a mesh using (par)Metis. Currently it is possible to do both online and offline partitioning. In online partitioning, each MPI rank stores only its own mesh partition. In offline partitioning (run as a single process, on a workstation), all the mesh partitions are stored and can be saved to disk, using the HDF5 filter, for later use during a parallel run.

Definition at line 75 of file MeshPartitioner.hpp.

Member Typedef Documentation

◆ mesh_Type

Definition at line 80 of file MeshPartitioner.hpp.

◆ meshPtr_Type

typedef std::shared_ptr<MeshType> meshPtr_Type

Definition at line 81 of file MeshPartitioner.hpp.

◆ idList_Type

typedef std::vector<Int> idList_Type

Definition at line 82 of file MeshPartitioner.hpp.

◆ graph_Type

typedef std::vector<idList_Type> graph_Type

Definition at line 83 of file MeshPartitioner.hpp.

◆ graphPtr_Type

typedef std::shared_ptr<graph_Type> graphPtr_Type

Definition at line 84 of file MeshPartitioner.hpp.

◆ partMesh_Type

typedef std::vector<meshPtr_Type> partMesh_Type

Definition at line 85 of file MeshPartitioner.hpp.

◆ partMeshPtr_Type

typedef std::shared_ptr<partMesh_Type> partMeshPtr_Type

Definition at line 86 of file MeshPartitioner.hpp.

Constructor & Destructor Documentation

◆ MeshPartitioner() [1/3]

Default empty constructor.

Definition at line 417 of file MeshPartitioner.hpp.

◆ MeshPartitioner() [2/3]

MeshPartitioner ( meshPtr_Type mesh,
std::shared_ptr< Epetra_Comm > &  comm,
Epetra_Map *  interfaceMap = 0,
Epetra_Map *  interfaceMapRep = 0 
)

Constructor.

This is a non-empty constructor. It takes as parameters the unpartitioned mesh (reference), the Epetra_Comm object in use (reference) and pointers to the Epetra interface and repeated interface maps. The constructor initializes the data members and calls a private method MeshPartitioner::execute which handles the mesh partitioning.

Parameters
mesh- Mesh& - the unpartitioned mesh
_comm- Epetra_Comm& - Epetra communicator object
interfaceMap- Epetra_Map*
interfaceMapRep- Epetra_Map*

Definition at line 424 of file MeshPartitioner.hpp.

◆ ~MeshPartitioner()

virtual ~MeshPartitioner ( )
inlinevirtual

Empty destructor.

Definition at line 112 of file MeshPartitioner.hpp.

◆ MeshPartitioner() [3/3]

MeshPartitioner ( const MeshPartitioner< MeshType > &  )
private

Member Function Documentation

◆ doPartition()

void doPartition ( meshPtr_Type mesh,
std::shared_ptr< Epetra_Comm > &  comm,
Epetra_Map *  interfaceMap = 0,
Epetra_Map *  interfaceMapRep = 0 
)

Partition the mesh.

It takes as parameters the unpartitioned mesh (reference), the Epetra_Comm object in use (reference) and pointers to the Epetra interface and repeated interface maps. The method initializes the data members and calls a private method MeshPartitioner::execute which handles the mesh partitioning.

Parameters
mesh- Mesh& - the unpartitioned mesh
_comm- Epetra_Comm& - Epetra communicator object
interfaceMap- Epetra_Map*
interfaceMapRep- Epetra_Map*
Note
This method is meant to be used with the empty constructor.

Definition at line 465 of file MeshPartitioner.hpp.

◆ setup()

void setup ( UInt  numPartitions,
std::shared_ptr< Epetra_Comm >  comm 
)

To be used with the new constructor.

Loads the parameters of the partitioning process from the simulation data file. Allocates and initializes data members according to the number of partitions specified in the data file.

Parameters
numPartitions- UInt - the number of partitions desired, in the offline case
_comm- Epetra_Comm& - reference of the Epetra communicator used

Definition at line 489 of file MeshPartitioner.hpp.

◆ update()

void update ( )

Call update() method after loading the graph, to rebuild all data structures.

This method is to be called after the partitioned graph is LOADED from a HDF5 file. Set M_nPartitions and rebuilds the M_graphVertexLocations data member, required to do mesh partitioning. !!! This method should be the first one to be called after loading the graph. !!!

Definition at line 521 of file MeshPartitioner.hpp.

◆ attachUnpartitionedMesh()

void attachUnpartitionedMesh ( meshPtr_Type mesh,
Epetra_Map *  interfaceMap = 0,
Epetra_Map *  interfaceMapRep = 0 
)

Stores a pointer to the unpartitioned mesh in the M_originalMesh data member.

Stores a pointer to the unpartitioned mesh in the M_originalMesh data member.

Parameters
mesh- Mesh& - the unpartitioned mesh (passed by reference)
interfaceMap- Epetra_Map* - pointer to the interface map (default value 0)
interfaceMapRep- Epetra_Map* - pointer to the repeated interface map (default value 0)

Definition at line 546 of file MeshPartitioner.hpp.

◆ releaseUnpartitionedMesh()

void releaseUnpartitionedMesh ( )

Releases the original unpartitioned mesh.

Releases the unpartitioned mesh so that it can be deleted, freeing A LOT of memory in some cases.

Definition at line 556 of file MeshPartitioner.hpp.

◆ doPartitionGraph()

void doPartitionGraph ( )

Executes the ParMETIS graph partitioning.

Executes the ParMETIS graph partitioning

Definition at line 564 of file MeshPartitioner.hpp.

◆ doPartitionMesh()

void doPartitionMesh ( )

Builds the partitioned mesh using the partitioned graph.

Builds the partitioned mesh using the partitioned graph

Definition at line 579 of file MeshPartitioner.hpp.

◆ fillEntityPID()

void fillEntityPID ( )

Initialize M_entityPID.

Definition at line 1529 of file MeshPartitioner.hpp.

◆ LIFEV_DEPRECATED()

const meshPtr_Type& LIFEV_DEPRECATED ( getPartition(Int k)  const)
inline

Return a pointer to the mesh partition with rank k.

Definition at line 189 of file MeshPartitioner.hpp.

◆ cleanUp()

void cleanUp ( )

Clean structures that are not needed after partitioning.

Definition at line 1629 of file MeshPartitioner.hpp.

◆ showMe()

void showMe ( std::ostream &  output = std::cout) const

Prints information about the state (data) of the object.

Definition at line 619 of file MeshPartitioner.hpp.

◆ vertexDistribution()

const std::vector<Int>& vertexDistribution ( ) const
inline

Return a reference to M_vertexDistribution.

Definition at line 204 of file MeshPartitioner.hpp.

◆ meshPartition() [1/2]

const meshPtr_Type& meshPartition ( ) const
inline

Return a const pointer to M_meshPartitions[0] - for parallel.

Definition at line 209 of file MeshPartitioner.hpp.

◆ meshPartition() [2/2]

meshPtr_Type& meshPartition ( )
inline

Definition at line 213 of file MeshPartitioner.hpp.

◆ meshPartitions()

const partMeshPtr_Type& meshPartitions ( ) const
inline

Return a pointer to M_meshPartitions.

Definition at line 218 of file MeshPartitioner.hpp.

◆ graphVertexLocations()

const std::vector<Int>& graphVertexLocations ( ) const
inline

Return a pointer to M_graphVertexLocations.

Definition at line 223 of file MeshPartitioner.hpp.

◆ elementDomains() [1/2]

const graphPtr_Type& elementDomains ( ) const
inline

Return a pointer to M_elementDomains.

Definition at line 228 of file MeshPartitioner.hpp.

◆ elementDomains() [2/2]

graphPtr_Type& elementDomains ( )
inline

Definition at line 232 of file MeshPartitioner.hpp.

◆ comm()

const std::shared_ptr<Epetra_Comm>& comm ( ) const
inline

Return a pointer to the communicator M_comm.

Definition at line 237 of file MeshPartitioner.hpp.

◆ setPartitionOverlap()

void setPartitionOverlap ( UInt const  overlap)
inline

Set M_buildOverlappingPartitions.

Definition at line 247 of file MeshPartitioner.hpp.

◆ operator=()

MeshPartitioner& operator= ( const MeshPartitioner< MeshType > &  )
private

◆ init()

void init ( )
private

Private Methods.

Initialize the parameters with default value.

Definition at line 434 of file MeshPartitioner.hpp.

◆ execute()

void execute ( )
private

Execute mesh partitioning using the configured MPI processes (online partitioning)

Executed the mesh partitioning using the number of MPI processes as the number of partitions. Sets current mesh element parameters: M_elementVertices, M_elementRidges, M_elementFacets, M_facetVertices Updates: M_elementDomains (indirectly) Other data members are changed indirectly by calling other private methods.

Definition at line 1464 of file MeshPartitioner.hpp.

◆ distributeElements()

void distributeElements ( UInt  numElements)
private

Build the graph vertex distribution vector.

Updates the member M_vertexDistribution according to the number of processors to be used by ParMETIS (the number of processes started for MPI

Parameters
numElements- UInt - number of elements in the mesh

Definition at line 630 of file MeshPartitioner.hpp.

◆ findRepeatedFacesFSI()

void findRepeatedFacesFSI ( )
private

Find faces on the boundaries between domains (FSI)

Identifies the element faces that are common to both the fluid and the solid meshes and creates a map between the faces that reside on the boundary between the two meshes and the processors used. Updates the members M_repeatedFacet and M_isOnProc.

Definition at line 655 of file MeshPartitioner.hpp.

◆ partitionConnectivityGraph()

void partitionConnectivityGraph ( UInt  numParts)
private

Partition the connectivity graph using ParMETIS.

Partitions the connectivity graph using ParMETIS. The result is stored in the member M_graphVertexLocations: this is a vector of integer values; its size is the number of elements in the unpartitioned mesh. Each value represents the number of the partition to which the element was assigned. Also creates M_elementDomains, the vector of elements in each subdomain. Updates: M_graphVertexLocations, M_elementDomains

Parameters
numParts- unsigned int - number of partitions for the graph cutting process

Definition at line 730 of file MeshPartitioner.hpp.

◆ matchFluidPartitionsFSI()

void matchFluidPartitionsFSI ( )
private

Updates the map between elements and processors in FSI.

Updates M_elementDomains during FSI modeling.

Definition at line 879 of file MeshPartitioner.hpp.

◆ redistributeElements()

void redistributeElements ( )
private

Redistribute elements among processes.

Redistributes elements among processes, when needed, after the connectivity graph partitioning phase. Updates M_elementDomains

Definition at line 988 of file MeshPartitioner.hpp.

◆ constructLocalMesh()

void constructLocalMesh ( )
private

Construct local mesh.

Constructs the data structures for the local mesh partition. Updates M_localNodes, M_localRidges, M_localFacets, M_localElements, M_globalToLocalNode.

Definition at line 1109 of file MeshPartitioner.hpp.

◆ constructNodes()

void constructNodes ( )
private

Construct vertices.

Adds vertices to the partitioned mesh object. Updates M_nBoundaryPoints, M_meshPartitions.

Definition at line 1168 of file MeshPartitioner.hpp.

◆ constructElements()

void constructElements ( )
private

Construct volumes.

Adds volumes to the partitioned mesh object. Updates M_globalToLocalElement, M_meshPartitions.

Definition at line 1203 of file MeshPartitioner.hpp.

◆ constructRidges()

void constructRidges ( )
private

Construct ridges.

Adds ridges to the partitioned mesh object. Updates M_nBoundaryRidges, M_meshPartitions.

Definition at line 1242 of file MeshPartitioner.hpp.

◆ constructFacets()

void constructFacets ( )
private

Construct facets.

Adds facets to the partitioned mesh object. Updates M_nBoundaryFacets, M_meshPartitions.

Definition at line 1293 of file MeshPartitioner.hpp.

◆ finalSetup()

void finalSetup ( )
private

Final setup of local mesh.

Updates the partitioned mesh object data members after adding the mesh elements (vertices, ridges, facets, volumes). Updates M_meshPartitions.

Definition at line 1415 of file MeshPartitioner.hpp.

◆ markGhostEntities()

void markGhostEntities ( )
private

Mark ghost entities.

Mark all ghost entities that have been added for overlapping partitions with the EntityFlags::GHOST flag in order to properly build the dof map in DOF::GlobalElements().

Definition at line 1578 of file MeshPartitioner.hpp.

Field Documentation

◆ M_numPartitions

UInt M_numPartitions
private

Private Data Members.

Definition at line 360 of file MeshPartitioner.hpp.

◆ M_meshPartitions

partMeshPtr_Type M_meshPartitions
private

Definition at line 361 of file MeshPartitioner.hpp.

◆ M_vertexDistribution

std::vector<Int> M_vertexDistribution
private

Definition at line 362 of file MeshPartitioner.hpp.

◆ M_adjacencyGraphKeys

std::vector<Int> M_adjacencyGraphKeys
private

Definition at line 363 of file MeshPartitioner.hpp.

◆ M_adjacencyGraphValues

std::vector<Int> M_adjacencyGraphValues
private

Definition at line 364 of file MeshPartitioner.hpp.

◆ M_comm

std::shared_ptr<Epetra_Comm> M_comm
private

Definition at line 365 of file MeshPartitioner.hpp.

◆ M_me

Int M_me
private

Definition at line 366 of file MeshPartitioner.hpp.

◆ M_localNodes

std::vector<std::vector<Int> > M_localNodes
private

Definition at line 368 of file MeshPartitioner.hpp.

◆ M_localRidges

std::vector<std::set<Int> > M_localRidges
private

Definition at line 369 of file MeshPartitioner.hpp.

◆ M_localFacets

std::vector<std::set<Int> > M_localFacets
private

Definition at line 370 of file MeshPartitioner.hpp.

◆ M_localElements

std::vector<std::vector<Int> > M_localElements
private

Definition at line 371 of file MeshPartitioner.hpp.

◆ M_globalToLocalNode

std::vector<std::map<Int, Int> > M_globalToLocalNode
private

Definition at line 372 of file MeshPartitioner.hpp.

◆ M_globalToLocalElement

std::vector<std::map<Int, Int> > M_globalToLocalElement
private

Definition at line 373 of file MeshPartitioner.hpp.

◆ M_nBoundaryPoints

std::vector<UInt> M_nBoundaryPoints
private

Definition at line 374 of file MeshPartitioner.hpp.

◆ M_nBoundaryRidges

std::vector<UInt> M_nBoundaryRidges
private

Definition at line 375 of file MeshPartitioner.hpp.

◆ M_nBoundaryFacets

std::vector<UInt> M_nBoundaryFacets
private

Definition at line 376 of file MeshPartitioner.hpp.

◆ M_originalMesh

meshPtr_Type M_originalMesh
private

Definition at line 379 of file MeshPartitioner.hpp.

◆ M_interfaceMap

Epetra_Map* M_interfaceMap
private

Definition at line 380 of file MeshPartitioner.hpp.

◆ M_interfaceMapRep

Epetra_Map* M_interfaceMapRep
private

Definition at line 381 of file MeshPartitioner.hpp.

◆ M_elementVertices

UInt M_elementVertices
private

Number of partitions handled. 1 for parallel (old way), != 1 for serial.

Definition at line 383 of file MeshPartitioner.hpp.

◆ M_elementFacets

UInt M_elementFacets
private

Definition at line 384 of file MeshPartitioner.hpp.

◆ M_elementRidges

UInt M_elementRidges
private

Definition at line 385 of file MeshPartitioner.hpp.

◆ M_facetVertices

UInt M_facetVertices
private

Definition at line 386 of file MeshPartitioner.hpp.

◆ M_repeatedFacet

std::shared_ptr<std::vector<Int> > M_repeatedFacet
private

Definition at line 387 of file MeshPartitioner.hpp.

◆ M_isOnProc

std::shared_ptr<std::vector<Int> > M_isOnProc
private

Definition at line 388 of file MeshPartitioner.hpp.

◆ M_graphVertexLocations

std::vector<Int> M_graphVertexLocations
private

Definition at line 389 of file MeshPartitioner.hpp.

◆ M_elementDomains

graphPtr_Type M_elementDomains
private

Definition at line 390 of file MeshPartitioner.hpp.

◆ M_serialMode

bool M_serialMode
private

Definition at line 391 of file MeshPartitioner.hpp.

◆ M_partitionOverlap

UInt M_partitionOverlap
private

Definition at line 392 of file MeshPartitioner.hpp.

◆ M_entityPID

struct LifeV::MeshPartitioner::EntityPIDList M_entityPID
private

The documentation for this class was generated from the following file: