Skip to content

Add instructions for profiling using TAU - #5580

Closed
dacreman wants to merge 29 commits into
releasefrom
dacreman/tau_install_docs
Closed

dacreman wants to merge 29 commits into
releasefrom
dacreman/tau_install_docs

Conversation

@dacreman

Copy link
Copy Markdown
Contributor

Description

This PR adds instructions for how to set up the TAU profiling tools on Ubuntu.

connorjward and others added 29 commits October 2, 2026 13:40
Co-authored-by: Connor Ward <c.ward20@imperial.ac.uk>
* add Action,Adjoint split tests

* add FormSum handler

* Remove redundant Tensor block cache

* Reuse Slate subblock extractor

* Move Slate circular imports to call sites

---------

Co-authored-by: Pablo Brubeck <brubeck@protonmail.com>
* form splitter: use `DAGTraverser` and handle `Action`, `Adjoint` (#5484)

* add Action,Adjoint split tests

* add FormSum handler

* Remove redundant Tensor block cache

* Reuse Slate subblock extractor

* Move Slate circular imports to call sites

---------

Co-authored-by: Pablo Brubeck <brubeck@protonmail.com>

* Use ExtractSubBlock for Slate block propagation

* Use unified Slate block traversal

* fixes

- fix block indexing
- use `reuse_if_untouched` for ScalarMul, Add, DiagonalTensor,
Reciprocal
- in `Mul`, pass blocks to the DAGTraverser instead of constructing
intermidiate Block nodes

---------

Co-authored-by: Leo Collins <leocollins511@gmail.com>
Co-authored-by: Leo Collins <collins.leo@proton.me>
* Store SNES weak reference on context construction

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Introduces the partition_rtree, which is an R-tree of bounding boxes covering each partition of the mesh. This R-tree is replicated on all ranks by allgathering the bounding boxes. The bounding boxes on each rank are obtained by traversing down the local R-tree. Each rank may submit more than one bounding box to the partition_rtree (we want to do this to minimise candidates since the mesh partitions may not be nice and convex, and might even be disconnected).

We choose the number of bounding boxes via a simple heuristic (we stop traversing when the total volume of the bounding boxes stop decreasing - see _box_ratio_heuristic). This should probably be improved to take into account the total number of bounding boxes so we don't blow up the allgather.

Constructing the partition rtree uses an allgather, although each rank normally submits O(10s) of boxes so I have not seen this kill us yet when testing up to 2048 ranks. If doing huge number of cores we might want to consider implementing some sort of hierarchy of partition rtrees.

Each rank queries this R-tree with the points local to that rank to determine the 'candidate' ranks for each point. We then build the candidate_sf which is a PETScSF mapping input points to their candidates. The local cell location process proceeds as usual, solving parallel conflicts with broadcasts/reductions via the candidate_sf.

This current implementation gets around 80% weak scaling efficiency in my tests up to 2048 cores.

---------

Co-authored-by: Connor Ward <c.ward20@imperial.ac.uk>
* Replace PyOP2 with pyop3
* Make extruded meshes just be a DMPlex
* Many other things
* Undo pin on islpy

* Use loopy main
)

* pyop3: Support MIN/MAX reductions in kernel calls and assignments

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Installing netgen-occt with --no-deps pulled netgen-occt 8.0.1, but
netgen-mesher 6.2.2608 requires netgen-occt==7.8.1, so `import netgen`
failed with "libTKBRep.so.8.0.1: cannot open shared object file".

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@dacreman dacreman closed this Oct 29, 2026
@dacreman

Copy link
Copy Markdown
Contributor Author

Should be branched from release not main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants