Skip to content

Extending DistributedField and DistributedFFTBasedPoissonSolver functionality to allow for distributed FFT preconditioner for ConjugateGradientPoissonSolver #4555

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 60 commits into
base: main
Choose a base branch
from
Open

Extending DistributedField and DistributedFFTBasedPoissonSolver functionality to allow for distributed FFT preconditioner for ConjugateGradientPoissonSolver #4555

wants to merge 60 commits into from

Conversation

xkykai
Copy link
Collaborator

@xkykai xkykai commented May 27, 2025 •
edited
Loading

With @simone-silvestri , this PR extends and fixes current functionality for DistributedField and DistributedFFTBasedPoissonSolver to allow ConjugateGradientPoissonSolver to be run in distributed settings with the DistributedFFTBasedPoissonSolver as a preconditioner.

Here are the changes we want to incorporate in this PR:

  1. Distributed norm, mean, and dot
  2. precondition! for DistributedFFTBasedPoissonSolver
  3. Extend DistributedFFTBasedPoissonSolver to allow for the solving of the screened Poisson equation, consistent with FFTBasedPoissonSolver

xkykai and others added 30 commits March 25, 2025 16:56
@xkykai
Refactor pressure correction calculations to remove timestep dependen…
37d0ad4 
…cy in velocity updates
@xkykai
pressure adjustment to remove dt
7b26e96
@xkykai
dividing and remultiplying dt in pNHS
b94fa3d
@xkykai
Merge branch 'main' into xk/pdt_timestep
685fedd
@xkykai
Clarify documentation for pressure correction velocities update in no…
f49da13 
…n-hydrostatic models
@tomchor
Merge branch 'main' into xk/pdt_timestep
6ba3bdb
@xkykai
Merge branch 'main' into xk/pdt_timestep
52cc076
@simone-silvestri
test it out
0b4f5da
@simone-silvestri
add methods
2b6933a
@simone-silvestri
this should work
c4a849c
@simone-silvestri
this should work
0a6ec07
@xkykai
Merge branch 'main' into xk/pdt_timestep
696ebd2
@xkykai
fix cg stability with kahan summation, method works now except wrong …
66f63d4 
…_mean implementation
@xkykai
refactor: update mean calculations in distributed fields and conjugat…
ad38ff9 
…e gradient solver
@simone-silvestri
add a distributed FFT preconditioner
6d8884f
@xkykai
updated test script
501b264
@xkykai
Merge branch 'xk-ss/distributed-cg-solver' of https://github.com/CliM…
a0fba4c 
…A/Oceananigans.jl into xk-ss/distributed-cg-solver
@simone-silvestri
correct one thing
4a761b6
@simone-silvestri
Merge branch 'xk-ss/distributed-cg-solver' of github.com:CliMA/Oceana…
d32625e 
…nigans.jl into xk-ss/distributed-cg-solver
@simone-silvestri
add distributed stuff
7a0d69f
@simone-silvestri
add the screened equarion
65a5539
@simone-silvestri
make it work on multiple CPUs
f0a0700
@xkykai
Merge remote-tracking branch 'origin/main' into xk-ss/distributed-cg-…
e776432 
…solver
@xkykai
test distributed fftpcg on gpus
7cec9db
@xkykai
Merge branch 'main' into xk/pdt_timestep
d43b9fb
@xkykai
fix test for dt being int64
213ed32
@xkykai
Merge branch 'main' into xk/pdt_timestep
33f2685
@xkykai
add documentation clarification
47ff726
@xkykai
fix eps (int) in solve_for_pressure!
17e11d4
@simone-silvestri
Merge branch 'main' into xk/pdt_timestep
94b5b3a
simone-silvestri
simone-silvestri reviewed May 28, 2025
View reviewed changes
xkykai and others added 22 commits May 28, 2025 06:56
@xkykai @simone-silvestri
Update src/DistributedComputations/distributed_fields.jl
51401eb 
Co-authored-by: Simone Silvestri <[email protected]>
@xkykai
Merge branch 'main' into xk-ss/distributed-cg-solver
791e8d0
@xkykai
Merge branch 'main' into xk/fix-cg-residual
2c479d7
@xkykai
validation script with vortex sheet
4eb3959
@xkykai
Merge branch 'main' into xk/fix-cg-residual
ea29715
@xkykai
enforcing gauge condition at the CG level
447dd8f
@xkykai
Merge branch 'main' into xk/fix-cg-residual
d3064bc
@xkykai
change enforce_gauge_condition! to be a function
ecf3cf4
@xkykai
fix mistake, add docstring
cbeb635
@xkykai
Merge branch 'main' into xk/fix-cg-residual
31b42a2
@xkykai @tomchor
Update src/Solvers/conjugate_gradient_poisson_solver.jl
d4b61bd 
Co-authored-by: Tomás Chor <[email protected]>
@xkykai
fix naming
c2069b6
@xkykai
fix docstring
15bc147
@xkykai
Add tests for Poisson solvers with immersed boundaries
712510e
@xkykai
Fix comment for pressure solver tests
5353ccf
@xkykai
fix libraries import
ef6c178
@xkykai
another fix
fe97955
@xkykai
Merge branch 'xk/fix-cg-residual' into xk-ss/distributed-cg-solver
bb405c5
@xkykai
Merge branch 'main' into xk-ss/distributed-cg-solver
dc39373
@xkykai
fix norm and preconditioner for fftpcg to work on immersed boundaries
e58e107
@xkykai
Merge branch 'main' into xk-ss/distributed-cg-solver
3d1186f
@xkykai
validation scripts for distributed fftpcg
00ae513
@xkykai xkykai marked this pull request as ready for review July 1, 2025 18:27
@xkykai
Copy link
Collaborator Author

xkykai commented Jul 1, 2025

I've now tested the validation scripts in distributed CPU and GPU settings and they are working. Should we write tests for distributed fields for norm, dot, mean, as well as ensuring that a distributed FFT-preconditioned CG solver works in one of the tests? Happy to work on them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@glwagner glwagner Awaiting requested review from glwagner

@simone-silvestri simone-silvestri Awaiting requested review from simone-silvestri

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@xkykai @simone-silvestri @tomchor