Skip to content

pin: make WalkDir harder to misuse and add Windows support #1736

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

lmb
Copy link
Collaborator

@lmb lmb commented Mar 31, 2025

The current WalkDir API requires the user to close each object that is iterated over. This isn't very intuitive, the two uses of the function in the tetragon code base end up leaking objects. It's also not straight forward to implement WalkDir on Windows, since BPFFS isn't really a thing on that platform.

Instead, turn WalkDir into an iterator which returns a new Pin object. By default, objects are closed once they have been iterated over. The user may call Pin.Take if they wish to retain the object. This is similar to the existing link.Iterator.Take.

One downside is that directories are not returned by the iteration anymore. The existing code in tetragon doesn't use them and is therefore not affected. There is also no more fine grained control over whether to descent into a directory. Aborting an iteration is still supported and as efficient as before.

Fixes: #1652

@lmb
Copy link
Collaborator Author

lmb commented Mar 31, 2025

@mtardy could you take a look whether this still fits your usecase in tetragon?

pin/pin.go Outdated
// Pin represents an object and its pinned path.
type Pin struct {
Path string
Object Pinner
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could also be io.Closer instead.

@mtardy
Copy link
Member

mtardy commented Apr 7, 2025

@mtardy could you take a look whether this still fits your usecase in tetragon?

will take a look shortly 👀 , was busy conferencing...

@mtardy
Copy link
Member

mtardy commented Apr 14, 2025

@mtardy could you take a look whether this still fits your usecase in tetragon?

will take a look shortly 👀 , was busy conferencing...

Looks good, am I using it correctly cilium/tetragon@38c1109 ?

@lmb
Copy link
Collaborator Author

lmb commented Apr 14, 2025

Yeah those changes look good! You can pass nil options to WalkDir I think. I wonder whether I should drop the return value from Take() for now, not sure it's particularly useful.

@mtardy
Copy link
Member

mtardy commented Apr 14, 2025

Yeah those changes look good! You can pass nil options to WalkDir I think.

ah yeah it seems ReadOnly: true doesn't work well with links

I wonder whether I should drop the return value from Take() for now, not sure it's particularly useful.

ah yeah maybe indeed

@lmb lmb marked this pull request as ready for review April 14, 2025 12:00
@lmb lmb requested a review from a team as a code owner April 14, 2025 12:00
The current WalkDir API requires the user to close each object that
is iterated over. This isn't very intuitive, the two uses of the
function in the tetragon code base end up leaking objects.
It's also not straight forward to implement WalkDir on Windows, since
BPFFS isn't really a thing on that platform.

Instead, turn WalkDir into an iterator which returns a new Pin object.
By default, objects are closed once they have been iterated over. The
user may call Pin.Take if they wish to retain the object. This is
similar to the existing link.Iterator.Take.

One downside is that directories are not returned by the iteration
anymore. The existing code in tetragon doesn't use them and is
therefore not affected. There is also no more fine grained control
over whether to descent into a directory. Aborting an iteration is
still supported and as efficient as before.

Fixes: cilium#1652
Signed-off-by: Lorenz Bauer <[email protected]>
Copy link
Member

@mtardy mtardy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this new version when using the iter with existing for := range.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pin: WalkDir makes it too easy to leak objects
2 participants