Comments (2)
Given the current kernel interface, I'm not sure that there's any particularly tidy API we can build for arbitrary use-cases :/ If the BPF side constantly deletes elements, then dump of such maps from the userland side will behave very poorly indeed.. but that's just a limitation of that particular use case.
In Cilium, we just use MaxElems
then have the ability to pass out some dump statistics at the end which includes various things, but in particular whether the dump ended up "incomplete", as measured by "we dumped MaxElems
but didn't reach the end of the map". So, as a user, you have to know that the dump may not be complete, and handle this.
I think in the end, we will have to define the map iteration interface in such a way that we communicate to library users the kinds of behaviour you can expect, depending on what kind of other interactions are occurring concurrently on the map [either from userspace via this library, or from BPF progs running in the kernel]. These include:
- Is it possible to dump the "same" element twice for a given iteration (for some definition of "same")
- How strong of a guarantee do I get about whether I have a complete view of the map once the dump is done?
- What kind of ordering can you expect from the map?
- Likely some other considerations I didn't think of in the moment..
With regards to your example in the summary, this forces users to consider the size of the map. One could imagine instead an API where you create a dump object first, which has optional "max iterations" (default: MaxElems
), then if someone really cares, they can limit the number of dumps. Then the actual iteration doesn't need to have this detail. This could be made optional via builder pattern, something like entries := NewDumper(...).WithMaxIterations(42)
. (Focus on the WithMaxIterations()
bit; I didn't think closely about what the dump object creation looks like).
from ebpf.
So, as a user, you have to know that the dump may not be complete, and handle this.
There seem to be two possible actions: punt to the user by logging / bumping some metric or retry. If the solution is retry, the max iteration limit can always be MaxElems.
What am I missing? What does Cilium do? Do you ever iterate more often than MaxElems?
I think in the end, we will have to define the map iteration interface in such a way that we communicate to library users the kinds of behaviour you can expect
This boils down to what the kernel implements for each given map type, no? As far as I understand, the iteration count is only a problem for hash maps, array-like maps are fine.
For example, your list of questions for Array:
- No element dumped twice
- Always get a complete view
- Keys are ordered
And for Hash:
- No guarantee
- No guarantee
- Keys are not ordered
And who knows what LRU maps guarantee.
With regards to your example in the summary, this forces users to consider the size of the map.
Yes and you're right that it's not very nice. Reading your comment makes me think that (a) limiting each iterator to MaxElems and (b) documenting that hash maps can't be reliably iterated is the best we can do.
from ebpf.
Related Issues (20)
- CI: TestMapBatch/Hash is flaky (arm64?) HOT 2
- dae can not recognize pppoe dial-up interface and route out correctly. HOT 1
- Kernel version detection does not work with vDSO disabled HOT 6
- Allow changing line info data in btf.Line HOT 11
- load program: invalid argument: unknown func bpf_redirect_peer#155 (51 line(s) omitted)
- With the program type raw_tracepoint, no data is generated.error: loading objects: field TraceSchedWakeup: program trace_sched_wakeup: load program: permission denied: 5: (61) r1 = *(u32 *)(r7 +2784): R7 invalid mem access 'inv' (5 line(s) omitted HOT 1
- Test TestPerfReaderWakeupEvents gets stuck on some runs HOT 9
- With Linux 4.9, loadBpfObjects() failed, error=argument list too long HOT 1
- program: relocation of program targeting a module fails if CONFIG_DEBUG_INFO_BTF_MODULES is disabled HOT 3
- Unusual `go` directive in `go.mod`
- flake: TestMapIteratorAllocations HOT 1
- TestHaveProgramType/Extension fails on kernels >6.7 HOT 2
- ci: arm64 tests fail HOT 4
- deprecation of directive `//go:linkname` HOT 3
- AttachXDP failed on Ubuntu 20.04, kernel version is 5.4 HOT 3
- program: make it harder to attach Kretprobe via link.Kprobe and vice versa
- btf: CO-RE: types from (other) kmod BTF are not available HOT 3
- map Lookup allocates when key does not exist HOT 3
- DATA RACE: github.com/cilium/ebpf.(*MapSpec).createMap HOT 1
- has to have BTF in order to use bpf_spin_lock HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ebpf.