workaround for hitting BPF max stack size about p4c-xdp

[mihaibudiu]

How about declaring hd as volatile?

[williamtu]

then it uses only 136 byte
But volatile generates too large program, so verifier says:
"BPF program is too large. Proccessed 65537 insn"

[mihaibudiu]

That is weird, you would not expect this program to be so large.

Maybe you can play more with volatile; perhaps instead of a second structure hd__ you can just use a volatile pointer to hd and use to make a read between the parser and the deparser.

[williamtu]

OK.
Actually the program size (number of instructions in the program) with and without volatile is roughly the same. Then it's weird that BPF verifier fails.

[williamtu]

I actually like the generated with volatile. With volatile, the compile spills the ethernet src/dst to stack using 1 byte instead of 8 byte, so it uses much less stack. However, there is no reordering at all, it's easier to read, but maybe have some performance overhead?

At lease when reading out the hd.x.y, it will read it from the stack and with the right size (no longer fixed to 8 byte).
With volatile, the size is 1232 line of instructions
Without volatile, it is 1363.

Now I suspect there might be an issue in verifier.

[kwjjyn]

Hi,
I using p4c-xdp to compile xdp7.p4 to get .c and .h files. And if I just compile the xdp7.c using clang directly，there's an error that

invalid stack off=-600 size=8

But when I add volatile before struct Headers hd in the xdp7.c , it would be no problem to load.
If there are any other ways to generate .c file with volatile from .p4 file using p4c-xdp compiler?
Otherwise I must add volatile before hd for ervery .c files which cannot be loaded into the kernel due to the hitting BPF stack.
Could you give me some advices?

[osinstom]

@williamtu did you find the suitable solution for this problem ? We're also facing it in our uBPF backend. The volatile keyword helps to overcome BPF stack size problem, but the verifier doesn't allow to inject program then.