Cap errors and faults on debug build for page_mapping benchmark

[JDuchniewicz]

When running sel4bench in non-release mode there is a deluge of logs like this:

<<seL4(CPU 0) [decodeUntypedInvocation/119 T0xffffff80bffde400 "" @400ab8]: Untyped Retype: Invalid destination address.>>
<<seL4(CPU 0) [decodeInvocation/646 T0xffffff80bffde400 "" @400af8]: Attempted to invoke a null cap #10.>>
<<seL4(CPU 0) [decodeUntypedInvocation/119 T0xffffff80bffde400 "" @400b78]: Untyped Retype: Invalid destination address.>>
<<seL4(CPU 0) [decodeInvocation/646 T0xffffff80bffde400 "" @400c08]: Attempted to invoke a null cap #11.>>
<<seL4(CPU 0) [decodeInvocation/646 T0xffffff80bffde400 "" @400c8c]: Attempted to invoke a null cap #11.>>
<<seL4(CPU 0) [decodeInvocation/646 T0xffffff80bffde400 "" @400d08]: Attempted to invoke a null cap #11.>>
Caught cap fault in send phase at address 0
while trying to handle:
vm fault on data at address 0x433060 with status 0x92000007at address 0x402928
With stack:
0x10011f30: 0x0
0x10011f38: 0x0
0x10011f40: 0x0
0x10011f48: 0x0
0x10011f50: 0x0
0x10011f58: 0x0
0x10011f60: 0x0
0x10011f68: 0x0
0x10011f70: 0x0
0x10011f78: 0x0
0x10011f80: 0x10011fa600000000
0x10011f88: 0x10011fa200000000
0x10011f90: 0x10011f9e00000000
0x10011f98: 0x31000000000000
0x10011fa0: 0x38000000390000
0x10011fa8: 0x0
<<seL4(CPU 0) [decodeUntypedInvocation/119 T0xffffff80bffde400 "" @400ab8]: Untyped Retype: Invalid destination address.>>
<<seL4(CPU 0) [decodeInvocation/646 T0xffffff80bffde400 "" @400af8]: Attempted to invoke a null cap #10.>>
<<seL4(CPU 0) [decodeUntypedInvocation/119 T0xffffff80bffde400 "" @400b78]: Untyped Retype: Invalid destination address.>>
<<seL4(CPU 0) [decodeUntypedInvocation/119 T0xffffff80bffde400 "" @400b78]: Untyped Retype: Invalid destination address.>>
<<seL4(CPU 0) [decodeInvocation/646 T0xffffff80bffde400 "" @400c08]: Attempted to invoke a null cap #11.>>
<<seL4(CPU 0) [decodeInvocation/646 T0xffffff80bffde400 "" @400c08]: Attempted to invoke a null cap #12.>>
<<seL4(CPU 0) [decodeInvocation/646 T0xffffff80bffde400 "" @400c8c]: Attempted to invoke a null cap #11.>>
<<seL4(CPU 0) [decodeInvocation/646 T0xffffff80bffde400 "" @400c8c]: Attempted to invoke a null cap #12.>>
<<seL4(CPU 0) [decodeInvocation/646 T0xffffff80bffde400 "" @400d08]: Attempted to invoke a null cap #11.>>
<<seL4(CPU 0) [decodeInvocation/646 T0xffffff80bffde400 "" @400d08]: Attempted to invoke a null cap #12.>>
Caught cap fault in send phase at address 0
while trying to handle:
vm fault on data at address 0x433060 with status 0x92000007at address 0x402928
With stack:
0x10011f30: 0x0
0x10011f38: 0x0
0x10011f40: 0x0
0x10011f48: 0x0
0x10011f50: 0x0
0x10011f58: 0x0
0x10011f60: 0x0
0x10011f68: 0x0
0x10011f70: 0x0
0x10011f78: 0x0
0x10011f80: 0x10011fa600000000
0x10011f88: 0x10011fa200000000
0x10011f90: 0x10011f9e00000000
0x10011f98: 0x32000000000000
0x10011fa0: 0x38000000390000
0x10011fa8: 0x0

Mine and @midnightveil 's debugging found out that the vm fault is caused when calling sel4_strerror (0x402928) (inside bench_proc) which then tries looking up sel4_errlist (0x433050) and looking up second enum there - seL4_InvalidCapability and that's why the nascent errors ensue.

It seems like we are accessing memory that is not remapped - .data section from looking up the ELF section mappings:

  [12] .data             PROGBITS         0000000000433000  00032000
       0000000000001324  0000000000000000  WA       0     0     16

It looks like this section is not copied when loading the ELF file:

sel4bench/apps/page_mapping/src/main.c

Line 276 in cfdd22d

benchmark_shallow_clone_process(env, &proc.process, seL4_MaxPrio,

Explicitly mentioned that only .text is copied over.

Platform tested: rockpro64 (and rock3b - where I initially thought that something is wrong on my side since I am still adding support for that one).

[JDuchniewicz]

Checked that it also reproduces on OdroidC4, so it is not related to Rockchip SoCs only.

[lsf37]

Thanks for reporting that, would indeed be nice to fix. One less thing for people to be worried about when they are porting things.

[Indanz]

Thanks for debugging this!

I fixed all other crashes/horrible bugs except for the page mapping one, I didn't get to the page mapping one yet.

[JDuchniewicz]

I spent some more time toying around with that. Seems like properly fixing that is a bit of a refactor of the test harness. Some further observations include:

• sel4bench will hang when you try logging before bench_spawn
• Logging anything after results in the fault while trying to access zf_log related data.

I tried writing a simple workaround where I would log hardcoded logs but it turns out that the logging backend itself does not get mapped in and therefore we get faults when trying to log from inside the process.

[Indanz]

All in all the quick fix seems to be finding those prints and removing them, assuming they're not in library code. (And if they are, they probably should propagate errors up instead of printing themselves anyway.)

The proper fix is to burn sel4bench to the ground and rewrite it cleanly.