Humility is the debugger for Hubris.
Hubris is the artifact that actually runs, and it must be allowed to be
optimized in terms of size, space, and run-time; in as much as contortions are
required for debuggability, they should be borne by Humility, not Hubris. As a
concrete example of this, Humility operates exclusively on release
builds of
Hubris, relying on (unloaded) DWARF information in the binaries to make sense of
the system.
Humility is Hubris-specific: it is not trying to be a generic in situ debugger, but rather specifically focused on debugging Hubris-based systems. Humility is therefore willing to encode Hubris-specific concepts like archives and tasks.
Debuggers must cut through abstractions, which often requires knowledge of underlying implementation detail. For Humility, this means being willing to take advantage of microcontroller-specific debug facilities where applicable. While Hubris may make decisions in the name of greater portability, Humility will generally make decisions to maximize debuggability, even where these facilities are highly specific to a particular MCU.
Humility is unafraid to offer device-specific functionality where that
functionality is useful for system debuggability. (For example,
humility i2c
functions as an I2C analyzer.) That said, all device
interaction uses Hubris as a proxy to actually communicate with the devices
themselves (e.g., via HIF);
Humility does not seek to create second I/O path.
Humility seeks to be useful, and therefore seeks to offer all manners of debugging: in situ, postmortem, dynamic instrumentation, static instrumentation, etc.
Humility operates by specifying a subcommand. There are options that are Humility-wide (that is, applying to every subcommand), as well as options that are specific to particular subcommands.
Humility needs to have some way of extracting information and/or controlling the microcontroller running Hubris. This is done through some variant of a debug probe, a separate microcontroller that speaks to debug-specific functionality on the target microcontroller. (For details of the mechanics of these probes on ARM parts, see RFD 65.)
For most evaluation boards, this debug probe is available on the board, and is connected to a host via USB, e.g. ST's STLink/V2 on the STM32F407 Discovery or the LPC-Link2 present on the LPCXpresso55S69. (On the Gemini board, there are two SWD headers, one for the LPC55S28 and the other for the STM32H753.) While Humility allows for direct attachment to an on-chip debugger, doing so precludes other connections (from, for example, OpenOCD), making it too disruptive to development workflows. To allow for easier development workflows, Humility also has the option to attach via OpenOCD.
The debug probe to use is specified to Humility via the -p
option
(long form --probe
) or the HUMILITY_PROBE
environment variable,
which can have the following values:
-
auto
(default): Automatically determine how to attach to the microcontroller. -
ocd
: Attach via OpenOCD, which is presumed to have the TCL interface available on localhost on port 6666 (its default). -
jlink
: Attach via Segger JLink, which is presumed to have the GDB interface available on localhost on port 2331 (its default). Note that when semihosting is being used by Hubris, the Segger JLink GDB server will become confused when Humility attaches to it -- and subsequent calls to semihosting will cause a halt. A subsequent Humility invocation will resume the target (directing semihosting output correctly to the running GDB instance), but subsequent semihosting output will again cause a halt. To recover from this condition, send an explicit ^C to the running GDB and continue from the resulting stop. -
usb
: Attach directly via USB to a debug probe. When multiple probes are plugged in via USB, a probe index must be specified as a suffix (e.g.,usb-0
,usb-1
, etc.) To determine which probe is which, examine the serial number in the output ofhumility probe
. -
vid:pid[:serial]
: In some cases, the automatic algorithm may either find the wrong thing, or timeout attempting to search for non-existent probes. In these cases, it can help to explicitly set the vendor ID (VID) and product ID (PID) of the debug probe, which should be colon-delimited, e.g.,0483:374e
. (Determining the VID and PID of an attached probe is platform-specific; on Linux one can use thelsusb
command.) In cases where there are multiple probes with the same VID and PID, the serial number of the probe (as reported viahumility probe
or found in theiSerialNumber
field of the USB device descriptor) can be postpended, also delimited by a colon, e.g.0483:374e:004000343137510939383538
.
Many Humility commands require the complete Hubris archive. This is a ZIP
archive created by the build process, and includes all binaries as well as the
app.toml
file used to configure the Hubris archive. The archive can be
found in the target
for Hubris, and will end with (.zip
), e.g.:
/path/to/hubris/target/demo-stm32h743-nucleo/dist/build-demo-stm32h743-nucleo.zip
.
The Hubris archive is specified via the -a
option or the HUMILITY_ARCHIVE
environment variable.
In the Humility examples in this documentation, unless otherwise specified,
the archive will be assumed to be set via -a
or HUMILITY_ARCHIVE
.
Many Humility commands are able to operate postmortem on a Hubris dump,
an ELF core file generated by the humility dump
command.
Dumps are offered in lieu of a probe and an archive and specified via
the -d
option (long form --dump
) or the HUMILITY_DUMP
environment
variable.
- humility apptable: print Hubris apptable
- humility dashboard: dashboard for Hubris sensor data
- humility diagnose: analyze a system to detect common problems
- humility doc: print command documentation
- humility dump: generate Hubris dump
- humility etm: commands for ARM's Embedded Trace Macrocell (ETM)
- humility extract: extract all or part of a Hubris archive
- humility flash: flash archive onto attached device
- humility gdb: Attach to a running system using GDB
- humility gpio: GPIO pin manipulation
- humility hash: Access to the HASH block
- humility hiffy: manipulate HIF execution
- humility i2c: scan for and read I2C devices
- humility itm: commands for ARM's Instrumentation Trace Macrocell (ITM)
- humility jefe: influence jefe externally
- humility lpc55gpio: LPC55 GPIO pin manipulation
- humility manifest: print archive manifest
- humility map: print memory map, with association of regions to tasks
- humility openocd: Run OpenOCD for the given archive
- humility pmbus: scan for and read PMBus devices
- humility probe: probe for any attached devices
- humility qspi: QSPI status, reading and writing
- humility readmem: read and display memory region
- humility readvar: read and display a specified Hubris variable
- humility registers: print Hubris registers
- humility rencm: query Renesas 8A3400X ClockMatrix parts
- humility rendmp: Renesas digitial muliphase controller operations
- humility ringbuf: read and display a specified ring buffer
- humility sensors: query sensors and sensor data
- humility spctrl: RoT -> SP control
- humility spd: scan for and read SPD devices
- humility spi: SPI reading and writing
- humility stackmargin: calculate and print stack margins by task
- humility stmsecure: change secure region settings on the stm32h7
- humility tasks: list Hubris tasks
- humility test: run Hubristest suite and parse results
- humility trace: trace Hubris operations
- humility validate: validate presence and operation of devices
- humility vsc7448: VSC7448 operations
This is a deprecated command that allows for the display of the app table
found in old Hubris archives; see humility manifest
to understand
the contents of an archive.
Provides a captive dashboard that graphs sensor values over time. (The
sensor
task is required for operation; see the documentation for
humility sensors
for more details.)
If -o
is provided, it specifies an output file for any raw sensor data
graphed by the dashboard.
This subcommand provides a "firmware engineer in a can" for scanning and reporting issues in a running system. It's mostly concerned with
- Noticing things that seem fishy, and
- Taking action to get good snapshots of those things.
This is application-independent logic, so it doesn't have any visibility into things the application may think are fishy -- only general behaviors at the OS level, like faults.
Provides detailed documentation for Humility and its commands. To
get documentation on Humility, run humility doc
; to get documentation
for a specific command (like this one!) run humility doc
and specify
the command name -- and run humility --help
to list all commands.
humility dump
takes a dump of the attached system, writing out an ELF
core file:
% humility dump
humility: attached via ST-Link
humility: core halted
humility: dumping to hubris.core.0
humility: dumped 1.12MB in 24 seconds
humility: core resumed
A dump file name may also be specified:
% humility dump hubris.core.`date +%s`
humility: attached via ST-Link
humility: core halted
humility: dumping to hubris.core.1600718079
humility: dumped 1.12MB in 24 seconds
humility: core resumed
The resulting dump can be used with many commands (including manifest
,
map
, readvar
, and tasks
) -- and need not be run on the same machine
as the debugged MCU, e.g.:
% humility -d hubris.core.0 tasks
humility: attached to dump
ID ADDR TASK GEN STATE
0 20000168 jefe 0 Healthy(InRecv(None))
1 200001d8 rcc_driver 0 Healthy(InRecv(None))
2 20000248 gpio_driver 0 Healthy(InRecv(None))
3 200002b8 usart_driver 0 Healthy(InRecv(None))
4 20000328 i2c_driver 0 Healthy(InRecv(None))
5 20000398 user_leds 0 Healthy(InRecv(None))
6 20000408 pong 0 Healthy(InRecv(None))
7 20000478 ping 40 Healthy(InReply(TaskId(0x3)))
8 200004e8 adt7420 0 Healthy(InRecv(Some(TaskId(0xffff))))
9 20000558 idle 0 Healthy(Runnable) <-
No documentation yet for humility etm
; pull requests welcome!
humility extract
extracts a file from either a Hubris archive or a
dump. By default, the contents of the file are sent to standard output,
e.g.:
% humility -a /path/to/my/hubris-archive.zip extract README.TXT
humility: extracting README.TXT to stdout
This is a build archive containing firmware build artifacts.
- app.toml is the config file used to build the firmware.
- git-rev is the commit it was built from, with optional dirty flag.
- info/ contains human-readable data like logs.
- elf/ contains ELF images for all firmware components.
- elf/tasks/ contains each task by name.
- elf/kernel is the kernel.
- img/ contains the final firmware images.
- debug/ contains OpenOCD and GDB scripts, if available.
To list all of the files in an archive, use the --list
(-l
) option:
% humility -a /path/to/my/hubris-archive.zip extract --list
SIZE NAME
462 README.TXT
46 git-rev
5289 app.toml
1937 chip.toml
197456 elf/task/jefe
208836 elf/task/sys
288440 elf/task/i2c_driver
309100 elf/task/spi_driver
1059092 elf/task/net
65156 elf/task/user_leds
87284 elf/task/ping
31228 elf/task/pong
178600 elf/task/udpecho
370756 elf/task/hiffy
310184 elf/task/hf
286556 elf/task/hash_driver
17612 elf/task/idle
201964 elf/task/rng_driver
555820 elf/kernel
1550 info/allocations.txt
4195 info/map.txt
632426 img/combined.srec
307060 img/combined.elf
862933 img/combined.ihex
307200 img/combined.bin
632426 img/final.srec
307060 img/final.elf
862933 img/final.ihex
307200 img/final.bin
1790 img/flash.ron
1047 debug/script.gdb
1586 debug/openocd.cfg
235 debug/openocd.gdb
Note that the file specified is treated as a substring; if the substring matches exactly one file, it will be extracted:
% humility -d ./hubris.core.79 extract map
humility: extracting info/map.txt to stdout
ADDRESS END SIZE FILE
08000000 08000298 298 target/demo-stm32h753-nucleo/dist/kernel
08000298 08004420 4188 target/demo-stm32h753-nucleo/dist/kernel
08004420 08005604 11e4 target/demo-stm32h753-nucleo/dist/kernel
08005a00 08005a58 58 target/demo-stm32h753-nucleo/dist/idle
08005c00 08005e10 210 target/demo-stm32h753-nucleo/dist/pong
08005e10 08005e88 78 target/demo-stm32h753-nucleo/dist/pong
08006000 080076f4 16f4 target/demo-stm32h753-nucleo/dist/jefe
080076f4 08007c08 514 target/demo-stm32h753-nucleo/dist/jefe
08007c08 08007c08 0 target/demo-stm32h753-nucleo/dist/jefe
08008000 08009798 1798 target/demo-stm32h753-nucleo/dist/udpecho
08009798 08009b90 3f8 target/demo-stm32h753-nucleo/dist/udpecho
08009b90 08009b90 0 target/demo-stm32h753-nucleo/dist/udpecho
08010000 08014020 4020 target/demo-stm32h753-nucleo/dist/hiffy
08014020 08015134 1114 target/demo-stm32h753-nucleo/dist/hiffy
08015134 0801574c 618 target/demo-stm32h753-nucleo/dist/hiffy
08018000 08019970 1970 target/demo-stm32h753-nucleo/dist/i2c_driver
08019970 08019ff4 684 target/demo-stm32h753-nucleo/dist/i2c_driver
08019ff4 0801a300 30c target/demo-stm32h753-nucleo/dist/i2c_driver
0801c000 0801de78 1e78 target/demo-stm32h753-nucleo/dist/spi_driver
0801de78 0801e898 a20 target/demo-stm32h753-nucleo/dist/spi_driver
0801e898 0801eca4 40c target/demo-stm32h753-nucleo/dist/spi_driver
08020000 0802a554 a554 target/demo-stm32h753-nucleo/dist/net
0802a554 0802d74c 31f8 target/demo-stm32h753-nucleo/dist/net
0802d74c 0802d7dc 90 target/demo-stm32h753-nucleo/dist/net
08040000 08041ea0 1ea0 target/demo-stm32h753-nucleo/dist/hf
08041ea0 080426a8 808 target/demo-stm32h753-nucleo/dist/hf
080426a8 080426a8 0 target/demo-stm32h753-nucleo/dist/hf
08044000 08044b8c b8c target/demo-stm32h753-nucleo/dist/ping
08044b8c 08044d34 1a8 target/demo-stm32h753-nucleo/dist/ping
08044d34 08044d34 0 target/demo-stm32h753-nucleo/dist/ping
08046000 080478b4 18b4 target/demo-stm32h753-nucleo/dist/hash_driver
080478b4 08047f68 6b4 target/demo-stm32h753-nucleo/dist/hash_driver
08047f68 08047f68 0 target/demo-stm32h753-nucleo/dist/hash_driver
08048000 08048fcc fcc target/demo-stm32h753-nucleo/dist/rng_driver
08048fcc 08049474 4a8 target/demo-stm32h753-nucleo/dist/rng_driver
08049474 08049474 0 target/demo-stm32h753-nucleo/dist/rng_driver
0804a000 0804a634 634 target/demo-stm32h753-nucleo/dist/sys
0804a634 0804a67c 48 target/demo-stm32h753-nucleo/dist/sys
0804a800 0804ac0c 40c target/demo-stm32h753-nucleo/dist/user_leds
0804ac0c 0804ac7c 70 target/demo-stm32h753-nucleo/dist/user_leds
20000400 20000400 0 target/demo-stm32h753-nucleo/dist/kernel
30000000 30000000 0 target/demo-stm32h753-nucleo/dist/net
If the substring matches more than one file, the command will fail and the matching files will be displayed:
% humility -d ./hubris.core.79 extract toml
humility extract failed: "toml" matches multiple files: app.toml, stm32h7.toml
To redirect output to a particular file, use the --output
(-o
) option.
To dump the entire archive, leave the file unspecified. (Note that extracting the entire archive requires the specification of an output file to prevent accidental blasts of binary content to the console.)
Flashes the target with the image that is contained within the specified
archive (or dump). This merely executes the underlying flashing
mechanism (either pyOCD or OpenOCD, depending on the target); if the
requisite software is not installed (or isn't in the path), this will
fail. Temporary files are created as part of this process; if they are to
be retained, the -R
(--retain-temporaries
) flag should be set.
To see what would be executed without actually executing any commands,
use the -n
(--dry-run
) flag. As a precautionary measure, if
the specified archive already appears to be on the target, humility flash
will fail unless the -F
(--force
) flag is set.
This command launches GDB and attaches to a running device.
By default, the user must be running openocd
or pyocd
in a separate
terminal.
The --run-openocd
option automatically launches openocd
based on the
openocd.cfg
file included in the build archive.
When using pyocd
, it must be launched with the --persist
option,
because humility gdb
connects to it multiple times (once to check the
app id, then again to run the console).
humility gpio
allows for GPIO pins to be set, reset, queried or
configured on STM32 targets. Commands:
--set
(-s
): Sets a pin (sets it high)--reset
(-r
): Resets a pin (sets it low)--toggle
(-t
): Toggles a pin (sets it high if low, low if high)--input
(-i
): Queries the state of a pin (or all pins if no pin is specified)--configure
(-c
): Configures a pin
To change the state of a pin (or pins), specify the pin (or pins) and the desired command. For example, to toggle the state on pin 14 on port B:
% humility gpio --toggle --pins B:14
humility: attached via ST-Link V3
[Ok([])]
To set pins B:0, B:14 and E:1:
% humility gpio --set --pins B:0,B:14,E:1
humility: attached via ST-Link V3
[Ok([]), Ok([]), Ok([])]
To reset pin E:1:
% humility gpio --reset --pins E:1
humility: attached via ST-Link V3
[Ok([])]
To get input values for a particular pin:
% humility gpio --input --pins B:0,B:14,E:1
humility: attached via ST-Link V3
B:0 = 1
B:14 = 1
E:1 = 0
To get input values for all pins, leave the pin unspecified:
% humility gpio --input
humility: attached via ST-Link V3
Pin 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
-----------------------------------------------------------------------
Port A 0 0 1 0 0 0 0 0 0 0 0 0 0 1 1 1
Port B 1 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0
Port C 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Port D 0 0 0 0 0 0 0 0 1 1 0 0 0 0 1 0
Port E 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Port F 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Port G 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Port H 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Port I 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Port J 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Port K 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
To configure a pin, the configuration should be specified as a colon-delimited 5-tuple consisting of:
- Mode:
Input
,Output
,Alternate
, orAnalog
- Output type:
PushPull
orOpenDrain
- Speed:
Low
,Medium
,High
, orVeryHigh
- Pull direction:
None
,Up
, orDown
- Alternate function: one of
AF0
throughAF15
For example, to configure pin 5 on port A as a push-pull output:
% humility gpio -c Output:PushPull:High:None:AF0 -p A:5
No documentation yet for humility hash
; pull requests welcome!
humility hiffy
allows for querying and manipulation of hiffy
, the
HIF agent present in Hubris. To list all Idol interfaces present in
Hubris, use the -l
(--list
) option:
% humility hiffy -l
humility: attached via ST-Link
TASK INTERFACE OPERATION ARG ARGTYPE
rcc_driver Rcc enable_clock_raw peripheral u32
disable_clock_raw peripheral u32
enter_reset_raw peripheral u32
leave_reset_raw peripheral u32
spi_driver Spi read device_index u8
write device_index u8
exchange device_index u8
lock device_index u8
cs_state CsState
release -
user_leds UserLeds led_on index usize
led_off index usize
led_toggle index usize
To enlist the Hubris agent to call a particular interface and operation,
use -c
(--call
), using -a
(--arguments
) to indicate any arguments,
e.g.:
% humility hiffy -c UserLeds.led_toggle -a index=0
humility: attached via ST-Link
UserLeds.led_toggle() = ()
To view the raw HIF functions provided to programmatic HIF consumers
within Humility, use -L
(--list-functions
).
On platforms that have I2C support, humility i2c
can be used
to scan a bus, scan a device, read a register, or write a register. Its
usage will be specific to the board being examined; to specify a
controller use the -c
, to specify a port (if necessary), use -p
; to
specify a mux and segment (if necessary), use -m
.
For example, on a Gimletlet, here is a scan of controller I2C3, revealing
one device at address 0x48
:
% humility i2c -s -c 3
humility: attached via ST-Link
Device scan on controller I2C3:
R = Reserved - = No device \o/ = Device found X = Timed out
ADDR 0x0 0x1 0x2 0x3 0x4 0x5 0x6 0x7 0x8 0x9 0xa 0xb 0xc 0xd 0xe 0xf
0x00 R R R R R R R R - - - - - - - -
0x10 - - - - - - - - - - - - - - - -
0x20 - - - - - - - - - - - - - - - -
0x30 - - - - - - - - - - - - - - - -
0x40 - - - - - - - - \o/ - - - - - - -
0x50 - - - - - - - - - - - - - - - -
0x60 - - - - - - - - - - - - - - - -
0x70 - - - - - - - - - - - - R R R R
To scan that device, specify its address via -d
:
% humility i2c -s -c 3 -d 0x48
humility: attached via ST-Link
Register scan for device 0x48 on I2C3:
- = No register ! = No device X = Timed out
ADDR 0x0 0x1 0x2 0x3 0x4 0x5 0x6 0x7 0x8 0x9 0xa 0xb 0xc 0xd 0xe 0xf
0x00 0b c8 80 00 20 00 05 00 49 80 05 cb 00 00 00 00
0x10 00 00 00 00 00 00 00 00 6c 00 06 00 00 00 00 00
0x20 0b c8 80 00 20 00 05 00 49 80 05 cb 00 00 00 00
0x30 00 00 00 00 00 00 00 00 6c 00 06 00 00 00 00 00
0x40 00 00 80 00 20 00 05 00 49 80 05 cb 00 00 00 00
0x50 00 00 00 00 00 00 00 00 6c 00 06 00 00 00 00 00
0x60 00 00 80 00 20 00 05 00 49 80 05 cb 00 00 00 00
0x70 00 00 00 00 00 00 00 00 6c 00 06 00 00 00 00 00
0x80 00 00 80 00 20 00 05 00 49 80 05 cb 00 00 00 00
0x90 00 00 00 00 00 00 00 00 6c 00 06 00 00 00 00 00
0xa0 00 00 80 00 20 00 05 00 49 80 05 cb 00 00 00 00
0xb0 00 00 00 00 00 00 00 00 6c 00 06 00 00 00 00 00
0xc0 00 00 80 00 20 00 05 00 49 80 05 cb 00 00 00 00
0xd0 00 00 00 00 00 00 00 00 6c 00 06 00 00 00 00 00
0xe0 00 00 80 00 20 00 05 00 49 80 05 cb 00 00 00 00
0xf0 00 00 00 00 00 00 00 00 6c 00 06 00 00 00 00 00
(This device is an ADT7420 temp sensor.) To look at a particular
register, elide -s
and specify the register of interest via -r
:
% humility i2c -c 3 -d 0x48 -r 0xb
humility: attached via ST-Link
Controller I2C3, device 0x48, register 0xb = 0xcb
To write a value to a register, specify the -w
flag, along with the
value to write, e.g. (for the ADT7420), the MSB of the THIGH
register:
% humility i2c -c 3 -d 0x48 -r 0x4 -w 0x1f
humility: attached via ST-Link
Controller I2C3, device 0x48, register 0x4 = 0x1f
Note that if registers are not writable, the write will (generally) be silently discarded by the device; it can be useful to read the register after writing it to confirm that the value is as expected:
% humility i2c -c 3 -d 0x48 -r 0x4
humility: attached via ST-Link
Controller I2C3, device 0x48, register 0x4 = 0x1f
humility itm
consumes data from the Instrumentation Trace Macrocell
(ITM) present in many ARM Cortex-M variants. ITM is problematic in many
dimensions: it is lossy; it requires knowledge of the target's clocking to
configure properly; it relies on functionality (SWO/SWV) that is often
buggy in chip debuggers; it isn't present everywhere (Cortex-M0+ in
particular doesn't have ITM). So in general, ITM isn't what Hubris
programmers should be looking for: those developing code and wishing to
see if and how that code is executed should prefer ring buffers to
ITM-based instrumentation. (See the documentation for humility ringbuf
for details.)
That said, ITM remains the best way to get certain messages from the
Hubris kernel (e.g., boot and panic messages); use humility itm -ea
to
enable ITM and attach to the connected device. For example, if running
with the ping
task, one will see messages from jefe
restarting it:
% humility -a /path/to/my/hubris-archive.zip itm -ea
humility: attached via ST-Link
humility: core halted
humility: core resumed
humility: ITM synchronization packet found at offset 6
Task #7 Divide-by-zero
Task #7 Memory fault at address 0x0
Task #7 Divide-by-zero
Humility allows for some (well-defined) manipulation of tasks via jefe
,
the Hubris supervisor. By default, jefe
will restart any faulting task;
when debugging, it can be useful to hold a task in the faulted state. This
is done via the -h
option:
% humility jefe -h ping
humility: attached via ST-Link
humility: successfully changed disposition for ping
This does not change the task, but rather jefe
's restart disposition; when
the task next faults, it will not be restarted, e.g.:
% humility tasks ping
humility: attached via ST-Link
system time = 26597
ID TASK GEN PRI STATE
8 ping 121 4 FAULT: divide by zero (was: ready)
This can be particularly useful to couple with either humility dump
or
humility tasks -sl
:
% humility tasks -sl ping
humility: attached via ST-Link
system time = 103879
ID TASK GEN PRI STATE
8 ping 121 4 FAULT: divide by zero (was: ready)
|
+---> 0x200065b0 0x0802a05e task_ping::divzero
@ /home/bmc/hubris/task/ping/src/main.rs:24
0x20006600 0x0802a0fe userlib::sys_panic
@ /home/bmc/hubris/sys/userlib/src/lib.rs:678
0x20006600 0x0802a0fe main
@ /home/bmc/hubris/task/ping/src/main.rs:35
To change the disposition of a task (back) to be restarted, use the -r
flag:
% humility jefe -r ping
humility: attached via ST-Link
humility: successfully changed disposition for ping
To inject a fault into a task, use the -f
flag, which will change the
disposition to injecting a fault. The injected fault has a dedicated type to
make clear that the fault originated outside of the task:
% humility jefe -f pong
humility: attached via ST-Link
humility: successfully changed disposition for pong
% humility tasks pong
humility: attached via ST-Link
system time = 191227
ID TASK GEN PRI STATE
7 pong 0 3 FAULT: killed by jefe/gen0 (was: recv, notif: bit0)
|
+---> 0x200063b8 0x08028c0a userlib::sys_recv_stub
@ /home/bmc/hubris/sys/userlib/src/lib.rs:288
0x20006400 0x080280ac userlib::sys_recv
@ /home/bmc/hubris/sys/userlib/src/lib.rs:236
0x20006400 0x080280ba main
@ /home/bmc/hubris/task/pong/src/main.rs:13
To restart a task that has had a fault injected, again use the -r
flag to
change its disposition back to restart.
Finally, to start a task that is not started by default, use the -s
flag.
No documentation yet for humility lpc55gpio
; pull requests welcome!
humility manifest
displays information about the Hubris archive. It
does not connect at all to a Hubris target to operate. In addition to
archive-wide attributes, humility manifest
displays the features enabled
for each task, as well as any device toplogy information present in
the archive, e.g.:
% humility manifest
version => hubris build archive v1.0.0
git rev => 753a57169eba699e73ee59e0cf5345eb1d6e1ae2-dirty
board => nucleo-h743zi2
target => thumbv7em-none-eabihf
features => h743, itm
total size => 140K
kernel size => 30K
tasks => 12
ID TASK SIZE FEATURES
0 hiffy 42.9K h743, stm32h7, itm, i2c, gpio, qspi
1 jefe 6.7K itm
2 i2c_driver 9.8K h743
3 spi_driver 10.7K spi3, h743
4 hf 8.6K h743
5 rcc_driver 4.7K h743
6 gpio_driver 5.8K h743
7 usart_driver 5.9K h743
8 user_leds 5.5K stm32h7
9 pong 4.7K
10 ping 5.2K uart
11 idle 0.1K
i2c buses => 1 controller, 1 bus
C PORT MODE NAME DESCRIPTION
2 F init - -
humility manifest
can operate on either an archive or on a dump.
One common pathology in Hubris tasks is a fault induced when a task
attempts to access a memory address outside of its designated regions.
(This can happen, for example, because the task attempted to access device
memory that was not allocated to it in the build description, because it
exceeded the device memory allocated to it, or because it allocated the
memory allocated to it, e.g., because of a stack overflow.) humility tasks
can be useful to see this happening; a non-zero generation count
will indicate that a task has been restarted -- and the log message from
jefe
will indicate the specific address, e.g.:
% humility itm -ea
humility: attached via OpenOCD
humility: core halted
humility: core resumed
humility: TPIU sync packet found at offset 1
humility: ITM synchronization packet found at offset 12
Task #7 Memory fault at address 0x200028fc
Task #7 Memory fault at address 0x200028fc
^C
To better understand the memory that a task is trying to access, one can
run the humility map
command, which shows the memory regions that have
been mapped into tasks, in address order:
% humility -a ~/hubris/target/demo/dist/build-demo.zip map
humility: attached via OpenOCD
DESC LOW HIGH SIZE ATTR ID TASK
0x08004864 0x08010000 - 0x08017fff 32KiB r-x-- 0 jefe
0x08004884 0x08018000 - 0x08019fff 8KiB r-x-- 1 rcc_driver
0x080048a4 0x0801c000 - 0x0801ffff 16KiB r-x-- 2 usart_driver
0x080048c4 0x08020000 - 0x08023fff 16KiB r-x-- 3 user_leds
0x080048e4 0x08024000 - 0x08025fff 8KiB r-x-- 4 ping
0x08004904 0x08026000 - 0x08027fff 8KiB r-x-- 5 pong
0x08004924 0x08028000 - 0x080280ff 256 r-x-- 6 idle
0x08004944 0x0802a000 - 0x0802bfff 8KiB r-x-- 7 oh_no
0x08004964 0x0802c000 - 0x0802dfff 8KiB r-x-- 8 oh_no2
0x08004874 0x20001000 - 0x200013ff 1KiB rwx-- 0 jefe
0x08004894 0x20001400 - 0x200017ff 1KiB rwx-- 1 rcc_driver
0x080048b4 0x20001800 - 0x20001bff 1KiB rwx-- 2 usart_driver
0x080048d4 0x20001c00 - 0x20001fff 1KiB rwx-- 3 user_leds
0x080048f4 0x20002000 - 0x200021ff 512 rwx-- 4 ping
0x08004914 0x20002400 - 0x200027ff 1KiB rwx-- 5 pong
0x08004934 0x20002800 - 0x200028ff 256 rwx-- 6 idle
0x08004954 0x20002900 - 0x200029ff 256 rwx-- 7 oh_no
0x08004974 0x20002a00 - 0x20002aff 256 rwx-- 8 oh_no2
0x08004824 0x40004400 - 0x400047ff 1KiB rw-d- 2 usart_driver
0x08004844 0x40020000 - 0x400203ff 1KiB rw-d- 2 usart_driver
0x08004854 0x40020c00 - 0x40020fff 1KiB rw-d- 3 user_leds
0x08004834 0x40023800 - 0x40023bff 1KiB rw-d- 1 rcc_driver
(In this case, task 7, oh_no
, has overflowed its stack -- which
we can see from the map
output has been sized to only 256 bytes.)
This command launches OpenOCD based on the config file in a build archive
No documentation yet for humility pmbus
; pull requests welcome!
humility probe
attempts to infer as much about the hardware state as it
can, e.g.:
% humility probe
humility: attached via ST-Link
humility: probe => STLink V3, VID 0483, PID 374e
humility: probe serial => 003700303137511139383538
humility: core => Cortex-M7
humility: manufacturer => STMicroelectronics
humility: chip => STM32H7, revision 0x2003
humility: status => executing
humility: debug units => CSTF(x2) CTI(x2) DWT ETM FPB ITM SCS SWO TMC TPIU
humility: CSTF => 0x5c004000, 0x5c013000
humility: CTI => 0x5c011000, 0xe0043000
humility: DWT => 0xe0001000
humility: ETM => 0xe0041000
humility: FPB => 0xe0002000
humility: ITM => 0xe0000000
humility: SCS => 0xe000e000
humility: SWO => 0x5c003000
humility: TMC => 0x5c014000
humility: TPIU => 0x5c015000
humility: ITM status => TRCENA enabled, TCR disabled, TER=0x0
humility: R0 => 0x20006000
humility: R1 => 0x20006000
humility: R2 => 0x0
humility: R3 => 0x0
humility: R4 => 0x0
humility: R5 => 0x0
humility: R6 => 0x0
humility: R7 => 0x0
humility: R8 => 0x0
humility: R9 => 0x0
humility: R10 => 0x0
humility: R11 => 0x0
humility: R12 => 0x0
humility: SP => 0x20006100
humility: LR => 0x802404f
humility: PC => 0x8024052
humility: xPSR => 0x61000000
humility: MSP => 0x20000f48
humility: PSP => 0x20006100
humility: SPR => 0x7000000
If provided a Hubris archive, humility probe
will display any register
contents symbolically, e.g.:
% humility -a ~/hubris/target/demo/dist/build-demo.zip probe
humility: attached via ST-Link
humility: probe => STLink V2-1, VID 0483, PID 374b
humility: probe serial => 066DFF383032534E43132614
humility: core => Cortex-M4
humility: manufacturer => STMicroelectronics
humility: chip => STM32F40x/STM32F41x, revision 0x1007
humility: debug units => DWT ETM FPB ITM SCS TPIU
humility: status => executing
humility: ITM => TRCENA enabled, TCR enabled, TER=0x3
humility: R0 => 0x0
humility: R1 => 0x0
humility: R2 => 0x1
humility: R3 => 0x20001bd4
humility: R4 => 0x20001bd4
humility: R5 => 0x801d988
humility: R6 => 0xb004
humility: R7 => 0x20001bf0
humility: R8 => 0x40004400
humility: R9 => 0x1
humility: R10 => 0x0
humility: R11 => 0xffff
humility: R12 => 0x0
humility: SP => 0x20001ba8
humility: LR => 0x801c12b <- main+0xef
humility: PC => 0x801d290 <- sys_recv_stub+0x1e
humility: xPSR => 0x61000000
humility: MSP => 0x20000f48
humility: PSP => 0x20001ba8
humility: SPR => 0x7000000
humility qspi
manipulates (and importantly, writes to) QSPI-attached
flash in Hubris. To read the device identifier, use the --id
(-i
)
option:
% humility qspi -i
humility: attached via ST-Link V3
DeviceIdData {
manufacturer_id: Micron(0x20)
memory_type: 3V(186)
memory_capacity: 33554432
uid_n: 16
# If a Micron device:
ext_device_id: 0b1000100
2nd device generation,
Standard BP scheme,
HOLD#/RESET#=HOLD
Additional HW RESET# is available,
Sector size is Uniform 64KB,
device_configuration_info: 0
uid: [9a, ec, 0b, 00, 19, f9, ff$, 39, 00, be, 69, 97, f4, a2]
}
[Ok([20, ba, 19, 10, 44, 0, 9a, ec, b, 0, 19, f9, ff, 39, 0, be, 69, 97, f4, a2])]
To write an image from a file, use the --writefile
(-W
) option:
% humility -W ./milan-spew-115k2-2dpc-0.4.1-dataeye.bin
humility: attached via ST-Link V3
humility: erasing 16777216 bytes...
humility: ... done
humility: flashed 16.00MB in 5 minutes
If writing similar images, it is much faster to write only those blocks
that differ. To perform a differential write, use the --diffwrite
(-D
)
option:
% humility qspi -D ./milan-spew-115k2-2dpc-0.4.1.bin
humility: attached via ST-Link V3
humility: erasing 65536 bytes...
humility: ... done
humility: hashed 16.00MB, wrote 64.00KB in 16 seconds
To read, write or hash a particular region, use the --read
(-r
),
--write
(-w
), or --hash
(-H
) respectively -- giving the address
via --address
(-a
) and the number of bytes via --nbytes
(-n
).
For example, to read 128 bytes from address 0x120000:
% humility qspi -r -a 0x120000 -n 128
\/ 1 2 3 4 5 6 7 8 9 a b c d e f
0x00120000 | 24 50 53 50 ee 7a 31 28 10 00 00 00 20 cd 48 20 | $PSP.z1(.... .H
0x00120010 | 00 00 00 00 40 04 00 00 00 30 12 00 00 00 00 00 | [email protected]......
0x00120020 | 01 00 00 00 00 54 01 00 00 40 12 00 00 00 00 00 | .....T...@......
0x00120030 | 03 00 00 00 00 54 01 00 00 a0 13 00 00 00 00 00 | .....T..........
0x00120040 | 08 00 00 00 40 ea 01 00 00 00 15 00 00 00 00 00 | ....@...........
0x00120050 | 09 00 00 00 40 06 00 00 00 f0 16 00 00 00 00 00 | ....@...........
0x00120060 | 0a 00 00 00 40 06 00 00 00 00 17 00 00 00 00 00 | ....@...........
0x00120070 | 0b 00 00 00 ff ff ff ff 01 00 00 00 00 00 00 00 | ................
To get the SHA256 hash for that same region:
% humility qspi -H -a 0x120000 -n 128
humility: attached via ST-Link V3
120000..000080: 4d07112733efe240f990fad785726c52de4335d6c5c30a33e60096d4c2576742
humility readmem
allows one to read a specified range of memory:
% humility readmem 0x00011b00
humility: attached via DAPLink
humility: reading at 0x11b00 for 256 bytes
\/ 1 2 3 4 5 6 7 8 9 a b c d e f
0x00011b00 | 00 0f 1a bf 81 54 bc f1 01 0f d0 bd 02 44 bc f1 | .....T.......D..
0x00011b10 | 02 0f 51 70 00 d1 d0 bd 91 70 d0 bd 69 00 01 00 | ..Qp.....p..i...
0x00011b20 | 04 00 00 00 04 00 00 00 17 01 01 00 6b 00 01 00 | ............k...
0x00011b30 | f1 00 01 00 65 78 70 6c 69 63 69 74 20 70 61 6e | ....explicit pan
0x00011b40 | 69 63 00 00 3c 1f 01 00 49 00 00 00 0a 00 00 00 | ic..<...I.......
0x00011b50 | 09 00 00 00 76 69 76 61 20 65 6c 20 6a 65 66 65 | ....viva el jefe
0x00011b60 | 0a 54 61 73 6b 20 23 20 50 61 6e 69 63 21 0a 00 | .Task # Panic!..
0x00011b70 | 61 1b 01 00 06 00 00 00 67 1b 01 00 08 00 00 00 | a.......g.......
0x00011b80 | 20 42 61 64 20 53 79 73 63 61 6c 6c 20 55 73 61 | Bad Syscall Usa
0x00011b90 | 67 65 20 0a 61 1b 01 00 06 00 00 00 80 1b 01 00 | ge .a...........
0x00011ba0 | 13 00 00 00 93 1b 01 00 01 00 00 00 20 53 74 61 | ............ Sta
0x00011bb0 | 63 6b 20 6f 76 65 72 66 6c 6f 77 20 61 74 20 61 | ck overflow at a
0x00011bc0 | 64 64 72 65 73 73 20 30 78 00 00 00 61 1b 01 00 | ddress 0x...a...
0x00011bd0 | 06 00 00 00 ac 1b 01 00 1d 00 00 00 93 1b 01 00 | ................
0x00011be0 | 01 00 00 00 20 4d 65 6d 6f 72 79 20 66 61 75 6c | .... Memory faul
0x00011bf0 | 74 20 61 74 20 61 64 64 72 65 73 73 20 30 78 00 | t at address 0x.
If an argument is present, it is the number of bytes to read:
$ humility readmem 0x00011d00 100
humility: attached via DAPLink
humility: reading at 0x11d00 for 100 bytes
\/ 1 2 3 4 5 6 7 8 9 a b c d e f
0x00011d00 | 20 62 6f 75 6e 64 73 3a 20 74 68 65 20 6c 65 6e | bounds: the len
0x00011d10 | 20 69 73 20 20 62 75 74 20 74 68 65 20 69 6e 64 | is but the ind
0x00011d20 | 65 78 20 69 73 20 30 30 30 31 30 32 30 33 30 34 | ex is 0001020304
0x00011d30 | 30 35 30 36 30 37 30 38 30 39 31 30 31 31 31 32 | 0506070809101112
0x00011d40 | 31 33 31 34 31 35 31 36 31 37 31 38 31 39 32 30 | 1314151617181920
0x00011d50 | 32 31 32 32 32 33 32 34 32 35 32 36 32 37 32 38 | 2122232425262728
0x00011d60 | 32 39 33 30 | 2930
Both arguments can be in either hex, decimal, octal or binary (addresses and contents will always be printed in hex):
humility readmem 0o216401 0b110
humility: attached via DAPLink
humility: reading at 0x11d01 for 6 bytes
0 \/ 2 3 4 5 6 7 8 9 a b c d e f
0x00011d00 | 62 6f 75 6e 64 73 | bounds
To display as half-words (16-bits) use -h
; to display as words (32-bits)
use -w
. (The addresses must be 2-byte and 4-byte aligned, respectively.)
$ humility readmem -w 0x20000000 0x40
humility: attached via DAPLink
humility: reading at 0x20000000 for 64 bytes
\/ 4 8 c
0x20000000 | 00000001 20000180 0000000b 00005020 | ....... .... P..
0x20000010 | 00000002 200001f0 00838042 00000000 | ....... B.......
0x20000020 | 00004db8 00004dc8 00004d28 00004d28 | .M...M..(M..(M..
0x20000030 | 00004d28 00004d28 00004d28 00004d28 | (M..(M..(M..(M..
A frequent use of readmem
is to read peripheral memory; as a
convenience, a peripheral name can be used in lieu of an address, provided
that an archive or dump is also specified:
$ humility -a ~/hubris/target/gemini-bu/dist/build-gemini-bu.zip readmem -w i2c4
0x30
humility: attached via ST-Link
\/ 4 8 c
0x58001c00 | 000000f7 00020490 00008000 00008000 | ................
0x58001c10 | 10c0ecff 00000000 00000021 00000000 | ........!.......
0x58001c20 | 00000000 00000080 00000000 00000000 | ................
Note that reading some peripheral memory may have side effects!
It can also be useful to interpret memory contents symbolically; to do this,
provide a dump or achive and specify the -s
option, e.g.:
$ humility -a ~/hubris/target/gemini-bu-rot/dist/build-gemini-bu-rot.zip readmem -s 0x20004b30 0x40
humility: attached via DAPLink
0x20004b30 | 0x20004bdc
0x20004b34 | 0x0000000a <- __EXCEPTIONS+0x2
0x20004b38 | 0x80000000
0x20004b3c | 0x00002e2e <- syscall_entry+0xcf6
0x20004b40 | 0x00000000
0x20004b44 | 0x0003c2e3 <- spi:main+0x5b
0x20004b48 | 0x0003ceda <- spi:sys_send_stub+0xe
0x20004b4c | 0x01000000
0x20004b50 | 0x00000000
0x20004b54 | 0x00000000
0x20004b58 | 0x00000000
0x20004b5c | 0x00000000
0x20004b60 | 0x00000000
0x20004b64 | 0x00000000
0x20004b68 | 0x00000000
0x20004b6c | 0x00000000
humility readvar
allows one to read a global static variable.
To list all such variables, use the -l
option:
% humility readvar -l
humility: MODULE VARIABLE ADDR SIZE
humility: kernel CORE_PERIPHERALS 0x20000000 1
humility: kernel CURRENT_TASK_PTR 0x20000018 4
humility: kernel DEVICE_PERIPHERALS 0x20000001 1
humility: kernel FAULT_NOTIFICATION 0x20000004 4
humility: kernel IRQ_TABLE_BASE 0x20000010 4
humility: kernel IRQ_TABLE_SIZE 0x20000014 4
humility: kernel TASK_TABLE_BASE 0x20000008 4
humility: kernel TASK_TABLE_SIZE 0x2000000c 4
humility: kernel __EXCEPTIONS 0x08000008 56
humility: kernel __INTERRUPTS 0x08000040 620
humility: kernel __RESET_VECTOR 0x08000004 4
humility: adt7420 TEMPS_BYMINUTE 0x2000b848 17288
humility: adt7420 TEMPS_BYSECOND 0x20008000 14408
To read a variable, specify it:
% humility readvar CURRENT_TASK_PTR
humility: attached via ST-Link
CURRENT_TASK_PTR (0x20000018) = Some(NonNull<kern::task::Task> {
pointer: 0x20000558 (*const kern::task::Task)
})
humility registers
displays the registers from either a live system
or a dump, e.g.:
% humility registers
humility: attached via ST-Link V3
R0 = 0x00000000
R1 = 0x0000000a
R2 = 0x80000000
R3 = 0x00000000
R4 = 0x00000000
R5 = 0x0000f406
R6 = 0x00002711
R7 = 0x20000310
R8 = 0x00000000
R9 = 0x00000000
R10 = 0x20000f68
R11 = 0x00000001
R12 = 0x200002b4
SP = 0x200002e8
LR = 0x0800414f
PC = 0x08004236
PSR = 0x4100000f <- 0100_0001_0000_0000_0000_0000_0000_1111
|||| | || | | |
|||| | || | | + Exception = 0xf
|||| | || | +------------ IC/IT = 0x0
|||| | || +-------------------- GE = 0x0
|||| | |+------------------------------ T = 1
|||| | +------------------------------- IC/IT = 0x0
|||| +--------------------------------- Q = 0
|||+----------------------------------- V = 0
||+------------------------------------ C = 0
|+------------------------------------- Z = 1
+-------------------------------------- N = 0
MSP = 0x200002e8
PSP = 0x20011ab0
SPR = 0x01000001 <- 0000_0001_0000_0000_0000_0000_0000_0001
||| | | |
||| | | + PRIMASK = 1
||| | +---------- BASEPRI = 0x0
||| +-------------------- FAULTMASK = 0
||+------------------------------ CONTROL.nPRIV = 1
|+------------------------------- CONTROL.SPSEL = 0
+-------------------------------- CONTROL.FPCA = 0
FPSCR = 0x00000000
If an archive is provided or if displaying registers from a dump, the symbol that corresponds to register's value (if any) is displayed, e.g.:
% humility -d ./hubris.core.81 registers
humility: attached to dump
R0 = 0x00000000
R1 = 0x0000000a
R2 = 0x80000000
R3 = 0x00000000
R4 = 0x00000000
R5 = 0x0000f406
R6 = 0x00002711
R7 = 0x20000310 <- kernel: 0x20000000+0x310
R8 = 0x00000000
R9 = 0x00000000
R10 = 0x20000f68 <- kernel: DEVICE_PERIPHERALS+0x0
R11 = 0x00000001
R12 = 0x200002b4 <- kernel: 0x20000000+0x2b4
SP = 0x200002e8 <- kernel: 0x20000000+0x2e8
LR = 0x0800414f <- kernel: write_str<cortex_m::itm::Port>+0xd
PC = 0x08004236 <- kernel: panic+0x36
PSR = 0x4100000f <- 0100_0001_0000_0000_0000_0000_0000_1111
|||| | || | | |
|||| | || | | + Exception = 0xf
|||| | || | +------------ IC/IT = 0x0
|||| | || +-------------------- GE = 0x0
|||| | |+------------------------------ T = 1
|||| | +------------------------------- IC/IT = 0x0
|||| +--------------------------------- Q = 0
|||+----------------------------------- V = 0
||+------------------------------------ C = 0
|+------------------------------------- Z = 1
+-------------------------------------- N = 0
MSP = 0x200002e8 <- kernel: 0x20000000+0x2e8
PSP = 0x20011ab0 <- pong: 0x20011800+0x2b0
SPR = 0x01000001 <- 0000_0001_0000_0000_0000_0000_0000_0001
||| | | |
||| | | + PRIMASK = 1
||| | +---------- BASEPRI = 0x0
||| +-------------------- FAULTMASK = 0
||+------------------------------ CONTROL.nPRIV = 1
|+------------------------------- CONTROL.SPSEL = 0
+-------------------------------- CONTROL.FPCA = 0
To display a stack backtrace, use the --stack
(-s
) option:
% humility -d ./hubris.core.81 registers --stack
humility: attached to dump
R0 = 0x00000000
...
R10 = 0x20000f68 <- kernel: DEVICE_PERIPHERALS+0x0
R11 = 0x00000001
R12 = 0x200002b4 <- kernel: 0x20000000+0x2b4
SP = 0x200002e8 <- kernel: 0x20000000+0x2e8
|
+---> 0x20000318 0x08004236 rust_begin_unwind
0x20000330 0x08000558 core::panicking::panic_fmt
0x20000358 0x08000ad8 core::panicking::panic
0x20000390 0x08003ba6 kern::arch::arm_m::safe_sys_tick_handler
0x20000390 0x08003ba6 kern::arch::arm_m::SysTick::{{closure}}
0x20000390 0x08003ba6 kern::arch::arm_m::with_task_table
0x20000390 0x08003bb0 SysTick
LR = 0x0800414f <- kernel: write_str<cortex_m::itm::Port>+0xd
...
To additionally display floating point registers on platforms that support
floating point, use the --floating-point
(-f
) option.
Query the Renesas 8A3400X ClockMatrix part -- or process a trace from Renesas configuration software.
No documentation yet for humility rendmp
; pull requests welcome!
humility ringbuf
reads and displays any Hubris ring buffers (as created
via the ringbuf!
macro in the Hubris ringbuf
crate). e.g.:
% humility -d ./hubris.core.5 ringbuf
ADDR NDX LINE GEN COUNT PAYLOAD
0x2000a288 552 92 1 5 (21.5, 70.69999694824219)
0x2000a298 553 92 1 1 (21.4375, 70.58749389648438)
0x2000a2a8 554 92 1 1 (21.5, 70.69999694824219)
0x2000a2b8 555 92 1 1 (21.4375, 70.58749389648438)
0x2000a2c8 556 92 1 5 (21.5, 70.69999694824219)
0x2000a2d8 557 92 1 1 (21.5625, 70.8125)
0x2000a2e8 558 92 1 15 (21.5, 70.69999694824219)
0x2000a2f8 559 92 1 1 (21.4375, 70.58749389648438)
0x2000a308 560 92 1 10 (21.5, 70.69999694824219)
0x2000a318 561 92 1 2 (21.4375, 70.58749389648438)
0x2000a328 562 92 1 2 (21.5, 70.69999694824219)
0x2000a338 563 92 1 1 (21.4375, 70.58749389648438)
0x2000a348 564 92 1 9 (21.5, 70.69999694824219)
0x2000a358 565 92 1 3 (21.4375, 70.58749389648438)
0x2000a368 566 92 1 4 (21.5, 70.69999694824219)
0x2000a378 567 92 1 1 (21.4375, 70.58749389648438)
...
If an argument is provided, only ring buffers that have a name that
contains the argument as a substring, or are in a task that contains
the argument as a substring will be displayed. For example,
to display every ring buffer that has i2c
in the name or the
containing task:
% humility -d ./hubris.core.76 ringbuf ksz
humility: attached to dump
humility: ring buffer ksz8463::__RINGBUF in net:
NDX LINE GEN COUNT PAYLOAD
2 134 89 1 Read(IADR5, 0x4000)
3 134 89 1 Read(IADR4, 0x0)
4 134 89 1 Read(P1MBSR, 0x780c)
5 148 89 1 Write(IACR, 0x1c00)
6 134 89 1 Read(IADR5, 0x4000)
7 134 89 1 Read(IADR4, 0x0)
8 148 89 1 Write(IACR, 0x1c14)
...
See the ringbuf
documentation for more details.
humility sensors
communicates with the sensor
Hubris task via its
Sensor
Idol interface to get sensor data. If there is no sensor
task
or if there are no sensors defined in the in Hubris application
description, this command will not provide any meaningful output. To list
all available sensors, use -l
(--list
); to summarize sensor values,
use -s
(--summarize
). To constrain sensors by type, use the -t
(--types
) option; to constrain sensors by device, use the -d
(--devices
) option; to constrain sensors by name, use the -n
(--named
) option. Within each option, multiple specifications serve as
a logical OR (that is, (-d raa229618,tmp117
would yield all sensors from
either device), but if multiple kinds of specifications are present, they
serve as a logical AND (e.g., -t thermal -d raa229618,tmp117
would yield
all thermal sensors from either device).
humility spctrl
runs commands on the RoT to control the SP.
You must run humility spctrl init
before any other commands
% humility spctrl init
[Ok([])]
You can read/write memory on the SP via the RoT
% humility spctrl -W read 0x08000000 64
humility: attached via CMSIS-DAP
\/ 4 8 c
0x08000000 | 20000400 08000299 08003b6d 08004271 | ... ....m;..qB..
0x08000010 | 08003c8d 08003ccd 08003cd3 00000000 | .<...<...<......
0x08000020 | 00000000 00000000 00000000 0800398b | .............9..
0x08000030 | 08003b6d 00000000 08003aa9 080039dd | m;.......:...9..
% humility spctrl -W read 0x00000000 64
humility: attached via CMSIS-DAP
\/ 4 8 c
0x00000000 | 3d0fbf49 991373d9 9107611c f6d84242 | I..=.s...a..BB..
0x00000010 | 742397db c7c60242 decc7515 ce719848 | ..#tB....u..H.q.
0x00000020 | b2d2639e faf8049b e202de8c 2ae12025 | .c..........% .*
0x00000030 | f739ba6f 20067a60 310c4e08 e42eca28 | o.9.`z. .N.1(...
% humility spctrl -W write 0x00000000 0x11,0x22,0x33,0x44,0x55,0x66,0x77,0x88
humility: attached via CMSIS-DAP
[Ok([])]
% humility spctrl -W read 0x00000000 64
humility: attached via CMSIS-DAP
\/ 4 8 c
0x00000000 | 44332211 88776655 9107611c f6d84242 | ."3DUfw..a..BB..
0x00000010 | 742397db c7c60242 decc7515 ce719848 | ..#tB....u..H.q.
0x00000020 | b2d2639e faf8049b e202de8c 2ae12025 | .c..........% .*
0x00000030 | f739ba6f 20067a60 310c4e08 e42eca28 | o.9.`z. .N.1(...
No documentation yet for humility spd
; pull requests welcome!
humility spi
can be used to read or write to attached SPI devices,
(as defined in the application TOML).
The SPI peripheral (block) to be used can be specified with the
--peripheral
(-p
) option. This should be a number that matches SPI
peripheral number; if it is not specified (and there is only one
SPI-controlling task found), the peripheral associated with that task will
be assumed.
Because of the full duplex nature of SPI, bytes will always be written
and read; to actually write specific bytes, the bytes to be written
should be specified via --write
(-w
). To report bytes read back,
--read
(-r
) should be specified, along with the number of bytes via
--nbytes
(-n
).
For example, to write the byte sequence 0x1
, 0x0
, 0x0
and then read
32 bytes (discarding the first three) from device 0 on SPI2:
% humility spi -p 2 --nbytes 32 --write 0x1,0x0,0x0 --read --discard 3
humility: attached to 0483:374e:003C00174741500520383733 via ST-Link V3
humility: SPI master is spi2_driver
\/ 1 2 3 4 5 6 7 8 9 a b c d e f
0x00000000 | 01 de aa 55 00 00 00 ff 06 00 00 00 1c 01 0b 00 | ...U............
0x00000010 | 00 00 00 00 ff ff ff 06 12 00 00 00 06 | .............
humility stackmargin
calculates and print stack margins by task. The
margin is determined by walking up each stack, looking for the first
word that does not contain the uninitialized pattern (0xbaddcafe
),
from which it infers maximum depth, and therefore margin:
% humility -d ./hubris.core.10 stackmargin
humility: attached to dump
ID TASK STACKBASE STACKSIZE MAXDEPTH MARGIN
0 jefe 0x20001000 1024 768 256
1 rcc_driver 0x20001400 1024 176 848
2 usart_driver 0x20001800 1024 216 808
3 user_leds 0x20001c00 1024 208 816
4 ping 0x20002000 512 224 288
5 pong 0x20002400 1024 208 816
6 idle 0x20002800 256 104 152
Note that the margin is only valid for the task's lifetime -- and in particular, will not be correct if the task has restarted due to a stack overflow!
Humility has support to manage the Root Security Services (RSS) and various flash options bits
A typical sequence to set the secure region at 0x08000000
humility stmsecure set-secure-bit
humility stmsecure set-secure-region 0x08000000 0xa000
To undo the secure region:
humility stmsecure unset-secure-region
humility stmsecure unset-secure-bit
The STM32 has support for flash bank swapping as well
humility stmsecure bank-swap
humility tasks
offers a ps-like view of a system, e.g.:
% humility tasks
humility: attached via ST-Link
system time = 1764993
ID TASK GEN PRI STATE
0 jefe 0 0 recv, notif: bit0 bit1(T+7)
1 rcc_driver 0 1 recv
2 gpio_driver 0 2 recv
3 usart_driver 0 2 recv, notif: bit0(irq39)
4 i2c_driver 0 2 recv
5 spi_driver 0 2 recv
6 user_leds 0 2 recv
7 pong 0 3 FAULT: killed by jefe/gen0 (was: recv, notif: bit0)
8 ping 14190 4 wait: send to pong/gen0
9 hiffy 0 3 notif: bit0(T+7)
10 hf 0 3 notif: bit0(T+18)
11 idle 0 5 RUNNING
To see every field in each task, you can use the -v
flag:
% humility -d hubris.core.4 tasks -v
humility: attached to dump
system time = 1791860
ID TASK GEN PRI STATE
...
7 pong 0 3 FAULT: killed by jefe/gen0 (was: recv, notif: bit0)
|
+-----------> Task {
save: SavedState {
r4: 0x200063c4,
r5: 0x10,
r6: 0x1,
r7: 0x0,
r8: 0x60003,
r9: 0x4,
r10: 0x200063d4,
r11: 0x1,
psp: 0x20006330,
exc_return: 0xffffffed,
...
},
priority: Priority(0x3),
state: Faulted {
fault: Injected(TaskId(0x0)),
original_state: InRecv(None)
},
...
...
To see a task's registers, use the -r
flag:
% humility tasks -r user_leds
humility: attached via ST-Link
system time = 1990498
ID TASK GEN PRI STATE
6 user_leds 0 2 recv
|
+---> R0 = 0x20005fc8 R1 = 0x0000000c R2 = 0x00000000 R3 = 0x20005fd8
R4 = 0x20005fc8 R5 = 0x0000000c R6 = 0x00000000 R7 = 0x00000000
R8 = 0x08027154 R9 = 0x00000000 R10 = 0xfffffe00 R11 = 0x00000001
R12 = 0x00000000 SP = 0x20005fa0 LR = 0x08026137 PC = 0x08026e42
To see a task's stack backtrace, use the -s
flag:
% humility tasks -s user_leds
humility: attached via ST-Link
system time = 2021382
ID TASK GEN PRI STATE
6 user_leds 0 2 recv
|
+---> 0x20005fc0 0x08026e42 userlib::sys_recv_stub
0x20006000 0x08026128 userlib::sys_recv
0x20006000 0x08026128 idol_runtime::dispatch
0x20006000 0x08026136 main
To additionally see line number information on a stack backtrace, also provide
-l
flag:
% humility tasks -sl user_leds
humility: attached via ST-Link
system time = 2049587
ID TASK GEN PRI STATE
6 user_leds 0 2 recv
|
+---> 0x20005fc0 0x08026e42 userlib::sys_recv_stub
@ /home/bmc/hubris/sys/userlib/src/lib.rs:288
0x20006000 0x08026128 userlib::sys_recv
@ /home/bmc/hubris/sys/userlib/src/lib.rs:236
0x20006000 0x08026128 idol_runtime::dispatch
@ /home/bmc/.cargo/git/checkouts/idolatry-1ebf1c2fd2f30300/6d18e14/runtime/src/lib.rs:137
0x20006000 0x08026136 main
@ /home/bmc/hubris/drv/user-leds/src/main.rs:110
These options can naturally be combined, e.g. humility tasks -slvr
.
When run against a test archive, humility test
kicks off the test suite
and parses its results via ITM.
humility: attached via ST-Link
humility: ITM synchronization packet found at offset 6
humility: expecting 22 cases
humility: running test_send ... ok
humility: running test_recv_reply ... ok
humility: running test_fault_badmem ... ok
humility: running test_fault_stackoverflow ... ok
humility: running test_fault_execdata ... ok
humility: running test_fault_illop ... ok
humility: running test_fault_nullexec ... ok
humility: running test_fault_textoob ... ok
humility: running test_fault_stackoob ... ok
humility: running test_fault_buserror ... ok
humility: running test_fault_illinst ... ok
humility: running test_fault_divzero ... ok
humility: running test_panic ... ok
humility: running test_restart ... ok
humility: running test_restart_taskgen ... ok
humility: running test_borrow_info ... ok
humility: running test_borrow_read ... ok
humility: running test_borrow_write ... ok
humility: running test_supervisor_fault_notification ... ok
humility: running test_timer_advance ... ok
humility: running test_timer_notify ... ok
humility: running test_timer_notify_past ... ok
humility: tests completed: pass
If a test fails, this will also create a complete report, e.g.:
humility: attached via ST-Link
humility: ITM synchronization packet found at offset 6
humility: expecting 22 cases
humility: running test_send ... ok
humility: running test_recv_reply ... fail
humility: running test_fault_badmem ... ok
...
humility: running test_timer_notify_past ... ok
humility: tests completed: fail
humility: test output dumped to hubris.testout.15
This output file will have (among other things) a section that has
complete test run information. For details on a failing test, look
for result: Fail
:
$ cat hubris.testout.15
...
==== Test results
[
...
TestCompletion {
case: "test_recv_reply",
result: Fail,
log: [
(
UserLog,
"assistant starting",
),
(
KernelLog,
"task @1 panicked: panicked at \'assertion failed: false\', test/test-suite/src/main.rs:124:5",
),
(
UserLog,
"Task #1 Panic!",
),
(
UserLog,
"assistant starting",
),
],
},
...
This shows the sequential ordering of all log messages while running the
test. The test report can also be useful even when tests pass; to always
dump a test report, use the -d
option to humility test
.
Note that humility test
relies on the ability to keep up with ITM data,
which can be lossy. In the event ITM data is lost, the failure mode is
unlikely to be a failing test, but rather a fatal error due to a misframed
packet:
humility: running test_fault_nullexec ... ok
humility: running test_fault_textoob ... ok
humility: running test_fault_stackoob ... humility: test output dumped to hubris.testout.21
humility: test failed: malformed datum: 0x74
Error: test failed
All received packet data will be dumped to the resulting output file, allowing these transient failures to be differentiated from deeper issues.
No documentation yet for humility trace
; pull requests welcome!
humility validate
uses the Hubris validate
task to validate the
correct presence of devices as described by the application TOML. To
view all devices, use the --list
option; to validate them, run
without any additional arguments:
% humility validate
humility: attached via ST-Link V3
ID VALIDATION C P MUX ADDR DEVICE DESCRIPTION
0 removed 2 F - 0x48 tmp117 Southwest temperature sensor
1 removed 2 F - 0x49 tmp117 South temperature sensor
2 removed 2 F - 0x4a tmp117 Southeast temperature sensor
3 present 2 F - 0x70 pca9545 U.2 ABCD mux
4 present 2 F - 0x71 pca9545 U.2 EFGH mux
5 present 2 F - 0x72 pca9545 U.2 IJ/FRUID mux
6 timeout 2 B - 0x73 pca9545 M.2 mux
7 timeout 2 B 1:4 0x4c tmp451 T6 temperature sensor
8 validated 3 H - 0x24 tps546b24a A2 3.3V rail
9 validated 3 H - 0x26 tps546b24a A0 3.3V rail
10 validated 3 H - 0x27 tps546b24a A2 5V rail
11 validated 3 H - 0x29 tps546b24a A2 1.8V rail
12 present 3 H - 0x3a max5970 M.2 hot plug controller
13 absent 3 H - 0x4c sbtsi CPU temperature sensor
14 present 3 H - 0x58 idt8a34003 Clock generator
15 validated 3 H - 0x5a raa229618 CPU power controller
16 validated 3 H - 0x5b raa229618 SoC power controller
17 validated 3 H - 0x5c isl68224 DIMM/SP3 1.8V A0 power controller
18 validated 4 F - 0x10 adm1272 Fan hot swap controller
19 validated 4 F - 0x14 adm1272 Sled hot swap controller
20 validated 4 F - 0x20 max31790 Fan controller
21 validated 4 F - 0x25 tps546b24a T6 power controller
22 removed 4 F - 0x48 tmp117 Northeast temperature sensor
23 removed 4 F - 0x49 tmp117 North temperature sensor
24 validated 4 F - 0x4a tmp117 Northwest temperature sensor
25 validated 4 F - 0x67 bmr491 Intermediate bus converter
No documentation yet for humility vsc7448
; pull requests welcome!