BEAM Crash Dumps Notes
I've been spending a lot of time with BEAM crash dumps lately, trying to debug something weird, and I noticed a few things about crash dump files that aren't actually captured in the ERTS documentation:
- You're going to miss binaries
- For some reason, not all binaries are emitted when a crash dump is created. I'm not sure if this is because there's a time limit for dumping them, but you won't get all of them all the time.
- Not all proc heap addresses are captured (maybe?)
- I've also noticed a few addresses in the
=proc_heap
sections aren't there. Not sure why, maybe I'm missing something.
- I've also noticed a few addresses in the
Unwinding the heap isn't hard, it's just annoying. The format for addresses is
<CODE><ADDRESS>
, and lines are<ADDRESS>:<TYPE>:<DATA>
such as:proc_heap:<0.184.0> FFFF454355A8:t5:AE:callback_cache,AA:supervisor,HFFFF45435670,HFFFF45435698,HFFFF454356C0 FFFF454356C0:E1D:g3F3CnN1cGVydmlzb3J3C2hhbmRsZV9pbmZvYQI
FFFF45435698:E1D:g3F3CnN1cGVydmlzb3J3C2hhbmRsZV9jYXN0YQI= FFFF45435670:E1D:g3F3CnN1cGVydmlzb3J3C2hhbmRsZV9jYWxsYQM= FFFF454355D8:tC:A5:state,HFFFF5529C8C8,A12:simpleoneforone,HFFFF454356E8,HFFFF552543A8,I3,I3600,N,I0,A5:never,A24:sslupgradeserversessioncachesup,N- So here FFFF454355A8 is an address, t5 is a tuple of size 5, tC is a tuple of size 12
AE
is an atom of length 14,AA
is an atom of length 10, andHFFFF
is a heap referenceI3
is an integer valueN
is a nil value- I think
E1D
is a reference to binary data? I'm not sure though
I started https://github.com/peixian/erl-crash-dump-parser to have a better dump parser. The current wxwidgets based one in
:observer
drives me mad, so I figured it'd be a good learning experience. https://github.com/peixian/erl-crash-dump-parser/blob/main/internal/parser/types.go is probably the most interesting section though, since it attempts to accurately capture what's observable in a crash dump.