BEAM Crash Dumps Notes

I've been spending a lot of time with BEAM crash dumps lately, trying to debug something weird, and I noticed a few things about crash dump files that aren't actually captured in the ERTS documentation:

  1. You're going to miss binaries
    • For some reason, not all binaries are emitted when a crash dump is created. I'm not sure if this is because there's a time limit for dumping them, but you won't get all of them all the time.
  2. Not all proc heap addresses are captured (maybe?)
    • I've also noticed a few addresses in the =proc_heap sections aren't there. Not sure why, maybe I'm missing something.
  3. Unwinding the heap isn't hard, it's just annoying. The format for addresses is <CODE><ADDRESS>, and lines are <ADDRESS>:<TYPE>:<DATA> such as:

    proc_heap:<0.184.0> FFFF454355A8:t5:AE:callback_cache,AA:supervisor,HFFFF45435670,HFFFF45435698,HFFFF454356C0 FFFF454356C0:E1D:g3F3CnN1cGVydmlzb3J3C2hhbmRsZV9pbmZvYQI FFFF45435698:E1D:g3F3CnN1cGVydmlzb3J3C2hhbmRsZV9jYXN0YQI= FFFF45435670:E1D:g3F3CnN1cGVydmlzb3J3C2hhbmRsZV9jYWxsYQM= FFFF454355D8:tC:A5:state,HFFFF5529C8C8,A12:simpleoneforone,HFFFF454356E8,HFFFF552543A8,I3,I3600,N,I0,A5:never,A24:sslupgradeserversessioncachesup,N

    • So here FFFF454355A8 is an address, t5 is a tuple of size 5, tC is a tuple of size 12
    • AE is an atom of length 14, AA is an atom of length 10, and HFFFF is a heap reference
    • I3 is an integer value
    • N is a nil value
    • I think E1D is a reference to binary data? I'm not sure though

    I started https://github.com/peixian/erl-crash-dump-parser to have a better dump parser. The current wxwidgets based one in :observer drives me mad, so I figured it'd be a good learning experience. https://github.com/peixian/erl-crash-dump-parser/blob/main/internal/parser/types.go is probably the most interesting section though, since it attempts to accurately capture what's observable in a crash dump.

Posted: 2025-01-04
Filed Under: erlang