Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- # Sanctum EDR process/thread enumeration crashdump analysis.
- ## Links
- https://github.com/0xflux/Sanctum/tree/main/crashdmp
- https://github.com/0xflux/Sanctum/commit/80611184c10262193d14077ea9144e1899be53b8
- https://x.com/sixtyvividtails/status/1931449604156920261
- ## Analysis
- Microsoft (R) Windows Debugger Version 10.0.27553.1004 AMD64
- Copyright (c) Microsoft Corporation. All rights reserved.
- Loading Dump File [C:\stuff\analysis\dumps\6E73FC25-5CAF-4678-B594-270CA255FAE6\060725-11406-01.dmp]
- Mini Kernel Dump File: Only registers and stack trace are available
- Windows 10 Kernel Version 26100 MP (12 procs) Free x64
- Product: WinNt, suite: TerminalServer SingleUserTS
- Kernel base = 0xfffff802`cdc00000 PsLoadedModuleList = 0xfffff802`ceaf4770
- Debug session time: Sat Jun 7 13:56:29.699 2025 (UTC - 7:00)
- System Uptime: 0 days 3:52:37.985
- For analysis of this file, run !analyze -v
- nt!KeBugCheckEx:
- fffff802`ce0b8b00 48894c2408 mov qword ptr [rsp+8],rcx ss:0018:fffff607`c4792120=0000000000000050
- ✨✨✨✨✨✨✨✨
- 1: kd> .sympath+ C:\stuff\analysis\dumps\6E73FC25-5CAF-4678-B594-270CA255FAE6
- ✨✨✨✨✨✨✨✨
- 1: kd> !analyze -v
- *******************************************************************************
- * *
- * Bugcheck Analysis *
- * *
- *******************************************************************************
- PAGE_FAULT_IN_NONPAGED_AREA (50)
- Invalid system memory was referenced. This cannot be protected by try-except.
- Typically the address is just plain bad or it is pointing at freed memory.
- Arguments:
- Arg1: fffffffffffffa8b, memory referenced.
- Arg2: 0000000000000000, X64: bit 0 set if the fault was due to a not-present PTE.
- bit 1 is set if the fault was due to a write, clear if a read.
- bit 3 is set if the processor decided the fault was due to a corrupted PTE.
- bit 4 is set if the fault was due to attempted execute of a no-execute PTE.
- - ARM64: bit 1 is set if the fault was due to a write, clear if a read.
- bit 3 is set if the fault was due to attempted execute of a no-execute PTE.
- Arg3: fffff8026410aeb8, If non-zero, the instruction address which referenced the bad memory
- address.
- Arg4: 0000000000000002, (reserved)
- Debugging Details:
- ------------------
- KEY_VALUES_STRING: 1
- Key : AV.Type
- Value: Read
- Key : Analysis.CPU.mSec
- Value: 5624
- Key : Analysis.Elapsed.mSec
- Value: 5625
- Key : Analysis.IO.Other.Mb
- Value: 2
- Key : Analysis.IO.Read.Mb
- Value: 1
- Key : Analysis.IO.Write.Mb
- Value: 0
- Key : Analysis.Memory.CommitPeak.Mb
- Value: 131
- Key : Bugcheck.Code.LegacyAPI
- Value: 0x50
- Key : Bugcheck.Code.TargetModel
- Value: 0x50
- Key : Dump.Attributes.AsUlong
- Value: 20008
- Key : Dump.Attributes.KernelGeneratedTriageDump
- Value: 1
- Key : Failure.Bucket
- Value: AV_R_(null)_sanctum!sanctum::alt_syscalls::AltSyscalls::configure_thread_for_alt_syscalls
- Key : Failure.Hash
- Value: {4faa5bf0-adf1-10af-ad55-e0dd0fdacc12}
- Key : Hypervisor.Enlightenments.ValueHex
- Value: 10fbe4
- BUGCHECK_CODE: 50
- BUGCHECK_P1: fffffffffffffa8b
- BUGCHECK_P2: 0
- BUGCHECK_P3: fffff8026410aeb8
- BUGCHECK_P4: 2
- FILE_IN_CAB: 060725-11406-01.dmp
- VIRTUAL_MACHINE: HyperV
- DUMP_FILE_ATTRIBUTES: 0x20008
- Kernel Generated Triage Dump
- READ_ADDRESS: Unable to get NonPagedPoolStart
- Unable to get NonPagedPoolEnd
- Unable to get PagedPoolStart
- Unable to get PagedPoolEnd
- fffffffffffffa8b
- MM_INTERNAL_CODE: 2
- IMAGE_NAME: sanctum.sys
- MODULE_NAME: sanctum
- FAULTING_MODULE: fffff802640d0000 sanctum
- BLACKBOXBSD: 1 (!blackboxbsd)
- BLACKBOXNTFS: 1 (!blackboxntfs)
- BLACKBOXPNP: 1 (!blackboxpnp)
- BLACKBOXWINLOGON: 1
- CUSTOMER_CRASH_COUNT: 1
- PROCESS_NAME: System
- TRAP_FRAME: fffff607c4792380 -- (.trap 0xfffff607c4792380)
- NOTE: The trap frame does not contain all registers.
- Some register values may be zeroed or incorrect.
- rax=fffffffffffffa88 rbx=0000000000000000 rcx=fffffffffffffa88
- rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000
- rip=fffff8026410aeb8 rsp=fffff607c4792510 rbp=fffff607c47958c0
- r8=ffffe58b2efb5a50 r9=00000000000000c2 r10=ffffe58b2d100140
- r11=ffff8481d4b83000 r12=0000000000000000 r13=0000000000000000
- r14=0000000000000000 r15=0000000000000000
- iopl=0 nv up ei pl zr na po nc
- sanctum!sanctum::alt_syscalls::AltSyscalls::configure_thread_for_alt_syscalls+0x28:
- fffff802`6410aeb8 8a4003 mov al,byte ptr [rax+3] ds:ffffffff`fffffa8b=??
- Resetting default scope
- STACK_TEXT:
- fffff607`c4792118 fffff802`cdf80bb5 : 00000000`00000050 ffffffff`fffffa8b 00000000`00000000 fffff607`c4792380 : nt!KeBugCheckEx
- fffff607`c4792120 fffff802`cde2daaf : ffffffff`fffffa8b 00000000`00001000 00000000`00000002 fffff802`cdc00000 : nt!MiSystemFault+0x735
- fffff607`c4792210 fffff802`ce2821cb : 00000020`7f9e908e fffff607`c4792400 00000020`7f9e9097 00001acf`af9dafc6 : nt!MmAccessFault+0x2ff
- fffff607`c4792380 fffff802`6410aeb8 : fffff802`6410f260 fffff607`c47926f0 ffffe58b`389940c0 ffffe58b`386460c0 : nt!KiPageFault+0x38b
- fffff607`c4792510 fffff802`6410ab86 : fffff802`cee01d40 00000000`007916d2 fffff802`ce67066e fffff802`cdc00000 : sanctum!sanctum::alt_syscalls::AltSyscalls::configure_thread_for_alt_syscalls+0x28 [C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs @ 174]
- fffff607`c4792560 fffff802`64109d05 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : sanctum!sanctum::alt_syscalls::AltSyscalls::walk_active_processes_and_set_bits+0x356 [C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs @ 322]
- fffff607`c4792830 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : sanctum!sanctum::alt_syscalls::AltSyscalls::initialise_for_system+0x7c5 [C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs @ 161]
- FAULTING_SOURCE_LINE: C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs
- FAULTING_SOURCE_FILE: C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs
- FAULTING_SOURCE_LINE_NUMBER: 174
- FAULTING_SOURCE_CODE:
- No source found for 'C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs'
- SYMBOL_NAME: sanctum!sanctum::alt_syscalls::AltSyscalls::configure_thread_for_alt_syscalls+28
- STACK_COMMAND: .cxr; .ecxr ; kb
- BUCKET_ID_FUNC_OFFSET: 28
- FAILURE_BUCKET_ID: AV_R_(null)_sanctum!sanctum::alt_syscalls::AltSyscalls::configure_thread_for_alt_syscalls
- OSPLATFORM_TYPE: x64
- OSNAME: Windows 10
- FAILURE_ID_HASH: {4faa5bf0-adf1-10af-ad55-e0dd0fdacc12}
- Followup: MachineOwner
- ---------
- ✨✨✨✨✨✨✨✨
- 1: kd> .trap 0xfffff607c4792380
- NOTE: The trap frame does not contain all registers.
- Some register values may be zeroed or incorrect.
- rax=fffffffffffffa88 rbx=0000000000000000 rcx=fffffffffffffa88
- rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000
- rip=fffff8026410aeb8 rsp=fffff607c4792510 rbp=fffff607c47958c0
- r8=ffffe58b2efb5a50 r9=00000000000000c2 r10=ffffe58b2d100140
- r11=ffff8481d4b83000 r12=0000000000000000 r13=0000000000000000
- r14=0000000000000000 r15=0000000000000000
- iopl=0 nv up ei pl zr na po nc
- sanctum!sanctum::alt_syscalls::AltSyscalls::configure_thread_for_alt_syscalls+0x28:
- fffff802`6410aeb8 8a4003 mov al,byte ptr [rax+3] ds:ffffffff`fffffa8b=?? 💥💥💥
- ✨✨✨✨✨✨✨✨
- 1: kd> k
- *** Stack trace for last set context - .thread/.cxr resets it
- # Child-SP RetAddr Call Site
- 00 fffff607`c4792510 fffff802`6410ab86 sanctum!sanctum::alt_syscalls::AltSyscalls::configure_thread_for_alt_syscalls+0x28 [C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs @ 174]
- 01 fffff607`c4792560 fffff802`64109d05 sanctum!sanctum::alt_syscalls::AltSyscalls::walk_active_processes_and_set_bits+0x356 [C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs @ 322]
- 02 fffff607`c4792830 fffff802`640d1b59 sanctum!sanctum::alt_syscalls::AltSyscalls::initialise_for_system+0x7c5 [C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs @ 161]
- 03 fffff607`c4795440 fffff802`640d155e sanctum!sanctum::initialise_sanctum+0x19 [C:\Users\ian\git\sanctum\driver\src\lib.rs @ 126]
- 04 fffff607`c4795660 fffff802`ce5aef08 sanctum!sanctum::driver_entry+0x12e [C:\Users\ian\git\sanctum\driver\src\lib.rs @ 114]
- 05 fffff607`c4795770 fffff802`ce5ad0d2 nt!PnpCallDriverEntry+0x54
- 06 fffff607`c47957c0 fffff802`ce641b7b nt!IopLoadDriver+0x6c6
- 07 fffff607`c4795990 fffff802`cdf09e32 nt!IopLoadUnloadDriver+0x7b
- 08 fffff607`c4795a00 fffff802`ce05904a nt!ExpWorkerThread+0x1b2
- 09 fffff607`c4795bb0 fffff802`ce2741c4 nt!PspSystemThreadStartup+0x5a
- 0a fffff607`c4795c00 00000000`00000000 nt!KiStartSystemThread+0x34
- ✨✨✨✨✨✨✨✨
- 1: kd> ub .
- sanctum!sanctum::alt_syscalls::AltSyscalls::configure_thread_for_alt_syscalls+0x9 [C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs @ 165]:
- fffff802`6410ae99 80e201 and dl,1
- fffff802`6410ae9c 88542437 mov byte ptr [rsp+37h],dl
- fffff802`6410aea0 48894c2438 mov qword ptr [rsp+38h],rcx
- fffff802`6410aea5 e85661fcff call sanctum!core::ptr::mut_ptr::impl$0::is_null<wdk_mutex::grt::Grt> (fffff802`640d1000)
- fffff802`6410aeaa a801 test al,1
- fffff802`6410aeac 7513 jne sanctum!sanctum::alt_syscalls::AltSyscalls::configure_thread_for_alt_syscalls+0x31 (fffff802`6410aec1)
- fffff802`6410aeae 488b442428 mov rax,qword ptr [rsp+28h] 🌽🌽🌽
- fffff802`6410aeb3 4889442440 mov qword ptr [rsp+40h],rax 🛤️🛤️🛤️
- 1: kd> u .
- sanctum!sanctum::alt_syscalls::AltSyscalls::configure_thread_for_alt_syscalls+0x28 [C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs @ 174]:
- fffff802`6410aeb8 8a4003 mov al,byte ptr [rax+3] 💥💥💥 🛰️🛰️🛰️
- fffff802`6410aebb 2404 and al,4 🛰️🛰️🛰️
- fffff802`6410aebd 3c04 cmp al,4
- fffff802`6410aebf 7505 jne sanctum!sanctum::alt_syscalls::AltSyscalls::configure_thread_for_alt_syscalls+0x36 (fffff802`6410aec6)
- fffff802`6410aec1 4883c448 add rsp,48h
- fffff802`6410aec5 c3 ret
- fffff802`6410aec6 8a442437 mov al,byte ptr [rsp+37h]
- fffff802`6410aeca 2401 and al,1
- ✨✨✨✨✨✨✨✨
- 1: kd> dps @rsp+40 L1 🛤️🛤️🛤️
- fffff607`c4792550 ffffffff`fffffa88 🌋🌋🌋
- 1: kd> dps @rsp+28 L1 🌽🌽🌽
- fffff607`c4792538 ffffffff`fffffa88 🌋🌋🌋
- ✨✨✨✨✨✨✨✨
- 1: kd> dt nt!_DISPATCHER_HEADER Mi*
- +0x003 Minimal : Pos 2, 1 Bit 🛰️🛰️🛰️
- ✨✨✨✨✨✨✨✨
- 1: kd> uf sanctum!sanctum::alt_syscalls::AltSyscalls::configure_thread_for_alt_syscalls
- sanctum!sanctum::alt_syscalls::AltSyscalls::configure_thread_for_alt_syscalls [C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs @ 165]:
- 165 fffff802`6410ae90 4883ec48 sub rsp,48h
- 165 fffff802`6410ae94 48894c2428 mov qword ptr [rsp+28h],rcx 🌽🌽🌽
- 165 fffff802`6410ae99 80e201 and dl,1
- 165 fffff802`6410ae9c 88542437 mov byte ptr [rsp+37h],dl
- ✨✨✨✨✨✨✨✨
- 1: kd> k3
- # Child-SP RetAddr Call Site
- 00 fffff607`c4792510 fffff802`6410ab86 sanctum!sanctum::alt_syscalls::AltSyscalls::configure_thread_for_alt_syscalls+0x28 [C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs @ 174]
- 01 fffff607`c4792560 fffff802`64109d05 sanctum!sanctum::alt_syscalls::AltSyscalls::walk_active_processes_and_set_bits+0x356 [C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs @ 322]
- 02 fffff607`c4792830 fffff802`640d1b59 sanctum!sanctum::alt_syscalls::AltSyscalls::initialise_for_system+0x7c5 [C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs @ 161]
- ✨✨✨✨✨✨✨✨
- 1: kd> .frame /c /r 1
- 01 fffff607`c4792560 fffff802`64109d05 sanctum!sanctum::alt_syscalls::AltSyscalls::walk_active_processes_and_set_bits+0x356 [C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs @ 322]
- rax=fffffffffffffa88 rbx=0000000000000000 rcx=fffffffffffffa88
- rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000
- rip=fffff8026410ab86 rsp=fffff607c4792560 rbp=fffff607c47958c0
- r8=ffffe58b2efb5a50 r9=00000000000000c2 r10=ffffe58b2d100140
- r11=ffff8481d4b83000 r12=0000000000000000 r13=0000000000000000
- r14=0000000000000000 r15=0000000000000000
- iopl=0 nv up ei pl zr na po nc
- cs=0010 ss=0018 ds=0000 es=0000 fs=0000 gs=0000 efl=00050246
- sanctum!sanctum::alt_syscalls::AltSyscalls::walk_active_processes_and_set_bits+0x356:
- fffff802`6410ab86 488b4c2450 mov rcx,qword ptr [rsp+50h] ss:0018:fffff607`c47925b0=fffffffffffffa88
- ✨✨✨✨✨✨✨✨
- 1: kd> ub .
- sanctum!sanctum::alt_syscalls::AltSyscalls::walk_active_processes_and_set_bits+0x333 [C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs @ 295]:
- fffff802`6410ab63 480f44c1 cmove rax,rcx
- fffff802`6410ab67 4883f800 cmp rax,0
- fffff802`6410ab6b 7443 je sanctum!sanctum::alt_syscalls::AltSyscalls::walk_active_processes_and_set_bits+0x380 (fffff802`6410abb0)
- fffff802`6410ab6d e99a000000 jmp sanctum!sanctum::alt_syscalls::AltSyscalls::walk_active_processes_and_set_bits+0x3dc (fffff802`6410ac0c)
- fffff802`6410ab72 488b4c2450 mov rcx,qword ptr [rsp+50h] 🌽🌽🌽
- fffff802`6410ab77 8a94248f000000 mov dl,byte ptr [rsp+8Fh]
- fffff802`6410ab7e 80e201 and dl,1
- fffff802`6410ab81 e80a030000 call sanctum!sanctum::alt_syscalls::AltSyscalls::configure_thread_for_alt_syscalls (fffff802`6410ae90)
- 1: kd> u .
- sanctum!sanctum::alt_syscalls::AltSyscalls::walk_active_processes_and_set_bits+0x356 [C:\Users\ian\git\sanctum\driver\src\alt_syscalls.rs @ 322]:
- fffff802`6410ab86 488b4c2450 mov rcx,qword ptr [rsp+50h]
- fffff802`6410ab8b e870030000 call sanctum!sanctum::alt_syscalls::AltSyscalls::configure_process_for_alt_syscalls (fffff802`6410af00)
- fffff802`6410ab90 488b8424e8000000 mov rax,qword ptr [rsp+0E8h]
- fffff802`6410ab98 4889442448 mov qword ptr [rsp+48h],rax
- fffff802`6410ab9d 4883e007 and rax,7
- fffff802`6410aba1 4883f800 cmp rax,0
- fffff802`6410aba5 0f84af020000 je sanctum!sanctum::alt_syscalls::AltSyscalls::walk_active_processes_and_set_bits+0x62a (fffff802`6410ae5a)
- fffff802`6410abab e9c2020000 jmp sanctum!sanctum::alt_syscalls::AltSyscalls::walk_active_processes_and_set_bits+0x642 (fffff802`6410ae72)
- ✨✨✨✨✨✨✨✨
- 1: kd> dps @rsp+50 L1 🌽🌽🌽
- fffff607`c47925b0 ffffffff`fffffa88 🌋🌋🌋
- ✨✨✨✨✨✨✨✨
- ✨✨✨✨✨✨✨✨
- fn walk_active_processes_and_set_bits(
- status: AltSyscallStatus,
- isolated_processes: Option<&[&str]>,
- ) {
- ...
- let current_process = unsafe { IoGetCurrentProcess() };
- ...
- let head = unsafe { (current_process as *mut u8).add(ACTIVE_PROCESS_LINKS_OFFSET) } as *mut LIST_ENTRY;
- let mut entry = unsafe { (*head).Flink };
- while entry != head {
- ...
- let thread_head = unsafe { (p_e_process as *mut u8).add(THREAD_LIST_HEAD_OFFSET) } as *mut LIST_ENTRY;
- let mut thread_entry = unsafe { (*thread_head).Flink };
- while thread_entry != thread_head {
- ...
- // 🌽🌽🌽
- let p_k_thread = unsafe { (thread_entry as *mut u8).sub(THREAD_LIST_ENTRY_OFFSET) } as *mut _KTHREAD;
- ...
- Self::configure_thread_for_alt_syscalls(p_k_thread, status); // 💥💥💥
- ...
- thread_entry = unsafe { (*thread_entry).Flink };
- }
- ...
- entry = unsafe { (*entry).Flink }
- }
- ...
- }
- ✨✨✨✨✨✨✨✨
- ✨✨✨✨✨✨✨✨
- ✨✨✨✨✨✨✨✨
- ## Analysis results
- Enumerating processes and threads as done in `walk_active_processes_and_set_bits` is inherently unsafe. It's unsafe because code doesn't hold locks that protect process and thread lists (like `PspActiveProcessLock`). As a result, the OS might unlink objects from these lists and destroy them while the Sanctum module is still using these objects.
- This appears to be exactly what happened in this case. While Sanctum was processing a thread object, that thread got destroyed, and its memory was reused for something else. Consequently, `thread_entry.Flink` was overwritten with an unrelated value (numerically close to 🌋🌋🌋, ffffffff`fffffa88). Sanctum then retrieved that unrelated value to get to the next thread, and attempted to access that "next" thread. Naturally, dereferencing "thread" at ffffffff`fffffa88 caused page fault, and subsequent bugcheck.
- ## Suggestions
- The proper way to enumerate processes and threads would be using `PsGetNextProcess` and `PsGetNextThread`. However, since these functions aren't exported, you might consider `ZwGetNextProcess` and `ZwGetNextThread` instead. While slightly slower, these syscalls are safe — they provide handles to actually existing processes and threads with properly adjusted refcounts, preventing objects from being destroyed during enumeration.
- Alternative approach is to use `ZwQuerySystemInformation` with infoclasses like `SystemProcessInformation`, `SystemExtendedProcessInformation`, or `SystemFullProcessInformation`. This method has the advantage of retrieving a snapshot of all running processes through a single syscall. However, it only provides process and thread IDs (not handles nor object pointers), creating a slight race between snapshot retrieval and attempt to open a process, exacerbated by potential PID reuse. But you can mitigate this by comparing CreateTime or re-retrieving full information after you've opened the process of interest.
Advertisement
Add Comment
Please, Sign In to add comment