custom-memory-heap-crash

Original source / Raw SKILL.md

---
name: custom-memory-heap-crash
description: Guide for debugging crashes related to custom memory heaps, particularly use-after-free issues caused by static destruction ordering, DEBUG vs RELEASE discrepancies, and custom allocator lifecycle problems. This skill should be used when investigating crashes that occur only in RELEASE builds, memory-related crashes involving custom allocators, static initialization/destruction order issues, or use-after-free bugs in C++ applications.
---

# Custom Memory Heap Crash Debugging

This skill provides systematic approaches for debugging crashes related to custom memory heaps, with emphasis on static destruction ordering issues, DEBUG vs RELEASE discrepancies, and memory lifecycle problems.

## Problem Recognition

Apply this skill when encountering:

- Crashes that occur only in RELEASE builds but not DEBUG builds
- Segmentation faults or access violations during program shutdown
- Use-after-free errors involving custom allocators
- Crashes in standard library code (locale, iostream, etc.) when custom heaps are involved
- Memory lifecycle issues during static destruction phase

## Investigation Approach

### Phase 1: Reproduce and Characterize

1. **Build both configurations**: Compile the application in both DEBUG and RELEASE modes to confirm the discrepancy
2. **Identify crash timing**: Determine if the crash occurs during:
   - Normal execution
   - Program shutdown (static destruction phase)
   - Library initialization
3. **Collect crash information**: Use GDB or equivalent debugger to obtain:
   - Full backtrace at crash point
   - Register values and memory state
   - The specific instruction causing the crash

### Phase 2: Understand Memory Lifecycle

1. **Map allocator lifecycle**: Document when custom heaps are:
   - Created (constructor timing)
   - Active (during main execution)
   - Destroyed (destructor timing)
2. **Identify static objects**: List all static/global objects that may allocate memory
3. **Trace allocation sources**: Determine which allocator (custom vs system) handles each allocation

### Phase 3: Analyze Static Destruction Order

1. **Review destruction sequence**: Static objects are destroyed in reverse order of construction
2. **Check cross-dependencies**: Identify objects that depend on the custom heap but may be destroyed after it
3. **Examine library internals**: Standard library components (locales, facets, streams) may register objects that outlive custom heaps

## Common Root Causes

### Use-After-Free During Static Destruction

**Pattern**: Objects allocated from a custom heap are accessed after the heap is destroyed.

**Symptoms**:
- Crash in destructor or cleanup code
- Backtrace shows standard library cleanup (e.g., locale facet destruction)
- Crash accesses memory that was valid earlier

**Investigation**:
- Check if standard library objects (locale facets, stream buffers) are allocated from the custom heap
- Verify the custom heap outlives all its allocations

### DEBUG vs RELEASE Allocation Differences

**Pattern**: Different allocation patterns between build configurations cause memory to come from different sources.

**Symptoms**:
- Works in DEBUG, crashes in RELEASE
- Memory addresses differ between configurations
- Conditional compilation affects allocator selection

**Investigation**:
- Examine preprocessor conditionals affecting memory allocation
- Check if DEBUG mode uses system allocator while RELEASE uses custom heap
- Look for `#ifdef DEBUG` or `#ifndef NDEBUG` blocks around allocation code

### Library-Internal Allocations

**Pattern**: Standard library internally allocates memory that gets routed through custom allocators.

**Symptoms**:
- Crash during library cleanup code
- Backtrace shows internal library functions
- No obvious user code involvement

**Investigation**:
- Trace which library operations trigger allocations
- Check if locale, iostream, or other library initializations use the custom heap
- Examine library source code if available

## Solution Strategies

### Strategy 1: Force Early Initialization

Trigger library initialization before the custom heap is created, ensuring library-internal allocations use the system allocator.

**Implementation approach**:
- Call library functions in `user_init()` or before heap creation
- For locale issues: instantiate `std::locale()` early
- For iostream issues: perform I/O operations early

### Strategy 2: Extend Heap Lifetime

Ensure the custom heap outlives all objects allocated from it.

**Implementation approach**:
- Use static local variables with guaranteed destruction order
- Implement explicit cleanup before heap destruction
- Consider lazy destruction or leaking the heap intentionally

### Strategy 3: Exclude Library Allocations

Prevent library-internal allocations from using the custom heap.

**Implementation approach**:
- Modify allocator selection logic to exclude certain allocation types
- Use thread-local flags during library initialization
- Implement allocation source tracking

## Verification Checklist

After implementing a fix:

1. **Build verification**: Confirm both DEBUG and RELEASE builds compile without errors
2. **Runtime verification**: Run both configurations without crashes
3. **Memory leak check**: Use Valgrind or equivalent to verify no memory leaks introduced
4. **Stress testing**: Run multiple iterations to catch intermittent issues
5. **Destruction order verification**: Confirm proper cleanup sequence with logging if needed

## Debugging Tools and Techniques

### GDB Commands for Memory Issues

```
# Get backtrace at crash
bt full

# Examine memory at address
x/16xg <address>

# Check if address is valid
info proc mappings

# Set breakpoint on destructor
break ClassName::~ClassName

# Watch memory location
watch *<address>
```

### Valgrind Usage

```
# Basic memory check
valgrind --leak-check=full ./program

# Track origins of uninitialized values
valgrind --track-origins=yes ./program

# Detect invalid reads/writes
valgrind --read-var-info=yes ./program
```

## Common Pitfalls

1. **Incomplete initialization**: Triggering partial library initialization may not allocate all necessary objects. Verify the specific code path that causes problematic allocations.

2. **Multiple initialization points**: Library components may be initialized from multiple code paths. Ensure all paths are covered.

3. **Thread safety assumptions**: Static initialization may involve thread-safety mechanisms that interact with custom allocators.

4. **Optimization effects**: Compiler optimizations may reorder or eliminate code that affects allocation timing.

5. **GDB command syntax**: When debugging, use proper quoting and escaping. Test commands interactively before scripting.

6. **Assuming single root cause**: Multiple allocation sources may contribute to the problem. Verify each fix addresses all crash scenarios.

## Decision Framework

When investigating, follow this systematic approach:

1. Can the crash be reproduced reliably? If not, add logging to capture crash state.
2. Is this a DEBUG vs RELEASE discrepancy? Check preprocessor conditionals.
3. Does the crash occur during shutdown? Focus on static destruction order.
4. Is library code involved? Investigate library initialization and cleanup.
5. Is memory being accessed after free? Trace the allocation source and lifetime.

Apply fixes incrementally and verify each change before proceeding.