ThordekkCore/COMPLETE_DEADLOCK_AUDIT.md

# COMPLETE DEADLOCK AUDIT - All Map Access Points from Worker Threads

## EXECUTION FLOW CONFIRMATION

**UpdateAI (line 314) → UpdateManagers (line 502) → ALL MANAGER UPDATE CALLS**

This means **ALL** managers listed below run on **WORKER THREADS** from the thread pool!

## CRITICAL FINDINGS

### Total ObjectAccessor Calls from Worker Threads: **31 calls**

| System | Files | ObjectAccessor Calls | Severity |
|--------|-------|---------------------|----------|
| **TargetScanner** | 1 | ✅ **FIXED** (was 3) | CRITICAL (was main bottleneck) |
| **ClassAI** | 2 | ✅ **PARTIALLY FIXED** (5 remaining in helpers) | HIGH |
| **QuestManager** | 3 | ❌ **11 calls** | CRITICAL |
| **GatheringManager** | 1 | ❌ **5 calls** | CRITICAL |
| **Strategy Systems** | 3 | ❌ **10 calls** | HIGH |

---

## 1. TargetScanner (FIXED ✅)

**File**: `src/modules/Playerbot/AI/Combat/TargetScanner.cpp`

**Status**: COMPLETELY FIXED - Returns GUIDs, BotAI resolves on main thread

**Previous Calls**:
- Line 328: `ObjectAccessor::GetUnit()` in FindAllHostiles() - **ELIMINATED**
- Line 183-264: FindBestTarget() - **ELIMINATED**
- Line 128-181: FindNearestHostile() - **ELIMINATED**

**Impact**: MASSIVE - Called hundreds of times per second across thousands of bots

---

## 2. ClassAI Systems (PARTIALLY FIXED ⚠️)

### UnholySpecialization.cpp (5 errors remaining)

**File**: `src/modules/Playerbot/AI/ClassAI/DeathKnights/UnholySpecialization.cpp`

**Status**: Main methods fixed, helper methods still broken

**Fixed**:
- ✅ Line 1555: UpdateGhoulManagement() - Now accepts target parameter
- ✅ Line 1603: CommandGhoulIfNeeded() - Uses target parameter

**Remaining Errors** (need target parameter passed through):
- ❌ Line 1085: HandleEmergencySurvival() - Uses `target` without parameter
- ❌ Line 1182: UpdateCombatPhase() - Uses `target` without parameter
- ❌ Line 1185: UpdateCombatPhase() - Uses `target` without parameter
- ❌ Line 1212: UpdateCombatPhase() - Uses `target` without parameter (2x)

### EvokerAI.cpp (FIXED ✅)

**File**: `src/modules/Playerbot/AI/ClassAI/Evokers/EvokerAI.cpp`

**Status**: COMPLETELY FIXED
- ✅ Line 656: UpdateEssenceManagement() - Now accepts target parameter

---

## 3. QuestManager Systems (NOT FIXED ❌)

**Execution Context**: Called from `BotAI::UpdateAI()` → `UpdateManagers()` line 1826
**Runs On**: WORKER THREADS via thread pool
**Severity**: CRITICAL - Quest operations are frequent

### QuestCompletion.cpp (6 calls)

**File**: `src/modules/Playerbot/Quest/QuestCompletion.cpp`

```cpp
Line 467:  Creature* creature = ObjectAccessor::GetCreature(*bot, guid);
Line 577:  Creature* creature = ObjectAccessor::GetCreature(*bot, guid);
Line 827:  auto* entity = ObjectAccessor::GetCreature(*bot, guid);
Line 902:  auto* entity = ObjectAccessor::GetCreature(*bot, guid);
Line 1003: auto* entity = ObjectAccessor::GetCreature(*bot, guid);
Line 1063: Creature* creature = ObjectAccessor::GetCreature(*bot, guid);
```

**Context**: Quest completion checks, turn-in validation, NPC interaction

### QuestPickup.cpp (3 calls)

**File**: `src/modules/Playerbot/Quest/QuestPickup.cpp`

```cpp
Line 242: Creature* creature = ObjectAccessor::GetCreature(*bot, ObjectGuid::Create<...>);
Line 248: GameObject* go = ObjectAccessor::GetGameObject(*bot, ObjectGuid::Create<...>);
Line 406: auto* entity = ObjectAccessor::GetCreature(*bot, guid);
```

**Context**: Quest giver detection, quest acceptance

### QuestTurnIn.cpp (2 calls)

**File**: `src/modules/Playerbot/Quest/QuestTurnIn.cpp`

```cpp
Line 400: Creature* creature = ObjectAccessor::GetCreature(*bot, guid);
```

**Context**: Quest turn-in NPC validation

---

## 4. GatheringManager (NOT FIXED ❌)

**Execution Context**: Called from `BotAI::UpdateAI()` → `UpdateManagers()` line 1842
**Runs On**: WORKER THREADS via thread pool
**Severity**: CRITICAL - Gathering operations are very frequent

### GatheringManager.cpp (5 calls)

**File**: `src/modules/Playerbot/Professions/GatheringManager.cpp`

```cpp
Line 242: target = ObjectAccessor::GetCreature(*GetBot(), node.guid);
          // In PerformGathering() - Skinning corpses

Line 260: GameObject* gameObject = ObjectAccessor::GetGameObject(*GetBot(), node.guid);
          // In PerformGathering() - Mining/Herbalism nodes

Line 604: Creature* creature = ObjectAccessor::GetCreature(*GetBot(), guid);
          // In ScanForSkinnableCorpses() - Finding skinnable mobs

Line 665: Creature* creature = ObjectAccessor::GetCreature(*GetBot(), node.guid);
          // In IsNodeValid() - Validating skinnable corpses

Line 670: GameObject* gameObject = ObjectAccessor::GetGameObject(*GetBot(), node.guid);
          // In IsNodeValid() - Validating gathering nodes
```

**Context**: Mining, Herbalism, Skinning - All professions that scan/interact with world objects

---

## 5. Strategy Systems (NOT FIXED ❌)

**Execution Context**: Strategies are executed during BotAI::UpdateAI() trigger processing
**Runs On**: WORKER THREADS via thread pool
**Severity**: HIGH - Combat/movement strategies run frequently

### CombatMovementStrategy.cpp (1 call)

**File**: `src/modules/Playerbot/AI/Strategy/CombatMovementStrategy.cpp`

```cpp
Line 417: Creature* creature = ObjectAccessor::GetCreature(*player, guid);
```

**Context**: Combat positioning, kiting, range management

### LootStrategy.cpp (6 calls)

**File**: `src/modules/Playerbot/AI/Strategy/LootStrategy.cpp`

```cpp
Line 286: Creature* creature = ObjectAccessor::GetCreature(*bot, corpseGuid);
          // Finding lootable corpses

Line 324: GameObject* object = ObjectAccessor::GetGameObject(*bot, objectGuid);
          // Finding lootable containers

Line 394: objA = ObjectAccessor::GetCreature(*bot, a);
          // Loot priority comparison

Line 396: objA = ObjectAccessor::GetGameObject(*bot, a);
          // Loot priority comparison

Line 399: objB = ObjectAccessor::GetCreature(*bot, b);
          // Loot priority comparison

Line 401: objB = ObjectAccessor::GetGameObject(*bot, b);
          // Loot priority comparison
```

**Context**: Loot scanning, priority calculation, corpse/container detection

### QuestStrategy.cpp (3 calls)

**File**: `src/modules/Playerbot/AI/Strategy/QuestStrategy.cpp`

```cpp
Line 875:  GameObject* go = ObjectAccessor::GetGameObject(*bot, ObjectGuid::Create<...>);
           // Quest objective interaction

Line 1375: ::Unit* target = ObjectAccessor::GetUnit(*bot, ObjectGuid::Create<...>);
           // Quest target validation

Line 1456: GameObject* gameObject = ObjectAccessor::GetGameObject(*bot, ObjectGuid::Create<...>);
           // Quest object interaction
```

**Context**: Quest objective tracking, interaction logic

---

## DEADLOCK MECHANISM

### How These Cause Deadlocks

1. **BotAI::UpdateAI()** runs on worker threads from thread pool
2. **UpdateManagers()** calls QuestManager, GatheringManager, etc. on worker threads
3. **Strategy execution** happens during trigger processing on worker threads
4. All **ObjectAccessor::Get*** methods access **Map::_objectsStore**
5. **Map::_objectsStore** is an unprotected **std::unordered_map** (NO MUTEXES)
6. Multiple worker threads accessing the same unprotected data structure → **DEADLOCK**

### Why Futures 3-14 Never Complete

With 5000+ bots:
- Thousands of worker threads executing UpdateAI() simultaneously
- Each thread calling QuestManager, GatheringManager, and Strategies
- Each manager making 1-11 ObjectAccessor calls
- **Total concurrent Map access**: 5000 bots × ~20 calls = ~100,000 concurrent Map accesses
- Map has NO protection → undefined behavior → futures hang forever

---

## FIX STRATEGY

### Pattern (Based on Successful TargetScanner Fix)

#### BEFORE (DEADLOCK):
```cpp
void SomeManager::Update(uint32 diff)
{
    // Running on worker thread!
    Creature* npc = ObjectAccessor::GetCreature(*bot, guid);  // DEADLOCK!
    if (npc)
        DoSomething(npc);
}
```

#### AFTER (THREAD-SAFE):
```cpp
void SomeManager::Update(uint32 diff)
{
    // Running on worker thread!
    ObjectGuid npcGuid = FindNPC();  // Use spatial grid snapshots

    if (!npcGuid.IsEmpty())
    {
        // Queue action with GUID - main thread will resolve
        sBotActionMgr->QueueAction(BotAction::InteractNPC(
            bot->GetGUID(),
            npcGuid,
            getMSTime()
        ));
    }
}
```

### Implementation Steps

1. **QuestManager Systems** (11 calls):
   - Change methods to work with GUIDs from spatial grid
   - Queue ACCEPT_QUEST/TURN_IN_QUEST/INTERACT_NPC actions
   - Main thread resolves GUIDs → pointers safely

2. **GatheringManager** (5 calls):
   - Change PerformGathering to accept GUID parameter
   - Change ScanForSkinnableCorpses to return GUIDs
   - Change IsNodeValid to accept GUID parameter
   - Queue INTERACT_OBJECT/LOOT_OBJECT actions

3. **Strategy Systems** (10 calls):
   - CombatMovementStrategy: Use position data from snapshots
   - LootStrategy: Return GUIDs, queue LOOT_OBJECT actions
   - QuestStrategy: Queue INTERACT_OBJECT/INTERACT_NPC actions

4. **UnholySpecialization Helper Methods** (5 remaining):
   - Pass target parameter through entire call chain
   - HandleEmergencySurvival(::Unit* target)
   - UpdateCombatPhase(::Unit* target)

---

## PRIORITY ORDER

### IMMEDIATE (Block Compilation):
1. ✅ Fix UnholySpecialization compilation errors (5 errors)

### CRITICAL (High Frequency):
2. ❌ Fix GatheringManager (5 calls) - Runs every frame for gathering bots
3. ❌ Fix QuestManager (11 calls) - Runs every frame for questing bots

### HIGH (Frequent):
4. ❌ Fix LootStrategy (6 calls) - Runs after every combat
5. ❌ Fix QuestStrategy (3 calls) - Runs during quest execution

### MEDIUM:
6. ❌ Fix CombatMovementStrategy (1 call) - Combat positioning

---

## EXPECTED RESULTS AFTER COMPLETE FIX

### With ALL 31 ObjectAccessor Calls Eliminated:

✅ **Zero Map access from worker threads**
✅ **100% lock-free spatial grid queries**
✅ **All GUID resolution on main thread only**
✅ **Action queue handles all Map-touching operations**

### Testing Targets:

- [ ] Build succeeds (fix UnholySpecialization errors first)
- [ ] Spawn 100 bots - verify futures complete
- [ ] Spawn 1000 bots - verify futures complete
- [ ] Spawn 5000 bots - **CRITICAL TEST: futures 3-14 MUST complete**
- [ ] Monitor for 60-second hangs - **MUST BE ZERO**
- [ ] Verify quest operations work
- [ ] Verify gathering operations work
- [ ] Verify combat rotations work
- [ ] Check server logs for deadlock warnings

---

## NEXT ACTIONS

1. **Fix UnholySpecialization compilation errors** (blocking build)
2. **Create automated refactoring script** for all 31 ObjectAccessor calls
3. **Implement fixes system-by-system** following TargetScanner pattern
4. **Build and test** with increasing bot counts (100 → 1000 → 5000)
5. **Verify futures 3-14 complete** with 5000+ bots

---

## SUMMARY

**CRITICAL DEADLOCK SOURCES IDENTIFIED**: 31 ObjectAccessor calls from worker threads

**SYSTEMS AFFECTED**: Quest (11), Gathering (5), Loot (6), ClassAI (5), Quest Strategy (3), Combat Movement (1)

**ROOT CAUSE**: All managers run on worker threads via BotAI::UpdateAI() → UpdateManagers()

**FIX PATTERN**: Return GUIDs instead of pointers, queue actions, resolve on main thread

**EXPECTED OUTCOME**: 100% deadlock elimination, futures 3-14 complete with 5000+ bots