Building a Fully Local AI Advisor for Civilization 7

Civilization 7 is a turn-based strategy game where you build an empire from ancient settlements to a modern nation. The decision space at any given turn spans hundreds of interacting variables: three eras (Antiquity through Modern) each with roughly 30-node tech and civic trees, government unlocks, city build queues, unit production, wonder timing, 25+ leaders with unique mechanics, five diverging victory paths, resource management, diplomacy, religion, and age transitions with crisis windows. Keeping it all in your head is the point of the game. Getting genuinely useful advice from an AI requires the system to actually know all of it, accurately.

The precision requirement is what makes this a real engineering problem. "Build Borobudur at some point" is useless advice. "Queue Borobudur at the start of Exploration era on a coastal tile, it gives +2 Food and +2 Happiness on all Quarters empire-wide, netting your specialist upkeep to zero" is actually useful. Getting there requires exact yields, prerequisite chains, era timing, and mechanic interactions. General-purpose LLMs have seen wiki content, but imprecisely. Tech trees from Civ 5 and 6 pattern-match onto Civ 7, leader abilities blur together, building yields skew toward the more impressive number. A hallucinating knowledge base produces confident noise.

The fix is a verified local knowledge base paired with retrieval. But verification is agent-intensive: generating 16 sections, running multi-round adversarial correction passes, cross-checking consistency. On cloud APIs, 50 agents costs $30-50 and you think hard before running it. On local hardware it costs electricity, so you run the workflow five times while tuning prompts instead of once hoping it's right. That difference is what this project runs on.

The constraint that shaped everything: Civ 7 uses 7-10 GB of VRAM on an RTX 5090 (32 GB total). The model I'm running, Qwen3.6-27B with a custom quantization recipe, needs about 24 GB once you include the KV cache for a 150K-token context window. That's tight. It works because the game usually sits around 7-8 GB, but at peak (10 GB) there's almost no margin.

For vision I went cloud: Claude Sonnet 5 via API, about $0.003 per screenshot. Over a full session (50-80 captures across 200 turns) that's about a dollar. Vision extraction at quality is the one piece I couldn't replicate locally. If the JSON is wrong, everything downstream is wrong.

How it works

Press a hotkey mid-game, get advice in about 20 seconds.

Why local inference changes the workflow

None of it was written by hand. A multi-agent pipeline generated the first drafts, and the first drafts were not reliable. I assumed AI-generated knowledge for a well-documented game would be accurate (there's tons of wiki content, the models have clearly seen it). They had not absorbed it cleanly:

• Hallucinated techs: "Astrology", "Archery", and "Mining" listed as Antiquity techs. None exist in Civ 7. They sound right because they existed in older games.
• Wrong era placement: Astronomy placed in Antiquity when it's actually Exploration. Pattern-matched from Civ 6's tech tree.
• Wrong yields: building yields off by 1-2, consistently biased toward the more impressive number.
• Mixed leader abilities: two different leaders' unique abilities swapped under the wrong names.
• Missing costs: no mention of the -2 Food / -2 Happiness per specialist. The advisor then tells you to stack six specialists and can't explain why your city is starving.

What made this tractable: running local models means you don't pay per token. A 50-agent verification pipeline on cloud APIs would cost $30-50 and require real deliberation about whether to run it. Running it locally costs electricity. I ran it five times while iterating on prompts. That changes how you build. You stop optimizing for fewer calls and start thinking about what the right workflow actually is. Web searches parallelized into verification pipelines, multi-source fact-checking, adversarial review passes, all of this becomes default practice when you're not metering tokens. Local inference doesn't just save money, it changes which workflows you're willing to run at all.

Inside the pipeline

What it produces

From a real session. Himiko / Maya Empire, Antiquity era, Turn 47, asking about specialist placement and Borobudur timing:

TURN 47 ADVISORY — Himiko / Maya Empire / Antiquity Era
Game State: 3 cities, 847 gold (+19/t), 12 sci/turn, 8 culture/turn
Research: Mining (68%) → next: Bronze Working
Age Progress: 76% — crisis window approaching

SPECIALIST MECHANICS

Per specialist on a Quarter:
  Base:       +2 Science, +2 Culture
  Adjacency:  +HALF of all adjacency bonuses on that tile
  Cost:       -2 Food, -2 Happiness per specialist

Science Quarter (University + Observatory):
  Base yield:           ~8-12 Science/turn
  Adjacency (6 tiles):  ~6 Science
  Specialist adds:      +5/turn (base + half adjacency)
  3 cities × 3 slots:   135-180+ Science/turn from specialists alone

BOROBUDUR (Exploration Wonder, Coast tile):
  +2 Food AND +2 Happiness on ALL Quarters empire-wide.
  Net specialist upkeep: ZERO.
  Queue it at the start of Exploration era.

CAPITAL BUILD ORDER:
  T47: Borobudur (4 turns at current production)
  T51: Complete → slot 2 scholars immediately
  T53: Bronze Working → 3rd specialist slot available
  T55: Artist slot if culture is secondary win path

Net after Borobudur + 3 specialists:
  Science: +12 → +21/turn
  Culture: +3/turn with 1 Artist

Age at 76%. ~10-15 turns before transition.
Complete Masteries for Codex — targeting 10 for Science Golden Age.

Specific yields, actual turn numbers, cost math that adds up. The specialist upkeep line is the kind of thing that only shows up in a well-verified knowledge base. Without it, the advisor stacks specialists and can't explain why the city starves.

Full automation

The current system is advisory-only. The next step is executing moves: the model emits structured commands, a second AHK layer translates them to actual inputs. The VRAM math already works (model + game fit within 32 GB with headroom). The state management system already logs decisions and outcomes per turn.

The mechanical side is straightforward. AutoHotkey v2 can watch pixel colors at known coordinates to detect when the "End Turn" button state flips, triggering an automatic capture cycle. Wiring structured model output to keyboard/mouse inputs is plumbing.

The hard part is UI state detection. Civ 7 has 15+ distinct UI states (main map, tech tree overlay, combat result dialog, narrative event card, age transition, trade popup) and each accepts different inputs. Issuing a move command while a combat result dialog is on screen does something completely wrong. I need a classifier that identifies the current UI state before any input, reliable enough to trust in an automated loop. That's the unsolved piece.