Phase 004 is not running the spec. Here’s exactly how.

Forensic findings, 2026-05-16. Same intake-tier gap as 001/002/003, plus 004-specific drift.

Finding 1 — Both 004 steps are tagged intake. Neither auto-fires.

pipeline/steps/cadence.py tags 004a and 004b as intake. The intake tier has runs_on: [] — never auto-fires. Same gap as the rest of intake.

Finding 2 — All three gold tables silently skip BQ when PIPELINE_RUN_ID isn’t set.

This is the third Phase in a row with the same gotcha. step_004b_join_gold.py at line 400 only writes gold_domains + gold_cids + gold_paid if PIPELINE_RUN_ID is in the environment:

if run_id and cid_records:
    ...write three gold tables...
elif not run_id:
    logger.info("BQ write skipped: no PIPELINE_RUN_ID (standalone mode)")

Every manual SSH fire of 004b since intake broke has silently skipped writing the gold tables. This is the highest-impact instance of the bug — gold_domains, gold_cids, gold_paid are the canonical inputs for every weekly/monthly/biweekly enrichment step in Phases 005-010.

Finding 3 — Specialty-key mismatch between 004a output and 004b’s search_keyword map.

004a’s fallback VALID_CATEGORIES set (line 76 of step_004a_specialties.py) uses long form like personal_injury_lawyer, family_law_attorney. 004b’s SPECIALTY_TO_SEARCH_KEYWORD map (line 147 of step_004b_join_gold.py) uses short form: personal_injury, family_law. These don’t match.

What actually happens at runtime depends on which list is authoritative. Two cases:

• If vertical_config.yaml uses short form (personal_injury) — the search_keyword map matches; the fallback hardcoded list in 004a is dead code that would never execute (because vertical config loads successfully and the fallback only kicks in when config is missing). The bug only surfaces if vertical config goes missing.
• If vertical_config.yaml uses long form (personal_injury_lawyer) — every domain ends up with search_keyword="lawyer" (the fallback default), losing all specialty-specific keyword routing. Downstream Phase 008/010 then fetches SERPs for generic “lawyer” for every firm regardless of practice area.

Unverified which case is real. Resolving this is Fix 2.

Finding 4 — The NON_V_OVERRIDE post-filter only catches 5 categories.

004b’s NON_VERTICAL_CATEGORIES set (line 123) is just 5 hardcoded GMaps categories: CPA, accountant, doctor, driving school, educational consultant. Any other lawyer-adjacent business that fooled 003c (e.g. mediator, paralegal services, legal aid clinic, court reporter) gets through into gold_domains as a confirmed firm.

This list is too short relative to the false-positive surface. Phase 003c is good but not perfect; the post-filter is the last line of defense and it has 5 entries.

Finding 5 — The category remap dict needs runtime measurement to confirm it’s catching real cases.

004a’s _CATEGORY_REMAP has 14 hardcoded mappings for categories Haiku tends to invent (wrongful_death_attorney → personal_injury_lawyer, etc.). Whether Haiku still invents these in current model versions is unverified. Some entries may be stale (mapping a category Haiku no longer produces) and some real invented categories may slip through unmapped.

The script does log every remap, so a single run’s log file would tell us: which remaps fire, which never fire, which invented categories show up that aren’t in the dict at all. That measurement has not been done.

Finding 6 — Without 004 completing, every downstream phase reads stale gold.

Phases 005-010 (weekly, monthly, biweekly) all read from gold_domains / gold_cids. If 004b never runs, those tables stop getting fresh rows for new run_ids. Downstream phases keep operating on the most recent successful gold-table run — which is whenever 004b last succeeded with PIPELINE_RUN_ID set.

That means every Monday/Thursday biweekly fire, every Monday weekly fire, every monthly fire is enriching the same fixed firm list. New firms that opened since the last successful 004b fire are invisible. Firms that closed are still being enriched as if they were live.

The bottom line