Investigate: structured feedback classification as a cross-cutting protocol

[abeltrano]

Motivation

PR #166 added a feedback classification rule to the spec-extraction-workflow template (Phase 3) that requires the LLM to categorize user feedback as Editorial, Correction, or Finding — defaulting to Finding when uncertain. This prevented a real failure mode where domain-expert feedback was silently under-promoted to a cosmetic change.

The underlying problem — LLMs default to the lowest-impact interpretation of user input — is not specific to spec extraction. Any interactive template where the user provides feedback mid-workflow is susceptible.

Investigation Questions

1. Which templates have interactive feedback loops? Candidates include:

   • interactive-design (Phase 1 reasoning loop)
   • extend-library (contributor guidance workflow)
   • engineering-workflow / collaborate-requirements-change (iterative change propagation)
   • evolve-protocol (protocol modification loop)
   • author-north-star (section-by-section drafting with user review)
   • author-presentation (multi-phase with user review)
   • maintenance-workflow (finding classification with user collaboration)
   • Any template using the iterative-refinement protocol

2. Should this be a protocol rather than per-template guidance? If the pattern applies broadly, it may belong as:

   • A new guardrail protocol (e.g., feedback-classification) applied to all interactive templates
   • An addition to the existing iterative-refinement protocol (since that protocol already governs how changes are applied during feedback loops)
   • A section in the anti-hallucination protocol (since under-promoting feedback is a form of information loss)

3. Does the classification taxonomy need to be richer for other contexts? The current three categories (Editorial / Correction / Finding) were designed for spec extraction. Other templates might need:

   • Scope change — user wants to expand or narrow the task
   • Constraint — user is adding a new constraint not previously stated
   • Preference — user is expressing a stylistic or approach preference (not a requirement)
   • Rejection — user is rejecting a proposed element entirely

4. What is the interaction with existing protocols?

   • iterative-refinement already says 'justify every change' — should feedback classification be embedded there?
   • requirements-elicitation already decomposes user input — should it also classify the input type?
   • adversarial-falsification already has confidence classification — is there overlap?

5. What form should it take?

   • A standalone guardrail protocol (protocols/guardrails/feedback-classification.md)
   • An extension to iterative-refinement protocol
   • Guidance in bootstrap.md for all interactive templates
   • A combination (protocol + bootstrap guidance)

Suggested Approach

1. Audit all interactive templates and the iterative-refinement protocol for feedback handling gaps
2. Draft a candidate protocol or protocol extension
3. Test it against 2-3 real interactive sessions to validate the taxonomy
4. Propose via extend-library template contribution workflow

Related

• spec-extraction-workflow: user feedback during Phase 3 can be silently under-promoted #165 — Original issue (spec-extraction-workflow specific fix)
• fix(templates): add feedback classification rule to spec-extraction-workflow #166 — PR implementing the spec-extraction-specific classification rule