RFC-026: User Feedback Signal — Reference Register
RFC-026: User Feedback Signal — Reference Register
© 2024-2026 Kanjani AI Research & Causum. All rights reserved.
Reinforcement Learning from Human Feedback (RLHF)
| # | Reference | Relevance to RFC |
|---|---|---|
| 1 | Kaufmann, T. et al. “Reinforcement Learning from Human Feedback: A Statistical Perspective.” arXiv:2604.02507, 2026. | Comprehensive statistical survey of RLHF — provides theoretical grounding for why feedback signals (like AIGP’s FEEDBACK message) are necessary for calibrating AI governance decisions against real user experience. |
| 2 | Casper, S. et al. “Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback.” arXiv:2307.15217, 2023. | Systematizes RLHF flaws including noisy/sparse human feedback — validates AIGP’s design choice to capture structured metadata (NLP analysis, specificity, expectation_gap) alongside binary thumbs_up/down to improve signal quality. |
| 3 | Li, S. et al. “A Survey of Reinforcement Learning from Human Feedback.” arXiv:2312.14925, 2024. | Surveys RLHF as a variant of RL learning from human feedback rather than engineered reward functions — contextualizes AIGP’s FEEDBACK message as a governance-layer feedback collection mechanism distinct from model training feedback. |
| 4 | Gao, H. et al. “AI Alignment through Reinforcement Learning from Human Feedback? Contradictions and Limitations.” arXiv:2406.18346, 2024. | Critically evaluates attempts to align AI with human values through feedback methods — validates why AIGP separates governance feedback (policy calibration) from model training feedback (RLHF), avoiding conflation of governance signals with training signals. |
| 5 | Zhong, H. et al. “Objective Mismatch in Reinforcement Learning from Human Feedback.” arXiv:2311.00168, 2024. | Demonstrates that optimizing for human preference feedback doesn’t necessarily optimize for downstream evaluation metrics — validates AIGP’s approach of using feedback for governance calibration (guardrail tuning) rather than direct model optimization. |
User Feedback Measurement and Design
| # | Reference | Relevance to RFC |
|---|---|---|
| 6 | Microsoft Data Science. “Beyond Thumbs Up and Thumbs Down: A Human-Centered Approach to Evaluation Design for LLM Products.” 2025. | Argues simplistic thumbs up/down mechanisms fail to capture nuances for meaningful improvements — directly motivates AIGP’s NLP analysis object (sentiment, specificity, expectation_gap, topics) augmenting binary ratings. |
| 7 | Tianpan.co. “Behavioral Signals That Actually Measure User Satisfaction in AI Products.” 2026. | Demonstrates that feedback widget placement and timing dramatically affect signal quality — validates AIGP’s design where feedback is captured while users are “still evaluating” rather than through delayed surveys. |
| 8 | Tianpan.co. “Why Your AI Metrics Are Lying to You: The Feedback Signal Timing Problem.” 2026. | Shows that immediate post-interaction feedback measures momentary state rather than actual utility — contextualizes AIGP’s session-level feedback aggregation (via session_id) for longitudinal signal quality. |
| 9 | Kalloori, S. et al. “Scalable User Feedback via Implicit Sentiment in Developer Prompts.” arXiv:2509.18361, 2025. | Addresses the challenge of evaluating AI satisfaction at scale when explicit ratings are sparse — validates AIGP’s dual approach: explicit ratings (thumbs_up/down) plus implicit NLP-derived signals from comment text. |
| 10 | Holmes, G. et al. “The Chatbot Usability Scale: Design and Pilot of a Usability Scale for Interaction with AI-Based Conversational Agents (BUS-15).” 2021. | Develops a validated instrument for measuring chatbot interaction quality — provides psychometric context for AIGP’s feedback measurement approach and validates the need for structured metrics beyond binary satisfaction. |
Privacy-Preserving User Analytics
| # | Reference | Relevance to RFC |
|---|---|---|
| 11 | Pilipiszyn, A. et al. “A General Pseudonymization Framework for Cloud-Based LLMs.” arXiv:2502.15233, 2025. | Proposes pseudonymization for LLM interactions to protect user privacy — directly validates AIGP’s pseudonym_id design (HMAC-derived, non-reversible, app-scoped) for feedback collection without user identification. |
| 12 | European Court of Justice. “Pseudonymised Data Classification under GDPR.” Case T-557/20 (SRB v EDPS), September 2025. | Landmark ruling that pseudonymised data is not automatically personal data for all parties — provides legal basis for AIGP’s pseudonym_id design where the governance server cannot re-identify users. |
| 13 | UK Information Commissioner’s Office. “Pseudonymisation Guidance.” ICO, 2025. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/data-sharing/anonymisation/pseudonymisation/ | Regulatory guidance on pseudonymisation as a risk reduction and security improvement technique — contextualizes AIGP’s design as a GDPR-compatible approach to behavioral analysis without identification. |
| 14 | Worklytics. “Building a GDPR-Compliant AI Usage Analytics Program Without Storing Personally Identifiable Data.” 2025. | Practical guidance on AI usage analytics that don’t create additional privacy risks — validates AIGP’s approach where governance server stores metadata only (no comment content, no prompt/response content). |
NIST AI RMF Alignment
| # | Reference | Relevance to RFC |
|---|---|---|
| 15 | NIST. “AI 100-1: Artificial Intelligence Risk Management Framework.” National Institute of Standards and Technology, 2023. | Defines MAP 3.2 (scientifically valid and reliable AI system evaluation) and MEASURE 2.6 (structured evaluation through human subject interaction) — these are the specific NIST requirements that RFC-026’s FEEDBACK message addresses. |
| 16 | NIST. “MEASURE-2.2: Human Subject Evaluations.” AI RMF Playbook, 2023. | Specifies that evaluations involving human subjects must be representative and meet applicable requirements — validates AIGP’s pseudonymized, consent-compatible feedback collection design. |
Parent-Child Evidence Relationships
| # | Reference | Relevance to RFC |
|---|---|---|
| 17 | W3C. “PROV-O: The PROV Ontology.” W3C Recommendation, 2013. https://www.w3.org/TR/prov-o/ | W3C standard for representing provenance information — AIGP’s parent-child relationship model (FEEDBACK_ON with parent_id/parent_ref) follows established provenance patterns for derivation and attribution. |
| 18 | Moreau, L. et al. “The Open Provenance Model Core Specification (v1.1).” Future Generation Computer Systems, 2011. | Foundational work on provenance graphs with causal relationships — validates AIGP’s three-level parent-child discovery (S3 path convention, bottom-up parent_id, top-down has_children) for evidence chain construction. |
Sentiment Analysis and NLP Heuristics
| # | Reference | Relevance to RFC |
|---|---|---|
| 19 | Liu, B. Sentiment Analysis: Mining Opinions, Sentiments, and Emotions. 2nd ed. Cambridge University Press, 2020. | Comprehensive treatment of sentiment analysis including intensity scoring — provides the theoretical basis for AIGP’s NLP analysis object (sentiment float -1.0 to 1.0, severity_language classification). |
| 20 | Devlin, J. et al. “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.” NAACL-HLT, 2019. | Foundation model enabling lightweight topic extraction and sentiment classification — contextualizes AIGP’s requirement for heuristic NLP analysis “without LLM call required” by referencing lightweight transformer-based approaches. |
Last updated: June 2026
Additional Sources (provided by author)
| # | Reference | Relevance to RFC |
|---|---|---|
| 21 | Xu, Y. et al. “Towards Boosting the Open-Domain Chatbot with Human Feedback.” Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), pp. 4060–4078, Toronto, 2023. https://aclanthology.org/2023.acl-long.224/ | Demonstrates how structured human feedback (beyond binary ratings) improves chatbot quality through iterative refinement — validates AIGP’s approach of capturing comment text with NLP analysis alongside thumbs_up/down for actionable governance signal. |
| 22 | Lindström, E. “Human Feedback Mechanisms for AI Systems.” Master’s Thesis, Linköping University, 2023. https://www.diva-portal.org/smash/get/diva2:1782678/FULLTEXT01.pdf (local copy: specification/papers/FULLTEXT01.pdf) |
Systematic analysis of human feedback mechanisms for AI systems — provides the empirical and design foundation for AIGP’s FEEDBACK message type, including discussion of feedback timing, modality (explicit vs. implicit), and privacy considerations in feedback collection. |