Qwen-InteractEval: An Interactive World Modeling for Boosting Human-Centered AI
输入“/”快速插入内容
Qwen-InteractEval: An Interactive World Modeling for Boosting Human-Centered AI
Qwen-InteractEval: An Interactive World Modeling for Boosting Human-Centered AI
1. Introduction: Beyond QA Toward Interactive World Modeling
The rapid advancement of foundation models has enabled a new generation of interactive AI systems—conversational agents, game NPCs, digital companions, therapeutic bots, and educational tutors—that engage humans in sustained, open-ended, real-time interaction. Yet the infrastructure for understanding, evaluating, and optimizing these systems has not kept pace. We identify three structural deficits:
Deficit 1: Temporal Blindness. Static benchmarks (MMLU, HumanEval, GPQA) and single-turn evaluations discard the temporal dynamics that define real-time interactive experience. Quality in sustained interaction is not a property of individual outputs but an emergent property of the trajectory—how states evolve, how conflicts develop and resolve, how engagement waxes and wanes across turns. No per-turn metric can capture whether an AI system sustains coherent behavior over a 20-turn interaction or whether its responses actively shape the interaction's future direction.
Deficit 2: Subjective Opacity. Pairwise preference rankings (Chatbot Arena, MT-Bench) compress rich multi-dimensional interaction quality into a single ordinal signal, offering no diagnostic decomposition of which interactive capabilities succeed or fail (Zheng et al., 2023). Holistic LLM-as-judge scores inherit the same opacity with the added risk of stylistic bias and position effects. We cannot determine whether a model's advantage reflects superior emotional responsiveness, stronger narrative agency, or simply more fluent prose.
Deficit 3: Evaluation–Optimization Disconnect. Even where evaluation identifies failures, the signal is too coarse to guide model improvement. A holistic "quality score" provides no actionable gradient—it cannot tell a developer whether to improve persona consistency, information absorption, emotional tracking, or proactive agency. The evaluation and optimization loops remain structurally disconnected: evaluation produces scores, but optimization requires structured, decomposed, causally-grounded feedback.
The core insight is that evaluation and optimization are two sides of the same coin—but this identity holds only under specific structural conditions that existing systems fail to meet. A holistic score ("7/10") is technically a reward signal, but it is non-decomposable (which capability should improve and how?), non-causal (what evidence triggered the failure?), and non-temporal (at which turn did quality degrade?). For evaluation to become optimization, three structural conditions must be satisfied simultaneously: (i) hierarchical decomposition—the evaluation signal must separate independent capability dimensions so that improving one does not require disrupting another; (ii) causal grounding—each evaluation judgment must be traceable to specific textual evidence and latent state transitions, not to holistic impressions; (iii) temporal continuity—evaluation must operate at both per-turn and trajectory levels, because interactive quality is an emergent property of sequences, not individual responses.
MarioLM addresses these deficits by proposing an interactive world architecture that serves a dual purpose—two co-equal, mutually-enabling frameworks:
1.
MarioEval (Evaluation): A process-oriented evaluation system built on a four-layer world model (Scene → Latent State → Event Chain → Rubric) that decomposes interactive quality into structured, causally-linked, hierarchical capability signals. The architecture provides diagnostic fingerprints that pinpoint which specific capability fails, at which turn, due to what causal evidence—replacing opaque holistic scores with auditable, structured evaluation.
2.
MarioOpt (Optimization): A two-algorithm structured optimization framework (§4.2) that exploits MarioEval's unique signal structure to achieve targeted, interpretable, and hierarchically-coherent model improvement. MarioOpt comprises Hierarchical Reward Decomposition with Evidence-Grounded State Reward (HRD, for continuous curriculum training with per-turn decomposed rewards) and Event-Chain Future Path Optimization (EFPO, for targeted recovery training from degrade trajectories)—converting MarioEval's decomposed evaluation signals into actionable training objectives with formal guarantees against capability regression.
MarioEval is, to our knowledge, the first architecture that satisfies all three conditions simultaneously, and MarioOpt is the first optimization framework that exploits the resulting signal structure. The same four-layer architecture that measures interactive quality also generates the multi-signal reward functions that improve it.
We instantiate this architecture in the domain of interactive role-playing dialogue, which we argue is a model problem for interactive AI more broadly. Role-playing scenarios compress the core challenges of real-time interactivity into a tractable setting: identity maintenance across turns, real-time absorption of novel information, psychologically plausible state evolution, adaptive communication, proactive narrative agency, and long-horizon goal consistency. These capabilities generalize directly to tutoring agents (tracking student understanding), therapeutic chatbots (maintaining relational coherence), and game NPCs (sustaining believable behavior). MarioEval's layered architecture is designed for portability: the Scene Layer generalizes to any structured interaction context, the Latent State Layer to any domain with trackable participant states, and the Rubric Layer to any setting where capabilities have logical dependencies.
Contributions. This work makes the following contributions, spanning conceptual, methodological, algorithmic, and empirical dimensions:
1.
Conceptual: The evaluation–optimization unification thesis (§1, §3, §4.2). We articulate and formalize the conditions under which an evaluation system's signals become directly usable as optimization objectives—hierarchical decomposition, causal grounding, and temporal continuity—and demonstrate that MarioEval is, to our knowledge, the first interactive AI architecture satisfying all three simultaneously. This reframes evaluation and optimization as a single design problem rather than two separate research agendas.
2.
Architectural: A cognition-first four-layer world model organized as three evaluation harnesses plus a novel adjudication logic (§3). We derive a four-layer architecture (Scene → Latent State → Event Chain → Rubric) from first-principles cognitive analysis rather than engineering convenience, with formal proofs of joint necessity and sufficiency. The first three layers constitute three independent evaluation harnesses, each responsible for a distinct dimension of world modeling:
•
Scene Harness — static world grounding: formalizes the interpretive context (persona, constraints, mechanisms, boundaries, objectives, initial situation) against which all dynamic evaluation is anchored.
•
Latent State Harness — dynamic psychological modeling: externalizes participants' evolving mental states (affect, relational dynamics, epistemic awareness, goals) as structured, evidence-grounded variables with formalized update rules, significance gating, and user interest tracking.
•
Event Chain Harness — causal trajectory reasoning: maintains an explicit causal model of how the interaction arrived at its current state (past causal graph), what tension drives the current moment (present conflict), and what consequences the current response will produce (future path prediction). The core function is causal attribution and inference—tracing why events happened and predicting what they will cause—not merely classifying trajectory outcomes.
The fourth layer constitutes a novel adjudication logic that consumes the three harnesses' outputs and produces actionable evaluation and optimization signals:
•
Rubric & Halt Adjudication — hierarchical capability scoring via the L1–L6 progressive cascade constraint (where each level's score is capped by the weakest foundation below it, enforcing the insight that higher-order interactive capabilities cannot exist without lower-order foundations); evidence-grounded expert-validated scoring definitions; and value-driven termination via user interest decay.
This harness+adjudication organization clarifies the architecture's modularity: each harness can be independently developed, validated, and improved; the adjudication logic consumes structured outputs from all three harnesses to produce integrated judgments.
3.
Algorithmic: MarioOpt—a two-algorithm structured optimization framework (§4.2). We propose HRD (hierarchical reward decomposition with evidence-grounded state reward, curriculum learning, and non-regression guarantees) and EFPO (future path optimization with recovery training and anti-stagnation regularization), organized as a two-stage training pipeline rather than a unified loss function. These algorithms exploit MarioEval's unique signal structure in ways that no existing optimization method (RLHF, DPO, Constitutional AI) can replicate, because they require the structured, causal, trajectory-level evaluation signals that only MarioEval produces.
4.
Empirical: Two theoretically significant findings (§5). (a) The Locally-Coherent-Globally-Degenerative (LCGD) pattern—we identify and theoretically explain the most common failure mode of frontier models in sustained interaction, attributing it to per-turn RLHF incentives and attention locality. (b) Hedonic bias in human evaluation—we demonstrate that human evaluators systematically overrate structurally stagnant interactions that are experientially pleasant, grounded in Kahneman's experiencing/remembering self distinction.
5.
Infrastructure: A trained user simulator and the Experiential World Game (§4.1, §4.3). We build (a) an SFT-trained, profile-conditioned user simulator grounded in the structural duality between MarioEval and MetaMind (MarioEval's harness naturally implements an Inverse-MetaMind: Response → ToM → Observation). It is trained on real production logs with a profile-embedding ladder and scaled from 35B to 122B, and in preliminary experiments across three benchmark frameworks (RealSim, SimulatorArena, HumanLM) it substantially outperforms frontier prompted baselines (GPT-5.4, Claude 4.7 Sonnet, Gemini 3.1 Pro) at faithful user simulation; ToM-RL and multi-turn RL are outlined as future extensions. We further propose (b) a persistent, multi-agent interactive world (the Experiential World Game) with a three-tier memory architecture that serves simultaneously as evaluation benchmark, training environment, and research platform.
2. Theoretical Foundations and Prior Work
MarioEval's design integrates theoretical frameworks from cognitive science, social psychology, and our own prior work on social intelligence evaluation and metacognitive modeling.
2.1 Theory of Mind and Metacognitive Social Reasoning
Humans evaluate interactive partners by continuously inferring their beliefs, desires, intentions, and emotional states—a process formalized in Theory of Mind (ToM) research (Premack & Woodruff, 1978; Baron-Cohen, 1997). MarioEval's Latent State Layer operationalizes this process, making explicit the mental state variables that a human evaluator would implicitly track.
Our prior work MetaMind (Zhang et al., 2025; NeurIPS 2025 Spotlight) provides direct theoretical and methodological foundations for this approach. MetaMind demonstrates that social understanding can be systematically decomposed into three collaborative cognitive stages: (1) a Theory-of-Mind Agent that generates hypotheses about user mental states (intent, emotion, belief), (2) a Moral Agent that refines these hypotheses using cultural norms and ethical constraints, and (3) a Response Agent that generates contextually appropriate responses while validating alignment with inferred intent. MetaMind achieves a 35.7% average improvement on social scenario tasks and enables LLMs to match average human performance on ToM benchmarks. MarioEval's Latent State Layer inherits MetaMind's core principle—that social cognition is decomposable into structured, verifiable mental state inferences—but extends it from a generation framework to an evaluation framework: rather than using ToM inferences to generate better responses, we use them to evaluate whether a model's responses are consistent with plausible mental state trajectories.
2.2 Social Intelligence in Interactive Agents: From Static Measurement to Process-Oriented Evaluation
Why social intelligence matters for interactive AI. Social intelligence (SI)—the capacity to perceive, interpret, and respond effectively to social situations—is the meta-capability that governs all sustained human-AI interaction. Unlike task-specific competence (coding, math, summarization), SI cannot be evaluated through isolated test items because it is inherently relational, temporal, and context-dependent: the same utterance may be socially brilliant in one context and disastrous in another, and its quality depends on the accumulated relational history, not just the current turn. For interactive AI agents—conversational companions, game NPCs, therapeutic bots, tutoring systems—SI is not one capability among many; it is the capability that determines whether the interaction feels authentic, productive, and worth sustaining. An agent with perfect factual accuracy but no SI (ignoring emotional cues, violating social norms, failing to adapt to relational dynamics) will produce interactions that humans abandon. This positions SI evaluation as a necessary prerequisite for any meaningful evaluation of interactive AI quality—and explains why MarioEval's architecture is designed around the cognitive structures of social cognition (Theory of Mind, appraisal theory, narrative engagement) rather than around linguistic features.
Our prior work SocialEval (Zhou et al., 2025; ACL 2025) establishes the paradigm of process-oriented SI evaluation. SocialEval introduces the "World Tree" structure—multi-plot narratives where an LLM makes decisions at critical junctures, with each decision reflecting specific interpersonal abilities from the BESSI framework (32 abilities across 5 domains). Its dual evaluation paradigm—outcome-oriented goal achievement plus process-oriented interpersonal ability assessment—demonstrates that SI is best measured through situated, interactive scenarios rather than static tests.
MarioEval extends SocialEval's paradigm in three critical directions. First, while SocialEval evaluates decisions at discrete branching points, MarioEval evaluates the continuous evolution of psychological states and causal chains across every turn—capturing not just what decision was made but how the interaction's internal dynamics evolved to produce it. Second, MarioEval introduces hierarchical capability decomposition (the L1–L6 ladder) that maps interactive failures to specific cognitive levels, going beyond SocialEval's flat ability dimensions. Third, MarioEval closes the evaluation-optimization loop by converting structured evaluation signals into actionable training feedback—a step SocialEval's design does not address.