Glossary
Glossary
Section titled “Glossary”This glossary provides formal definitions for all terms used throughout the IOA-ORM specification. Terms defined here are linked automatically when they appear in other sections.
IOA-ORM (Interactive Oral Assessment Ontology and Reference Model) : The primary artifact. A multi-layered specification for interactive oral assessment systems, comprising a domain ontology (shared vocabulary), a reference model (system abstraction), and an executable specification (machine-readable schema). See also: IOA Domain Ontology, IOA Reference Model, IOA Executable Specification.
IOA Domain Ontology : The conceptual layer of IOA-ORM. Defines the shared vocabulary, domain entities, and assessment-theoretic constructs (e.g., AssessmentProfile, EvidenceTarget, CompletionPolicy). Grounded in Joughin (1998), Akimov & Malin (2020), Bloom (1956), and Fenton (2025).
IOA Reference Model : The system abstraction layer of IOA-ORM. Defines the architecture (node graph, policies, evidence ledger), design principles, and runtime semantics for interactive oral assessment systems.
IOA Executable Specification : The machine-readable layer of IOA-ORM. Comprises TypeScript schema, validation rules, and the compilation pipeline from authoring to runtime execution.
Intermediate Representation (IR) : The engineering role played by the IOA Executable Specification — as a compilation target from authoring tools and a compilation source for runtime engines. “IR” describes a role, not the artifact’s primary identity.
Assessment Profile : A structured declaration of the exam’s position on Joughin’s (1998) six dimensions of oral assessment. Captures design parameters that determine what the exam measures and how validity/reliability claims are supported.
Assessment Purpose : Whether the exam is formative (practice/feedback), summative (graded), or diagnostic (placement). Affects evidence handling, feedback timing, and recording policy.
Calibration Profile : References to calibration exercises and measured accuracy metrics for the AI examiner. Ensures consistent assessment quality across sessions.
Candidate Command : A structured input from the candidate that the runtime controller MUST process (e.g., repeat, pause, clarification). These are runtime primitives, not UI decorations.
Completion Policy : Rules governing when a node is “done” — how many turns, what evidence is required, time limits. Machine-enforceable by the runtime controller.
Content Type : Joughin’s (1998) four primary categories of what oral assessment can measure: knowledge/understanding, applied problem solving, interpersonal competence, intrapersonal qualities.
Context Policy : Rules governing what exam context (rubric, previous nodes, candidate history) the AI examiner may access at each node.
Evidence Ledger : The authoritative, structured collection of all evidence signals produced during a session. First-class output consumed by the marking runtime.
Evidence Signal : A runtime-emitted record that a specific evidence target was (or was not) demonstrated, with confidence and provenance.
Evidence Target : A rubric-aligned definition of what the exam is trying to assess at a given node.
Exam : A published, versioned oral assessment with defined structure, policies, and evidence targets.
Exam Runtime Package
: The canonical machine-readable artifact representing a complete published exam. The single source of truth for all runtime configurations. Type name: ExamRuntimePackage.
Fairness Audit : Structured analysis of assessment outcomes across demographic dimensions to detect systematic disparities.
Follow-Up Policy : Rules governing how many follow-ups the examiner may issue, and under what conditions.
Marking Runtime : The downstream system that reads the evidence ledger and produces assessment scores.
Moderation Policy : Rules for human review of AI-generated evidence signals. Supports inter-rater reliability.
Pipecat Adapter Output : The compiled Pipecat-specific configuration (FlowManager config + NodeConfig) generated from the IOA-ORM specification.
Prompting Level : A classification of examiner follow-up moves based on Pearce & Chiavaroli’s (2020) taxonomy: from neutral presentation to leading guidance.
Question Pool : A set of equivalent question variants from which one or more are drawn per session. Enables inter-case reliability.
Recovery Policy : Rules governing how the runtime handles anomalies — silence, unclear answers, off-topic, anxiety, network issues.
Runtime Event : An immutable record of a significant state change during a session.
Runtime Node : A discrete unit of the exam flow — a question, task, scenario segment, or transition point.
Runtime Session : A single candidate’s attempt at an exam. One exam may have many sessions.
Runtime State : The mutable, per-session state tracked by the runtime controller during execution.
Scaffolding Budget : Maximum scaffolding intensity permitted at a node (0–10). The amount of scaffolding provided is itself evidence of candidate competence (Fenton, 2025).
Telemetry Policy : Rules governing what operational data is emitted and where.
Transcript Turn : A single utterance in the conversation, attributed to examiner or candidate, with timing and node context.
Transition Policy : Rules governing how and when the runtime moves from one node to another.
Validity Claim : A structured declaration of how the exam addresses face, content, construct, concurrent, inter-rater, inter-case, or fairness validity.