References

Bibliography and Knowledge Base for the IOA-ORM Specification

Oral Assessment Theory

• Joughin, G. (1998). “Dimensions of Oral Assessment.” Assessment & Evaluation in Higher Education, 23(4), 367–378. — Six dimensions of oral assessment (content type, interaction, authenticity, structure, examiners, orality). Primary theoretical grounding for AssessmentProfile.

• Joughin, G. (2010). A short guide to oral assessment. Leeds Met Press in association with University of Wollongong. ISBN 978-1-907240-09-6. — Three-way classification (presentations, interrogations, applications). Validity/reliability criteria. Extends the 1998 dimensions framework with practical implementation guidance.

• Akimov, N. & Malin, M. (2020). “When Old Becomes New: Oral Examination as an Online Assessment Tool.” Interactive Learning Environments. — Validity/reliability/fairness matrix. Question banking. Recording and moderation. Identity verification. Grounding for ValidityClaim, ModerationPolicy, FairnessAudit.

• Bayley, R. et al. (2024). “Back to the Future: Implementing Large-Scale Oral Exams.” — ConVOE model for 600+ students. Parallel administration, batch grading, practice sessions. Grounding for scalability constructs.

• Fenton, A. (2015). “Reconsidering oral exams and assessments.” — Earlier work on oral assessment as authentic evaluation method. Characterizes oral assessment as conversation, not interrogation. Cited by collaborator’s literature review.

• Fenton, A. (2025). “Reconsidering the Use of Oral Exams and Assessments.” — IOA components. Prompting taxonomy (Pearce & Chiavaroli, 2020). Formative vs. summative. Scaffolding-as-evidence. Grounding for PromptingLevel, scaffoldingBudget, assessmentPurpose.

• Sotiriadou, S. et al. (2020). “Interactive oral” definition. — “A form of assessment asking students to perform real-world tasks to demonstrate meaningful application of necessary knowledge and skills.” Grounding for assessmentType and task-based assessment design.

• Bloom, B.S. (1956). Taxonomy of Educational Objectives: The Classification of Educational Goals. — Six cognitive levels (Remember → Create). Grounding for BloomLevel enum and cognitiveLevel on EvidenceTarget.

Bloom’s Taxonomy & AI Assessment

• Nguyen, H. et al. (2023). — Bloom’s taxonomy and AI capability argument. AI performs well at lower Bloom levels but struggles at Create.

Oral Assessment Challenges

• Huxham, M. et al. (2012). — Student anxiety in oral assessments.

• Sayre, E. (2014). — Open-book assessment design.

• Ward, R. et al. (2024). — Interactive oral assessment patterns.

Design Science Research

• Hevner, A.R., March, S.T., Park, J., & Ram, S. (2004). “Design Science in Information Systems Research.” MIS Quarterly, 28(1), 75–105. — 7 Guidelines for DSR. Evaluation methods taxonomy.

• Gregor, S. & Hevner, A.R. (2013). “Positioning and Presenting Design Science Research for Maximum Impact.” MIS Quarterly, 37(2), 337–355. — Knowledge Contribution Framework (2×2 matrix). Ω and Λ knowledge. Contribution levels.

• Gregor, S. & Jones, D. (2007). “The Anatomy of a Design Theory.” JAIS, 8(5), 312–335. — 8 components of ISDT. Design theory template.

• Venable, J., Pries-Heje, J., & Baskerville, R. (2016). “FEDS: A Framework for Evaluation in Design Science Research.” EJIS, 25(1), 77–89. — Evaluation strategies (Technical Focus, Field Strategy, Lab Experiment, Informed Argument).

• Peffers, K. et al. (2007). “A Design Science Research Methodology for Information Systems Research.” JMIS, 24(3), 45–77. — 6-step DSRM Process Model.

Design Space & Methodology

• MacLean, A. et al. (1991). “Questions, Options, and Criteria: Elements of Design Space Analysis.” Human-Computer Interaction, 6(3-4), 201–250. — QOC methodology for documenting design decisions.

• Nickerson, R.C. et al. (2013). “A Method for Taxonomy Development and Its Application in Information Systems.” EJIS, 22(2), 125–155. — Taxonomy development methodology.

• Grisold, T. et al. (2021). “Process Redesign: The Grammar of Re-designing.” Business Process Management Journal. — Design space exploration via morphological analysis.

• Jansson, D.G. & Smith, S.M. (1991). “Design Fixation.” Design Studies, 12(1), 3–11. — Design fixation prevention through systematic exploration.

IS / Agent Systems

• van der Aalst, W.M.P. et al. (2003). “Workflow Patterns.” Distributed and Parallel Databases, 14(1), 5–51. — Workflow patterns informing state machine design.

• Young, R. (2010). “CQRS: Command Query Responsibility Segregation.” — CQRS pattern informing separation of evidence collection and evaluation.

• Yao, S. et al. (2023). “ReAct: Synergizing Reasoning and Acting in Language Models.” ICLR 2023. — ReAct pattern informing agent autonomy gradient.

• Schick, T. et al. (2023). “Toolformer: Language Models Can Teach Themselves to Use Tools.” — Tool-use hallucination risk informing single-function communication model.

AI Safety / Governance

• Bai, Y. et al. (2022). “Constitutional AI: Harmlessness from AI Feedback.” — Constitutional principles informing agent boundary model.

• Greshake, K. et al. (2023). “Not What You’ve Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection.” — Prompt injection risks informing context policy and defense-in-depth.

• Rebedea, T. et al. (2023). “NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails.” — Proxy architecture informing Runtime Controller design.

Software Engineering

• Bass, L., Clements, P., & Kazman, R. (2012). Software Architecture in Practice, 3rd ed. — Layered architecture, separation of concerns.

• Gamma, E. et al. (1995). Design Patterns: Elements of Reusable Object-Oriented Software. — Adapter pattern for Pipecat decoupling.

• Fowler, M. (2005). “Event Sourcing.” — Event sourcing pattern for evidence ledger and audit trail.

• Hohpe, G. & Woolf, B. (2003). Enterprise Integration Patterns. — Messaging patterns for event protocol design.

• Lattner, C. et al. (2020). “MLIR: Scaling Compiler Infrastructure for Domain Specific Computation.” — Multi-level intermediate representation concept for multi-target compilation.

Provenance & Audit

• Buneman, P. et al. (2001). “Why and Where: A Characterization of Data Provenance.” — Provenance model (why/where/how) for evidence signals.

• Moreau, L. et al. (2009). “The Open Provenance Model.” — OPM for runtime event provenance.

• Nath, R. et al. (2024). “Chain of Custody for Digital Evidence.” — Chain-of-custody patterns for evidence ledger.