WisPaper
WisPaper
学术搜索
学术问答
价格
TrueCite
OntoAgent: Moving Beyond Chat to Professional Requirements Interviews with Experience Ontologies
总结
问题
方法
结果
要点
摘要

OntoAgent is a novel agentic framework for automating requirements elicitation interviews by integrating Large Language Models (LLMs) with a domain-specific "Experience Ontology." It transforms free-form chatbot interactions into structured, systematic interviews, significantly outperforming baselines by 33% in Implicit Requirements Elicitation (IRE) and 21% in questioning efficiency (TKQR).

TL;DR

Automating requirements elicitation is no longer just about prompt engineering. OntoAgent introduces a structured "Experience Ontology" that guides LLMs through a systematic interview process. By treating an interview as a tree-traversal problem rather than a free-form chat, it achieves a 33% boost in uncovering implicit requirements and nearly solves the "blind spot" LLMs have for non-functional requirements like style and aesthetics.

The Problem: The "Chat" Trap in Requirements Engineering

Traditional requirements elicitation relies on the intuition and experience of human analysts. When we replace humans with LLMs, we often encounter the "Chat Trap":

  1. Redundancy: Asking the same questions in different ways.
  2. Breadth over Depth: LLMs tend to stay at the surface level (e.g., "Do you want a login?") while missing critical implementation slots (e.g., "OAuth vs. Password recovery").
  3. The "Style" Blind Spot: Without guidance, LLMs rarely ask about non-functional requirements such as design, accessibility, or branding.

Motivation Comparison Fig 1. Analysts use structured experience (Right) while LLMs often default to ad-hoc, inefficient chat (Left).

Methodology: The Ontology-Guided Approach

OntoAgent shifts the paradigm by decoupling "what to ask" (handled by the Ontology) from "how to ask" (handled by the LLM).

1. Experience Ontology Induction

OntoAgent builds a three-level hierarchical tree from domain requirements:

  • Aspects: High-level domains (e.g., Interaction, Content, Style).
  • Dimensions: Specific functionalities (e.g., Login, Search, Reporting).
  • Slots: Clarifiable details (e.g., Sorting rules, filtering mechanisms).

2. The Decision Loop

The "Agentic" part of OntoAgent involves a four-step iterative logic:

  • ScoreOnto: Prioritizes nodes based on the user's initial prompt.
  • ReRankOnto: Dynamically updates priorities based on the latest answer.
  • GatePrune: The most critical feature. If a user says "I don't need a login," the agent immediately prunes the entire "Login" branch, preventing redundant questions.
  • QuestionGen: The LLM synthesizes a natural question based on the selected "Slot."

OntoAgent Framework Fig 2. The OntoAgent architecture, showing the pipeline from ontology induction to the real-time interview loop.

Experimental Breakthroughs

The authors tested OntoAgent against 5 state-of-the-art (SOTA) baselines, including Mistake-guided Prompting and LLMREI.

Performance Gains

  • Effectiveness (IRE): OntoAgent reached 0.69, whereas standard "Chain of Thought" (CoT) prompting lingered at a dismal 0.08. This proves that the ontology is the primary engine of effectiveness, not just the model's reasoning.
  • Efficiency (TKQR): By ranking high-value questions earlier, OntoAgent ensures that if an interview is cut short, the most vital requirements are already captured.

The "Style" Breakthrough

One of the most impressive results is in the Style aspect. Previous LLM methods scored near 0.00 in style elicitation. OntoAgent jumped to 0.55 because the ontology explicitly forced the agent to explore aesthetic requirements.

Performance Tables Table 1. Aspect-level analysis showing the massive improvement in Interaction (IRE_int), Content (IRE_con), and Style (IRE_sty).

Critical Insight: Why Does This Work?

OntoAgent works because it addresses the Inductive Bias of LLMs. LLMs are trained on conversations, which are naturally rambling and unstructured. Requirements elicitation is a search problem over an unknown requirement space. By providing the LLM with a "map" (the ontology) and a "navigation algorithm" (Score/ReRank/Prune), the agent transforms into a professional analyst.

Limitations & Future Work

The current version is evaluated primarily on the web application domain. While the authors demonstrated scalability by increasing the training data, the quality of the "Experience Ontology" still depends on having a diverse set of initial requirement documents to learn from. Future research could explore how these agents handle conflicting requirements between multiple stakeholders.

Conclusion

OntoAgent represents a significant step toward Systematic Requirements Engineering. It shows that for LLMs to succeed in high-stakes professional roles, we must equip them with explicit structures of human experience.

Final takeaway: If you want a professional AI agent, stop chatting with it and start guiding it with an ontology.

发现相似论文

试试这些示例

  • Search for recent papers that utilize formal ontologies to constrain Large Language Model agents in software engineering tasks.
  • Which paper introduced the ReqElicitGym evaluation environment, and what were the primary weaknesses identified in zero-shot LLM interviewers there?
  • Explore research applying ontology-managed dialogue systems to other requirements engineering phases such as conflict resolution or specification generation.
目录
OntoAgent: Moving Beyond Chat to Professional Requirements Interviews with Experience Ontologies
1. TL;DR
2. The Problem: The "Chat" Trap in Requirements Engineering
3. Methodology: The Ontology-Guided Approach
3.1. 1. Experience Ontology Induction
3.2. 2. The Decision Loop
4. Experimental Breakthroughs
4.1. Performance Gains
4.2. The "Style" Breakthrough
5. Critical Insight: Why Does This Work?
5.1. Limitations & Future Work
6. Conclusion