Sequence Modelling for Sentence Classification: Navigating the Rhetorical Structure of Law

TL;DR

This research tackles the challenge of identifying the "rhetorical role" of sentences in legal judgments—distinguishing between background facts, legal arguments, and final rulings. By moving from independent sentence classification to sequence modelling using Maximum Entropy taggers, the authors significantly improved the precision of document structuring, paving the way for more coherent and flexible legal summaries.

Context: Beyond Simple Extraction

Most early summarization systems relied on "sentence extraction"—picking the most important lines based on keywords or location. However, a legal judgment isn't just a list of facts; it's a structured argument. To summarize it well, a system must understand why a sentence is there. Is it a summary of lower court proceedings, or is it the Law Lord's final disposal of the case?

The authors leverage the HOLJ Corpus (House of Lords Judgments), identifying seven key rhetorical roles: FACT, PROCEEDINGS, BACKGROUND, FRAMING, DISPOSAL, TEXTUAL, and OTHER.

The Problem: The Independence Flaw

Prior works often treated sentences as independent data points. However, discourse is naturally sequential. If sentence $n$ is part of the FACTS section, sentence $n+1$ is highly likely to also be a FACT. Standard classifiers like Naive Bayes or C4.5 struggle to capture this "clustering" effect because they lack a memory of previous labels.

Methodology: The Power of the Sequence

The core innovation here is the transition from standard classification to a Sequence Labelling Framework.

1. Feature Engineering

The authors used a rich set of features including:

• Location: Position relative to the paragraph and the specific "Law Lord" speech.
• Linguistic Cues: Tense, aspect, and modality of the main verb (e.g., "I would allow..." vs. "The defendant said...").
• Named Entities: Identifying persons, organizations, and legal citations.

2. Maximum Entropy Sequence Tagger

Instead of predicting a label $y$ based only on features $x$, the sequence tagger calculates the probability of a sequence of tags ($y_1...y_n$) given the sentences ($s_1...s_n$).

The mathematical intuition behind the log-linear model used here is to allow arbitrary, overlapping features without the strong independence assumptions required by Naive Bayes. By adding features related to the previous two tags, the model effectively learns the "grammar" of legal arguments.

Experiments and Results

The study compared C4.5, Naive Bayes, Winnow, SVM, and Maximum Entropy (ME).

• Standalone Performance: SVM and ME were top contenders. Interestingly, C4.5 performed well on location alone but failed to integrate complex linguistic features effectively.
• The Sequence Boost: As shown in the table below, adding sequence information (SEQ) consistently outperformed the standard independent models (ME).

The system was particularly successful at identifying TEXTUAL sentences (85.4 F-score) due to their distinct length and lack of complex verbs. However, it faced challenges with FACT sentences, which are lexically diverse, often being confused with PROCEEDINGS.

Critical Insight: Why it Works

The success of the Maximum Entropy approach in this paper underscores a fundamental truth in Natural Language Processing: Context is king. By modeling the dependencies between labels, the system filters out logically impossible transitions (e.g., jumping from a background citation directly into a final ruling without framing).

Limitations

• NER Dependence: The system relies heavily on hand-coded Named Entity Recognition rules, which are difficult to scale to other domains without significant manual labor.
• The "Other" Category: Performance on rare categories remains poor (0.0 F-score for OTHER), suggesting that even sequence models need more balanced data or synthetic oversampling.

Conclusion

Hachey and Grover demonstrate that legal documents have a predictable rhetorical "rhythm." By capturing this rhythm through sequence modelling, we can move closer to automated systems that don't just "shorten" text, but actually understand the communicative intent of the law.