Legal Natural Language Processing
? Datasets
Legal Judgement Prediction (LJP)
| Dataset |
Links |
Domain |
Language |
Size |
| FSCS (Niklaus et al., 2021) |
? ? |
Swiss court judgments |
?? ?? ?? |
85K cases w/ 2 outcomes |
| ECtHR (Chalkidis et al., 2021) |
? ? |
EU court judgments |
?? |
11K cases w/ 11 outcomes |
| ECHR (Aletras et al., 2019) |
? ? |
EU court judgments |
?? |
11.5K cases w/ 11 outcomes |
| CAIL (Xiao et al., 2018) |
? |
Chinese court judgements |
?? |
2.6M cases w/ 6 outcomes |
Legal Text Classification (LTC)
| Dataset |
Links |
Domain |
Language |
Size |
| GLC (Papaloukas et al., 2021) |
? ? |
Greek legislation |
?? |
47.5K laws w/ 2.7K labels |
| CUAD (Hendrycks et al., 2021) |
? ? |
Contracts |
?? |
510 contracts w/ 41 classes |
| MultiEURLEX (Chalkidis et al., 2021) |
? ? |
EU legislation |
?? ?? ?? ?? ?? (18+) |
65K laws w/ 4.5K labels |
| LEDGAR (Tuggener et al., 2020) |
? ? |
Contracts |
?? |
60.5K contracts w/ 12.6K labels |
| Contract Discovery (Borchmann et al., 2020) |
? |
Contracts |
?? |
2.6K clauses w/ 21 classes |
| EURLEX-57K (Chalkidis et al., 2019) |
? ? |
EU legislation |
?? |
57K laws w/ 4.3K labels |
| Unfair-ToS (Lippi et al., 2018) |
? ? |
Contracts |
?? |
9.4K sentences w/ 9 classes |
| Contract Elements (Chalkidis et al., 2017) |
? ? |
Contracts |
?? |
2.4K contracts w/ 10 classes |
| OPP-115 (Wilson et al., 2016) |
? ? |
Privacy laws |
?? |
115 policies w/ 23K labels |
Legal Information Retrieval (LIR)
| Dataset |
Links |
Domain |
Language |
Size |
| BSARD (Louis et al., 2022) |
? ? |
Belgian legislation |
?? |
1.1K questions w/ 22.6K candidate statutory articles |
| EU2UK (Chalkidis et al., 2021) |
? ? |
EU & UK legislation |
?? |
2K query documents w/ 52.5K candidate documents |
| UK2EU (Chalkidis et al., 2021) |
? ? |
EU & UK legislation |
?? |
2.1K query documents w/ 3.9K candidate documents |
| COLIEE-Case-Law-Retrieval (Rabelo et al., 2020) |
? ? |
Canadian precedents |
?? |
650 query cases w/ 128K candidate cases |
| COLIEE-Statute-Law-Retrieval (Rabelo et al., 2020) |
? ? |
Japanese legislation |
?? ?? |
808 questions w/ 768 candidate statutory articles |
| CAIL2019-SCM (Xiao et al., 2019) |
? |
Chinese court judgements |
?? |
8.9K triplets of cases |
Legal Question Answering (LQA)
| Dataset |
Links |
Domain |
Language |
Size |
| CaseHOLD (Zheng et al., 2021) |
? |
US case holdings |
?? |
53.1K multiple-choice questions |
| JEC-QA (Zhong et al., 2019) |
? ? |
Chinese law |
?? |
26.3K multiple-choice questions |
| CJRC (Duan et al., 2019) |
? |
Chinese court judgements |
?? |
50K question-answers from 10K documents |
| PrivacyQA (Ravichander et al., 2019) |
? |
Privacy policies |
?? |
1.7K question-answers from 35 documents |
Legal Textual Entailment (LTE)
| Dataset |
Links |
Domain |
Language |
Size |
| COLIEE-Case-Law-Entailment (Rabelo et al., 2020) |
? ? |
Canadian precedents |
?? |
425 cases w/ related case |
| COLIEE-Statute-Law-Entailment (Rabelo et al., 2020) |
? ? |
Japanese legislation |
?? ?? |
808 questions w/ related statutory article |
Legal Text Summarization (LTS)
| Dataset |
Links |
Domain |
Language |
Size |
| UK-Abs (Shukla et al., 2022) |
? ? |
UK court cases |
?? |
793 pairs of (case, abastractive summary) from the UK Supreme Court |
| IN-Abs (Shukla et al., 2022) |
? ? |
Indian court cases |
?? |
7.1K pairs of (case, abastractive summary) from the Indian Supreme Court |
| IN-Ext (Shukla et al., 2022) |
? ? |
Indian court cases |
?? |
50 pairs of (case, extractive summary) from the Indian Supreme Court |
| TOS;DR (Keymanesh et al., 2020) |
? |
Terms of service |
?? |
1.6K pairs of (agreement text, summary) from data privacy policies |
| BillSum (Kornilova et al., 2019) |
? ? |
US Congressional bills |
?? |
22.2K pairs of (bill, summary) |
| TL;DRLegal (Manor et al., 2019) |
? |
Terms of service |
?? |
84 pairs of (agreement text, summary) from software licenses |
| TOS;DR (Manor et al., 2019) |
? |
Terms of service |
?? |
421 pairs of (agreement text, summary) from data privacy policies |
| BVA Cases (Zhong et al., 2019) |
? |
US court cases |
?? |
92 pairs of (case, summary) from the US Board of Veterans' Appeal |
| LCR (Galgani et al., 2012) |
? ? |
Australian court cases |
?? |
3.9K pairs of (case, catchphrases) |
Legal Language Modeling (LLM)
| Dataset |
Links |
Language |
Size |
| Pile of Law (Henderson et al., 2022) |
? ? |
?? |
~256GB of legal and administrative legal text |
Benchmarks
| Dataset |
Task |
Language |
Tasks |
| FairLex (Chalkidis et al., 2022) |
? ? |
?? ?? ?? ?? ?? |
Clasification (x1), legal judgement prediction (x3) |
| LexGLUE (Chalkidis et al., 2022) |
? ? |
?? |
Classsification (x6), multiple-choice QA (x1) |
Models
| Model |
Links |
Language |
Size |
| Legal-HeBERT (Chriqui et al., 2022) |
? ? |
?? |
110M |
| PoL-BERT-Large (Henderson et al., 2022) |
? ? |
?? |
336M |
| Italian-LEGAL-BERT (Licari and Comande, 2022) |
? ? |
?? |
110M |
| JuriBERT (Douka et al., 2021) |
? ? |
?? |
{6M, 15M, 42M, 110M} |
| Custom-LEGAL-BERT (Zheng et al., 2021) |
? ? |
?? |
110M |
| LEGAL-BERT (Chalkidis et al., 2020) |
? ? |
?? |
{35M, 110M} |
| LEGAL-GPT-{1,2} (Borchmann et al., 2020) |
? |
?? |
{117M, 1.5B} |
Books
- [
2017] Artificial Intelligence and Legal Analytics: New Tools for Law Practice in the Digital Age, K. Ashley. [link]
? Surveys
- [
2020-05] How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence, H. Zhong et al. [pdf]
- [
2019-09] A Brief History of the Changing Roles of Case Prediction in AI and Law, K. Ashley [pdf]
- [
2018-12] Deep learning in law: early adaptation and legal word embeddings trained on large corpora, I. Chalkidis et al. [pdf]
? Talks
- [
2019-06] Law as Data: The Promise and Challenges of Natural Language Processing for Legal Research, A. Dyevre. [slides]
- [
2019-04] Artificial Intelligence and Law – An Overview and History, H. Surden. [video]
? Conferences & Workshops
- The Natural Legal Language Processing (NLLP) Workshop [website]
- The International Conference on Artificial Intelligence and Law (ICAIL) [website]
- The International Conference on Legal Knowledge and Information Systems (JURIX) [website]
- The EXplainable AI in Law (XAILA) Workshop [website]
- The International Workshop on Juris-informatics (JURISIN) [website]
- The Competition on Legal Information Extraction/Entailment (COLIEE) [website]
- The International Workshop on Legal Information Retrieval [website]
