Article

Legal Question Answering Using Ranking SVM and Syntactic/Semantic Similarity

Authors:

Randy GoebelAuthors Info & Claims

New Frontiers in Artificial Intelligence: JSAI-isAI 2014 Workshops, LENLS, JURISIN, and GABA, Kanagawa, Japan, October 27–28, 2014, Revised Selected Papers

Pages 244 - 258

https://doi.org/10.1007/978-3-662-48119-6_18

Published: 27 October 2014 Publication History

Abstract

We describe a legal question answering system which combines legal information retrieval and textual entailment. We have evaluated our system using the data from the first competition on legal information extraction/entailment (COLIEE) 2014. The competition focuses on two aspects of legal information processing related to answering yes/no questions from Japanese legal bar exams. The shared task consists of two phases: legal ad hoc information retrieval and textual entailment. The first phase requires the identification of Japan civil law articles relevant to a legal bar exam query. We have implemented two unsupervised baseline models (tf-idf and Latent Dirichlet Allocation (LDA)-based Information Retrieval (IR)), and a supervised model, Ranking SVM, for the task. The features of the model are a set of words, and scores of an article based on the corresponding baseline models. The results show that the Ranking SVM model nearly doubles the Mean Average Precision compared with both baseline models. The second phase is to answer “Yes” or “No” to previously unseen queries, by comparing the meanings of queries with relevant articles. The features used for phase two are syntactic/semantic similarities and identification of negation/antonym relations. The results show that our method, combined with rule-based model and the unsupervised model, outperforms the SVM-based supervised model.

References

[1]

Jones KS Willett P A statistical interpretation of term specificity and its application in retrieval Document Retrieval Systems 1988 London Taylor Graham Publishing 132-142

[2]

Blei DM, Ng AY, and Jordan MI Latent dirichlet allocation J. Mach. Learn. Res. 2003 3 993-1022

[3]

Joachims, T.: Optimizing search engines using clickthrough data. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2002, pp. 133–142. ACM, New York (2002)

[4]

Wei, X., Croft, W.B.: LDA-based document models for ad-hoc retrieval. In: Proceeding of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2006, pp. 178–185. ACM, New York (2006)

[5]

Maxwell, K.T., Oberlander, J., Croft, W.B.: Feature-based selection of dependency paths in ad hoc information retrieval. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, vol. 1, Long Papers, pp. 507–516. Association for Computational Linguistics, Sofia, August 2013

[6]

Kim, M-Y., Xu, Y., Goebel, R., Satoh, K.: Answering yes/no questions in legal bar exams. In: JURISIN (2013)

[7]

Jikoun, V., de Rijke, M.: Recognizing textual entailment using lexical similarity. In: Proceedings of the PASCAL Challenges Workshop on RTE (2005)

[8]

MacCartney, B., Grenager, T., de Marneffe, M.-C., Cer, D., Manning, C.D.: Learning to recognize features of valid textual entailments. In Proceedings of HLT-NAACL (2006)

[9]

Sno, R., Vanderwende, L., Menezes, A.: Effectively using syntax for recognizing false entailment. In: Proceedings of HLT-NAACL (2006)

[10]

Lai, A., Hockenmaier, J.: Illinois-LH: a denotational and distributional approach to semantics. In: Proceedings of SemEval 2014: International Workshop on Semantic Evaluation (2014)

[11]

Ohno S and Hamanishi M New Synonym Dictionary 1981 Tokyo Kadokawa Shoten

[12]

Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, and Witten IH The WEKA data mining software: an update SIGKDD Explor. 2009 11 1 10-18

[13]

Kim, M-Y., Kang, S-J., Lee, J-H.: Resolving ambiguity in inter-chunk dependency parsing. In: Proceedings of 6th Natural Language Processing Pacific Rim Symposium, pp. 263–270 (2001)

[14]

Walas M Gelbukh A How to answer yes/no spatial questions using qualitative reasoning? Computational Linguistics and Intelligent Text Processing 2012 Heidelberg Springer 330-341

[15]

Marelli, M., Bentivogli, L., Baroni, M., Bernardi, R., Menini, S., Zamparelli, R.: SemEval-2014 task 1: evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment. In: Proceedings of SemEval 2014: International Workshop on Semantic Evaluation (2014)

[16]

Bdour WN and Gharaibeh NK Development of yes/no arabic question answering system Int. J. Artif. Intell. Appl. 2013 4 1 51-63

[17]

Kouylekov, M., Magnini, B.: Tree edit distance for recognizing textual entailment: estimating the cost of insertion. In: Proceedings of the second PASCAL Challenges Workshop on RTE (2006)

[18]

Vanderwende, L., Menezes, A., Snow, R.: Microsoft research at RTE-2: syntactic contributions in the entailment task: an implementation. In: Proceedings of the Second PASCAL Challenges Workshop on RTE (2006)

[19]

Nielsen, R.D., Ward, W., Martin, J.H.: Toward dependency path based entailment. In: Proceedings of the Second PASCAL Challenges Workshop on RTE (2006)

[20]

Zanzotto, F.M., Moschitti, A., Pennacchiotti, M., Pazienza, M.T.: Learning textual entailment from examples. In: Proceedings of the Second PASCAL Challenges Workshop on RTE (2006)

[21]

Harmeling, S.: An extensible probabilistic transformation-based approach to the third recognizing textual entailment challenge. In: Proceedings of ACL PASCAL Workshop on Textual Entailment and Paraphrasing (2007)

[22]

Marsi, E., Krahmer, E., Bosma, W.: Dependency-based paraphrasing for recognizing textual entailment. In: Proceedings of ACL PASCAL Workshop on Textual Entailment and Paraphrasing (2007)

Cited By

Martinez-Gil J(2023)A survey on legal question–answering systemsComputer Science Review10.1016/j.cosrev.2023.10055248:COnline publication date: 1-May-2023
https://dl.acm.org/doi/10.1016/j.cosrev.2023.100552
Sulis EHumphreys LAudrito DDi Caro L(2021)Exploiting Textual Similarity Techniques in Harmonization of LawsAIxIA 2021 – Advances in Artificial Intelligence10.1007/978-3-031-08421-8_13(185-197)Online publication date: 1-Dec-2021
https://dl.acm.org/doi/10.1007/978-3-031-08421-8_13
Yang ZWang PZhang LShou LXu W(2019)A Recurrent Attention Network for Judgment PredictionArtificial Neural Networks and Machine Learning – ICANN 2019: Text and Time Series10.1007/978-3-030-30490-4_21(253-266)Online publication date: 17-Sep-2019
https://dl.acm.org/doi/10.1007/978-3-030-30490-4_21

Recommendations

Question Answering of Bar Exams by Paraphrasing and Legal Text Analysis
New Frontiers in Artificial Intelligence
Abstract
Our legal question answering system combines legal information retrieval and textual entailment, and exploits paraphrasing and sentence-level analysis of queries and legal statutes. We have evaluated our system using the training data from the ...
Two-step cascaded textual entailment for legal bar exam question answering
ICAIL '17: Proceedings of the 16th edition of the International Conference on Articial Intelligence and Law

Our legal question answering system combines legal information retrieval and textual entailment, and exploits semantic information using a logic-based representation. We have evaluated our system using the data from the competition on legal information ...
Answering Yes/No Questions in Legal Bar Exams
New Frontiers in Artificial Intelligence
Abstract
The development of Question Answering (QA) systems has become important because it reveals research issues that require insight from a variety of disciplines, including Artificial Intelligence, Information Extraction, Natural Language Processing, ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

New Frontiers in Artificial Intelligence: JSAI-isAI 2014 Workshops, LENLS, JURISIN, and GABA, Kanagawa, Japan, October 27–28, 2014, Revised Selected Papers

Oct 2014

352 pages

ISBN:978-3-662-48118-9

DOI:10.1007/978-3-662-48119-6

Editors:
Tsuyoshi Murata
Tokyo Institute of Technology, Tokyo, Japan
,
Koji Mineshima
Ochanomizu University, Tokyo, Japan
,
Daisuke Bekki
Department Information Science, Faculty of Science, Ochanomizu University , Tokyo, Japan

© Springer-Verlag Berlin Heidelberg 2015.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 27 October 2014

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 31 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Martinez-Gil J(2023)A survey on legal question–answering systemsComputer Science Review10.1016/j.cosrev.2023.10055248:COnline publication date: 1-May-2023
https://dl.acm.org/doi/10.1016/j.cosrev.2023.100552
Sulis EHumphreys LAudrito DDi Caro L(2021)Exploiting Textual Similarity Techniques in Harmonization of LawsAIxIA 2021 – Advances in Artificial Intelligence10.1007/978-3-031-08421-8_13(185-197)Online publication date: 1-Dec-2021
https://dl.acm.org/doi/10.1007/978-3-031-08421-8_13
Yang ZWang PZhang LShou LXu W(2019)A Recurrent Attention Network for Judgment PredictionArtificial Neural Networks and Machine Learning – ICANN 2019: Text and Time Series10.1007/978-3-030-30490-4_21(253-266)Online publication date: 17-Sep-2019
https://dl.acm.org/doi/10.1007/978-3-030-30490-4_21

View Options

View options

Figures

Tables

Media

View Table of Conten