Cited By
View all- Martinez-Gil J(2023)A survey on legal question–answering systemsComputer Science Review10.1016/j.cosrev.2023.10055248:COnline publication date: 1-May-2023
Question Answering of Bar Exams by Paraphrasing and Legal Text Analysis
Authors: 
Mi-Young Kim, Ying Xu, Yao Lu, Randy GoebelAuthors Info & Claims
Mi-Young Kim, Ying Xu, Yao Lu, Randy GoebelAuthors Info & ClaimsPages 299 - 313
Abstract
Our legal question answering system combines legal information retrieval and textual entailment, and exploits paraphrasing and sentence-level analysis of queries and legal statutes. We have evaluated our system using the training data from the competition on legal information extraction/entailment (COLIEE)-2016. The competition focuses on the legal information processing required to answer yes/no questions from Japanese legal bar exams, and it consists of three phases: legal ad-hoc information retrieval (Phase 1), textual entailment (Phase 2), and a combination of information retrieval and textual entailment (Phase 3). Phase 1 requires the identification of Japan civil law articles relevant to a legal bar exam query. For this phase, we have used an information retrieval approach using TF-IDF and a Ranking SVM. Phase 2 requires decision on yes/no answer for previously unseen queries, which we approach by comparing the approximate meanings of queries with relevant articles. Our meaning extraction process uses a selection of features based on a kind of paraphrase, coupled with a condition/conclusion/exception analysis of articles and queries. We also identify synonym relations using word embedding, and detect negation patterns from the articles. Our heuristic selection of attributes is used to build an SVM model, which provides the basis for ranking a decision on the yes/no questions. Experimental evaluation show that our method outperforms previous methods. Our result ranked highest in the Phase 3 in the COLIEE-2016 competition.
References
[1]
Jones KS Willett P A statistical interpretation of term specicity and its application in retrieval Document Retrieval Systems 1988 London Taylor Graham Publishing 132-142
[2]
Joachims, T.: Optimizing search engines using clickthrough data. In: Proceedings of 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2002, pp. 133–142. ACM, New York (2002)
[3]
Maxwell, K.T., Oberlander, J., Croft, W.B.: Feature-based selection of dependency paths in ad hoc information retrieval. In: Proceedings of 51st Annual Meeting of the Association for Computational Linguistics, (vol. 1: Long Papers), pp. 507–516. Association for Computational Linguistics, Sofia, August 2013
[4]
Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, and Witten IH The WEKA data mining software: an update SIGKDD Explor. 2009 11 1 10-18
[5]
Bdour WN and Gharaibeh NK Development of yes/no Arabic question answering system Int. J. Artif. Intell. Appl. 2013 4 1 51-63
[6]
Nielsen, R.D., Ward, W., Martin, J.H.: Toward dependency path based entailment. In: Proceedings of 2nd PASCAL Challenges Workshop on RTE (2006)
[7]
Zanzotto, F.M., Moschitti, A., Pennacchiotti, M., Pazienza, M.T.: Learning textual entailment from examples. In: Proceedings of 2nd PASCAL Challenges Workshop on RTE (2006)
[8]
Harmeling, S.: An extensible probabilistic transformation-based approach to the third recognizing textual entailment challenge. In: Proceedings of ACL PASCAL Workshop on Textual Entailment and Paraphrasing (2007)
[9]
Marsi, E., Krahmer, E., Bosma, W.: Dependency-based paraphrasing for recognizing textual entailment. In: Proceedings of ACL PASCAL Workshop on Textual Entailment and Paraphrasing (2007)
[10]
Kim, M.-Y., Xu, Y., Goebel, R.: Alberta-KXG: legal question answering using ranking SVM and syntactic/semantic similarity. In: 8th International Workshop on Juris-Informatics (JURISIN), 2014
[11]
Kim, M.-Y., Xu, Y., Goebel, R.: A convolutional neural network in legal question answering. In: JURISIN Workshop (2015)
[12]
Sultan MA, Bethard S, and Sumner T Back to basics for monolingual alignment: exploiting word similarity and contextual evidence Trans. Assoc. Comput. Linguist. 2014 2 219-230
[13]
Berant, J., Percy, L.: Semantic parsing via paraphrasing. In: Proceedings of Conference of the Association for Computational Linguistics (ACL), pp. 1415–1425 (2014)
[14]
Zhang, W., Ming, Z., Zhang, Y., Liu, T., Chua, T.S.: Exploring key concept paraphrasing based on pivot language translation for question retrieval. In: AAAI, pp. 410–416 (2015)
[15]
Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S.J., McClosky, D.: The Stanford CoreNLP natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55–60 (2014)
[16]
Kim, M.-Y., Goebel, R., Kano, Y., Satoh, K.: COLIEE-2016: evaluation of the competition on legal information extraction and entailment. In: Tenth International Workshop on Juris-Informatics (JURISIN) (2016)
[17]
Carvalho, D.S., Tran, V.D., Tran, K.V., Lai, V.D., Nguyen, M.-L.: Lexical to discourse-level corpus modeling for legal question answering. In: Tenth International Workshop on Juris-Informatics (JURISIN) (2016). (Submission ID: JNLN1)
[18]
Taniguchi, R., Kano, Y.: Legal yes/no question answering system using case-role analysis. In: Tenth International Workshop on Juris-Informatics (JURISIN) (2016). (Submission ID: KIS)
[19]
Kim, K., Heo, S., Jung, S., Hong, K., Rhim, Y.-Y.: An ensemble based legal information retrieval and entailment system. In: Tenth International Workshop on Juris-Informatics (JURISIN) (2016). (Submission ID: iLis7)
[20]
Do, P.-K., Nguyen, H.-T., Tran, C.-X., Nguyen, M.-T., Minh, N.L.: Legal question answering using ranking SVM and deep convolutional neural network. In: Tenth International Workshop on Juris-Informatics (JURISIN) (2016). (Submission ID: JNLN3)
[21]
Onodera, D., Yoshioka, M.: Civil code article information retrieval system based on legal terminology and civil code article structure. In: Tenth International Workshop on Juris-Informatics (JURISIN) (2016). (Submission ID: HUKB)
[22]
Nguyen, T.-S., Phan, V.-A., Nguyen, T.-H., Trieu, H.-L., Chau, N.-P., Pham, T.-T., Nguyen, L.-M.: Legal information extraction/entailment using SVM-ranking and tree-based convolutional neural network. In: Tenth International Workshop on Juris-Informatics (JURISIN) (2016). (Submission ID: JNLN2)
[23]
John, A.K., Di Caro, L., Boella, G., Bartolini, C.: Team-normas’ participation at the COLIEE 2016 bar legal exam competition. In: Tenth International Workshop on Juris-Informatics (JURISIN) (2016). (Submission ID: N01)
Index Terms
- Question Answering of Bar Exams by Paraphrasing and Legal Text Analysis
Index terms have been assigned to the content through auto-classification.
Recommendations
Textual Entailment in Legal Bar Exam Question Answering Using Deep Siamese Networks
New Frontiers in Artificial IntelligenceAbstractEvery day a large volume of legal documents are produced, and lawyers need support for their analysis, especially in corporate litigation. Typically, corporate litigation has the aim of finding evidence for or against the litigation claims. ...
Answering Yes/No Questions in Legal Bar Exams
New Frontiers in Artificial IntelligenceAbstractThe development of Question Answering (QA) systems has become important because it reveals research issues that require insight from a variety of disciplines, including Artificial Intelligence, Information Extraction, Natural Language Processing, ...
Legal Question Answering Using Ranking SVM and Syntactic/Semantic Similarity
New Frontiers in Artificial IntelligenceAbstractWe describe a legal question answering system which combines legal information retrieval and textual entailment. We have evaluated our system using the data from the first competition on legal information extraction/entailment (COLIEE) 2014. The ...
Comments
Information & Contributors
Information
Published In
Nov 2016
346 pages
ISBN:978-3-319-61571-4
DOI:10.1007/978-3-319-61572-1
© Springer International Publishing AG 2017.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 14 November 2016
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 31 Jan 2025
Other Metrics
Citations
Cited By
View all- Martinez-Gil J(2023)A survey on legal question–answering systemsComputer Science Review10.1016/j.cosrev.2023.10055248:COnline publication date: 1-May-2023