In recent years, there has been a remarkable transformation in our interaction with legal texts due to the widespread utilization and adoption of natural language processing technology. This technology has advanced the analysis and enhanced the understanding of complex legal terminology and contexts. Moreover, the emergence of recent generative large language models (LLMs), particularly ChatGPT, has also introduced a revolutionary contribution to the way legal texts can be processed and comprehended. This paper focuses on the development of a cooperative legal question-answering LLM-based chatbot. Our work involves formulating a set of legal questions pertaining to Palestinian cooperatives and their associated regulations. We compare the auto-generated answers provided by the chatbot with correspondences prepared by a legal expert.To assess the chatbot’s performance, we evaluate its responses to 50 queries generated by the legal expert and compare them to their relevance judgments. The results indicate that the chatbot achieved an impressive overall accuracy rate of 82% in answering the queries, with an F1 score equivalent to 79%.
Authors
Mohammed Maree
Rabee Al-Qasem
and Banan Tantour
Pages From
551
Pages To
558
ISSN
2511-2112
Journal Name
International Journal of Information Technology
Volume
16
Issue
1
Keywords
Large language models, Artificial intelligence, Chatbots, Natural language processing, Question answering, Legal text
Abstract