A Text Mining Approach for Automated Case Classification of Judicial Judgment
DOI:
https://doi.org/10.53560/PPASA(62-1)880Keywords:
Judicial Judgment, Judgment Classification, Pakistani Court, Machine Learning, Legal KnowledgeAbstract
Court judgments are based on legal reasoning, evidence, and judicial decisions. However, many judgments are difficult to understand due to their length and complex language. Judges and attorneys often cite legal rulings to interpret regulations effectively. Legal education for lawyers, judges, and trainees benefits from judicial insights into legal reasoning, legislative interpretation, and legal standards. Accurate classification of legal judgments requires sophisticated methodologies. The increasing number of new and pending cases adds to the courts' workload, highlighting the need for efficient classification. Since only a limited number of court rulings are decided each year, many cases remain unresolved and are carried forward to the following year. Despite efforts to improve classification accuracy across judicial judgment datasets, Pakistani court judgments still lack a proper classification system. To address this gap, we compiled a dataset of criminal judgments from the Pakistani High and Supreme Courts. We developed a machine learning model using neural networks and a transformer architecture to achieve accurate classification. Our approach incorporated Support Vector Machine (SVM), Long Short-Term Memory (LSTM), and DistilBERT models, taking into account the dataset’s volume and unique characteristics. Our research explores optimal classification algorithms tailored to Pakistan’s legal landscape. After comparison, the DistilBERT transformer model outperformed the others, achieving an accuracy of 98%. It demonstrated an exceptional ability to understand contextual semantics and effectively handle the complexities of multi-label classification in legal judgments. The contributions of our work include the development of a new dataset of Pakistani court rulings, advancements in legal research, and an automated case classification system designed to streamline judicial procedures and enhance access to justice.
References
W. Hwang, D. Lee, K. Cho, H. Lee, and M. Seo. A multi-task benchmark for korean legal language understanding and judgement prediction. Advances in Neural Information Processing Systems 35: 32537-32551 (2022).
V. Vaissnave and P. Deepalakshmi. A keyword-based multi-label text categorization in the Indian legal domain using bi-LSTM. Soft Computing: Theories and Applications: In Soft Computing: Theories and Applications. Advances in Intelligent Systems and Computing. T.K. Sharma, A.W. Ahn, O.P. Verma, and B.K. Panigrahi (Eds.). Volume 1380. Springer, Singapore pp. 213-227 (2022).
S. Agrawal. How To Read Court Judgments Quickly and Efficiently (2023).
https://www.scribd.com/document/628208457/How-to- Read-Court-Judgments-Quickly-and-Efficiently
B. Shukla, S. Gupta, A.K. Yadav, and D. Yadav. Challenges and issues in legal documents classification. AIP Conference Proceedings 2754(1): 020001 (2023).
J. Li, G. Zhang, L. Yu, and T. Meng. Research and design on cognitive computing framework for predicting judicial decisions. Journal of Signal Processing Systems 91(10): 1159-1167 (2019).
G. Sukanya and J. Priyadarshini. A meta-analysis of attention models on legal judgment prediction system. International Journal of Advanced Computer Science and Applications 12(2): 531-538 (2021).
A.S. Kwak, C. Jeong, J.W. Lim, and B. Min. A Korean Legal Judgment Prediction Dataset for Insurance Disputes. arXiv preprint arXiv 2401: 14654 (2024).
A.J. Rawat, S. Ghildiyal, and A.K. Dixit. Topic Modeling Techniques for Document Clustering and Analysis of Judicial Judgements. International Journal of Engineering Trends and Technology 70(11): 163-169 (2022).
A. Barman, D. Roy, D. Paul, I. Dutta, S.K. Guha, S. Karmakar, and S.K. Naskar. Convolutional Neural Networks can achieve binary bail judgement classification. arXiv preprint arXiv 2401: 14135 (2024).
K. Shirsat, A. Keni, P. Chavan, and M. Gosavi. Legal judgement prediction system. International Research Journal of Engineering and Technology 8(5): 734-738 (2021).
R.A. Shaikh, T.P. Sahu, and V. Anand. Predicting outcomes of legal cases based on legal factors using classifiers. Procedia Computer Science 167: 2393-2402 (2020).
V. Malik, R. Sanjay, S.K. Nigam, K. Ghosh, S.K. Guha, A. Bhattacharya, and A. Modi. ILDC for CJPE: Indian legal documents corpus for court judgment prediction and explanation. arXiv preprint arXiv 2105: 13562 (2021).
N. Sivaranjani, V.K.S. Ragavan, and G. Jawaherlalnehru. A Neural Network model for Indian Supreme Court legal judgment prediction. Journal of Intelligent & Fuzzy Systems 1: 235936 (2024).
S.S. Megala. Classification of Legal Judgement Summary using Conditional Random Field Algorithm. International Journal of Computer Sciences and Engineering 6(5): 23-33 (2018).
R. Sil and A. Roy. A novel approach on argument based legal prediction model using machine learning. IEEE International Conference on Smart Electronics and Communication, (10-12 September 2020), Trichy, India pp. 487-490 (2020).
P. Bhilare, N. Parab, N. Soni, and B. Thakur. Predicting outcome of judicial cases and analysis using machine learning. International Research Journal of Engineering and Technology 6(3): 326-330 (2019).
K. Javed and J. Li. Artificial intelligence in judicial adjudication: Semantic biasness classification and identification in legal judgement. Heliyon 10(9): e30184 (2024).
A. Fan, S. Wang, and Y. Wang. Legal Document Similarity Matching Based on Ensemble Learning. IEEE Access 12: 33910-33922 (2024).
Z. Yang, J. Ge, T. Hu, W. Yu, Y. Zheng, and Y. Dong. Text Classification of Judgement Documents Considering Sample Imbalance. 17th IEEE Conference on Industrial Electronics and Applications (16-19 December 2022), Chengdu, China pp. 1459-1462 (2022).
Y. Wang, J. Gao, and J. Chen. Deep learning algorithm for judicial judgment prediction based on BERT. 5th IEEE International Conference on Computing, Communication and Security (14-16 October 2020), Patna, India pp. 1-6 (2020).
B. Chen, Y. Li, S. Zhang, H. Lian, and T. He. A deep learning method for judicial decision support. 19th IEEE International Conference on Software Quality, Reliability and Security Companion (22-26 July 2019), Sofia, Bulgaria pp. 145-149 (2019).
S. Abbara, M. Hafez, A. Kazzaz, A. Alhothali, and A. Alsolami. ALJP: An Arabic Legal Judgment Prediction in Personal Status Cases Using Machine Learning Models. arXiv preprint arXiv 2309: 00238 (2023).
B. Strickson and B.D.L. Iglesia. Legal judgement prediction for UK courts. 3rd International Conference on Information Science and Systems (19 - 22 March, 2020), Cambridge, United Kingdom pp. 204-209 (2020).
A. Shelar, M. Moharir, V. Nama, J. Jaikumar, V. Patil, and A. Edupuganti. SmartLawAnnotator: A Machine Learning-Based Annotation Recommendation for Judgment Prediction. In: International Conference on Advances in Data-driven Computing and Intelligent Systems. Springer, Singapore pp. 415-430 (2023).
F. de Arriba-Pérez, S. García-Méndez, F.J. González-Castaño, and J. González-González. Explainable machine learning multi-label classification of Spanish legal judgements. Journal of King Saud University-Computer and Information Sciences 34(10): 10180-10192 (2022).
J. Zahir. Prediction of court decision from a rabic documents using deep learning. Expert Systems 40(6): e13236 (2023).
M. Medvedeva, A. Üstün, X. Xu, M. Vols, and M. Wieling. Automatic judgement forecasting for pending applications of the European Court of Human Rights. Proceedings of the Fifth Workshop on Automatec Semantic Analysis of Information in Legal Text. CEUR Workshop Proceedings pp. 12-23 (2021).
C. O'Sullivan and J. Beel. Predicting the outcome of judicial decisions made by the european court of human rights. arXiv preprint arXiv 1912: 10819 (2019).
M. Medvedeva, M. Vols, and M. Wieling. Judicial decisions of the European Court of Human Rights: Looking into the crystal ball. 13th Annual Conference on empirical legal studies (9-10 November, 2018), University of Michigan Law School, USA pp. 24 (2018).
M. Medvedeva, M. Vols, and M. Wieling. Using machine learning to predict decisions of the European Court of Human Rights. Artificial Intelligence and Law 28(2): 237-266 (2020).
X. Zheng, Y. Liu, J. Jiang, L.M. Thomas, and N. Su. Predicting the litigation outcome of PPP project disputes between public authority and private partner using an ensemble model. Journal of Business Economics and Management 22(2): 320-345 (2021).
Y. Fang, X. Tian, H. Wu, S. Gu, Z. Wang, F. Wang, and Y. Weng. Few-shot learning for Chinese legal controversial issues classification. IEEE Access 8: 75022-75034 (2020).
L. Liu, D. An, Y. Wang, X. Ma, and C. Jiang. Research on legal judgment prediction based on Bert and LSTM-CNN fusion model. 3rd IEEE world symposium on artificial intelligence (18-20 June 2021), Guangzhou, China pp. 41-45 (2021).
Y. Liu and Y. Chen. A two-phase sentiment analysis approach for judgement prediction. Journal of Information Science 44(5): 594-607 (2018).
https://www.pakistanlawsite.com/Login/MainPage
M. Omar, S. Choi, D. Nyang, and D. Mohaisen. Robust natural language processing: Recent advances, challenges, and future directions. IEEE Access 10: 86038-86056 (2022).
T.N. Fitria. QuillBot as an online tool: Students’ alternative in paraphrasing and rewriting of English writing. Englisia: Journal of Language, Education, and Humanities 9(1): 183-196 (2021).
K. Clark, U. Khandelwal, O. Levy, and C.D. Manning. What Does Bert Look At? An Analysis of Bert’s Attention. arXiv preprint arXiv 1906: 04341 (2019).
S. Sarica and J. Luo. Stopwords in technical language processing. Plos One 16(8): e0254937 (2021).
R.C. Staudemeyer and E.R. Morris. Understanding LSTM-a tutorial into long short-term memory recurrent neural networks. arXiv preprint arXiv 1909: 09586 (2019).
G. Sukanya and J. Priyadarshini. Analysis on word embedding and classifier models in legal analytics. AIP Conference Proceedings 2802(1): 140001 (2024).
D.W. Otter, J.R. Medina, and J.K. Kalita. A survey of the usages of deep learning for natural language processing. IEEE transactions on Neural Networks and Learning Systems 32(2): 604-624 (2020).
