Detecting AI-Generated Essays with Lexical Stylometric Features using Machine Learning
DOI:
https://doi.org/10.31436/ijpcc.v12i2.699Keywords:
Machine Learning, lexical features, stylometric, AI-Generated essays, academic writing, ChatGPTAbstract
The increasing use of generative artificial intelligence (AI) in academic writing has intensified concerns about authorship authenticity. This study investigates whether lexical stylometric features alone can distinguish AI-generated essays from human-written essays. A balanced dataset of 200 undergraduate essays (100 AI-generated and 100 human-written) was analyzed using vocabulary richness measures, lexical density, word usage distributions, and word-level n-grams. Logistic Regression, Decision Tree, and Support Vector Machine classifiers were applied to evaluate performance. Results indicate that lexical features provide strong discriminatory capability across the three models with accuracy ranging between 90 – 97.5%, suggesting that effective AI authorship detection can be achieved without complex feature engineering or ensemble methods. The findings support the feasibility of a lightweight and interpretable detection framework for educational contexts.
References
T. Kehkashan, R. A. Riaz, A. S. Al Shamayleh, A. Akhunzada, N. Ali, M. Hamza, and F. Akbar, “AI generated text detection: A comprehensive review of methods, datasets, and applications,” Computer Science Review, vol. 58, 100793, 2025.
B. Guo, X. Zhang, Z. Wang, M. Jiang, J. Nie, Y. Ding, J. Yue, and Y. Wu, “How close is ChatGPT to human experts? Comparison corpus, evaluation, and detection,” Cornell University, 2023. Preprint at arXiv. arXiv:2301.07597v1
H. Alamleh, A. A. S. AlQahtani and A. ElSaid, "Distinguishing human-written and ChatGPT-generated text using machine learning," Systems and Information Engineering Design Symposium (SIEDS), Charlottesville, VA, USA, pp. 154-158, 2023, doi: 10.1109/SIEDS58326.2023.10137767.
N. Krawczyk, B. Probierz and J. Kozak, “Towards AI-generated essay classification using numerical text representation,” Applied Sciences, vol. 14(21), 9795, 2024.
A. Masih, B. Afzal, S. Firdoos, J. Mahmood, A. Ali, M. S. Abdulnabi and D. M. Balungu, “Classifying human vs. AI text with machine learning and explainable transformer models,” Scientific Reports, vol. 15, 43310, 2025.
A. Agarwal and P. Sharma, “A comparative study of stylometric features for authorship discrimination using machine learning,” Proceedings of the 2023 International Conference on Learning Representations (ICLR 2023), pp. 1–12, 2023.
A. Abbasi, A. R. Javed, F. Iqbal, Z. Jalil, T. R. Gadekallu, and N. Kryvinska, “Authorship identification using ensemble learning,” Scientific Reports, vol. 14 (1), 6789, 2024.
A. Yadagiri, L. Shree, S. Parween, A. Raj, S. Maurya, and P. Pakray, “Detecting AI-generated text with pre-trained models using linguistic features,” Proceedings of the 21st International Conference on Natural Language Processing (ICON), pp.188-196, AU-KBC Research Centre, Chennai, India, NLP Association of India (NLPAI), 2024.
H. Desaire, A. E. Chua, M. Isom, R. Jarosova, D. Hua, “Distinguishing academic science writing from humans or ChatGPT with over 99% accuracy using off-the-shelf machine learning tools,” Cell Reports Physical Science, vol. 4 (6), 101426, 2023.
L. J. A. Chang, “Detecting AI generated text: A Comparative study of machine learning algorithms,” in Proceedings of the 29th World Multi Conference on Systemics, Cybernetics and Informatics (WMSCI 2025), pp. 294–299, 2025.
S. Ramnath, K. Pandey, E. Boschee, and X. Ren, “CAVE: Controllable authorship verification explanations,” in Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2025 – Long Papers), pp. 8939–8961, Association for Computational Linguistics, 2025.
J. Zu, M. Fauss, and C. Li, “Effects of generation model on detecting AI-generated essays in a writing test,” Educational Testing Service, Princeton, NJ, 2024.
J. Kubrusly, E. D. dos Santos, and L. S. Pelegrino, “Detecting AI-generated texts using machine learning models,” Communications in Statistics: Case Studies, Data Analysis and Applications, vol. 11 (4), pp. 495-512, 2025.
C. Chaka, “Reviewing the performance of AI detection tools in differentiating between AI-generated and human-written texts: A literature and integrative hybrid review,” Journal of Applied Learning & Teaching, vol. 7(1), pp. 115–126, 2024.
