The Language of AI and Human Poetry: A Comparative Lexicometric Study

Afendi Hamat

Abstract


This study conducts a lexicometric analysis to compare the lexical richness and diversity in poetry generated by AI models with that of human poets. Employing a robust dataset that includes 1,333 AI-generated poems and 517 human-authored poems across seven distinct poetic eras, six key lexical metrics—Maas Index, MTLD, MATTR, HD-D, Hapax Legomenon Ratio, and Lexical Density—were applied for comparative analysis. The lexical characteristics of the poems were studied through a series of statistical tests and machine learning techniques, including Mann-Whitney U tests, Cliff's Delta, and Random Forest classification. The findings reveal a marked lexical superiority in human poetry, evidenced by significant differences and large effect sizes in all metrics except Lexical Density. HD-D emerged as the most discriminating factor, adeptly differentiating human poetry from its AI-generated counterpart. Further analysis identified the GPT-4 model as exhibiting the closest alignment to human poetry in terms of lexical attributes. The study discusses these outcomes in the context of AI's evolving linguistic competencies, shedding light on the inherent challenges and future prospects of AI in creative writing. Thus, this research provides an empirical framework for assessing AI’s language generation abilities and sets the stage for further interdisciplinary exploration into the frontiers of artificial creativity.  

 

Keywords: artificial intelligence; lexicometry; machine learning; lexical analysis; poetry


Full Text:

PDF

References


Abu-Rabiah, E. (2023). Evaluating L2 Vocabulary Development Features Using Lexical Density And Lexical Diversity Measures. LLT Journal: Journal on Language and Language Teaching, 26(1), 168–182. https://doi.org/10.24071/llt.v26i1.5841

Alowedi, N. A., & Al-Ahdal, A. A. M. H. (2023). Artificial Intelligence based Arabic-to-English machine versus human translation of poetry: An analytical study of outcomes. Journal of Namibian Studies : History Politics Culture, 33. https://doi.org/10.59670/jns.v33i.800

Atkinson, P., & Barker, R. (2023). AI and the social construction of creativity. Convergence: The International Journal of Research into New Media Technologies, 29(4), 1054–1069. https://doi.org/10.1177/13548565231187730

Attak, E. H. (2023). Tax Policy and Entrepreneurship (pp. 421–442). https://doi.org/10.4018/978-1-6684-8781-5.ch019

Ayinde, L., Wibowo, M. P., Ravuri, B., & Bin Emdad, F. (2023). ChatGPT as an important tool in organisational management: A review of the literature. Business Information Review, 40(3), 137–149. https://doi.org/10.1177/02663821231187991

Belfi, A. M., Vessel, E. A., & Starr, G. G. (2018). Individual Ratings of Vividness Predict Aesthetic Appeal in Poetry. Psychology of Aesthetics, Creativity, and the Arts, 12(3), 341–350. https://doi.org/10.1037/aca0000153

Benoit, K. (2020). The SAGE Handbook of Research Methods in Political Science and International Relations (L. Curini & R. Franzese, Eds.; Vol. 2). SAGE Publications Ltd. https://doi.org/10.4135/9781526486387

Bestgen, Y. (n.d.). Measuring Lexical Diversity in Texts: The Twofold Length Problem. https://doi.org/https://doi.org/10.48550/arXiv.2307.04626

Brglez, M., & Vintar, Š. (2022). Lexical Diversity in Statistical and Neural Machine Translation. Information. https://doi.org/10.3390/info13020093

Chen, R., & Liu, H. (2014). Quantitative Aspects of Journal of Quantitative Linguistics. Journal of Quantitative Linguistics, 21(4), 299–340. https://doi.org/10.1080/09296174.2014.944327

Collentine, J. (2004). The Effects of Learning Contexts on Morphosyntactic and Lexical Development. Studies in Second Language Acquisition. https://doi.org/10.1017/s0272263104262040

Cramer, J. (2016, May 16). Can Robot Artists Create Human-Quality Work? Not Yet. https://home.dartmouth.edu/news/2016/05/can-robot-artists-create-human-quality-work-not-yet

Fergadiotis, G., Wright, H. H., & West, T. M. (2013). Measuring lexical diversity in narrative discourse of people with aphasia. American Journal of Speech-Language Pathology, 22(2). https://doi.org/10.1044/1058-0360(2013/12-0083)

Hakami, A., Alqarni, R., Almutairi, M., & Alhothali, A. (2021). Arabic Poems Generation using LSTM, Markov-LSTM and Pre-Trained GPT-2 Models. Advances in Machine Learning, 139–147. https://doi.org/10.5121/csit.2021.111512

Heng, R., Pu, L., & Liu, X. (2023). The Effects of Genre on the Lexical Richness of Argumentative and Expository Writing by Chinese EFL Learners. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2022.1082228

Hong, J.-W., & Curran, N. M. (2019). Artificial Intelligence, Artists, and Art. ACM Transactions on Multimedia Computing, Communications, and Applications, 15(2s), 1–16. https://doi.org/10.1145/3326337

Hutson, J., & Schnellmann, A. (2023). The Poetry of Prompts: The Collaborative Role of Generative Artificial Intelligence in the Creation of Poetry and the Anxiety of Machine Influence. Faculty Scholarship, 462. https://digitalcommons.lindenwood.edu/faculty-research-papers/462/

Jarvis, S. (2013). Capturing the Diversity in Lexical Diversity. Language Learning, 63(SUPPL. 1), 87–106. https://doi.org/10.1111/j.1467-9922.2012.00739.x

Kantosalo, A., & Riihiaho, S. (2019). Quantifying co-creative writing experiences. Digital Creativity, 30(1), 23–38. https://doi.org/10.1080/14626268.2019.1575243

Köbis, N., & Mossink, L. D. (2021). Artificial intelligence versus Maya Angelou: Experimental evidence that people cannot differentiate AI-generated from human-written poetry. Computers in Human Behavior, 114. https://doi.org/10.1016/j.chb.2020.106553

Kubi, B. (2018). Bemoaning of Love: An Aspect of Ga Women’s Discourse on Love in Adaawe Song- Texts. International Journal of Comparative Literature and Translation Studies, 6(2), 43. https://doi.org/10.7575/aiac.ijclts.v.6n.2p.43

Labaca-Castro, R. (2023). Generative Adversarial Nets. In Machine Learning under Malware Attack (pp. 73–76). Springer Fachmedien Wiesbaden. https://doi.org/10.1007/978-3-658-40442-0_9

Lee, H.-K. (2022). Rethinking creativity: creative industries, AI and everyday creativity. Media, Culture & Society, 44(3), 601–612. https://doi.org/10.1177/01634437221077009

Lo, K.-L., Ariss, R., & Kurz, P. (2022). GPoeT-2: A GPT-2 Based Poem Generator. http://arxiv.org/abs/2205.08847

Mandják, T., Lavissière, A., Hofmann, J., Bouchery, Y., Lavissière, M. C., Faury, O., & Sohier, R. (2019). Port marketing from a multi-disciplinary perspective: A systematic literature review and lexicometric analysis. Transport Policy, 84, 50–72. https://doi.org/10.1016/j.tranpol.2018.11.011

McCarthy, P. M., & Jarvis, S. (2010). MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, 42(2), 381–392. https://doi.org/10.3758/BRM.42.2.381

Meng, Q. (2021). The Pedagogy of Corpus-Aided English-Chinese Translation From a Critical &Amp; Creative Perspective. Theory and Practice in Language Studies. https://doi.org/10.17507/tpls.1101.04

Obermeier, C., Menninghaus, W., von Koppenfels, M., Raettig, T., Schmidt-Kassow, M., Otterbein, S., & Kotz, S. A. (2013). Aesthetic and Emotional Effects of Meter and Rhyme in Poetry. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2013.00010

Parsons, L. T., & Pinkerton, L. (2022). Poetry and Prose as Methodology: A Synergy of Knowing. Methodological Innovations. https://doi.org/10.1177/20597991221087150

Pulvirenti, G., & Gambino, R. (2022). Einbildungskraft (Imagination). Goethe-Lexicon of Philosophical Concepts, 2(1). https://doi.org/10.5195/glpc.2022.59

Rahmeh, H. (2023). Digital Verses Versus Inked Poetry: Exploring Readers’ Response to AI-Generated and Human-Authored Sonnets. Scholars International Journal of Linguistics and Literature, 6(09), 372–382. https://doi.org/10.36348/sijll.2023.v06i09.002

Rezwana, J., & Maher, M. L. (2023). Designing Creative AI Partners with COFI: A Framework for Modeling Interaction in Human-AI Co-Creative Systems. ACM Transactions on Computer-Human Interaction, 30(5), 1–28. https://doi.org/10.1145/3519026

Richter, A., Ng, K., & Fallah, B. (2019). Bibliometric and text mining approaches to evaluate landfill design standards. Scientometrics, 118. https://doi.org/10.1007/s11192-019-03011-4

Rockmore, D. (2020, January 7). What Happens When Machines Learn to Write Poetry. The New Yorker. https://www.newyorker.com/culture/annals-of-inquiry/the-mechanical-muse

Sennrich, R., Haddow, B., & Birch, A. (2016). Neural Machine Translation of Rare Words with Subword Units. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1715–1725. https://doi.org/10.18653/v1/P16-1162

Sugunan, D. (2022). Multifarious nature in Bharathy’s Lyrical Literature. International Research Journal of Tamil, 4(SPL 2), 1–7. https://doi.org/10.34256/irjt22s21

Sujatna, E. T. S., Heriyanto, H., & Andri, S. (2021). Lexical Density and Variation in Indonesian Folklores in English Student Textbooks: An SFL Study. Leksika Jurnal Bahasa Sastra Dan Pengajarannya. https://doi.org/10.30595/lks.v15i2.11102

Sunico, R. C. (2021). The Poetry of Simple Words. Perspectives in the Arts and Humanities Asia. https://doi.org/10.13185/paha2020.10208

Swett, B. A., Hahn, E. N., & Llorens, A. J. (2021). Designing Robots for the Battlefield: State of the Art. In Robotics, AI, and Humanity (pp. 131–146). Springer International Publishing. https://doi.org/10.1007/978-3-030-54173-6_11

Thorndike, E. L. (1921). The Teacher’s Word Book. Teacher’s College, Columbia University. https://pure.mpg.de/rest/items/item_2395369_2/component/file_2395368/content

Uccelli, P., Galloway, E. P., Barr, C. D., Meneses, A., & Dobbs, C. L. (2015). Beyond Vocabulary: Exploring Cross-Disciplinary Academic-Language Proficiency and Its Association With Reading Comprehension. Reading Research Quarterly. https://doi.org/10.1002/rrq.104

Van de Cruys, T. (2020). Automatic Poetry Generation from Prosaic Text. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2471–2480. https://doi.org/10.18653/v1/2020.acl-main.223

Vitta, J. P., Nicklin, C., & Albright, S. W. (2023). Academic Word Difficulty and Multidimensional Lexical Sophistication: An English‐for‐academic‐purposes‐focused Conceptual Replication of Hashimoto and Egbert (2019). Modern Language Journal. https://doi.org/10.1111/modl.12835

Wassiliwizky, E., Koelsch, S., Wagner, V., Jacobsen, T., & Menninghaus, W. (2017). The emotional power of poetry: neural circuitry, psychophysiology and compositional principles. Social Cognitive and Affective Neuroscience, 12(8), 1229–1240. https://doi.org/10.1093/scan/nsx069

Yi, X., Li, R., & Sun, M. (2018). Chinese Poetry Generation with a Salient-Clue Mechanism. Proceedings of the 22nd Conference on Computational Natural Language Learning, 241–250. https://doi.org/10.18653/v1/K18-1024

Zenker, F., & Kyle, K. (2021). Investigating minimum text lengths for lexical diversity indices. Assessing Writing, 47. https://doi.org/10.1016/j.asw.2020.100505

Zhang, Y., & Wu, W. (2021). How effective are lexical richness measures for differentiations of vocabulary proficiency? A comprehensive examination with clustering analysis. Language Testing in Asia, 11(1). https://doi.org/10.1186/s40468-021-00133-6




DOI: http://dx.doi.org/10.17576/3L-2024-3002-01

Refbacks

  • There are currently no refbacks.


 

 

 

eISSN : 2550-2247

ISSN : 0128-5157