Application of various communication tactics with a chatbot for text rewriting: effectiveness evaluation

Мұқаба

Дәйексөз келтіру

Толық мәтін

Ашық рұқсат Ашық рұқсат
Рұқсат жабық Рұқсат берілді
Рұқсат жабық Рұқсат ақылы немесе тек жазылушылар үшін

Аннотация

ChatGPT is an effective tool for text rewriting, enabling modifications to structure and phrasing, error correction, and improved readability. This study attempts to compare the efficiency of interaction and the “quality” of texts after rewriting by formulating requests to ChatGPT using three communication tactics: cooperative, informative, and persuasive. The effectiveness of such interaction was evaluated using scales measuring the ease of chatbot interaction and expert assessments of text quality. A comparison of the different communication tactics with ChatGPT revealed significant differences and trends when transitioning from cooperative to informative, and subsequently to persuasive tactics. An upward trend was observed in dialogue duration, the number of iterations, and text length, accompanied by increased logical coherence, structural clarity, and detail, as well as improved accuracy and fluency of the text. A downward trend was noted in metrics related to the ease of use and user support provided by the chatbot. The informative tactic occupied an intermediate position, delivering structured, complete, and comprehensive responses while reducing flexibility and coherence. The specificity of interaction in the informative tactic led to the identification of four latent variables explaining the variance across the three communication tactics: quality of interaction and user support in dialogue, user satisfaction with text quality, duration and volume of dialogue, and the substantive and semantic completeness of the text. The study illustrates the positioning of the examined communication tactics with ChatGPT within the space of these latent variables.

Толық мәтін

Рұқсат жабық

Авторлар туралы

A. Voronin

Institute of Psychology, Russian Academy of Sciences; State Academic University for Humanities

Хат алмасуға жауапты Автор.
Email: oroninan@bk.ru

ScD (Psychology), Professor, Chief Research Fellow of the Laboratory of Psychology of Speech and Psycholinguistics

Ресей, 129366, Moscow, Yaroslavskaya str., 13, bldg. 1; 19049, Moscow, Maronovsky lane, 26

V. Palenova

State Academic University for Humanities

Email: violetta.palenova@yandex.ru

Graduate student

Ресей, 19049, Moscow, Maronovsky lane, 26

Әдебиет тізімі

  1. Alomari E.A. Unlocking the Potential: A Comprehensive Systematic Review of ChatGPT in Natural Language Processing Tasks // Computer Modeling in Engineering & Sciences. 2024. V. 141. № 1. P. 43–85. doi: 10.32604/cmes.2024.052256.
  2. Bedington A., Halcomb E., McKee H., Sargent T., Smith A. Writing with generative AI and human-machine teaming: Insights and recommendations from faculty and students // Computers and Composition. 2024. V. 71. Article 102833. doi: 10.1016/j.compcom.2024.102833.
  3. Bernotienė L., Kaprizkinas A. Linguistic Strategies of Communicative Functions in Business Discourse // Res Humanitariae. 2019. V. 24. P. 172–185. doi: 10.15181/rh.v24i0.1913.
  4. Berriche L., Larabi-Marie-Sainte S. Unveiling ChatGPT text using writing style // Heliyon. 2024. V. 10. № 12. Article e32976. doi: 10.1016/j.heliyon.2024.e32976.
  5. Boella G., Hulstijn J., van der Torre L. Persuasion Strategies in Dialogue. 2004.
  6. Borsci S., Malizia A., Schmettow M., et al. The Chatbot Usability Scale: the Design and Pilot of a Usability Scale for Interaction with AI-Based Conversational Agents // Personal and Ubiquitous Computing. 2022. V. 26. P. 95–119. doi: 10.1007/s00779-021-01582-9.
  7. Cascella M., Montomoli J., Bellini V., Bignami E. Evaluating the Feasibility of ChatGPT in Healthcare: an Analysis of Multiple Clinical and Research Scenarios // Journal of Medical Systems. 2023. V. 47. № 1. P. 1–5. doi: 10.1007/s10916-023-01925-4.
  8. Chen J., Guo F., Ren Z., Li M., Ham J. Effects of Anthropomorphic Design Cues of Chatbots on Users’ Perception and Visual Behaviors // International Journal of Human-Computer Interaction. 2023. P. 1–19. doi: 10.1080/10447318.2023.2193514.
  9. Croes E.A.J., Antheunis M.L. Can we be friends with Mitsuku? A longitudinal study on the process of relationship formation between human and a social chatbot // Journal of Social and Personal Relationships. 2021. V. 38. № 1. P. 279–300. doi: 10.1177/0265407520959463.
  10. Cuayáhuitl H., Lee D., Ryu S., Cho Y., Choi S., Indurthi S., Yu S., Choi H., Hwang I., Kim J. Ensemble-Based Deep Reinforcement Learning for Chatbots. Neurocomputing. 2019. V. 366. P. 118–130. doi: 10.1016/j.neucom.2019.08.007.
  11. Curry N., Baker P., Brookes G. Generative AI for corpus approaches to discourse studies: A critical evaluation of ChatGPT // Applied Corpus Linguistics. 2024. V. 4. Article 100082. doi: 10.1016/j.acorp.2023.100082.
  12. Deng J., Lin Y. The Benefits and Challenges of ChatGPT: An Overview // Frontiers in Computing and Intelligent Systems. 2023. V. 2. № 2. P. 81–83. doi: 10.54097/fcis.v2i2.4465.
  13. Drouin M., Sprecher S., Nicola R., Perkins T. Is chatting with a sophisticated chatbot as good as chatting online or FTF with a stranger? // Computers in Human Behavior. 2022. V. 128. Article 107100. doi: 10.1016/j.chb.2021.107100.
  14. Fanning J.E., Escobar-Domingo M.J., Foppiani J., Lee D., Miller A.S., Janis J.E., Lee B.T. Improving Readability and Automating Content Analysis of Plastic Surgery Webpages With ChatGPT // Journal of Surgical Research. 2024. V. 299. P. 103–111. doi: 10.1016/j.jss.2024.04.006.
  15. Guler N., Kirshner S., Vidgen R. A literature review of artificial intelligence research in business and management using machine learning and ChatGPT // UNSW Business School Research Paper Forthcoming. 2023. doi: 10.2139/ssrn.4540834.
  16. Guo C., Lu Y., Dou Y., Wang F.-Y. Can ChatGPT boost artistic creation: The need of imaginative intelligence for parallel art // IEEE/CAA Journal of Automatica Sinica. 2023. V. 10. № 4. P. 835–838. doi: 10.1109/JAS.2023.123555.
  17. Hutson M. Could AI help you to write your next paper? // Nature. 2022. V. 611. P. 192–193. doi: 10.1038/d41586-022-03479-w.
  18. Janson A. How to leverage anthropomorphism for chatbot service interfaces: The interplay of communication style and personification // Computers in Human Behavior. 2023. V. 149. Article 107954. doi: 10.1016/j.chb.2023.107954.
  19. Koltovskaia S., Rahmati P., Saeli H. Graduate students’ use of ChatGPT for academic text revision: Behavioral, cognitive, and affective engagement // Journal of Second Language Writing. 2024. V. 65. doi: 10.1016/j.jslw.2024.101130.
  20. Kooli C. Chatbots in Education and Research: A Critical Examination of Ethical Implications and Solutions // Sustainability. 2023. V. 15. № 7. Article 5614. doi: 10.3390/su15075614.
  21. Kosaka T., Kamito A. When ChatGPT-4o Is (Less) Human-Like: Preliminary Subjective Rating Tests for Psycholinguistic Research, 2024. doi: 10.31234/osf.io/d9wmp.
  22. Kumar V., Ashraf A., Nadeem W. AI-powered marketing: What, where, and how? // International Journal of Information Management. 2024. V. 77. Article 102783. doi: 10.1016/j.ijinfomgt.2024.102783.
  23. Lee J.Y. Can an artificial intelligence chatbot be the author of a scholarly article? // Journal of Educational Eva- luation for Health Professions. 2023. V. 20. Article 6. doi: 10.3352/jeehp.2023.20.6.
  24. Lin Z. Why and how to embrace AI such as ChatGPT in your academic life. Royal Society Open Science, 2023. doi: 10.1098/rsos.230658.
  25. Malyuga E.N., Tomalin B. Communicative strategies and tactics of speech manipulation in intercultural business discourse // Training, Language and Culture. 2017. V. 1. № 1. P. 28–45. doi: 10.29366/2017tlc.1.1.2.
  26. Nizhnik L. Epistemic words and communicative strategies of cooperation. Naukovì zapiski Nacìonalʹnogo unì- versitetu “Ostrozʹka akademìâ”. Serìâ “Fìlologìâ”, 2022. doi: 10.25264/2519-2558-2022-13(81)-49-53.
  27. Orji R. Why are persuasive strategies effective? Exploring the strengths and weaknesses of socially-oriented persuasive strategies. In: Persuasive Technology: Development and Implementation of Personalized Technologies to Change Attitudes and Behaviors, 2017. P. 253–266. doi: 10.1007/978-3-319-55134-0_20.
  28. Pentina I., Hancock T., Xie T. Exploring relationship development with social chatbots: A mixed-method study of Replika // Computers in Human Behavior. 2023. V. 140. Article 107600. doi: 10.1016/j.chb.2022.107600.
  29. Pigg S. Research writing with ChatGPT: A descriptive embodied practice framework // Computers and Composition. 2024. V. 71. Article 102830. doi: 10.1016/j.compcom.2024.102830.
  30. Poldrack R.A., Baker C.I., Durnez J. et al. Scanning the horizon: towards transparent and reproducible neuroimaging research // Nature Reviews Neuroscience. 2017. V. 18. № 2. P. 115–126. doi: 10.1038/nrn.2016.167.
  31. Radyuk A.V. Cooperative Communicative Strategies and Tactics as Means of English Business Discourse Harmonization // MGIMO Review of International Relations. 2013. № 1(28). P. 236–239. doi: 10.24833/2071-8160-2013-1-28-236-239.
  32. Rahman M.M.M., Watanobe Y. ChatGPT for Education and Research: Opportunities, Threats, and Strategies // Applied Sciences. 2023. V. 13. № 9. Article 5783. doi: 10.3390/app13095783.
  33. Retkowsky J., Hafermalz E., Huysman M. Managing a ChatGPT-empowered workforce: Understanding its affordances and side effects // Business Horizons. 2024. V. 67. № 5. P. 511–523. doi: 10.1016/j.bushor.2024.04.009.
  34. Rhee C.E., Choi J. Effects of personalization and social role in voice shopping: An experimental study on product recommendation by a conversational voice agent // Computers in Human Behavior. 2020. V. 109. Article 106359. doi: 10.1016/j.chb.2020.106359.
  35. Salvagno M., Taccone F.S., Gerli A.G. Can artificial intelligence help for scientific writing? // Critical Care. 2023. V. 27. Article 75. doi: 10.1186/s13054-023-04380-2.
  36. Shpomer Е.А. Communicative Strategies and Tactics as an Effective Communication Factor // Vestnik Hakasskogo gosudarstvennogo universiteta im. N.F. Katanova. 2013. № 4. P. 109–113.
  37. Sohail S.S., Madsen D.Ø., Himeur Y., Ashraf M. Using ChatGPT to Navigate Ambivalent and Contradictory Research Findings on Artificial Intelligence // SSRN Electronic Journal. 2023. doi: 10.2139/ssrn.4413913.
  38. Thorp H.H. ChatGPT is fun, but not an author. Science. 2023. V. 379. P. 313. doi: 10.1126/science.adg7879.
  39. Valgina N.S. Theory of Text. Moscow, Logos, 2003. 280 p. (in Russian)
  40. van Dis E.A.M., Bollen J., Zuidema W. et al. ChatGPT: five priorities for research // Nature. 2023. V. 614. № 7947. P. 224–226. doi: 10.1038/d41586-023-00288-7.
  41. Vettorel P. Communication strategies and co-construction of meaning in ELF: Drawing on “Multilingual Resource Pools” // Journal of English as a Lingua Franca. 2019. V. 8. P. 179–210. doi: 10.1515/jelf-2019-2019.
  42. Xu Y., Zhang J., Deng G. Enhancing customer satisfaction with chatbots: The influence of communication styles and consumer attachment anxiety // Frontiers in Psychology. 2022. V. 13. doi: 10.3389/fpsyg.2022.902782.
  43. Ying L., Liu Z., Fang H. et al. Text summarization with ChatGPT for drug labeling documents // Drug Discovery Today. 2024. V. 29. № 6. Article 104018. doi: 10.1016/j.drudis.2024.104018.
  44. Zhang A., Li C.X.R., Piper M., et al. ChatGPT for Improving Postoperative Instructions in Multiple Fields of Plastic Surgery // Journal of Plastic, Reconstructive & Aesthetic Surgery. 2024. doi: 10.1016/j.bjps.2024.08.065.
  45. Zhang Y., Prebensen N.K. Co-creating with ChatGPT for tourism marketing materials // Annals of Tourism Research Empirical Insights. 2024. V. 5. № 1. doi: 10.1016/j.annale.2024.100124.

Қосымша файлдар

Қосымша файлдар
Әрекет
1. JATS XML
Жүктеу
2. Fig. 1. Eigenvalue graph.

Жүктеу (196KB)
Метадеректер
3. Fig. 2. Mean and standard deviations for quality of interaction and user support in the dialogue of three communication tactics.

Жүктеу (78KB)
Метадеректер
4. Fig. 3. Mean and standard deviations for user satisfaction with the quality of the final text of three communication tactics.

Жүктеу (81KB)
Метадеректер
5. Fig. 4. Mean and standard deviations for the duration and volume of the dialogue of three communication tactics.

Жүктеу (76KB)
Метадеректер
6. Fig. 5. Mean and standard deviations for the content and semantic completeness of the text of three communication tactics.

Жүктеу (76KB)
Метадеректер
7. Table 3

Жүктеу (1MB)
Метадеректер

© Russian Academy of Sciences, 2025