Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. Can Language Models Recognize Convincing Arguments?
 
conference paper

Can Language Models Recognize Convincing Arguments?

Rescala, Paula Dolores  
•
Ribeiro, Manoel Horta
•
Hu, Tiancheng
Show more
Al-Onaizan, Yaser
•
Bansal, Mohit
Show more
2024
EMNLP 2024 - 2024 Conference on Empirical Methods in Natural Language Processing, Findings of EMNLP 2024
Conference on Empirical Methods in Natural Language Processing (EMNLP 2024)

The capabilities of large language models (LLMs) have raised concerns about their potential to create and propagate convincing narratives.Here, we study their performance in detecting convincing arguments to gain insights into LLMs' persuasive capabilities without directly engaging in experimentation with humans.We extend a dataset by Durmus and Cardie (2018) with debates, votes, and user traits and propose tasks measuring LLMs' ability to (1) distinguish between strong and weak arguments, (2) predict stances based on beliefs and demographic characteristics, and (3) determine the appeal of an argument to an individual based on their traits.We show that LLMs perform on par with humans in these tasks and that combining predictions from different LLMs yields significant performance gains, surpassing human performance.The data and code released with this paper contribute to the crucial effort of continuously evaluating and monitoring LLMs' capabilities and potential impact.(https://go.epfl.ch/persuasion-llm).

  • Files
  • Details
  • Metrics
Loading...
Thumbnail Image
Name

2024.findings-emnlp.515.pdf

Type

Main Document

Version

http://purl.org/coar/version/c_970fb48d4fbd8a85

Access type

openaccess

License Condition

CC BY

Size

582.09 KB

Format

Adobe PDF

Checksum (MD5)

6536c558264015af158ed461f04fcd99

Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés