Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Journal articles
  4. Leveraging large language models for predictive chemistry
 
Loading...
Thumbnail Image
research article

Leveraging large language models for predictive chemistry

Jablonka, Kevin Maik  
•
Schwaller, Philippe  
•
Ortega-Guerrero, Andres  
Show more
February 6, 2024
Nature Machine Intelligence

Machine learning has transformed many fields and has recently found applications in chemistry and materials science. The small datasets commonly found in chemistry sparked the development of sophisticated machine learning approaches that incorporate chemical knowledge for each application and, therefore, require specialized expertise to develop. Here we show that GPT-3, a large language model trained on vast amounts of text extracted from the Internet, can easily be adapted to solve various tasks in chemistry and materials science by fine-tuning it to answer chemical questions in natural language with the correct answer. We compared this approach with dedicated machine learning models for many applications spanning the properties of molecules and materials to the yield of chemical reactions. Surprisingly, our fine-tuned version of GPT-3 can perform comparably to or even outperform conventional machine learning techniques, in particular in the low-data limit. In addition, we can perform inverse design by simply inverting the questions. The ease of use and high performance, especially for small datasets, can impact the fundamental approach to using machine learning in the chemical and material sciences. In addition to a literature search, querying a pre-trained large language model might become a routine way to bootstrap a project by leveraging the collective knowledge encoded in these foundation models, or to provide a baseline for predictive tasks.|Machine learning techniques are widely employed in chemical science, but are application specific and their development requires dedicated expertise. Jablonka and colleagues fine-tune the GPT-3 model and show that it can provide surprisingly accurate answers to a wide range of chemical questions.

  • Files
  • Details
  • Metrics
Loading...
Thumbnail Image
Name

document.pdf

Type

Publisher's Version

Access type

openaccess

License Condition

CC BY

Size

2.16 MB

Format

Adobe PDF

Checksum (MD5)

35f0eaca6ac79588d33cadf6d18ab6e9

Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés