Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Journal articles
  4. Conditional LSTM-GAN for Melody Generation from Lyrics
 
research article

Conditional LSTM-GAN for Melody Generation from Lyrics

Yu, Yi
•
Srivastava, Abhishek
•
Canales, Simon  
April 1, 2021
Acm Transactions On Multimedia Computing Communications And Applications

Melody generation from lyrics has been a challenging research issue in the field of artificial intelligence and music, which enables us to learn and discover latent relationships between interesting lyrics and accompanying melodies. Unfortunately, the limited availability of a paired lyrics-melody dataset with alignment information has hindered the research progress. To address this problem, we create a large dataset consisting of 12,197 MIDI songs each with paired lyrics and melody alignment through leveraging different music sources where alignment relationship between syllables and music attributes is extracted. Most importantly, we propose a novel deep generative model, conditional Long Short-Term Memory (LSTM) Generative Adversarial Network for melody generation from lyrics, which contains a deep LSTM generator and a deep LSTM discriminator both conditioned on lyrics. In particular, lyrics-conditioned melody and alignment relationship between syllables of given lyrics and notes of predicted melody are generated simultaneously. Extensive experimental results have proved the effectiveness of our proposed lyrics-to-melody generative model, where plausible and tuneful sequences can be inferred from lyrics.

  • Details
  • Metrics
Type
research article
DOI
10.1145/3424116
Web of Science ID

WOS:000641174200015

Author(s)
Yu, Yi
Srivastava, Abhishek
Canales, Simon  
Date Issued

2021-04-01

Published in
Acm Transactions On Multimedia Computing Communications And Applications
Volume

17

Issue

1

Start page

35

Subjects

Computer Science, Information Systems

•

Computer Science, Software Engineering

•

Computer Science, Theory & Methods

•

Computer Science

•

lyrics-conditioned melody generation

•

conditional lstm-gan

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
LTS5  
Available on Infoscience
June 5, 2021
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/178591
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés