Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. Code-Aware LLM Prompting in Deductive Qualitative Analysis: A Study in Multi-framework Analysis of Learning Designs
 
conference paper

Code-Aware LLM Prompting in Deductive Qualitative Analysis: A Study in Multi-framework Analysis of Learning Designs

Rodríguez Triana, María Jesús
•
Saban, Mohamed
•
Asensio-Pérez, Juan I.
Show more
Tammets, Kairit
•
Sosnovsky, Sergey
Show more
2026
Two Decades of TEL. From Lessons Learnt to Challenges Ahead - 20th European Conference on Technology Enhanced Learning, EC-TEL 2025, Newcastle upon Tyne and Durham, UK, September 15–19, 2025, Proceedings, Part I
20th European Conference on Technology Enhanced Learning

Large Language Models (LLMs) have demonstrated potential for (semi-)automating the qualitative analysis of unstructured data, particularly in deductive qualitative coding using codebooks. While prior research has shown the feasibility of this technology, model performance seems to vary depending on the nature of the constructs being coded. However, existing approaches typically apply a single LLM prompting strategy across entire datasets, often of discourse transcripts or questionnaire data, using a single coding scheme or theoretical frame. This paper introduces an adaptive prompting approach where LLM prompts are customized based on researcher-defined or data-driven rules for specific codes. We apply this approach to analyze the description of 35 multimedia learning designs (comprising 758 items/activities) created by teachers in an inquiry-based learning digital platform. Two human coders and an open-weights LLM (Llama 3.3) coded the dataset attending to three different pedagogical frameworks. Our results indicate that data-driven adaptive prompting outperforms uniform prompting approaches (namely, zero-shot, few-shot with/without context) in terms of agreement with human coders and cost. While the improvement is not statistically significant, the approach offers potential advantages for large datasets (especially, considering the costs), highlighting opportunities for human-AI collaboration in educational data analysis.

  • Details
  • Metrics
Type
conference paper
DOI
10.1007/978-3-032-03870-8_29
Scopus ID

2-s2.0-105016166610

Author(s)
Rodríguez Triana, María Jesús

Universidad de Valladolid

Saban, Mohamed

Universidad de Valladolid

Asensio-Pérez, Juan I.

Universidad de Valladolid

Prieto, Luis P.

Universidad de Valladolid

Haba-Ortuo, Inmaculada

Universidad de Valladolid

Villa-Torrano, Cristina

Universidad de Valladolid

Gillet, Denis  

École Polytechnique Fédérale de Lausanne

Editors
Tammets, Kairit
•
Sosnovsky, Sergey
•
Ferreira Mello, Rafael
•
Pishtari, Gerti
•
Nazaretsky, Tanya
Date Issued

2026

Publisher

Springer Science and Business Media Deutschland GmbH

Published in
Two Decades of TEL. From Lessons Learnt to Challenges Ahead - 20th European Conference on Technology Enhanced Learning, EC-TEL 2025, Newcastle upon Tyne and Durham, UK, September 15–19, 2025, Proceedings, Part I
DOI of the book
https://doi.org/10.1007/978-3-032-03870-8
ISBN of the book

978-3-032-03869-2

978-3-032-03870-8

Total of pages

XXVI, 569

Series title/Series vol.

Lecture Notes in Computer Science; 16063 LNCS

ISSN (of the series)

1611-3349

0302-9743

Start page

427

End page

441

Subjects

Human-AI collaboration

•

Large Language Models

•

Learning Analytics

•

Learning Design

•

Qualitative coding

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
SCI-STI-DG  
Event nameEvent acronymEvent placeEvent date
20th European Conference on Technology Enhanced Learning

Newcastle upon Tyne and Durham, UK

2025-09-15 - 2025-09-19

FunderFunding(s)Grant NumberGrant URL

Regional Government of Castile and Leon

European Union

RYC2021-032273-I

ERDF

PID2023-146692OB-C32,VA176P23

Show more
Available on Infoscience
September 30, 2025
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/254478
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés