DESIGNING LITERACY LEARNING WITH MULTIMODALITY IN THE AGE OF GENERATIVE AI
Main Article Content
Abstract
This paper examines the implications of generative artificial intelligence (GenAI) for literacy learning through the lens of multimodality and pedagogy as design. I argue that the rapid proliferation of digital technologies has fundamentally reshaped the epistemic conditions of literacy, necessitating a reconceptualisation of literacy beyond traditional notions of reading and writing as well as necessitating the role of the teacher to be a designer of learning experiences. While GenAI offers significant potential in supporting literacy learning, it also raises critical concerns related to bias, authorship, and reliability. These tensions underscore the need for critical multimodal literacy, enabling students to engage reflexively with AI-generated content. Building on these insights, I outline key educational implications for curriculum, pedagogy, and assessment. Specifically, I advocate for a shift towards multimodal and process-oriented assessment, the integration of GenAI as a pedagogical aide, and the development of students’ semiotic capital.
Downloads
Article Details
Section
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
How to Cite
References
Adami, E., Diamantopoulou, S., & Lim, F. V. (2022). Design in Gunther Kress’s social semiotics. London Review of Education, 20(1), Article 41. https://doi.org/10.14324/LRE.20.1.41
Araújo, A. D., Gualberto, C., Pinheiro, M. S., & Lim, F. V. (2024). Multimodality, AI, and education: An interview with Professor Fei Victor Lim. Linguagem em Foco (Language in Focus) Journal, 16(2), 174-178. https://doi.org/10.46230/lef.v16i2.13856
Burrell, J. (2016). How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big Data & Society, 3(1), 1-12. https://doi.org/10.1177/2053951715622512
Chandel, P., & Lim, F. V. (2024). Generative AI and literacy development in the literacy classroom: A Systematic review of literature. Ubiquitous Learning: An International Journal, 18(2), 31-49. https://doi.org/10.18848/1835-9795/CGP/v18i02/31-49
Cope, B., & Kalantzis, M. (2015). An introduction to the pedagogy of multiliteracies. In B. Cope & M. Kalantzis (Eds.), A pedagogy of multiliteracies: Learning by design (pp. 1-36). Palgrave Macmillan.
Cope, B., & Kalantzis, M. (2022). Artificial intelligence in the long view: From mechanical intelligence to cyber-social systems. Discover Artificial Intelligence, 2(1), Article 13. https://doi.org/10.1007/s44163-022-00029-1
Cope, B., Kalantzis, M., & Zapata, G. C. (2026). Literacies in the age of AI: Teaching and learning in the digital era. Wiley.
Djonov, E., & van Leeuwen, T. (2018). The power of semiotic software: A critical multimodal perspective. In J. Flowerdew & J. E. Richardson (Eds.), The Routledge handbook of critical discourse studies (pp. 566-581). Routledge.
Jewitt, C. (Ed.). (2014). The Routledge handbook of multimodal analysis (2nd ed.). Routledge.
Jewitt, C., Bezemer, J., & O’Halloran, K. L. (2016). Introducing multimodality. Routledge.
Koh, E., Jonathan, C., Lim, F. V., Kadir, M., & Tan, J. P-L. (2023). Value and challenges in using a collaborative critical reading and learning analytics system. Proceedings of the 16th International Conference on Computer-Supported Collaborative Learning.
Koh, E., Lim, F. V., & Jonathan, C. (2026). Twenty-first Century Competencies assessment using formative learning analytics in K-12: Case examples from Singapore. In K. L. Chue, A. Yeo, K. H. K. Tan, & T. H. Yong (Eds.), Fundamentals of assessment: Principles and practices for the classroom (pp 195-205). Routledge.
Kress, G. (2010). Multimodality a social semiotic approach to contemporary communication. Routledge.
Kress, G., Jewitt, C., Bourne, J., Franks, A., Hardcastle, J., Jones, K., & Reid, E. (2005). English in urban classrooms: A multimodal perspective on teaching and learning (2nd ed.). Routledge.
Kress, G., & Selander, S. (2012). Multimodal design, learning and cultures of recognition. The Internet and Higher Education, 15(4), 265-268. https://doi.org/10.1016/j.iheduc.2011.12.003
Lim, F. V. (2021). Designing learning with embodied teaching: Perspectives from multimodality. Routledge.
Lim, F. V. (2023). Editorial: Transduction in evolving contexts of intermodal meaning making: Moving meaning across media, modes and multiple realities. Frontiers in Communication, 8, Article 1138753. https://doi.org/10.3389/fcomm.2023.1138753
Lim, F. V. (2024a). The multimodal turn in higher education. In V. Beltrán-Palanques & E. Bernad-Mechó (Eds.), Current trends in EMI and multimodality in higher education (pp. 9-25). Routledge.
Lim, F. V. (2024b). Multimodality and learning: Desiderata for designing social futures. In V. Tavares (Ed.), Empowering language learners in a changing world through pedagogies of multiliteracies (pp. 21-39). Springer.
Lim, F. V. (2024c). Designing synchronous online learning experiences with social media as semiotic technologies. In I. Pineda & R. Bosso (Eds.), Virtual English as a lingua franca (pp. 127-146). Routledge.
Lim, F. V. (in press). Designing learning and learning as design: Pedagogical and social change for the digital age. In V. Tavares (Ed.). The Routledge handbook of language education and social change. Routledge.
Lim, F. V., & Bernad-Mechó, E. (n.d.). Using generative artificial intelligence for digital multimodal composing in the pre-service literacy classroom [Manuscript under review].
Lim, F. V., Cope, B., & Kalantzis, M. (2022). A metalanguage for learning: Rebalancing the cognitive with the socio-material. Frontiers in Communication, 7, Article 830613. https://doi.org/10.3389/fcomm.2022.830613
Lim, F. V., Cope, B., & Kalantzis, M. (in press). Reimagining pedagogies for multiliteracies in the age of Generative Artificial Intelligence, Special Issue, Pedagogies.
Lim, F. V., Gilje, Ø., & Djonov, E. (2025). Editorial for special issue on digital multimodal composing in the era of Artificial Intelligence, Computers and Composition, 75, Article 102911. https://doi.org/10.1016/j.compcom.2024.102911
Lim, F. V., & Nguyen, T. T. H. (2022). Design-based research approach for teacher learning: A case study from Singapore. ELT Journal, 76(4), 452-464. https://doi.org/10.1093/elt/ccab035
Lim, F. V. & Nguyen, T. T. H. (2023). ‘If you have the freedom, you don’t need to even think hard’ – Considerations in designing for students’ agency through digital multimodal composing in the language classroom. Language and Education, 37(4), 409-427. https://doi.org/10.1080/09500782.2022.2107875
Lim, F. V., & Querol-Julian, M. (Eds.). (2024). Designing Learning with digital technologies: Perspectives from multimodality. Routledge.
Lim, F. V., & Tan-Chia, L. (2023). Designing learning for multimodal literacy: Teaching viewing and representing. Routledge.
Lim, F. V., & Tan, X. (2025). “If ChatGPT was used … then it shouldn’t be graded”: Understanding pre-service teachers’ perceptions towards Generative Artificial Intelligence for English teaching and learning. In G. Zapata & J. Jones (Eds.). Generative AI technologies, multiliteracies, and language education. Routledge.
Lim, F. V., & Toh, W. (2026). Designing learning with digital play: A multimodal perspective. Routledge.
Lim, F. V., & Unsworth, L. (2023). Multimodal composing in the English classroom: recontextualising the curriculum to learning. English in Education, 57(2), 102-119. https://doi.org/10.1080/04250494.2023.2187696
Lim, F. V., & Unsworth, L. (in press). Handbook of educational semiotics: Multimodality and learning. Bloomsbury.
Lim, F. V., & Yee, J. (forthcoming). Investigating digital literacies: Reflections from developing the student digital literacies profiling tool. In E. Baker (Ed.), Research handbook on digital literacies. Edward Elgar Publishing.
Lim, F. V., Yee, J. E., & Quek, J. (2025, October 1). What our body’s response reveals about AI in the classroom. University World News, https://www.universityworldnews.com/post.php?story=20251001143334646
Moje, E. B., & Luke, A. (2009). Literacy and identity: Examining the metaphors in history and contemporary research. Reading Research Quarterly, 44(4), 415-437. https://doi.org/10.1598/RRQ.44.4.7
New London Group. (1996/2025). Reprint: A pedagogy of multiliteracies: Designing social futures, Harvard Educational Review, 95(1), 102-134. https://doi.org/10.17763/1943-5045-95.1.102
O’Halloran, K. L. (Ed.). (2004). Multimodal discourse analysis: Systemic functional perspectives. Continuum.
Peters, M. A., Jackson, L., Papastephanou, M., Jandrić, P., Lazaroiu, G., Evers, C. W., … Fuller, S. (2024). AI and the future of humanity: ChatGPT-4, philosophy and education – Critical responses. Educational Philosophy and Theory, 56(9), 828–862. https://doi.org/10.1080/00131857.2023.2213437
Street, B. V. (1995). Social literacies: Critical approaches to literacy in development, ethnography and education. Longman.
UNESCO, IRCAI. (2024). Challenging systematic prejudices: An investigation into gender bias in large language models.
Yee, J., Lim, F. V., & Quek, J. (2025). Moods, bots, and bodies: University students’ emotional and physiological responses to human vs. GenAI chatbots [Preprint]. Research Square.
Weninger, C. (2020). From language skills to literacy: Broadening the scope of English language education through media literacy. Routledge
Zhao, S., Djonov, E., & van Leeuwen, T. (2014). Semiotic technology and practice: A multimodal social semiotic approach to PowerPoint. Text & talk, 34(3), 349-375. https://doi.org/10.1515/text-2014-0005