Critical multimodal literacies in world language education

Digital storytelling project

Yuri Kumagai, Smith College, & Keiko Konoeda, University of Massachusetts Amherst

Click the video frame above to view the presentation. To ask the presenters a question about their presentation, please add a comment at the bottom of this page between October 6 and October 10. Presenters will check for and reply to questions at the end of each symposium day.


The purpose of this presentation is to look into a digital storytelling project, which applied the Pedagogy of Multiliteracies (New London Group, 1996) and aimed at fostering students’ critical multimodal literacies in a Japanese as a world language classroom at a college in the US. World language education has long privileged narrow definitions of written and spoken language; however, there is an increased and urgent need to widen what is regarded as “literacy” and to develop students’ understanding of multimodal aspects of communication, in light of the changing landscapes of literacy practices that we engage in (Kress, 2003). The recent advancement of digital communication has necessitated a heightened awareness on how to make meanings with multiple modes such as linguistic, visual, aural, spatial, and gestural modes.

In this presentation, we report on the pedagogical model of the digital storytelling project for critical multimodal literacies, and examine the students’ meta-awareness of multimodality in classroom discussions, post-project survey and interviews, and two focal students’ digital story products. We designed the project based on the essential pedagogical elements the New London Group (1996) advocated—i.e., situated practice, overt instruction, critical framing, and transformed practice (also, Cope and Kalantzis 2008)—starting with critical multimodal reading of digital stories, in order to scaffold raising awareness of multiple modes. Based on the analysis of multiple data sources (e.g., classroom discussions, students’ writing of drafts and final video products, post-project surveys, and individual interviews), we demonstrate how students further developed their meta-awareness of the significant role that multimodality plays in communicating messages and became critical evaluators of multimodal literacy through explicit attention on multimodality in the project.

We argue that the digital storytelling project afforded the opportunity for the language learners to draw on a wider variety of available designs than they would in a single mode activity, and to Design their language use for their own purposes. We cannot emphasize enough the importance of such projects that encourage students to be intentional and purposeful in Designing a text, so that the students can become the designers of their future selves as language users.

Yuri Kumagai

Senior Lecturer
Smith College

Keiko Konoeda

Doctoral Candidate
University of Massachusetts Amherst

31 thoughts on “Critical multimodal literacies in world language education”

  1. Excellent presentation with great application to all foreign languages at different levels of proficiency. What did you find more surprising from collecting the multimodal data and the learners’ interview responses?

    1. Thank you very much for your comment and question, Sylvia! I was first surprised at how much students picked up features of Japanese multimodal story in the classroom discussion (Phase 1) before their actual discussion. Later on I was personally most surprised at Genni’s comments about how she wrote in the digital story what she normally doesn’t talk about, and the fact that this project allowed her to present herself differently, and us to see her another side. Yuri may have different things that she was surprised at, so let’s see how she would respond. Thanks again!

      1. Dear Sylvia,

        Thank you for your comments and question.
        For me, the most surprising (and delightful) thing that I found from interviewing the students was how serious and invested they were in working on this project. I have been doing this project for many years with my 2nd year students. And as you can imagine, the project requires a lot of time and effort on the part of students, but they are always so proud of what they have accomplished and says that this is the most rewarding project they have even done in learning Japanese language. They gained the sense of ownership of the language they are learning with this project.

  2. Hello, very well-organized presentation and fascinating contents. I really enjoy your work. I’m also interested in multimodality and multiliteracies so I find your project very relevant and insightful. I have a couple of questions for you:

    1) Did you use any specific software for the digital storytelling? I just wonder about its practicality and ease of use because Ping seems to be much more competent in this kind of visual art project.

    2) Do you consider your students in an intermediate or elementary level of language proficiency? I’m asking because I mostly deal with beginner learners and it’s always my concern about the feasibility of directing class discussion and self-reflection in their L2.

    3) The project seemed to work beautifully but I was wondering if you have encountered any eminent problem/concern during any phase of project that you think is worth sharing? It would be very beneficial for future implication.

    Thank you so much. Excellent presentation!

    1. Thank you for your comments and questions, Nat! I am glad to hear that you find our presentation relevant and insightful. Please let us know if you have presentations, publications, ongoing/future projects that we can learn about your work, if you don’t mind.

      Regarding the software tools, we gave a workshop on Audacity for voice recording, and Photo Story (for PC) and iMovie (for Mac) for movie editing. Some students like Ping additionally used Photoshop, which we didn’t give a workshop on, so that was the skill that she brought to the classroom with her. It would depend on the students that you have, but we have found that our students are increasingly (each year) becoming faster at finding their ways with these tools.

      Our students would most likely be assessed as intermediate. At the time of the project they were finishing up the two-volume elementary textbook, and were able to express & engage in discussion on familiar topics. In classroom discussion before the production, the self-reflection, and research interviews, use of English (and L2) was encouraged, because we also evaluated that these tasks require language functions and vocabulary beyond their repertoire in Japanese. Do you feel that L2 needs to be used exclusively throughout the project?

      In the past few years of project, we have found a few problems, some of which we have addressed. Some students spend countless hours trying to make a “perfectly embellished” video with sophisticated technology (like animation), and we have tried to focus students’ attention more on the content (message, purpose driven language use & visual & music choice). Another challenge is the aspect of conveying a message from a personal story, and not just describing a sequence of events – and we have pushed students in the revision of the first draft and have conducted individual conferences when necessary. These are the top two I can think of now. Do you foresee other challenges? Yuri may be able to add other problems that she has encountered.

      Thank you again!

      1. Dear Nat,
        Thank you for your comment and question.
        As Keiko mentioned, the only problem I encountered in the past was that the students sometimes spent too much time on visual aspect of the project. Although it is certainly important to ‘design’ multimodal elements (besides language use) in communicating message, but as a part of FL learning, students do need to think deeply as to how they want to tell a story linguistically as well. I find it very important to communicate this point to students at the very beginning of the project.

  3. Thank you very much for your comment and question, Sylvia! I was first surprised at how much students picked up features of Japanese multimodal story in the classroom discussion (Phase 1) before their actual discussion. Later on I was personally most surprised at Genni’s comments about how she wrote in the digital story what she normally doesn’t talk about, and the fact that this project allowed her to present herself differently, and us to see her another side. Yuri may have different things that she was surprised at, so let’s see how she would respond. Thanks again!

  4. Thank you very much for this presentation, I enjoyed it very much. Just out of curiosity, can you tell me what level of language class this was? It’s hard to imagine my 101 level FL students coming up with comparable presentations. Another question I have is this: When we have only a short amount of teaching time with students (50 minutes 4x a week), how could one give an assignment such as this knowing that students will have widely varying abilities when it comes to digital media and presentation development?

    1. Thank you for your comment and questions, Luella!

      First question, about the level of language class, this was done in Japanese 220, Second Year First Semester (or the third semester from the beginning). I do have experience with a production in a first semester Japanese classroom (not this one), but it was of a much smaller scale and was more structured. Yet I do believe that it is important that lower level language students get the experience to use the language for authentic purposes in a medium and modes that are relevant to them (of course also meeting the objectives of the course), and I don’t think they should have to wait until they become 3rd or 4th year students to become language users.

      Reading your second question, I am curious what holds you back – short amount of teaching time, or the nature of the assignment? This Second Year Japanese course meets 50 min weekly, five days a week, and being an intensive language course, we have expectations for students to spend 10 hours outside of class weekly. Sure, finding the time is tricky, especially when there is much to cover for lower language course. The way we initially found the time was by replacing the end-of-the-semester presentation with this. From there we (instructors) discussed what steps were needed and how early we had to start, given the course schedule and the workload that we can expect from the students in our context. It is critical to give much scaffold and time for each step, because this is not a single assignment, but is actually a project with many assignments and guided work that spans four skills and more, which are planned so that the project ends with a final video production. Initially we use two class meetings (squeezing teaching items) for the Phase 1 (Critical Reading) where students discussed their media experiences, watched multimodal stories, and analyzed them. Then we had another class meeting for sharing story idea (beginning of Phase 2), writing assignments (first draft and revision), an in-class workshop for technology, out-of-classroom work to complete editing, and two class meetings for in-class presentations.

      I hope I answered your questions. Please let me know if this helps you imagine planning a project like this in your context.

  5. I really appreciated this great presentation. The examples from the two students were so impressive in that the authors delivered the lived experiences literally. What made you concentrate on the two focal students? Did they show any specific characteristics that are in a line with your research questions?

    1. Dear Sangshin,
      Thank you for your comment and question.
      First of all, I would like to say that we could have chosen any of the student’s videos (there were 16 videos in total created) in order to discuss similar things that we have discussed in this presentation.
      We have chosen, however, these two videos as examples because the ways Ginna and Ping designed their stories show some different emphasis of multimodal features. Ginna paid attentions to the effect of (abstract) images, especially colors and tones, to communicate her message, while Ping’s video was unique in her use of written mode (use of words on video), moving images, and sound effect. All these features were discussed during Critical Multimodal Reading phase as important elements to consider, and they took advantage of different elements depending on their interests and expertise. We thought their videos are good representatives in terms of demonstrating what they have learned from the class discussions and also showing individual creativity.

  6. I really enjoyed this presentation. What a great project! I was wondering about the technical logistics of students’ work on the projects. Did students have access to computers with the necessary software outside of the classroom, or did you use class time to work on the video presentations? I see that you mentioned above that students participated in a workshop to learn how to use some useful programs, but did any of the students struggle with the computer techniques required to complete the project? If so, what kinds of resources were available for them to overcome that obstacle? Thank you so much!

    1. Dear Jeannette,
      Thank you for your comment and question.
      In terms of software, our students used either Photo Story (for PC, freeware) or iMovie (for Mac). During the workshop, an educational technology staff showed students how to use both softwares. Photo Story is specifically developed for creating a digital story, and is extremely easy to use. We have only spent one to two class meetings for them to work on the project in class (We have two workshops for the given class the presentation was based on, but many students stated in the post-project survey that “one workshop is necessary for students at this age!”). They have access to both softwares in all of the computer labs on campus at our college, but we think they worked on the project on their own computer at home.

      We understand that depending on the students population and the computer equipments available in schools, assistant students need would differ. In our case, we found in the past that our students are becoming more and more computer savvy and do not need much assistance from us. Also, there are always some students who are especially knowledgeable about computer in class and they help each other solving problems. However, if they need any further help, we (instructors) as well as computer technology staff on campus were ready to offer help. I hope this answer are helpful.

  7. Thank you for your presentation. The examples you show alone attest to the amount of creativity that manifests when you give students a chance to use multiple resources for making meaning. I was wondering if you could say a little about where you see this project going next. Do you have plans to continue to develop this work?

    1. Dear Chantelle,
      Thank you for watching and for your comment. I agree that it is very important, especially in world language education, to encourage that creative meaning-making. In regards to your question about the future of this project, we feel that pedagogically we have reached our goals considering the context of the course we embed this project in, but we would like to analyze the data focusing on other themes than multimodality, for example, learners’ self-representation in such a project. Thank you again!

  8. Hello,

    Thank you for sharing your poster. I thought the student reflections were very interesting. I was wondering whether you specified in their projects whether the intended audience for their videos was other students, or whether they were allowed to come up with their own audience. What benefits or problems do you foresee in allowing and encouraging students to form their own intended audience (besides classmates) to design a video for?

    Secondly, an interesting feature about Youtube is that it allows you to follow the statistics of how many times the video has been watched and also audience retention. Can you think of any ways in which these analytical tools can be incorporated into the lesson?

    Thanks again. Your presentation was very interesting.

    –Christian Ruvalcaba

    1. Dear Christian,
      Thank you for watching and for your questions. First, regarding the intended audience, at the beginning of the project, the students were told that they would have an in-class presentation at the end of the semester, so they were very aware that their classmates would be included in the audience, and that very likely had an impact on what they decided to tell (message, self-disclosure, etc) but also how they decided to tell (stay linguistically simple). Then the students watched YouTube digital story as a sample for analysis, and learned that their digital video is not just for the classmates but for a whole world if they choose to share (which was by each student’s choice). I feel that it is important for the students to decide their audience on their own (besides classmates), and decide what and how to tell, which is what we all do outside of classroom.

      Second, we have not made uploading to YouTube as a requirement. If we were to teach with a focus on media literacy and take up YouTube as one type, we would look at the statistics of the sample story, study what kinds of videos are watched, and produce a video with the YouTube audience in mind. However, that was not what we intended, and we do not foresee that in this context. Thanks for your suggestion.

  9. I thoroughly enjoyed your presentation. I can’t imagine the amount of work that went into that project. Very well organized and insightful!
    What I really liked most was Genni’s story. It not only proves that students have the potential to communicate successfully through a variety of multimodal designs, but that they can give great,constructive evaluations.
    I feel motivated to give it a try. I’m worried though about how to begin to awaken my 101 and 102 FL students multimodal senses?

    1. Dear Miriam,
      Thank you for your comments, and I am excited to hear that you feel motivated to try. For lower level language students, you may have different goals and shapes than ours. I would recommend that you look at the learners’ literacies (including L1 and popular culture literacies) for their multimodality, and design the project adding multimodality to your current curricular goals. It may take the form of a poster, comic strip making, or short slides with a voice-over, and you may choose a topic that your students will have the vocabulary for from the textbook unit (but is also significant enough for them). Good luck, and hope to hear how you teach for multimodality in your 101 and 102 classrooms!

  10. Hello
    This presentation is very good.I have learnt a lot of new things as well as increased my knowledge relating to some concepts I have been exposed to in the multiliteracies framework.My curiosity is however to know at what level the lesson was taught? What the language of instruction the target language(japanese) or in english?I also would want to how this can be implemented in a beginning FL course where students are coming into contact with the language for the first?Thank you!

    1. Dear Abraham,
      Thank you for your comment and question.
      The level of the course that this project was implemented was a second year (3rd semester) Japanese course. We use Japanese language to teach the course with occasional and minimum use of English for grammar explanations, etc. The instruction for the project was mostly done in Japanese, however the handout we created had some English words supplemented for difficult vocabulary. During the class discussion for the analysis of a sample video, we allowed them to use English if they couldn’t express their thoughts only in Japanese. We believe purposeful use of English is appropriate and effective in FL instruction.
      Although I haven’t taught the very beginning level for a while, my colleagues have done a similar project in the very first semester Japanese course. For that course, the assigned topic was to introduce “your college life (and its neighborhood)”. Naturally what they were able to express with the target language alone at that point was limited; however, they have done a wonderful job drawing on their knowledge of various genres (e.g., a quiz show, documentary, music video, etc.) and creatively using the language with the help of other modes. As long as the assigned topics are appropriate for the level, you will be amazed what students can do in a project such as this one.

  11. Yuri-san and Keiko-san, Thank you for your great presentation. As you also mentioned in the presentation, exercising multi-modal literacies seems to be providing opportunities for them to express as a competent speaker/user of Japanese, not as a second class user. Having a feeling of achievement in language learning becomes crucial, and this learner-center approach nurtures the agency of students. Have you done any similar project with 1st or 2nd semester learners?

    1. Dear Kayo-san,
      Thanks for watching and leaving a comment. I haven’t personally done the similar project with 1st or 2nd semester learners, but my colleagues have done a similar project with the very first semester Japanese course (with topics that were appropriate for their level, such as ‘introducing your college life’, and the students worked in a pair/small group). The language they used were of course very simple, but combining that with other modes (visual images, music, movement, etc.) they created amazing videos.

  12. Thank you for an excellent presentation! One of the biggest problems I encounter with my students is the desire to “be themselves” while learning a L2. This project shows how being creative and using multi-modality in a second language can allow the student to maintain their sense of identity and even discover more about themselves through L2 use.
    I wonder, did you have any problems with student motivation, or did the class embrace the project from the beginning?

    1. Dear Clare,
      Thanks for your comment and question.
      Yes, it was really rewarding to find out that the students deeply reflected on their life, and learned something new about themselves through this project. They were quite impressed with themselves what they could actually do with the language that they are learning.
      In answering your question, luckily for us, we haven’t had students who lack motivation. It is probably because we do not have a language requirement in our college and they are serious about their learning of Japanese. Sometimes they are too motivated and spend too much time on this project…

  13. I loved hearing about these projects. I think it’s a truly lovely approach to language teaching and learning. I’ve been learning about multiliteracies and multimodality for the first time this semester, and it’s really useful for me to see an example of how students can use different modes to help express themselves in a foreign language. I imagine that the audience was able to gain a sense of the speakers’ personalities not just from their spoken story but from the images they chose. Just in the two examples you discussed, there’s such a contrast between the type of art used in the presentations. I’m buzzing with ideas for my own classes now. Thank you!

    1. Dear Holly,
      Thanks for watching our presentation and sharing your thoughts.
      We are glad to know that this gives you ideas for some projects that you would try in your teaching!

Comments are closed.

a hybrid symposium on research and practice

%d bloggers like this: