Multimodal texts

In this paper, we propose applying a jointly developed model to the task of multilingual visual question answering. Specifically, we conduct experiments on a multimodal sequence-to-sequence transformer model derived from the T5 encoder-decoder architecture. Text tokens and Vision Transformer (ViT) dense image embeddings are inputs to an encoder then we used a decoder to automatically anticipate discrete text tokens.

11p dianmotminh02 03-05-2024 6 1   Download

This article aims to provide some theoretical bases on exercises, competence development-oriented exercises, the process of building and using exercises and applying exercises to develop the teaching reading comprehension multimodal texts competence for Philology pedagogical students.

12p viberkshire 09-08-2023 7 3   Download

This article aims to unify the concept from which to develop the teaching reading comprehension multimodal texts competence assessment standards to meet educational innovation in general and competence development of the teachers in particular.

16p viantman 07-04-2023 8 4   Download

The participants in the study also highly perceived the positive effects of multimodal texts on their English language learning. Finally, some implications were drawn from the findings so as to enhance the quality of culture and communication teaching and learning at the upper secondary level.

16p viharry 15-12-2022 12 7   Download

Trong quá trình khai thác nội dung từ cuốn sách giáo khoa Literature 6, bài viết "Phương pháp dạy học văn bản đa phương thức ở trường trung học qua khảo sát sách giáo khoa Literature 6, Hoa Kì" rút ra được nhiều chỉ dẫn tiến bộ từ phương pháp dạy học văn bản đa phương thức ở trường phổ thông của Hoa Kì để có thể áp dụng vào trong thực tiễn, trên cơ sở đó là kênh tham chiếu quan trọng trong tiến trình đổi mới chương trình và sách giáo khoa ởViệt Nam.

4p runordie3 27-06-2022 15 2   Download

Năng lực giao tiếp bằng văn bản đa phương thức bao gồm: đọc hiểu văn bản đa phương thức, viết văn bản đa phương thức, trình bày (nói) văn bản đa phương thức, nghe hiểu văn bản đa phương thức. Trong đó, đọc hiểu văn bản đa phương thức có vai trò rất quan trọng. Nó không chỉ là cơ sở cho học sinh học tốt môn Ngữ văn mà còn là cơ sở giúp học sinh đọc hiểu các môn học khác (các môn khoa học tự nhiên và khoa học xã hội). MỜi các bạn cùng tham khảo bài viết "Sử dụng văn bản đa phương thức trong dạy học đọc hiểu văn bản ở nhà trường phổ thông" để biết thêm nội dung chi tiết.

3p runordie3 27-06-2022 26 2   Download

In recent years, the task of Visual Question Answering (VQA) has evolved into a very attractive research field. Normally, this task requires a simultaneous understanding of both the visual content of the image and the textual content of the question.

7p visherylsandberg 18-05-2022 7 2   Download

This study further supports this theory by the analysis of the posters of fifteen top grossing American romantic comedies. Based on Kress and Van Leeuwen’s framework (2006), the research indicates the similarities and differences in using semiotic elements to create meaning and promote the movies. The results not only identify the patterns in designing movie posters but also examine the effectiveness of the framework in analyzing multimodal texts.

15p meyerowitz 25-12-2021 12 0   Download

Research purpose: The research purpose of the study is to propose teaching measures for multimodal text reading comprehension in Philology in secondary school.

27p extraenglish 26-05-2021 18 3   Download

The present panorama of communication features the co-employment of language and other semiotic resources. This paper addresses this fledging field, multimodal discourse analysis, by investigating a genre targeted at children. Specifically, it studies how meanings in comics for children are constructed both verbally and visually. The data for the study is one comic - Little Red Riding Hood, which is presented via colored images and verbal texts in English.

12p quenchua1 11-11-2019 29 1   Download

Multimodal interfaces combining, e.g., natural language and graphics take advantage of both the individual strength of each communication mode and the fact that several modes can be employed in parallel, e.g., in the text-picture combinations of illustrated documents. It is an important goal of this research not simply to merge the verbalization results of a natural language generator and the visualization results of a knowledge-based graphics generator, but to carefully coordinate graphics and text in such a way that they complement each other. ...

7p buncha_1 08-05-2013 39 3   Download

The growing popularity of multimedia documents requires language technologies to approach automatic language analysis and generation from yet another perspective: that of its use in multimodal communication. In this paper, we present a support tool for COSMOROE, a theoretical framework for modelling multimedia dialectics. The tool is a text-based search interface that facilitates the exploration of a corpus of audiovisual files, annotated with the COSMOROE relations.

4p bunthai_1 06-05-2013 51 2   Download

This paper describes the NECA MNLG; a fully implemented Multimodal Natural Language Generation module. The MNLG is deployed as part of the NECA system which generates dialogues between animated agents. The generation module supports the seamless integration of full grammar rules, templates and canned text. The generator takes input which allows for the specification of syntactic, semantic and pragmatic constraints on the output.

4p bunthai_1 06-05-2013 42 3   Download

In this talk, we will, show how techniques for planning text and discourse can be generalized to plan the structure and content of multimodal communications, that integrate natural language, pointing, graphics, and animations. The central claim of this talk is that the generation of multimodal discourse can be considered as an incremental planning process that aims to achieve a given communicative goal.

2p bunmoc_1 20-04-2013 32 1   Download

Mobile interfaces need to allow the user and system to adapt their choice of communication modes according to user preferences, the task at hand, and the physical and social environment. We describe a multimodal application architecture which combines finite-state multimodal language processing, a speech-act based multimodal dialogue manager, dynamic multimodal output generation, and user-tailored text planning to enable rapid prototyping of multimodal interfaces with flexible input and adaptive output. ...

8p bunmoc_1 20-04-2013 46 2   Download

We describe how context-sensitive, usertailored output is specified and produced in the COMIC multimodal dialogue system. At the conference, we will demonstrate the user-adapted features of the dialogue manager and text planner. three-dimensional walkthrough of the finished bathroom. We will focus on how context-sensitive, usertailored output is generated in the third, guidedbrowsing phase of the interaction. Figure 2 shows a typical user request and response from COMIC in this phase.

4p bunbo_1 17-04-2013 44 1   Download

Software to translate English text into American Sign Language (ASL) animation can improve information accessibility for the majority of deaf adults with limited English literacy. ASL natural language generation (NLG) is a special form of multimodal NLG that uses multiple linguistic output channels. ASL NLG technology has applications for the generation of gesture animation and other communication signals that are not easily encoded as text strings.

6p bunbo_1 17-04-2013 46 2   Download

We discuss Image Sense Discrimination (ISD), and apply a method based on spectral clustering, using multimodal features from the image and text of the embedding web page. We evaluate our method on a new data set of annotated web images, retrieved with ambiguous query terms. Experiments investigate different levels of sense granularity, as well as the impact of text and image features, and global versus local text features.

8p hongvang_1 16-04-2013 55 1   Download

This paper explores the relationship between discourse segmentation and coverbal gesture. Introducing the idea of gestural cohesion, we show that coherent topic segments are characterized by homogeneous gestural forms and that changes in the distribution of gestural features predict segment boundaries. Gestural features are extracted automatically from video, and are combined with lexical features in a Bayesian generative model. The resulting multimodal system outperforms text-only segmentation on both manual and automaticallyrecognized speech transcripts. ...

9p hongphan_1 15-04-2013 73 2   Download

This collective work deals with the analysis of audiovisual numerical texts or corpora, which may e.g. form part of an audiovisual library or archive. The development of methods, tools and conceptual frameworks (or models) for the concrete analysis of audiovisual texts or corpora is one of the most important issues for multimedia (audiovisual) digital libraries, archives, collections, etc. and also for any project or program to compile and disseminate knowledge heritage (e.g. cultural, scientific etc.).

309p namde01 10-04-2013 68 6   Download