MUTASK: Multimodal Translation and Adaptation of Scientific Knowledge for Global Accessibility (Chist-Era 2026-2029)
This project develops European AI-driven methods to translate and adapt scholarly content – bridging linguistic gaps, shaping accessible knowledge for different audiences, and ensuring that vital research truly reaches the people who need it. We confront three main hurdles: (1) numerous researchers and students are restricted by language barriers, limiting engagement with important scientific findings; (2) even when translation exists, content often remains too dense or too specialized for lay audiences; and (3) there is a lack of effective workflows that carry these adapted materials to their intended communities.
To tackle these challenges, our consortium integrates state-of-the-art machine translation, automated summarization, semantic indexing, and video-based storytelling into one streamlined workflow. A core goal is to reshape complex academic documents – research articles, conference proceedings, outreach texts – into dynamically narrated videos, detailed yet understandable summaries, and localized translations accessible at multiple expertise levels. Users can pause videos to request clarifications, ensuring a truly adaptive learning experience. This “human touch” sits at the heart of our approach: we emphasize media education, audience research, and strategic communication, so that each translated and transformed output genuinely resonates with target groups ranging from schoolchildren to lay citizens and policy makers.
By joining specialists in computational linguistics, media education, and strategic communication from Poland, France, and Switzerland, we will deliver practical tools that fit current publishing systems while upholding Open Science standards. Crucially, we plan to involve the ultimate audiences – students, journalists, nonprofits – in testing these solutions. The result is a production system that seamlessly converts cutting-edge science into richly engaging, clearly communicated, and culturally adapted outputs for broader society. Ultimately, we foster more inclusive, democratic, and trusted science by ensuring that knowledge in any language or format remains comprehensible and widely disseminated.
With the growth of the information in different media such as TV programs or internet, a new issue arises. How a user can access to the information which is expressed in a foreign language? The idea of the project is to develop a multilingual help system of understanding without any human being intervention. What we would like to do, is to help people understanding broadcasting news, presented in a foreign language and to compare it to the corresponding one available in the mother tongue of the user. The concept of understanding is approached in this project by giving access to any information whatever the language in which it is presented. In fact, with the development of internet and satellite TV, tens of thousands shows and broadcasting news are available in different languages, it turns out that even high educated people, do not speak more than two or three languages while the majority speaks only one, which makes this huge amount of information inaccessible. Consequently, the majority of TV and radio programs as well as information on internet are inaccessible for the majority of people. And yet, one would like to listen to news in his own language and compare it to what has been said on the same topic in another language. For instance, how the topic of AIDS is presented in SAUDI-ARABIA and in USA? What is the opinion of The Jerusalem-Post about Yasser-Arafat? And how it is presented in Al-Quds ? To access to various information and to make available different and sometimes opposite information, we propose to develop AMIS (Access to Multilingual Information and Opinions). As a result, AMIS will permit to have another side of story of an event. The understanding process is considered here to be the comprehension of the main ideas of a video. The best way to do that, is then to summarize the video for having access to the essential information. Henceforth, AMIS will focus on the most relevant information by summarizing it and by translating it to the user if necessary. Another aspect of AMIS is to compare two summaries produced by this system, from two languages on the same topic whatever their support is: video, audio or text and to present the difference between their contents in terms of information, sentiments, opinions, etc. Furthermore, the coverage of the web and social media will be exploited in order to strengthen or weaken the retrieved opinions. AMIS could be incorporated in a TV remote control or such as software associated to any internet browser. In conclusion AMIS will address the following research points:
- Text, audio and video summarization
- Automatic Speech Recognition (ASR)
- Machine Translation
- Cross-lingual sentiment analysis
- Achieving successful synergy between the previous research topics
TRAM: TRanslation of Arab Music (Projet de Coopération scientifique inter-universitaires 2016-2018)
The consortium is composed of the University of Jordan, the univeristy of Lorraine and the university of Faculty of Sciences Beyrouth Lebanon.
The objective of TRAM is to show the feasibility of an automatic accompaniment of Arab vocal improvisation. The idea is to propose an automatic instrumental response to an Arab singer who executes a Mawwal (or Istikhbar). The originality of the project is to investigate an approach based on Machine Translation (MT) in studying the accompaniment of Arab vocal improvisation. This approach considers the mutual interaction between the singer and the instrumentalist as a question and answer: vocal sentence (question) and instrumental response (answer). In Machine translation, we need a parallel corpus composed of a source and a target language. The training process allows then to associate each phrase of the source sentence to its corresponding phrase in the target language. To deal with this project, we propose a consortium composed of experts in music and in machine translation and more generally on machine learning process. This project necessitates collecting data which will be a considerable resource for researchers and which will provided freely to our research community. This bootstrapping project will probably help us to apply in the near future to H2020.
