Machine Translation for Videos

K4A developed a large scale automatic transcription and translation system capable of generating automatic subtitles for academic lectures or talks recorded on video.

How can you automatically create subtitles for millions of academic videos?

Firstly, automatic transcription tools provide verbatim same-language subtitles, thereby allowing the hard-of-hearing and, for example, non-native speakers, access to this content. Meanwhile, machine translation tools make these subtitles available in languages other than that in which the video was recorded.

Secondly, given that current automatic systems (ours included) are not yet capable of producing subtitles that are sufficiently free of errors as to be considered fully usable, we worked on the implementation of a system for intelligently editing automatic output. The aim here is to make the use of automatic tools time- and cost-effective, and therefore sustainable over any kinds of university or company based video collections.

We integrated our tools into VideoLectures.Net, Polimeda and the Matterhorn platform, where we were faced with two very different challenges. In the first instance, we needed to develop and implement a communication system between the Matterhorn platform and our tools to allow automatic captions to be generated for the videos added to the platform. Then, we will needed to consider the best way to implement our intelligent editing system, through which users will be able to correct any errors incurred during the transcription/translation process. The idea lead into the incorporation of new features typical of subtitle editors into the Matterhorn Engage Player.

In order to allow communication between Matterhorn and our automatic transcription/translation system, we have implemented a workflow operation that allows the necessary data to be sent from the platform to the transcription system, making use of a specially-designed web service. The developed software for the translation and transcription of video lectures is provided directly by one of K4A members, University Politecnica Valencia:

• TLK: The transLectures-UPV toolkit za Automatic Speech Recognition (ASR) https://www.translectures.eu/web/tlk/
• TLP: The transLectures Platform for integracion of all components into the users repository https://www.translectures.eu/web/tlp/
• Testing service with language models at: http://ttp.mllp.upv.es/

The offer for transcriptions is for the following languages:

• English
• Spanish
• Catalan
• French
• Estonian
• Italian
• Dutch
• Portuguese
• Slovenian
• German (in development)

and translation:

• English ↔ Spanish, Catalan, French, Italian
• English → Slovenian
• Estonian, Dutch, Portuguese → English
• Spanish ↔ Catalan