K4A developed a large scale automatic transcription and translation system capable of generating automatic subtitles for academic lectures or talks recorded on video.

How can you automatically create subtitles for millions of academic videos?

Firstly, automatic transcription tools provide verbatim same-language subtitles, thereby allowing the hard-of-hearing and, for example, non-native speakers, access to this content. Meanwhile, machine translation tools make these subtitles available in languages other than that in which the video was recorded.

Secondly, given that current automatic systems (ours included) are not yet capable of producing subtitles that are sufficiently free of errors as to be considered fully usable, we worked on the implementation of a system for intelligently editing automatic output. The aim here is to make the use of automatic tools time- and cost-effective, and therefore sustainable over any kinds of university or company based video collections.

We integrated our tools into VideoLectures.Net, Polimeda and the Matterhorn platform, where we were faced with two very different challenges. In the first instance, we needed to develop and implement a communication system between the Matterhorn platform and our tools to allow automatic captions to be generated for the videos added to the platform. Then, we will needed to consider the best way to implement our intelligent editing system, through which users will be able to correct any errors incurred during the transcription/translation process. The idea lead into the incorporation of new features typical of subtitle editors into the Matterhorn Engage Player.

In order to allow communication between Matterhorn and our automatic transcription/translation system, we have implemented a workflow operation that allows the necessary data to be sent from the platform to the transcription system, making use of a specially-designed web service. The developed software for the translation and transcription of video lectures is provided directly by one of K4A members, University Politecnica Valencia:

The offer for transcriptions is for the following languages:

  • English
  • Spanish
  • Catalan
  • French
  • Estonian
  • Italian
  • Dutch
  • Portuguese
  • Slovenian
  • German (in development)

and translation:

  • English ↔ Spanish, Catalan, French, Italian
  • English → Slovenian
  • Estonian, Dutch, Portuguese → English
  • Spanish ↔ Catalan

Credentify is a decentralized micro-credentials clearinghouse powered by a blockchain network across European universities allowing safe transfer of millions of micro-credentials as smaller units summing up into ECTS credits.

Credentify clearing house for digital credentials
Credentify clearing house for digital credentials

This empowers European students, educational workers and universities across Europe to make the accreditation of their traditional learning experience fast, dynamic, safe, reliable, transparent and accountable.

Credentify ensures that micro-credentials are certified and mapped to European qualifications frameworks and can scale into other forms of Higher Education. Credentify therefore empowers students and universities with equitable knowledge accreditation by allowing it to be more fair and flexible in its delivery.

Credentify is built on native European technologies, extensive policy and research analysis and is integrated with ESCO to maximize impact in the European Education Area and Digital Single Market.

Translexy is an API that enables you to translate your content. It is already integrated into MOOC platforms and video repositories to create new educational experiences.

It also offers tools for developers and researchers. Translexy API is the first free and open translation service to use Neural Machine Translation Models (NMT) for educational content, which immensely improves translation results. NMT has recently emerged as a disruptive technology and has become the dominant paradigm in machine translation.

Discover here how you can use, integrate and customize this service in your own application or MOOC solution. Translexy API can be used on its own or its results can be customized for pre-publishing with our partners.

Translexy API provides translation from English into nine European and two BRIC, languages, namely German (DE), Italian (IT), Portuguese (PT), Dutch (DU), Bulgarian (BG), Greek (EL), Polish (PL), Czech (CS), Croatian (CR), Russian (RU) and Chinese (ZH).

Problem Description

PASCAL has been very successful in supporting and promoting the development of the videolectures site (www.videolectures.net). Knowledge 4 All Foundation has been established to carry forward the development of this activity as well as other legacies of the PASCAL Network. One problem created by the success of videolectures is the difficulty that individual users have in identifying the best video for their needs among the vast range of possibilities afforded by the site. Each video has a particular mix of content and style of presentation with implicit assumptions about background knowledge and level of expertise of its intended audience.

On the other hand the video consumer has an approximate understanding of his/her abilities and material that he/she would like to learn about. For example, they may have a background in basic classiffication methods (eg SVMs) applied to text, some knowledge of probability theory, but not know about Bayesian reasoning. He/she would like to learn about Topic Models. The question of which sequence of videos would be most appropriate to help him/her to attain the desired knowledge would also depend on the style of presentation he/she prefers and so on. Currently Videolectures provides a contextualisation service that is only based on keywords extracted from the lecture titles.

It provides a recommendation of videos that are related to a given video. The relation would typically be based on the topic of the video or the lecturer. Furthermore, the system does not adapt its responses to the interests or background of the user. Videolectures has begun to collect information about individual users, though it is currently limited to associations with the keywords of lectures that they have viewed. This information is currently only being used for o -line analytics.

Proposed Harvest Project

The La Vie project would develop a proof-of-concept system that would provide users with advice on suitable videos for their needs. The key additional components that the project will bring to videolectures are:

  1. topic extraction and modeling based on text extracted from associated slides and audio transcriptions. This will ensure that the devised user models can capture semantic level interests of the users.
  2. inclusion of the information currently being logged about individual users in the recommender system running live on the videolectures site.
  3. visualisation of the developed recommender system. This will provide a topic landscape that will enable users to see the available videos emphasising those likely to be of interest to the user.

The La Vie system will be broken down into ve phases. First the text extraction modules will use text mining methods to extract information from the content and meta-data associated with a particular video. The second phase will involve topic modeling from the extracted text in order to develop a richer semantic representation. Phase three will use the developed topic models and other relevant information (such as formalised ontologies, taxonomies, structured information sources such as wikipedia, linked open data (LOD), and other contextual information) to populate a semantic representation of individual videos and users. The fourth phaseinformation retrieval module will provide relevant recommendations by linking the enriched semantic representations of users and videos. The nal phase collects feedback from users about the operation and e ectiveness of the recommendations that have been given. This information will be used to update the user and other models. All of these components will be integrated into videolectures through the user interface that will provide visualisation and interactivity.

More detailed descriptions of these components are given below. The project will provide a framework into which new modules can be plugged either to replace old ones or to enhance functionality. For example, a simple k-means algorithm would be used initially for the topic modeling, while clearly more advanced methods should provide improved performance and may be included in the system during the rst phase of the project if time permits. As an example of enhancing functionality the representation of user interests as a distribution over meta-data attributes would make it straightforward to include new modules re ning these distributions through interactions with a user applying say a bandit algorithm inspired by content-based image retrieval.

This framework approach to the development of the La Vie project ensures that the risks associated with the project are kept to a minimum. Once the framework is in place and the basic system operational, further incremental developments can be included without risking the overall success of the project results. Several components already exist including Qminer for user modeling, Enrycher for text enrichment, DocumentAtlas for visualisation, etc., that will make this approach likely to deliver a basic working system within approximately six weeks given the manpower envisaged below. Here we have taken into account the time required for the team to accumulate the necessary expertise in the di erent existing modules as well as the signi cant effort required to design and program the new framework and modules. In the second part of the project speci c advanced modules will be developed chosen based on the skills of the participants and potential for enhancing the overall performance of the system. These modules could include probabilistic topic models, bandit style algorithms for optimising user interaction, improved visualisation algorithms, etc.

The Harvest project La Vie has developed a proof-of-concept recommendation system to provide users of VideoLectures.Net with advice on suitable videos for their needs.

One problem created by the success of videolectures is the difficulty that individual users have in identifying the best video for their needs among the vast range of possibilities offered by the site. Each video has a particular mix of content and style of presentation with implicit assumptions about background knowledge and level of expertise of its intended audience. On the other hand the video consumer has an approximate understanding of his/her abilities and material that he/she would like to learn about.

For example, they may have a background in basic classification methods (e.g. SVMs) applied to text, some knowledge of probability theory, but do not know about Bayesian reasoning. He/she would like to learn about Topic Models. The question of which sequence of videos would be most appropriate to help him/her to attain the desired knowledge would also depend on the style of presentation he/she prefers and so on.

The Harvest project La Vie has developed a proof-of-concept recommendation system to provide users with advice on suitable videos for their needs. The system was broken down into six phases. First the text extraction modules uses text mining methods to extract information from the content and meta-data associated with a particular video. The second phase uses information retrieval techniques to retrieve related data from online external resources, i.e. Wikipedia.

Phase three employs automatic speech recognition techniques from the EU transLectures project in order to obtain the automatic transcriptions of videos. The fourth phase involves topic and user modeling from the extracted text in previous phases in order to develop a richer semantic representation. Phase five provides relevant recommendations by linking the enriched semantic representations of users and videos on a basis of a SVM classifier. The final phase collects feedback from users about the operation and effectiveness of the recommendations that have been given. This information is used to update our models.

All these components were integrated into VideoLectures.Net through the user interface that provides visualization and interactivity. In addition, a system update module was developed in order to synchronize our database with VideoLectures.Net, specially for including recently published videos to our recommendation engine.

Currently, the recommender is working on the VideoLectures.Net’s development machine and providing significantly better recommendations, based on a informal human evaluation. It is planned to deploy the recommender into the real VideoLectures.Net web site in the next weeks. This will allow us to perform a real-life evaluation of the La Vie recommender and to assess its quality and benefits against the old one.

20.000 academic talks enhanced by Machine Translations and Transcriptions

VideoLectures was born in 2001 as an internally-funded project at the Jozef Stefan Institute, Slovenia. The pilot project involved videoing the weekly Solomon Lectures held at the institute – regular lectures open to the public on artificial intelligence and general computer science topics. These were made available online in an attempt to enable students and researchers around the world to become part of a global audience. These initial experiments were so successful that the team behind VideoLectures began to collaborate with a series of European projects. One of the first and main contributors (both financially and in terms of lectures) was PASCAL, helping VideoLectures to grow rapidly. PASCAL’s many workshops and links to major conferences contributed several hundred new lectures for the site (three times more than from any other project) providing a valuable resource for  PASCAL researchers.

Soon the use of increasingly advanced video streaming technology made VideoLectures the collaborator of choice for many. As VideoLectures was seen to provide quality recordings for existing projects, the team were increasingly approached to become involved with new projects. Over the next few years, VideoLectures.Net joined forces with a series of Framework 5, 6 and 7 projects. The team also approached major conferences and events and recorded them in full. Such extensive involvement with so many projects quickly transformed the online resource into a truly global phenomenon covering an impressive range of topics. VideoLectures is now run by the dedicated Center for Transfer in Information Technologies at the Jozef Stefan Institute, led by Mitja Jermol since 2003. Enabled by a move to the Django web toolkit in early 2007, PowerPoint slides now appear next to the video, timed to change as the video plays. Talks on similar topics are linked. Viewers can comment on the lectures, leading to online discussions about the material. The number of downloads are listed, and lectures are ranked according to their popularity.

These innovations have created a real change to the style and quality of presentations being made by academics. “People check their ratings and ask themselves why someone less famous than they are is getting more downloads,” says Jermol. “This encourages them to improve their style.” As the technology improves and is made increasingly consistent over all operating systems and browsers, the number of visits from people across the world increases (see map). Feedback from viewers is also excellent, with complimentary comments from as far afield as Africa and Australia. Today VideoLectures.Net is one of the leading web-based educational portals of its kind. It has now been recognised by the 2013 World Summit Award for being one of the most outstanding examples of creative and  innovative e-Content in the world in the last decade.

In 2013 another recognition came, this time the WSA Online Jury evaluated 200 WSA winners from the last decade and selected what they consider to be the 8 all-time bests: the WSIS+10 Global Champions. VideoLectures.Net was selected as the winner in the “e- Science & Technology” category.

The World Summit Award (WSA) is a part of the United Nations Summit on the Information Society. It represents a unique global competition for the recognition of best e-Content and global creativity. It is a global not-for-profit activity promoting the most outstanding achievements as a flagship partnership initiative of the UN’s Global Alliance for ICT and Development and in close collaboration with UNESCO, UNIDO, ISOC and a world wide network of partners.

These two recognitions and our work and research in Open Education and Open Software were a pre-text in discussing and realising the unique partnership with UNESCO via establishing the Chair on Open Technologies for OER and Open Learning.