In the Challenge we have an experimental setup, where the test subject is first shown a question, followed by ten sentences. Five of the sentences are “relevant” to the question (they are of the same topic as the question) and five of the sentences are irrelevant (they have no relation to the topic of the question). One of the relevant sentences is the correct answer to the question. The experimental setup is designed to resemble a real-life information retrieval scenario as closely possible while at the same time retaining a controlled setup where the ground truth is known. Thus, in the Challenge the meaning of “relevant” is defined in terms of this experimental setup. The objective of the Challenge is to find the best methods and features that can be used to predict the relevance from the eye movement measurements.

The goal of the “BCI Competition III” is to validate signal processing and classification methods for Brain-Computer Interfaces (BCIs). Compared to the past BCI Competitions, new challanging problems are addressed that are highly relevant for practical BCI systems, such as

• session-to-session transfer (data set I)
• small training sets, maybe to be solved by subject-to-subject transfer (data set IVa),
• non-stationarity problems (data set IIIb, data set IVc),
• multi-class problems (data set IIIa, data set V, data set II,),
• classification of continuous EEG without trial structure (data set IVb, data set V).

Also this BCI Competition includes for the first time ECoG data (data set I) and one data set for which preprocessed features are provided (data set V) for competitors that like to focus on the classification task rather than to dive into the depth of EEG analysis.

The aim of this challenge is to encourage work on automated construction and population of ontologies. For the purposes of this challenge, an ontology consists of a set of concepts and a set of instances. An instance can be assigned to one or more concepts. The concepts are connected into a hierarchy. Several types of tasks are included in this challenge:

• Ontology construction: given a set of documents, construct an ontology with these documents as instances.
• Ontology extension: given a partial ontology and a set of instances, extend the ontology with new concepts using the given instances.
• Ontology population: given a partially populated hierarchy of concepts, develop a model that can assign new instances to concepts.
• Concept naming: given a set of instances and the assignment of instances to concepts, suggest user-friendly labels for the concepts.

Evaluation is based on comparing the results to a “golden standard” ontology prepared by human editors.

The goal of this challenge is to recognise objects from a number of visual object classes in realistic scenes (i.e. not pre-segmented objects). It is fundamentally a supervised learning learning problem in that a training set of labelled images will be provided. The four object classes that have been selected are:

• motorbikes
• bicycles
• people
• cars

There will be two main competitions:

• For each of the 4 classes, predicting presence/absence of an example of that class in the test image.
• Predicting the bounding box and label of each object from the 4 target classes in the test image.

Contestants may enter either (or both) of these competitions, and can choose to tackle any (or all) of the four object classes. The challenge allows for two approaches to each of the competitions:

• Contestants may use systems built or trained using any methods or data excluding the provided test sets.
• Systems are to be built or trained using only the provided training data.

The intention in the first case is to establish just what level of success can currently be achieved on these problems and by what method; in the second case the intention is to establish which method is most successful given a specified training set.

The goal of the proposed challenge is to assess the current situation concerning Machine Learning (ML) algorithms for Information Extraction (IE) from documents, identifying future challenges and to foster additional research in the field. The aim is to:

• Define a methodology for fair comparison of ML algorithms for IE.
• Define a publicly available resource for evaluation that will exist and be used beyond the lifetime of the challenge; such framework will be ML oriented, not IE oriented as so far proposed in other similar evaluations.
• Perform actual tests of different algorithms in controlled situations so to understand what works and what does not and therefore identify new future challenges.

Efficient approximate inference in large Hybrid Networks (graphical models with discrete and continuous variables) is one of the major unsolved problems in machine learning, and insight into good solutions would be beneficial in advancing the application of sophisticated machine learning to a wide range  of real-world problems. Such research would benefit potentially applications in Speech Recognition, Visual Object Tracking and Machine Vision, Robotics, Music Scene Analysis, Analysis of complex Times series, understanding and modelling complex computer networks, Condition monitoring, and other complex phenomena. This theory challenge specifically addresses a central component area of PASCAL, namely Bayesian Statistics and statistical modelling, and is also related to the other central areas of Computational Learning, Statistical Physics and Optimisation techniques. One aim of this challenge is to bring together leading researchers in graphical models and related areas to develop and improve on existing methods for tackling the fundamental intractability in HNs. We do not believe that there will necessarily emerge a single best approach, although we would expect that successes in one application area should be transferable to related areas. Many leading machine learning researches are currently working on applications that involve HNs, and we invite participants to suggest their own applications. Ideally, this would be in the form of a dataset along the lines of PASCAL.

Background: The idea of a multilingual world is becoming truly a reality, as sophisticated monolingual, cross-lingual and multilingual language technologies have been created and have immensely optimized the translation quality and language/topic coverage in real-life situations.

However, a major challenge resides in respecting the plural diversity of all world regions residing in their languages and cultures yet avoiding all forms of education, research and business fragmentation linked to those same assets. Another is the relationship between the EU’s language technologies for the Digital Single Market and its connection with other language markets, which is also a major opportunity the field had in the last couple of years.

Specific Challenge: The challenge is to facilitate multilingual online communication in developing countries specifically in the domain of education, and enable it with the technologies developed in the EU, currently leading in this field, by taking down existing language barriers. These barriers hamper wider penetration of cross-border education, commerce, social communication and exchange of cultural content.

Additionally, current machine translation solutions typically perform well only for a limited number of target languages, and for a given text type. The potential for a value added global action in creating access to educational content with machine translation acting as a bridge between national educational systems is enormous.

Specific Solution: The Knowledge 4 All Foundation solved the problem of mass translation in education by developing TransLexy, a robust service that provides translation from English into nine European and two BRIC, languages, namely:

1. English → Bulgarian (Български)
2. English → Czech (Čeština)
3. English → German (Deutsch)
4. English → Greek (Ελληνικά)
5. English → Croatian (Hrvatski)
6. English → Italian (Italiano)
7. English → Dutch (Nederlands)
8. English → Polish (Polszczyzna)
9. English → Portuguese (Português)
10. English → Russian (Русский)
11. English → Chinese (漢語, 汉语)

The platform is intended to overcome the existing language barriers in education, and can deal with huge volumes, high variety of languages and education text styles, and deliver results in reasonable time (in most cases, instantly).

Moving forward: The Foundation will add to its portfolio in partnership with the University of Edinburgh the following language pairs:

• English → Afaan Oromo
• English → Tigrinya
• English → Igbo
• English → Yoruba
• English → Gujarati
• English → Punjabi
• Kurdish→ English
• North Korean → English
• Hausa → English
• Swahili→ English

The Probabilistic Automata learning Competition (PAutomaC) is the first on-line challenge about learning non-deterministic probabilistic finite state machines (HMM, PFA, …)

The competition is over. In addition of the technical report describing the competition that was written at the beginning, an article published in the proceedings of ICGI’12 is available. It contains a lot of details about the competition and the results. In the same proceedings, one can find a short paper from the winning team of the competition. Two articles from other participants to the competition can also be found here (team Hulden) and here (team Kepler).

News

December 11th 2012: the code of the winning team is avalable here.

September 11th 2012: following the discussion during the workshop, the target and solutions files of the competition phase data are available in the download section.

September 7th 2012: the PAutomaC workshop at ICGI’12 is on.

July 3rd: After a storm that postponed for 2 days the end of competition, PAutomaC is finished. The winner is the team of Chihiro Shibata and Ryo Yoshinaka, Congratulation to them! And thanks to all participants: this have been a great competition.

May 20th: Phase 2 is launched: The data of the real competition are available! The data sets of the training phase are still available but you cannot submit your results any more. However, the files containing the true probabilities (obtained with the target automata) are available.

March 20th: 16 new data sets are available!

March 8th: The website is fully operational, the first data set is available.

Why this competition?

Finite state automata (or machines) are well-known models for characterizing the behaviour of systems or processes. They have been used for several decades in computer and software engineering to model the complex behaviours of electronic circuits and softwares. A nice feature of an automaton model is that it is easy to interpret, but unfortunately in many applications the original design of a system is unknown. That is why learning approaches has been used, for instance:

• To modelize DNA or protein sequences in bioinformatics.
• To find patterns underlying different sounds for speech processing.
• To develop morphological or phonological rules for natural language processing.
• To modelize unknown mechanical processes in Physics
• To discover the exact environment of robots.
• To detect Anomaly for detecting intrusions in computer security.
• To do behavioural modelling of users in applications ranging from web systems to the automotive sector.
• To discover the structure of music styles for music classification and generation.
• …

In all such cases, an automaton model is learned from observations of the system, i.e., a finite set of strings. As the data gathered from observations is usually unlabelled, the standard method of dealing with this situation is to assume a probabilistic automaton model, i.e., a distribution over strings. In such a model, different states can generate different symbols with different probabilities: the two main formalisms are Hidden Markov Models (HMM) and Probabilistic Finite State Automata (PFA).

This is what this competition is about.

How is it working?

We automatically generated PFAs in a way that is described here. We then generated two data sets from each PFA: one is the training set and the second is the test set (where duplicate strings have been removed). The idea is to train your learning algorithm on the first data set in order to assign probabilities to the strings in the test set.

Details about how to participate can be found in the participate section.

There also will be some real world data sets during the second phase of the competition.

According to the World Health Organisation, cardiovascular diseases (CVDs) are the number one cause of death globally: more people die annually from CVDs than from any other cause. An estimated 17.1 million people died from CVDs in 2004, representing 29% of all global deaths. Of these deaths, an estimated 7.2 million were due to coronary heart disease. Any method which can help to detect signs of heart disease could therefore have a significant impact on world health. This challenge is to produce methods to do exactly that. Specifically, we are interested in creating the first level of screening of cardiac pathologies both in a Hospital environment by a doctor (using a digital stethoscope) and at home by the patient (using a mobile device).

The problem is of particular interest to machine learning researchers as it involves classification of audio sample data, where distinguishing between classes of interest is non-trivial. Data is gathered in real-world situations and frequently contains background noise of every conceivable type. The differences between heart sounds corresponding to different heart symptoms can also be extremely subtle and challenging to separate. Success in classifying this form of data requires extremely robust classifiers. Despite its medical significance, to date this is a relatively unexplored application for machine learning.

Classifying Heart Sounds Challenge

Classifying Heart Sounds Challenge

Classifying Heart Sounds Challenge