/ Projects / 4th Recognizing Textual Entailment Challenge

Goals of the organizers

The goal of the “BCI Competition IV” is to validate signal processing and classification methods for Brain-Computer Interfaces (BCIs). Compared to the past BCI Competitions, new challanging problems are addressed that are highly relevant for practical BCI systems, such as

  • classification of continuous EEG without trial structure (data sets 1).
  • classification of EEG signals affected by eye movement artifacts (data sets 2).
  • classification of the direction of wrist movements from MEG (data sets 3).
  • discrimination requiring fine grained spatial resolution in ECoG (data sets 4).


The organizers are aware of the fact that by such a competition it is impossible to validate BCI systems as a whole. But nevertheless we envision interesting contributions to ultimately improve the full BCI and to provide a challenging data base for the research community.

Goals for the participants

For each data set specific goals are given in the respective description. Technically speaking, each data set consists of single-trials of spontaneous brain activity, one part labeled (calibration or training data) and another part unlabeled (evaluation or test data), and a performance measure. The goal is to infer labels (or their probabilities) for the evaluation data sets from calibration data that maximize the performance measure for the true (but to the competitors unknown) labels of the evaluation data. Results will be announced at a workshop of the NIPS 2008 conference and on this web site. For each data set, the competition winner gets a chance to publish the algorithm in an individual article that will appear in a volume devoted to this competition.

Data sets

Data sets 1: ‹motor imagery, uncued classifier application› (description)
provided by the Berlin BCI group: Technische Universität Berlin (Machine Learning Laboratory) and Fraunhofer FIRST (Intelligent Data Analysis Group) (Klaus-Robert Müller, Benjamin Blankertz, Carmen Vidaurre, Guido Nolte), and Campus Benjamin Franklin of the Charité – University Medicine Berlin, Department of Neurology, Neurophysics Group (Gabriel Curio)
EEG, motor imagery (2 classes of left hand, right hand, foot); evaluation data is continuous EEG which contains also periods of idle state
[64 EEG channels (0.05-200Hz), 1000Hz sampling rate, 2 classes (+ idle state), 7 subjects]

Data sets 2a: ‹4-class motor imagery› (description)
provided by the Institute for Knowledge Discovery (Laboratory of Brain-Computer Interfaces), Graz University of Technology, (Clemens Brunner, Robert Leeb, Gernot Müller-Putz, Alois Schlögl, Gert Pfurtscheller)
EEG, cued motor imagery (left hand, right hand, feet, tongue)
[22 EEG channels (0.5-100Hz; notch filtered), 3 EOG channels, 250Hz sampling rate, 4 classes, 9 subjects]

Data sets 2b: ‹motor imagery› (description)
provided by the Institute for Knowledge Discovery (Laboratory of Brain-Computer Interfaces), Graz University of Technology, (Robert Leeb, Clemens Brunner, Gernot -Müller-Putz, Alois Schlögl, Gert Pfurtscheller)
EEG, cued motor imagery (left hand, right hand)
[3 bipolar EEG channels (0.5-100Hz; notch filtered), 3 EOG channels, 250Hz sampling rate, 2 classes, 9 subjects]

Data sets 3: ‹hand movement direction in MEG› (description)
provided by the Brain Machine Interfacing Initiative, Albert-Ludwigs-University Freiburg, the Bernstein Center for Computational Neuroscience Freiburg and the Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen (Stephan Waldert, Carsten Mehring, HubertPreissl, Christoph Braun)
The data set contains directionally modulated low-frequency MEG activity that was recorded while subjects performed wrist movements in four different directions.
[10 MEG channels (filtered to 0.5-100Hz), 400Hz sampling rate, 4 classes, 2 subjects]

Data sets 4: ‹finger movements in ECoG› (description)
provided by Departments of Physics and Medicine of the University of Washington, Seattle (Kai J. Miller) and Wadsworth Center, NYS Department of Health (Gerwin Schalk)
ECoG data during individual flexions of the five fingers; movements acquired with a data glove.
[48 – 64 ECoG channels (0.15-200Hz), 1000Hz sampling rate, 5 classes, 3 subjects]

[ top ]

Schedule

July 3rd 2008: launching of the competition
November 21st 2008, midnight CET to Nov 22nd: deadline for submissions
December 12th 2008: announcement of the results at a workshop of NIPS 2008 and on this web site

[ top ]

Submission

Submissions to a data set are to be sent to the responsible contact person as stated in the data set description. The submission has to comprise the estimated labels, names and affiliations of all involved researchers and a short note on the involved processing techniques. We send confirmations for each submission we get. If you do not receive a confirmation within 2 days please resend your email with CC to the other organizing committee members, ‹benjamin.blankertz@tu-berlin.de›, ‹alois.schloegl@tugraz.at›, ‹waldert@bccn.uni-freiburg.de›, ‹kjmiller@u.washington.edu›.

One researcher may NOT submit multiple results to one data set. She/he has to decide for her/his favorite one. However: From one research group multiple submissions to one data set are possible. The sets of involved researchers do not have to be disjoint, but (1) the ‘first author’ (contributor) should be distinct, and (2) approaches should be substantially different.

For details on how to submit your results please refer to the description of the respective data set. If questions remain unanswered send an email to the responsable contact person for the specific data set which is indicated in the description.

Submissions are evaluated for each group of data sets separately. There is no need to submit for all data sets of the competition in order to participate, however each submission must provide results for all data sets of one group (e.g., for all subjects provided in Data sets 2a).

Each participant agrees to deliver an extended description (4-6 pages) of the used algorithm for publication until Februar 1st 2009 in case she/he is the winner for one of the data sets.

Organizers

Berlin: Benjamin Blankertz, Carmen Vidaurre, Michael Tangermann, Klaus-Robert Müller

Graz: Clemens Brunner, Robert Leeb, Gernot Müller-Putz, Alois Schlögl, Gert Pfurtscheller

Freiburg/Tübingen: Stephan Waldert, Carsten Mehring, Ad Aertsen, Niels Birbaumer

Washington/Albany: Kai J. Miller, Gerwin Schalk

[ top ]

References

Short History of past BCI Competitions

The first BCI Competition was announced at NIPS 2001, and the second at NIPS 2002. The first competition was a first try to see how such a thing would work and it was only announced in a smaller community. Accordingly there were not such much submissions, but nevertheless many researchers showed great interest when the results were published (first in the internet and then in IEEE Trans Neural Sys Rehab Eng, 2003, vol 11(2), pp. 184-185 [ draft ]). For the second competition data sets were provided by four of the leading groups in EEG-based BCIs. Here we received 59 submissions. A review of the 2nd competition appeared in IEEE Trans Biomed Eng, 51(6):1044-1051, 2004 [ draft ] and articles of all winning teams of the competition were published in the same issue which provides a good overview of the state of art in classification techniques for BCI. The 3rd BCI Competition involved data sets from five BCI labs and we received 99 submissions. It was reviewed in IEEE Trans Neural Sys Rehab Eng, 14(2):153-159, 2006 [ draft ] and individual articles of the competition winners appeared in different journals.

References to papers that analyze competition data sets

can be found here. Please help us to make the list complete and keep it up to date by reporting unlisted papers to ‹benjamin.blankertz@tu-berlin.de›, preferably PubMed ID (PMID) or in BibTex format.

References to papers about past BCI Competitions

 

References to overviews of BCI research.

  • Gerwin Schalk. Brain-Computer symbiosis J Neural Eng 5 P1-P15, 2008.
  • Guido Dornhege, José del R. Millán, Thilo Hinterberger, Dennis McFarland, and Klaus-Robert Müller, editors. Toward Brain-Computer Interfacing. MIT Press, Cambridge, MA, 2007.
  • Kübler A, Kotchoubey B. Brain-computer interfaces in the continuum of consciousness. Curr Opin Neurol 20(6):643-9, 2007.
  • B.Z. Allison, E.W. Wolpaw, and Wolpaw J.R.. Brain-computer interface systems: progress and prospects. Expert Rev Med Devices, 4(4):463-474, 2007.
  • Eleanor A. Curran and Maria J. Stokes. Learning to control brain activity: A review of the production and control of EEG components for driving brain-computer interface (BCI) systems. Brain Cogn., 51:326-336, 2003.
  • Jonathan R. Wolpaw, Niels Birbaumer, Dennis J. McFarland, Gert Pfurtscheller, and Theresa M. Vaughan. Brain-computer interfaces for communication and control. Clin. Neurophysiol., 113:767-791, 2002.
  • José del R. Millán. Brain-computer interfaces. In M.A. Arbib (ed.), “Handbook of Brain Theory and Neural Networks, 2nd ed.” Cambridge: MIT Press, 2002.
  • Andrea Kübler, Boris Kotchoubey, Jochen Kaiser, Jonathan Wolpaw, and Niels Birbaumer. Brain-computer communication: Unlocking the locked in. Psychol. Bull., 127(3):358-375, 2001.

 

References to BCI Special Issues.

  • IEEE Signal Proc Magazine, 25(1), 2008.
  • IEEE Trans. Biomed. Eng., 51(6), 2004.
  • IEEE Trans. Neural Sys. Rehab. Eng., 11(2), 2003.
  • IEEE Trans. Rehab. Eng., 8(2), 2000.


Links to General Interest BCI Sites.

 

Logo

Follow us:

© Knowledge 4 All Foundation Ltd.