BioASQ challenge on large-scale biomedical semantic indexing and question answering

BioASQ challenge on large-scale biomedical semantic indexing and question
answering
(BioASQ workshop to be collocated with CLEF 2013 in Valencia, Spain on
September 27, 2013)

Web site: http://bioasq.org/
twitter: https://twitter.com/bioasq

The BioASQ challenge aims to push for a solution to the information access
problem of biomedical experts. It will evaluate the ability of systems to
perform various tasks in the biomedical QA process:
1. large-scale classification of biomedical documents onto ontology concepts
(semantic indexing),
2. classification of biomedical questions onto relevant concepts,
3. retrieval of relevant document snippets, concepts and knowledge base
triples, and
4. delivery of the retrieved information in a concise and
user-understandable form.

In particular, the challenge will comprise two main tasks:

Task 1a. Large-scale online biomedical semantic indexing
Large-scale semantic indexing will be evaluated on the whole of PubMed. In
particular, participants will be asked to classify incoming documents before
the human curators do:
* BioASQ will distribute new unclassified PubMed documents.
* Participants will have a limited response time to attach MeSH terms.

Task 1a will run for three consecutive periods (batches) of 6 weeks each.
The first batch will start on April 15, 2013. Separate winners will be
announced for each batch. Participation in the task can be partial, i.e. in
some of the batches.

Task 1b. Biomedical semantic QA
The systems will be evaluated against gold answers created by a team of
biomedical experts. The task will run in two phases:
In Phase A:
* BioASQ will transmit simultaneously questions from the benchmark.
* Participants will have limited time to respond with concepts, snippets,
triples.
In Phase B:
* BioASQ will distribute questions + concepts, snippets, triples.
* Participants will respond with facts, summaries, etc.

The two phases will run consecutively, i.e. Phase B will start after the end
of phase A. Phase A will start on June 3, 2013. Separate winners will be
announced for each phase and each task target, i.e. concepts, snippets,
triples, etc. Participation in the task can be partial, i.e. in one of the
two phases and only for some of the targets.

Prizes will be awarded to the winners. Details about the prizes will be
announced on the Web site of the challenge.

Important dates:
March 18, 2013: Training data available for task 1a. Dry-run data available
for task 1b.
April 15, 2013: Start of task 1a.
June 3, 2013: Start of task 1b.
July 15, 2013: Submission of papers for BioASQ workshop
August 30, 2013: End of challenge
September 27, 2013: BioASQ workshop, collocated with CLEF 2013

The BioASQ challenge and workshop are organised by the BioASQ project,
supported by the European Commission within the 7th Framework Programme
(Grant Agreement No. 318652).