Discrete optimization problems and combinatorial structures are becoming increasingly important in machine learning. In the endeavour to process more and more complex data, one may quickly find oneself working with graphs, relations, partitions, or sparsity structure. In addition, we may want to predict structured, sparse estimators, or combinatorial objects such as permutations, trees or other graphs, group structure and so on.

While complex structures enable much richer applications, they often come with the caveat that the related learning and inference problems become computationally very hard. When scaling to large data, combinatorial problems also add some further challenges to compact representations, streaming and distributed computation.

Despite discouraging theoretical worst-case results, many practically interesting problems can be much more well behaved (when modeled appropriately), or are well modeled by assuming properties that make them so. Such properties are most often structural and include symmetry, exchangeability, sparsity, or submodularity. Still, not all of these structures and their application ranges are well understood. The DISCML workshop revolves around such structures in machine learning, their applications and theory.

Organizers

• Andreas Krause, ETH Zurich
• Jeff A. Bilmes, University of Washington
• Stefanie Jegelka, UC Berkeley

A key ambition of AI  is to render computers able to evolve in and interact with the real world. This can be made possible only if the machine is able to produce a correct interpretation of its available modalities (image, audio, text, …), upon which it would then build a reasoning to take appropriate actions. Computational linguists use the term “semantics” to refer to the possible interpretations (concepts) of natural language expressions, and showed some interest in “learning semantics”, that is finding (in an automated way) these interpretations. However, “semantics” are not restricted to natural language modality, and are also pertinent for speech or vision modalities. Hence, knowing visual concepts and common relationships between them would certainly bring a leap forward in scene analysis and in image parsing akin to the improvement that language phrase interpretations would bring to data mining, information extraction or automatic translation, to name a few.

Progress in learning semantics has been slow mainly because this involves sophisticated models which are hard to train, especially since they seem to require large quantities of precisely annotated training data. However, recent advances in learning with weak and limited supervision lead to the emergence of a new body of research in semantics based on multi-task/transfer learning, on learning with semi/ambiguous supervision or even with no supervision at all. The goal of this workshop is to explore these new directions and, in particular, to investigate the following questions:

• How should meaning representations be structured to be easily interpretable by a computer and still express rich and complex knowledge?
• What is a realistic supervision setting for learning semantics? How can we learn sophisticated representations with limited supervision?
• How can we jointly infer semantics from several modalities?

Organizers

• Antoine Bordes – CNRS – Université de Technologie de Compiègne
• Jason Weston – Google
• Ronan Collobert – IDIAP
• Léon Bottou – Microsoft

Program Committee 

• Y-Lan Boureau – NYU/INRIA
• Nicolas Le Roux – INRIA
• Marc’Aurelio Ranzato – Google
• Ilya Sutskever – University of Toronto
• Graham Taylor – NYU
• Nicolas Usunier – UPMC

Preference learning has been studied for several decades and has drawn increasing attention in recent years due to its importance in web applications, such as ad serving, search, and electronic commerce. In all of these applications, we observe (often discrete) choices that reflect relative preferences among several items, e.g. products, songs, web pages or documents. Moreover, observations are in many cases censored. Hence, the goal is to reconstruct the overall model of preferences by, for example, learning a general ordering function based on the partially observed decisions.
Choice models try to predict the specific choices individuals (or groups of individuals) make when offered a possibly very large number of alternatives. Traditionally, they are concerned with the decision process of individuals and have been studied independently in machine learning, data and web mining, econometrics, and psychology. However, these diverse communities have had few interactions in the past. One goal of this workshop is to foster interdisciplinary exchange, by encouraging abstraction of the underlying problem (and solution) characteristics.

The workshop is motivated by the two following lines of research:

1. Large scale preference learning with sparse data: There has been a great interest and take-up of machine learning techniques for preference learning in learning to rank, information retrieval and recommender systems, as supported by the large proportion of preference learning based literature in the widely regarded conferences such as SIGIR, WSDM, WWW, and CIKM. Different paradigms of machine learning have been further developed and applied to these challenging problems, particularly when there is a large number of users and items but only a small set of user preferences are provided.

2. Personalization in social networks: recent wide acceptance of social networks has brought great opportunities for services in different domains, thanks to Facebook, LinkedIn, Douban, Twitter, etc. It is important for these service providers to offer personalized service (e.g., personalization of Twitter recommendations). Social information can improve the inference for user preferences. However, it is still challenging to infer user preferences based on social relationship.

As such, we especially encourage submissions on theory, methods, and applications focusing on large-scale preference learning and choice models in social media. In order to avoid a dispersed research workshop, we solicit submissions (papers, demos and project descriptions) and participation that specifically tackle the research areas as below:

• Preference elicitation
• Ranking aggregation
• Choice models and inference
• Statistical relational learning for preferences
• Link prediction for preferences
• Learning Structured Preferences
• Multi-task preference learning
• (Social) collaborative filtering

Program Committee:

• Nir Ailon, Israel Institute of Technology
• Edwin Bonilla, NICTA-ANU
• Tiberio Caetano, NICTA - ANU
• François Caron, INRIA
• Jonathan Chung-Kuan Huang, Carnegie Mellon University
• Chris Dance, Xerox Research Centre Europe
• Jo­hannes Fürnkranz, TU Darmstadt
• John Guiver, Microsoft Research Cambridge
• Eyke Hüllermeier, Universität Marburg
• Hang Li, Microsoft Research Asia
• Robert Nowak, University of Wisconsin-Madison
• Filip Radlinski, Microsoft
• Chu Wei, Yahoo! Labs
• Markus Weimer, Yahoo! Labs
• Kai Yu, NEC Labs
• Zhao Xu, Fraunhofer IAIS

Program Co-Chair:

• Jean-Marc Andreoli, Xerox Research Centre Europe
• Cedric Archambeau, Xerox Research Centre Europe
• Guillaume Bouchard, Xerox Research Centre Europe
• Shengbo Guo, Xerox Research Centre Europe
• Kristian Kersting, Fraunhofer IAIS -- University of Bonn
• Scott Sanner, NICTA-ANU
• Martin Szummer, Microsoft Research Cambridge
• Paolo Viappiani, Aalborg University
• Onno Zoeter, Xerox Research Centre Europe

The fields of machine learning and pattern recognition can arguably be considered as a modern-day incarnation of an endeavor which has challenged mankind since antiquity. In fact, fundamental questions pertaining to categorization, abstraction, generalization, induction, etc., have been on the agenda of mainstream philosophy, under different names and guises, since its inception. Nowadays, with the advent of modern digital computers and the availablity of enormous amount of raw data, these questions have taken a computational flavor.

As it often happens with scientific research, in the early days of machine learning there used to be a genuine interest around philosophical and conceptual issues, but over time the interest shifted almost entirely to technical and algorithmic aspects and became driven mainly by practical applications. In recent years, however, there has been a renewed interest around the foundational and/or philosophical problems of machine learning and pattern recognition, from both the computer scientist's and the philosopher's camps. This suggests that the time is ripe to initiating a long-term dialogue between the philosophy and the machine learning communities with a view to foster cross-fertilization of ideas.

In particular, we do feel the present moment is appropriate for reflection, reassessment and eventually some synthesis, with the aim of providing the machine learning field a self-portrait of where it currently stands and where it is going as a whole, and hopefully suggesting new directions. The aim of this workshop is precisely to consolidate research efforts in this area, and to provide an informal discussion forum for researchers and practitioners interested in this important yet diverse subject.

The workshop is planned to be a one-day meeting. The program will feature invited as well as contributed presentations. We feel that the more informal the better and we would like to solicit open and lively discussions and exchange of ideas from researchers with different backgrounds and perspectives. Plenty of time will be allocated to questions, discussions, and breaks.

This workshop will focus on relations between machine learning problems. The idea is that by better understanding how different machine learning problems relate to each other, we will be able to better understand the field as a whole.

The idea of a relation is quite general. In includes such notions as reductions between learning problems, but is not restricted to that. Our goal can be explained by an analogy with functional analysis - rather than studying individual functions, functional analysis focusses on the transformations between different functions. This high level of abstraction led to enormous advances in mathematics.

The motivation for the workshop is several-fold:

• End users typically only care about solving their problem, not the technique used. Many machine learning techniques still require a detaile dunderstanding of their operation in order to use them
• ML as a service Much modern software is evolving to being delivered via the web as a service. What does it mean  for Machine Learning to be delivered as a service? One question that needs resolving is how to describe what the service does (ideally in a declarative manner). Understanding relations between machine learning problems can be thus seen as analogous to the composition of (Machine Learning) web services
• Reinvention Many machine learning solutions are reinvented / rediscovered. This is hardly surprising since the focus is often on techniques and not problems. If you can not describe your problem in a manner that others can easily understand and search, then it is hard to figure whether solutions to seemingly new problems already exist.
• Modularity A feature of mature engineering disciplines is modularity, which has enormous design and economic advantages. Understanding relations between problems seems important to achieve greater modularity.
• Conceptual simplicity Finally, if one can understand the field using a smaller number of primitives and combination operations, then this has an intrinsic appeal (apply Occam's razor at the meta-level!)

Organisers

Bob Williamson
John Langford
Ulrike von Luxburg
Mark Reid
Jennifer Wortman Vaughan

In recent years, computational photography (CP) has emerged as a new field that has put forward a new understanding and thinking of how to image and display our environment. Besides addressing classical imaging problems such as deblurring or denoising by exploiting new insights and methodology in machine learning as well as computer and human vision, CP goes way beyond traditional image processing and photography.

By developing new imaging systems through innovative hardware design, CP not only aims at improving existing imaging techniques but also aims at the development of new ways of perceiving and capturing our surroundings. However, CP is not only about to redefine "everyday" photography but also aims at applications in scientific imaging, such as microscopy, biomedical imaging, and astronomical imaging, and can thus be expected to have a significant impact in many research areas.

After the great success of last year's workshop on CP at NIPS, this workshop proposal tries to accommodate the strong interest in a follow-up workshop expressed by many workshop participants last year. The objectives of this workshop are: (i) to give an introduction to CP, present current approaches and report about the latest developments in this fast-progressing field, (ii) spot and discuss current limitations and present open problems of CP to the NIPS community, and (iii) to encourage scientific exchange and foster interaction between researchers from machine learning, neuro science and CP to advance the state of the art in CP.

The tight interplay between both hardware and software renders CP an exciting field of research for the whole NIPS community, which could contribute in various ways to its advancement, be it by enabling new imaging devices that are possible due to the latest machine learning methods or by new camera and processing designs that are inspired by our neurological understanding of natural visual systems.

Thus the target group of participants are researchers from the whole NIPS community (machine learning and neuro science) and researchers working on CP and related fields.

Driven by cheap commodity storage, fast data networks, rich structured models, and the increasing desire to catalog and share our collective experiences in real-time, the scale of many important learning problems has grown well beyond the capacity of traditional sequential systems. These “Big Learning” problems arise in many domains including bioinformatics, astronomy, recommendation systems, social networks, computer vision, web search and online advertising.

Simultaneously, parallelism has emerged as a dominant widely used computational paradigm in devices ranging from energy efficient mobile processors, to desktop supercomputers in the form of GPUs, to massively scalable cloud computing services. The Big Learning setting has attracted intense interest across industry and academia, with active research spanning diverse fields ranging from machine learning and databases to large scale distributed systems and programming languages. However because the Big Learning setting is being studied by experts of these various communities, there is a need for a common venue to discuss recent progress, to identify pressing new challenges, and to exchange new ideas.

This workshop aims to:

* Bring together parallel and distributed system builders in industry and academia, machine learning experts, and end users to identify the key challenges, opportunities, and myths of Big Learning. What REALLY changes from the traditional learning setting when faced with terabytes or petabytes of data?
* Solicit practical case studies, demos, benchmarks and lessons-learned presentations, and position papers.
* Showcase recent and ongoing progress towards parallel ML algorithms
* Provide a forum for exchange regarding tools, software, and systems that address the Big Learning problem.
* Educate the researchers and practitioners across communities on state-of-the-art solutions and their limitations, particularly focusing on key criteria for selecting task- and domain-appropriate platforms and algorithms.

Focal points for discussions and solicited submissions include but are not limited to:

1. Case studies of practical applications that operate on large data sets or computationally intensive models; typical data and workflow patterns; machine learning challenges and lessons learned.
2. Insights about the end users for large-scale learning: who are they, what are their needs, what expertise is required of them?
3. Common data characteristics: is it more typical for data to appear in streams or in batches? What are the applications that demand online or real-time learning, and how can the engineering challenges for deploying autonomously adaptive systems be overcome? Which analytic and learning problems are more appropriate for (or even require) analysis in the cloud, and when is “desktop” learning on sub-sampled or compressed data sufficient?
4. Choices in data storage and management, e.g., trade-offs between classical RDBMS and NoSQL platforms from a data analysis and machine learning perspectives.
5. The feasibility of alternate structured data storage: object databases, graph databases, and streams.
6. Suitability of different distributed system platforms and programming paradigms: Hadoop, DryadLINQ, EC2, Azure, etc.
7. Applicability of different learning and analysis techniques: prediction models that require large-scale training, vs. simpler data analysis (e.g., summary statistics), which is needed when.
8. Computationally intensive learning and inference: Big Learning doesn’t just mean 9. Big Data it also can mean massive models or structured prediction tasks.
Labeling and supervision: scenarios for large-scale label availability and appropriate learning approaches. Making use of diverse labeling strategies (curated vs. noisy/crowd-sourced/feedback-based labeling)
10. Real-world deployment issues: initial prototyping requires quickly-implemented-and-expandable solutions, along with the ability to easily incorporate new features/data sources.
11. Practicality of high-performance hardware for large-scale learning (e.g., GPUs, FPGAs, ASIC). GPU vs. CPU processors: programming strategies and performance opportunities and tradeoffs.
12. Unifying the disparate data structures and software libraries that have emerged in the GP-GPU community.
13. Evaluation methodology and trade-offs between machine learning metrics (predictive accuracy), computational performance (throughput, latency, speedup), and engineering complexity and cost.
14. Principled methods for dealing with huge numbers of features. As the number of data points grow, often times so do the number of features as well as their dependence structure. Does Big Learning require, for example, better ways of doing multiple hypothesis testing than FDR?
15. Determination of when is an answer good enough. How can we efficiently estimate confidence intervals over Big Data?

Target audience includes industry and academic researchers from the various subfields relevant to large-scale machine learning, with a strong bias for either position talks that aim to induce discussion, or accessible overviews of the state-of-the-art. We will solicit paper submissions in the form of short, long and position papers as well as demo proposals. Papers that focus on emerging applications or deployment case studies will be particularly encouraged, while demos of operational toolkits and platforms will be considered for inclusion in the primary program of the workshop.

Cosmology aims at the understanding of the universe and its evolution through scientific observation and experiment and hence addresses one of the most profound questions of human mankind. With the establishment of robotic telescopes and wide sky surveys cosmology already now faces the challenge of evaluating vast amount of data. Several projects will image large fractions of the sky in the next decade; for example the Dark Energy Survey will culminate in a catalogue of 300 million objects extracted from peta-bytes of observational data, while the Large Synoptic Survey Telescope is designed to image the entire observable Southern sky every few nights for 10 years. The importance of automatic data evaluation and analysis tools for the success of these surveys is undisputed.

Many problems in modern cosmological data analysis are tightly related to fundamental problems in machine learning, such as classifying stars and galaxies and cluster finding of dense galaxy populations. Other typical problems include data reduction, probability density estimation, how to deal with missing data and how to combine data from different surveys. An increasing part of modern cosmology aims at the development of new statistical data analysis tools and the study of their behaviour and systematics often not aware of recent developments in machine learning and computational statistics.

Therefore, the objectives of this workshop are two-fold:

• To bring together experts from the Machine Learning and Computational Statistics community with experts in the field of cosmology to promote, discuss and explore the use of machine learning techniques in data analysis problems in cosmology and to advance the state of the art.
• By presenting current approaches, their possible limitations, and open data analysis problems in cosmology to the NIPS community, this workshop aims to encourage scientific exchange and to foster collaborations among the workshop participants.

The workshop is held as a one-day workshop organised jointly by experts in the field of empirical inference and cosmology. The target group of participants are researchers working in the field of cosmological data analysis as well as researchers from the whole NIPS community sharing the interest in real-world applications in a fascinating, fast-progressing field of fundamental research. Due to the mixed participation of computer scientists and cosmologists the invited speakers will be asked to give talks with tutorial character and make the covered material accessible for both computer scientists and cosmologists.

Model order selection, which is a trade-off between model resolution and its statistical reliability, is one of the fundamental questions in machine learning. It was studied in detail in the context of supervised learning with i.i.d. samples, but received relatively little attention beyond this domain. The goal of our workshop is to raise attention to the question of model order selection in other domains, share ideas and approaches between the domains, and identify perspective directions for future research. Our interest covers ways of defining model complexity in different domains, examples of practical problems, where intelligent model order selection yields advantage over simplistic approaches, and new theoretical tools for analysis of model order selection. The domains of interest span over all problems that cannot be directly mapped to supervised learning with i.i.d. samples, including, but not limited to, reinforcement learning, active learning, learning with delayed, partial, or indirect feedback, and learning with submodular functions.

An example of first steps in defining complexity of models in reinforcement learning, applying trade-off between model complexity and empirical performance, and analyzing it can be found in [1-4]. An intriguing research direction coming out of these works is simultaneous analysis of exploration-exploitation and model order selection trade-offs. Such an analysis enables to design and analyze models that adapt their complexity as they continue to explore and observe new data. Potential practical applications of such models include contextual bandits (for example, in personalization of recommendations on the web [5]) and Markov decision processes.

References:
[1] N. Tishby, D. Polani. "Information Theory of Decisions and Actions", Perception-Reason-Action Cycle: Models, Algorithms and Systems, 2010.
[2] J. Asmuth, L. Li, M. L. Littman, A. Nouri, D. Wingate, "A Bayesian Sampling Approach to Exploration in Reinforcement Learning", UAI, 2009.
[3] N. Srinivas, A. Krause, S. M. Kakade, M. Seeger, "Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design", ICML, 2010.
[4] Y. Seldin, P. Auer, F. Laviolette, J. Shawe-Taylor, R. Ortner, "PAC-Bayesian Analysis of Contextual Bandits", NIPS, 2011.
[5] A. Beygelzimer, J. Langford, L. Li, L. Reyzin, R. Schapire, "Contextual Bandit Algorithms with Supervised Learning Guarantees", AISTATS, 2011.

Background

Technological advances to profile medical patients have led to a change of paradigm in medical prognoses. Medical diagnostics carried out by medical experts is increasingly complemented by large-scale data collection and quantitative genome-scale molecular measurements. Data that are already available as of today or are to enter medical practice in the near future include personal medical records, genotype information, diagnostic tests, proteomics and other emerging ‘omics’ data types.

This rich source of information forms the basis of future medicine and personalized medicine in particular. Predictive methods for personalized medicine allow to integrate these data specific for each patient (genetics, exams, demographics, imaging, lab, genomic etc.), both for improved prognosis and to design an individual-specific optimal therapy.

However, the statistical and computational approaches behind these analyses are faced with a number of major challenges. For example, it is necessary to identify and correcting for structured influences within the data; dealing with missing data and the statistical challenges that come along with carrying out millions of statistical tests. Also, to render these methods useful in practice computational efficiency and scalability to large-scale datasets are an integral requirement. Finally, any computational approach needs to be tightly integrated with medical practice to be actually used and the experiences gained need to be fed back into future development and improvements. To both address these technical difficulties ahead and to allow for an efficient integration and application in a medical context, it is necessary to bring the communities of statistical method developers, medics and biological investigators together.

Purpose of Workshop

The purpose of this 2nd cross-discipline workshop is to bring together machine learning, statistical genetics and healthcare researchers interested in problems and applications of predictive models in the field of personalized medicine. The goal of the workshop will be to bridge the gap between the theory of predictive models and statistical genetics with respect to medical applications and the pressing needs of the healthcare community. The workshop will promote an exchange of ideas, helping to identify important and challenging applications as well as the discovery of possible synergies. Ideally, we hope that such discussion will lead to interdisciplinary collaborations with resulting collaborative grant submissions. The emphasis will be on the statistical and engineering aspects of predictive models and how it relates to practical medical and biological problems.

Although related in a broad sense, the workshop does not directly overlap with the fields of Bioinformatics and Biostatistics. While predictive modeling for healthcare has been explored by biostatisticians for several decades, the focus of this workshop is on substantially different needs and problems that are better addressed by modern machine learning technologies. For example, how should we organize clinical trials to validate the clinical utility of predictive models for personalized therapy selection? How can we integrate and combine heterogenious data while accounting for confounding influences? How can we ensure computational efficiency that render these methods useful in practice? The focus of this workshop will be methods to address these and related questions.

The focus is not on questions of basic science; rather, we will focus on predictive models that combine available patient data while resolving the technical and statistical challenges through modern machine learning. The workshop program will combine presentations by invited speakers from both machine learning, statistical genetics and personalized medicine fields and by authors of extended abstracts submitted to the workshop. In addition, we will reserve sufficient room for discussion both in the forms of an open panel as well as in the context of poster presentations.