Discrete optimization problems and combinatorial structures are becoming increasingly important in machine learning. In the endeavour to process more and more complex data, one may quickly find oneself working with graphs, relations, partitions, or sparsity structure. In addition, we may want to predict structured, sparse estimators, or combinatorial objects such as permutations, trees or other graphs, group structure and so on.

While complex structures enable much richer applications, they often come with the caveat that the related learning and inference problems become computationally very hard. When scaling to large data, combinatorial problems also add some further challenges to compact representations, streaming and distributed computation.

Despite discouraging theoretical worst-case results, many practically interesting problems can be much more well behaved (when modeled appropriately), or are well modeled by assuming properties that make them so. Such properties are most often structural and include symmetry, exchangeability, sparsity, or submodularity. Still, not all of these structures and their application ranges are well understood. The DISCML workshop revolves around such structures in machine learning, their applications and theory.

Discrete Optimization in Machine Learning (DISCML) 2012 — Structure and Scalability Workshop

Discrete Optimization in Machine Learning (DISCML) 2012 — Structure and Scalability Workshop

Discrete Optimization in Machine Learning (DISCML) 2012 — Structure and Scalability Workshop

One of the main practical goals of machine learning is to identify relevant trade-offs in different problems, formalize, and solve them. We have already achieved fairly good progress in addressing individual trade-offs, such as model order selection or exploration-exploitation. In this workshop we would like to focus on problems that involve more than one trade-off simultaneously. We are interested both in practical problems where "multi-trade-offs" arise and in theoretical approaches to their solution.

Obviously, many problems in life cannot be reduced to a single trade-off and it is highly important to improve our ability to address multiple trade-offs simultaneously. Below we provide several examples of situations, where multiple trade-offs arise simultaneously. The goal of the examples is to provide a starting point for a discussion, but they are not limiting the scope and any other multi-trade-off problem is welcome to be discussed at the workshop.

Multi-trade-offs arise naturally in interaction between multiple learning systems or when a learning system faces multiple tasks simultaneously; especially when the systems or tasks share common resources, such as CPU time, memory, sensors, robot body, and so on. For a concrete example, imagine a robot riding a bicycle and balancing a pole. Each task individually (cycling and pole balancing) can be modeled as a separate optimization problem, but their solutions have to be coordinated, since they share robot resources and robot body. More generally, each learning system or system component has its own internal trade-offs, which have to be balanced against trade-offs of other systems, whereas shared resources introduce external trade-offs that enforce cooperation. The complexity of interaction can vary from independent systems sharing common resources to systems with various degrees of relation between their inputs and tasks. In multi-agent systems communication between the agents introduces an additional trade-off.

We are also interested in multi-trade-offs that arise within individual systems. For example, model order selection and computational complexity [1], or model order selection and exploration-exploitation [2]. For a specific example of this type of problems, imagine a system for real-time prediction of the location of a ball in table tennis. This system has to balance between at least three objectives that interact in a non-trivial manner: (1) complexity of the model of flight trajectory, (2) statistical reliability of the model, (3) computational requirements. Complex models can potentially provide better predictions, but can also lead to overfitting (trade-off between (1) and (2)) and are computationally more demanding. At the same time, there is also a trade-off between having fast crude predictions or slower, but more precise estimations (trade-off between (3) and (1)+(2)). Despite the complex nature of multi-trade-offs, there is still hope that they can be formulated as convex problems, at least in some situations [3].

References:
[1] Shai Shalev-Shwartz and Nathan Srebro. "SVM Optimization: Inverse Dependence on Training Set Size", ICML, 2008.
[2] Yevgeny Seldin, Peter Auer, François Laviolette, John Shawe-Taylor, and Ronald Ortner. "PAC-Bayesian Analysis of Contextual Bandits", NIPS, 2011.
[3] Andreas Argyriou, Theodoros Evgeniou and Massimiliano Pontil. Convex multi-task feature learning. Machine Learning, 2008, Volume 73, Number 3.

Multi-Trade-offs in Machine Learning

Multi-Trade-offs in Machine Learning

Multi-Trade-offs in Machine Learning

Since its inception for describing the laws of communication in the 1940's, information theory has been considered in fields beyond its original application area and, in particular, it was long attempted to utilize it for the description of intelligent agents. Already Attneave (1954) and Barlow (1961) suspected that neural information processing might follow principles of information theory and Laughlin (1998) demonstrated that information processing comes at a high metabolic cost; this implies that there would be evolutionary pressure pushing organismic information processing towards the optimal levels of data throughput predicted by information theory. This becomes particularly interesting when one considers the whole perception-action cycle, including feedback. In the last decade, significant progress has been made in this direction, linking information theory and control. The ensuing insights allow to address a large range of fundamental questions pertaining not only to the perception-action cycle, but to general issues of intelligence, and allow to solve classical problems of AI and machine learning in a novel way.

The workshop will present recent work on progress in AI, machine learning, control, as well as biologically plausible cognitive modeling, that is based on information theory.

Information in Perception and Action Workshop

Information in Perception and Action Workshop

Information in Perception and Action Workshop

The traditional remit of machine learning is problems of inference on complex data. At the computational bottlenecks of our algorithms, we typically find a numerical problem: optimization, integration, sampling. These inner routines are often treated as a black box, but many of these tasks in numerics can be viewed as learning problems:

• How can optimizers learn about the objective function, and how should they update their search direction? [1]
• How should a quadrature method estimate an integral given observations of the integrand, and where should these methods put their evaluation nodes? [2,3,4]
• How should MCMC samplers adapt their proposal distributions given past evaluations of the unnormalised density? [5]
• Can approximate inference techniques be applied to numerical problems? [6]?

Many of these problems can be seen as special cases of decision theory, active learning, or reinforcement learning. Work along these lines was pioneered twenty years ago by Diaconis [7] and O'Hagan [2]. But modern desiderata for a numerical algorithm differ markedly from those common elsewhere in machine learning: Numerical methods are "inner-loop" algorithms, used as black boxes by large groups of users on wildly different problems. As such, robustness, computation and memory costs are more important here than raw prediction power or convergence speed. Availability of good implementations also matters. These kind of challenges can be well addressed by machine learning researchers, once the relevant community is brought together to discuss these topics.

Some of the algorithms we use for numerical problems were developed generations ago. They have aged well, showing impressively good performance over a broad spectrum of problems. Of course, they also have a variety of shortcomings, which can be addressed by modern probabilistic models (see some of the work cited above). In the other direction, the numerical mathematics community, a much wider field than machine learning, is bringing experience, theoretical rigour and a focus on computational performance to the table. So there is great potential for cross-fertilization to both the machine learning and numerical mathematics community. The main goals of this workshop are

• to bring numerical mathematicians and machine learning researchers into contact to discuss possible contributions of machine learning to numerical methods.
• to present recent developments in probabilistic numerical methods.
• to discuss future directions, performance measures, test problems, and code publishing standards for this young community.

Probabilistic Numerics Workshop

Automatic text understanding has been an unsolved research problem for many years. This partially results from the dynamic and diverging nature of human languages, which ultimately results in many different varieties of natural language. This variations range from the individual level, to regional and social dialects, and up to seemingly separate languages and language families.

However, in recent years there have been considerable achievements in data driven approaches to computational linguistics exploiting the redundancy in the encoded information and the structures used. Those approaches are mostly not language specific or can even exploit redundancies across languages.

This progress in cross-lingual technologies is largely due to the increased availability of multilingual data in the form of static repositories or streams of documents. In addition parallel and comparable corpora like Wikipedia are easily available and constantly updated. Finally, cross-lingual knowledge bases like DBpedia can be used as an Interlingua to connect structured information across languages. This helps at scaling the traditionally monolingual tasks, such as information retrieval and intelligent information access, to multilingual and cross-lingual applications.

From the application side, there is a clear need for such cross-lingual technology and services. Available systems on the market are typically focused on multilingual tasks, such as machine translation, and don't deal with cross-linguality. A good example is one of the most popular news aggregators, namely Google News that collects news isolated per individual language. The ability to cross the border of a particular language would help many users to consume the breadth of news reporting by joining information in their mother tongue with information from the rest of the world.

In the anti-corruption strand, a workshop on the advance use of modern information technologies to fight corruption was held by Mitja Jermol, Head of the Centre for Knowledge Transfer at the Jožef Stefan Institute in Ljubljana, Gaber Cerle, Director of OurSpace, Slovenia and three other expertst. The topic is part of a large-scale project which will expand to the EU level. It was first presented in the framework of the Romanian Anti-Corruption Directorate’s (DNA) seminar in Bucharest this June, and another working group is expected in 2013, coordinated by the DNA and the Slovenian Commission for the Prevention of Corruption.

Advance use of modern information technologies to fight corruption Workshop

Advance use of modern information technologies to fight corruption Workshop

The goal of this ICDM 2012 workshop is to help closing the gap between data mining practice and theory. To this end, we intend to explore what is the essence of exploratory data mining and how to formalize it in a useful but theoretically well-founded way.

The workshop is motivated by a widely perceived discrepancy between theoretical data mining prototypes and practitioners’ requirements. A notable example is frequent pattern mining. Despite its attractive theoretical foundations, the practical use of frequent pattern mining methods has been limited. This is due to a difficulty to overcome issues, such as the pattern explosion problem and a discrepancy between usefulness and frequency. These issues have been addressed to some extent in the past 15 years, through heuristic post-processing steps and through rigorously motivated adaptations. The multitude of possible solution strategies has unfortunately to a large extent undermined the original elegance, and made it hard for practitioners to understand how to use these techniques.

The problem is however not restricted to frequent pattern mining alone. The multitude of available methods for typical exploratory data mining problems such as (subspace) clustering and dimensionality reduction is such that practitioners face a daunting task in selecting a suitable method. Additionally to the usability issues, less attention has been given on pattern mining methods for relational databases. Although most real world databases are relational, most pattern mining research has focused on one-table data.

We believe the core reasons for these difficulties are:

• Different users inevitably have different prior beliefs and goals, whereas most exploratory data mining algorithms have a rigid objective function and do not consider this.
• Formally comparing the quality of different data mining patterns is hard due to their widely varying nature (e.g. comparing a dimensionality reduction with a frequent itemset), unless their 'interestingness' can be quantified in a comparable manner.
• The iterative process of data mining is often not considered.
• Data mining in complex relational data is hard to fit into standard data mining prototypes.
• More generally, data mining methods tend to be rigid, defined for highly specific tasks, for highly specific and idealized data, and for very specific types of patterns.

The purpose of this workshop will be to serve as a forum of exchanging ideas on how to formalize exploratory data mining in order to make it useful in practice. This workshop will survey (through invited as well as contributed talks and posters) some existing attempts at addressing the problems mentioned above. We particularly encourage papers that present principled theoretical contributions motivated by real world requirements.

Practical Theories for Exploratory Data Mining Workshop

Practical Theories for Exploratory Data Mining Workshop

Practical Theories for Exploratory Data Mining Workshop

Practical Theories for Exploratory Data Mining Workshop

COMMPER 2012 is a workshop in conjunction with The European Conference on Machine Learning and Practice of Knowledge Discovery in Databases (ECML PKDD), which will take place in Bristol, UK from September 24th to 28th, 2012. The First COMMPER Workshop took place in conjuction with the IEEE ICDM Conference in vancouver, Canada.

Data mining and knowledge discovery in social networks has advanced significantly over the past several years, due to the availability of a large variety of online and online social network systems. The focus of COMMPER is on two main streams of social networks: community mining and people recommenders.

The first focus of this workshop is on mining communities in social networks and in particular in scientific collaboration networks. Consider, for example, a dataset of scientific publications along with information about each publication and the complete citation network. Many data-analysis questions arise: what are the underlying communities, who are the most influential authors, what are the set-skills of individual authors, what are the observed collaboration patterns, how does interest on popular topics propagates, who does the network evolve in terms of collaborations, topics, citations, and so on. In this workshop we indent to bring domain experts, such as bibliometricians, closer to researchers from the fields of data mining and social networks. The expected outcome is to strengthen the collaboration of these communities aiming at high impact-research contributions and discussions. We aspire that the workshop will lead to the development of new insights and data mining methodologies that could be employed for the analysis of communities, models of human collaboration, topic discovery, evolution of social networks, and more.

People recommenders, the second main topic of this workshop, deal with the problem of finding meaningful relationships among people or organizations. In online social networks, relationships can be friends on Facebook, professional contacts on LinkedIn, dates on an online dating site, jobs or workers on employment websites, or people to follow on Twitter. The nature of these domains makes people-to-people recommender systems to be significantly different from traditional item-to-people recommenders. One basic difference in the people recommender domain is the benefit or requirement of reciprocal relationships. Another difference between these domains is that people recommenders are likely to have rich user profiles available. The goal of this workshop is to build a community around people recommenders and instigate discussion about this emerging area of research for recommender systems. With this workshop, we want to reach out to research done in both academia and industry.

Topics

We encourage that papers submitted to COMMPER focus on, but are not limited to the following topics:

• analysis of scientific communities
• collaboration networks
• bibliometrics and data mining
• analysis of co-authorship networks
• analysis of citation networks
• communities in social networks
• dynamic networks
• formation of teams
• learning skills of individuals
• topic and community evolution and dynamics
• comparative studies of community networks
• people recommendation in social networks
• community recommendations in social networks
• mentor/mentee recommendations in tutoring systems
• expert search and expertise recommendation
• employee/employer recommendations
• online dating recommendations
• people search in the enterprise
• team recommendations
• reviewer assignment
• location-aware people recommendation

The Second International Workshop on Mining Communities and People Recommenders

The Second International Workshop on Mining Communities and People Recommenders

The Second International Workshop on Mining Communities and People Recommenders

The 22nd MLSP workshop in the series organized by the Signal Processing Society MLSP Technical Committee will present the most recent and exciting advances in data analysis for signal processing problems through tutorials, keynote talks, as well as special and regular singletrack sessions. The workshop will take place at The Royal Palace of the Magdalena, Santander, Spain.

22nd MLSP workshop

22nd MLSP workshop

22nd MLSP workshop