————————————————————————————————–
NUMML 2010
Numerical Mathematical Challenges in Machine Learning
NIPS*2010 Workshop
December 11th, 2010, Whistler, Canada
URL: http://numml.kyb.tuebingen.mpg.de/
————————————————————————————————–
Call for Contributions
——————————
We invite high-quality submissions for presentation as posters at the
workshop. The poster session will be designed along the lines of the poster
session for the main NIPS conference. There will probably be a spotlight
session (2 min./poster), although this depends on scheduling details not
finalized yet. In any case, authors are encouraged (and should be motivated)
to use the poster session as a means to obtain valuable feedback from experts
present at the workshop (see “Invited Speakers” below).
Submissions should be in the form of an extended abstract, paper (limited to 8
pages), or poster. Work must be original, not published or in submission
elsewhere (a possible exception are publications at venues unknown to machine
learning researchers, please state such details with your submission).
Authors should make an effort to motivate why the work fits the goals of the
workshop (see below) and should be of interest to the audience. Merely
resubmitting a submission rejected at the main conference, without adding such
motivation, is strongly discouraged.
Important Dates
————————
* Deadline for submission: 21st October 2010
* Notification of acceptance: 27th October 2010
* Workshop date: 11th December 2010
Submission:
—————–
Please email your submissions to: suvadmin(at)googlemail.com
NOTE:
———
At least one author of each accepted submission must attend to present the
poster/potential spotlight at the workshop. Further details regarding the
submission process are available from the workshop homepage.
What follows is a synopsis about workshop goals, invited speakers, expected
audience. This information can also be obtained from the workshop homepage.
—————————————————————————————————————–
Abstract
————
Most machine learning (ML) methods are based on numerical mathematics (NM)
concepts, from differential equation solvers over dense matrix factorizations
to iterative linear system and eigen-solvers. As long as problems are of
moderate size, NM routines can be invoked in a black-box fashion. However, for
a growing number of real-world ML applications, this separation is insufficient
and turns out to be a severe limit on further progress.
The increasing complexity of real-world ML problems must be met with layered
approaches, where algorithms are long-running and reliable components rather
than stand-alone tools tuned individually to each task at hand. Constructing
and justifying dependable reductions requires at least some awareness about NM
issues. With more and more basic learning problems being solved sufficiently
well on the level of prototypes, to advance towards real-world practice the
following key properties must be ensured: scalability, reliability, and
numerical robustness. Unfortunately, these points are widely ignored by many
ML researchers, preventing applicability of ML algorithms and code to complex
problems and limiting the practical scope of ML as a whole.
Goals, Potential Impact
———————————-
Our workshop addresses the abovementioned concerns and limitations. By
inviting numerical mathematics researchers with interest in *both* numerical
methodology *and* real problems in applications close to machine learning, we
will probe realistic routes out of the prototyping sandbox. Our aim is to
strengthen dialog between NM and ML. While speakers will be encouraged to
provide specific high-level examples of interest to ML and to point out
accessible software, we will also initiate discussions about how to best
bridge gaps between ML requirements and NM interfaces and terminology; the
ultimate goal would be to figure out how at least some of NM’s high standards
of reliability might be transferred to ML problems.
The workshop will reinforce the community’s awakening attention towards
critical issues of numerical scalability and robustness in algorithm design
and implementation. Further progress on most real-world ML problems is
conditional on good numerical practices, understanding basic robustness and
reliability issues, and a wider, more informed integration of good numerical
software. As most real-world applications come with reliability and scalability
requirements that are by and large ignored by most current ML methodology, the
impact of pointing out tractable ways for improvement is substantial.
General Topics of Interest
————————————-
A basic example for the NM-ML interface is the linear model (or
Gaussian Markov random field), a major building block behind sparse estimation,
Kalman smoothing, Gaussian process methods, variational approximate inference,
classification, ranking, and point process estimation. Linear model computations
reduce to solving large linear systems, eigenvector approximations, and matrix
factorizations with low-rank updates. For very large problems, randomized or
online algorithms become attractive, as do multi-level strategies. Additional
examples include analyzing global properties of very large graphs arising in
social, biological, or information transmissing networks, or robust filtering
as a backbone for adaptive exploration and control.
We welcome and seek contributions on the following subtopics (although we do
not limit ourselves to these):
A) Large to huge-scale numerical algorithms for ML applications
* Eigenvector approximations: Specialized variants of the Lanczos algorithm,
randomized algorithms. Application examples are:
– The linear model (covariance estimation);
– Spectral clustering, graph Laplacian methods,
– PCA, scalable graph analysis (social networks),
– Matrix completion (consumer-preference prediction)
* Randomized algorithms for low-rank matrix approximations
* Parallel and distributed algorithms
* Online and streaming numerical algorithms
B) Solving large linear systems:
* Iterative solvers
* Preconditioners, especially those based on model/problems structure which
arise in ML applications
* Multi-grid / multi-level methods
* Exact solvers for very sparse matrices
Application examples are:
– Linear models / Gaussian MRF (mean computations),
– Nonlinear optimization methods (trust-region, Newton steps, IRLS)
C) Numerical linear algebra packages relevant to ML
* LAPACK, BLAS, GotoBLAS, MKL, UMFPACK, PETSc, MPI
D) Exploiting matrix/model structure, fast matrix-vector multiplication
* Matrix decompositions/approximations
* Multi-pole methods
* Nonuniform FFT, local convolutions
E) How can numerical methods be improved using ML technology?
* Reordering strategies for sparse decompositions
* Preconditioning based on model structure
* Distributed parallel computing
Target audience:
Our workshop is targeted towards practitioners from NIPS, but is of interest
to numerical linear algebra researchers as well.
Workshop
————–
The workshop will feature talks (tutorial style, as well as technical) on
topics relevant to the workshop. Because the explicit purpose of our workshop
is to foster cross-fertilization between the NM and ML communities, we also
plan to hold a discussion session, which we will help to structure by raising
concrete questions based on the topics and concerns outlined above.
To further bolster active participation, we will set aside time for poster and
spotlight presentations, which will offer participants a chance to get
feedback about their work.
Invited Speakers
————————
Inderjit Dhillon University of Texas, Austin
Dan Kushnir Yale University
Michael Mahoney Stanford University
Richard Szeliski Microsoft Research
Alan Willsky Massachusetts Institute of Technology
Workshop URL
———————
http://numml.kyb.tuebingen.mpg.de
Workshop Organizers
——————————
Suvrit Sra
Max Planck Institute for Biological Cybernetics, Tuebingen
Matthias W. Seeger
Max Planck Institute for Informatics and Saarland University, Saarbruecken
Inderjit Dhillon
University of Texas at Austin, Austin, TX
——————————————————————————
