Applications are invited for a Ph.D. studentship on the general area of “Sequential Decision-making under Uncertainty” at INRIA Lille – Team SequeL. Below is the detail of this call.
—
Title: Sequential Decision-Making with Big Data
Keywords: sequential decision-making, reinforcement learning, learning and planning in MDPs and POMDPs, exploration/exploitation dilemma, bandit algorithms, adaptive resource allocation, regret minimization, optimization
Research Program:
The candidate is expected to conduct research on both theoretical and applied aspects of the problem of “Sequential Decision-making with Big Data” (see the description below), collaborate with researchers and Ph.D. students at INRIA and outside, and publish the results of her/his research in conferences and journals. The candidate will work with Mohammad Ghavamzadeh (http://chercheurs.lille.inria.fr/~ghavamza) and other researchers at Team SequeL (https://sequel.lille.inria.fr).
This Ph.D. program is focused on the problem of dealing with big data and limited resources in sequential decision-making under uncertainty.
– Big Data: Sequential decision-making applications that need to handle Big Data can be classified into three categories, which define related research problems.
1) Very large number of data points: This is a typical case in time series data that are fairly simple, but sampled at high frequency, such as user clicks on the web and financial data. In this scenario, the most important issue is the computational cost.
2) Very high-dimensional input space: This case arises when each data point consists of a lot of measurements, leading to a curse of dimensionality. Examples are customer information in online marketing problems and problems with complex sensors (such as Kinect cameras). The best way to solve this type of problem is to leverage intrinsic regularities (e.g., smoothness, sparsity, dependencies in features) to reduce the dimensionality.
3) Partially observable input space: Often, the observed input measurements do not have sufficient information for accurate decision-making, but one can leverage the history of the observations to improve the situation. This often requires projecting the problem into a high-dimensional representation.
– Limited Resources: In many real-world sequential decision-making applications we only have a limited budget of resources such as number of samples or access to a system’s simulator etc. When the available resources (sample or computation) are limited and/or access to more resources is costly, it would be absolutely necessary to allocate the available resources (or ask for more resources) efficiently in order to find good strategies. The problem of adaptive resource allocation has been studied in bandits, planning, and stochastic optimization, but there still exist many open problems and challenges in this area that require further investigation.
– Other Related Problems that arise in real-world applications of sequential decision-making: (i) how to evaluate a policy learned from a batch of historical data (generated with a different policy) with minimum interaction with the real-world environment, (ii) learning risk-sensitive and robust strategies, (iii) learning interpretable policies (i.e., policies that are understandable by experts of the problem at hand, who do not necessarily know much about machine learning, like medical doctors or financial managers) etc.
—
Requirements:
The applicant will have a Master’s (or equivalent) degree in Computer Science, Statistics, or related fields, with background in reinforcement learning, bandit algorithms, statistics, and optimization. Programming skills will be considered as a plus. The working language of the group is English, so the candidate is expected to have good communication skills in English.
—
About INRIA and Team SequeL:
SequeL (https://sequel.lille.inria.fr) is one of the most dynamic teams at INRIA (http://www.inria.fr), with over 25 researchers and Ph.D. students working on several aspects of machine learning from theory to application, including statistical learning, reinforcement learning, and sequential decision-making. The SequeL team is involved in national and European research projects and has collaboration with international research groups. This allows the Ph.D. candidate to collaborate with leading researchers in the field at top universities in Europe and North America such as University College of London (UCL), University of Alberta, and McGill University. Lille is the capital of the north of France, a metropolis with over one million inhabitants, and with excellent train connection to Brussels (30min), Paris (1h) and London (1h30).
—
Benefits:
– Duration: 36 months – starting date of the contract : October 2013, 15th
– Salary: 1957.54 Euros the first two years and 2058.84 Euros the third year
– Monthly salary after taxes: around 1597.11 Euros the first two years and 1679,76 Euros the 3rd year (benefits included)
– Possibility of French courses
– Help for housing
– Participation for transportation
– Scientific Resident card and help for husband/wife visa
—
Application Submission:
The application should include a brief description of the applicant’s research interests and past experience, plus a CV that contains her/his degrees, GPAs, relevant publications, name and contact information of up to three references, and other relevant documents. Please send your application to mohammad.ghavamzadeh@inria.fr. The deadline for the application is April 15 but the applicants are encouraged to submit their application as soon as possible.
—
This call has also been posted on
1) my webpage at
http://chercheurs.lille.inria.fr/~ghavamza/phd-ad-2013.html
2) the INRIA website at:
http://www.inria.fr/institut/recrutement-metiers/offres/theses/campagne-2013/%28view%29/details.html?id=PGTFK026203F3VBQB6G68LONZ&LOV5=4509&LG=FR&Resultsperpage=20&nPostingID=7222&nPostingTargetID=12647&option=52&sort=DESC&nDepartmentID=10
