Data science & artificial intelligence
For digitalized industry & services
Since its inception in 1985, AISTATS has been an interdisciplinary gathering of researchers at the intersection of artificial intelligence, machine learning, statistics and related areas.
The academic team of the Data Science and Artificial Intelligence for Digitalized Industry and Services chair presented four papers at the 2021 edition, which was held online last April:
Nonlinear Functional Output Regression: A Dictionary Approach
Dimitri Bouche, Marianne Clausel, François Roueff, Florence d’Alché-Buc
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:235-243
When OT meets MoM: Robust estimation of Wasserstein Distance
Guillaume Staerman, Pierre Laforgue, Pavlo Mozharovskyi, Florence d’Alché-Buc
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:136-144
Nearest Neighbour Based Estimates of Gradients: Sharp Nonasymptotic Bounds and Applications
Guillaume Ausset, Stephan Clémençon, François Portier
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:532-540
Learning Fair Scoring Functions: Bipartite Ranking under ROC-based Fairness Constraints
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:784-792
More info on the AISTATS conference
The DSAIDIS chair will welcome several interns in 2021. Some of them will have the opportunity to continue with a PhD Thesis.
Here are some Master 2 programs that have interest in these internships. Please, contact us if you run a Master program with interest on our topics.
This edition of the international conference NeurIPS will be entirely online. The DSAIDIS Chair members will present five papers this year.
Heavy-tailed Representations, Text Polarity Classification & Data Augmentation
Hamid JALALZAI (Télécom Paris) · Pierre Colombo (Télécom Paris) · Chloé Clavel (Télécom Paris) · Eric Gaussier (Université Joseph Fourier, Grenoble) · Giovanna Varni (Télécom Paris) · Emmanuel Vignon (IBM) · Anne Sabourin (Télécom Paris)
Hausdorff Dimension, Heavy Tails, and Generalization in Neural Networks
Umut Simsekli (Télécom Paris/ University of Oxford) · Ozan Sener (Intel Labs) · George Deligiannidis (Oxford) · Murat Erdogdu (University of Toronto)
Quantitative Propagation of Chaos for SGD in Wide Neural Networks
Valentin De Bortoli (ENS Paris-Saclay) · Alain Durmus (ENS Paris Saclay) · Xavier Fontaine (ENS Paris-Saclay) · Umut Simsekli (Télécom Paris/ University of Oxford)
Explicit Regularisation in Gaussian Noise Injections
Alexander Camuto (University of Oxford & The Alan Turing Institute) · Matthew Willetts (University of Oxford) · Umut Simsekli (Télécom Paris/ University of Oxford) · Stephen J Roberts (University of Oxford) · Chris C Holmes (University of Oxford)
Statistical and Topological Properties of Sliced Probability Divergences
Kimia Nadjahi (Télécom Paris) · Alain Durmus (ENS Paris Saclay) · Lénaïc Chizat (CNRS) · Soheil Kolouri (HRL Laboratories LLC) · Shahin Shahrampour (Texas A&M University) · Umut Simsekli (Télécom Paris/ University of Oxford)
Umut Şimşekli (Télécom Paris), Levent Sagun (EPFL) and Mert Gürbüzbalaban (Rutgers University)’s paper entitled: “A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks” received the “best paper honorable mention” award at ICML 2019.
The gradient noise (GN) in the stochastic gradient descent (SGD) algorithm is often con-sidered to be Gaussian in the large data regime by assuming that theclassicalcentral limittheorem (CLT) kicks in. This assumption is often made for mathematical convenience, sinceit enables SGD to be analyzed as a stochastic differential equation (SDE) driven by a Brow-nian motion. We argue that the Gaussianity assumption might fail to hold in deep learningsettings and hence render the Brownian motion-based analyses inappropriate. Inspired bynon-Gaussian natural phenomena, we consider the GN in a more general context and invokethegeneralizedCLT (GCLT), which suggests that the GN converges to aheavy-tailedα-stablerandom variable. Accordingly, we propose to analyze SGD as an SDE driven by a Lévy motion.Such SDEs can incur ‘jumps’, which force the SDEtransitionfrom narrow minima to widerminima, as proven by existing metastability theory. To validate theα-stable assumption, weconduct extensive experiments on common deep learning architectures and show that in allsettings, the GN is highly non-Gaussian and admits heavy-tails. We further investigate thetail behavior in varying network architectures and sizes, loss functions, and datasets. Ourresults open up a different perspective and shed more light on the belief that SGD preferswide minima.
Call for candidature: postdoctoral positions in Machine Learning/Artificial Intelligence at Télécom Paris, Institut Polytechnique de Paris, France.
We are currently opening two postdoctoral positions in Machine Learning to join the chair “Data Science and Artificial Intelligence for Digitalized Industry and Services” at Télécom Paris (in the newly created Institut Polytechnique de Paris, France) to work on one of the following topics:
Télécom Paris is one of the Top 4 Engineering Schools in France, member of Institut Mines-Télécom and founding member of the Institut Polytechnique de Paris. Created in 1878, it has evoled to become the leading engineering school in digital technology in France. Its excellent teaching and innovative pedagogy places Télécom Paris at the heart of a unique ecosystem of innovation built upon the interactions between its initial and continuous education, its research center and its two business incubators.
Its research activity is assessed exceptional in quality with publications in the best journals and conferences by the HCERES (Evaluation Agency for Research and Higher Education) and has received the Carnot TSN label (Telecom and Digital Society).
Its research area covers the entire field of digital technology and develops expertise in six strategic axis on a European scale, among which the Data Science and Artificial Intelligence axis. This major theme gathers over 40 researchers and 80 PhD students. In 2017, there were 24 PhD thesis defense on these topics and its academic team published over 220 scientific papers.
The “Data Science and Artificial Intelligence for Digitalized Industry and Services” chair, initiated in the Signal, Statistics and Machine Learning group, was officially launched in April 2019, with a team of 21 permanent researchers and the sponsoring of five major companies: Airbus Defence & Space, Engie, Idemia, Safran and Valeo. This chair is holded by Pr. Florence d’Alché-Buc.
The postdocs are expected to have a strong background in both theoretical and computational aspects of Machine Learning. They will lead researches on the cited topics with the team and participate to the supervision of PhD thesis. They will also actively contribute to the scientific animation of the Chair and will collaborate with the industrial partners on applications.
The successful candidate will hold a PhD in machine learning, statistics/biostatistics, computer science, have a strong publishing experience in peer-reviewed journals and conferences in Machine Learning and an excellent background in applied mathematics/statistics. Strong interest for computational aspects of Machine learning is expected as very good skills in programming (Python).
Positions are available from July 2019 or later, and for a duration of two years, with possibility of extension for a third year.
To apply, please send via email, with the sentence “Postdoc application” as a subject, a single file containing a statement of research interests, a CV, a copy of relevant certificates, (p)reprints of two publications, and a list of two references to Pr. Florence d’Alché-Buc (florence.dalche(at)enst.fr) before July 10, 2019.
AISTATS is the International Conference on Artificial Intelligence and Statistics. Its 22nd edition took place in Naha, Okinawa, Japan from 16 April to 18 April 2019. Since its inception in 1985, AISTATS has been an interdisciplinary gathering of researchers at the intersection of artificial intelligence, machine learning, statistics and related areas.
The 2019 AISTATS edition featured 360 different papers. It is thus smaller than NeurIPS and ICML, making it easier to interact with authors, while preserving a tough selection on the submissions. Moreover, focus is mainly put on the statistical analysis of the proposed methods, which is at the core of the Image-Data-Signal department activity.
Pierre Laforgue and Alex Lambert, PhD students at Télécom Paris, have presented two papers:
Infinite Task Learning in RKHSs
Romain R Brault (Télécom Paris), Alex Lambert (Télécom Paris), Zoltan Szabo (Ecole Polytechnique), Florence d’Alché-Buc (Télécom Paris), Maxime Sangnier (Sorbonne University)
Machine learning has witnessed tremendous success in solving tasks depending on a single hyperparameter. When considering simultaneously a finite number of tasks, multi-task learning enables one to account for the similarities of the tasks via appropriate regularizers. A step further consists of learning a continuum of tasks for various loss functions. A promising approach, called Parametric Task Learning, has paved the way in the continuum setting for affine models and piecewise-linear loss functions. In this work, we introduce a novel approach called Infinite Task Learning whose goal is to learn a function whose output is a function over the hyperparameter space. We leverage tools from operator-valued kernels and the associated vector-valued RKHSs that provide an explicit control over the role of the hyperparameters, and also allows us to consider new type of constraints. We provide generalization guarantees to the suggested scheme and illustrate its efficiency in cost-sensitive classification, quantile regression and density level set estimation.
Autoencoding any Data through Kernel Autoencoders
Pierre Laforgue (Télécom Paris), Stephan Clémençon (Télécom Paris), Florence d’Alche-Buc (Télécom Paris)
This paper investigates a novel algorithmic approach to data representation based on kernel methods. Assuming that the observations lie in a Hilbert space X , the introduced Kernel Autoencoder (KAE) is the composition of mappings from vector-valued Reproducing Kernel Hilbert Spaces (vv-RKHSs) that minimizes the expected reconstruction error. Beyond a first extension of the autoencoding scheme to possibly infinite dimensional Hilbert spaces, KAE further allows to autoencode any kind of data by choosing X to be itself a RKHS. A theoretical analysis of the model is carried out, providing a generalization bound, and shedding light on its connection with Kernel Principal Component Analysis. The proposed algorithms are then detailed at length: they crucially rely on the form taken by the minimizers, revealed by a dedicated Representer Theorem. Finally, numerical experiments on both simulated data and real labeled graphs (molecules) provide empirical evidence of the KAE performances.
On 3 April 2019, Télécom Paris launched its new teaching and research chair, “Data Science and Artificial Intelligence for Digitalized Industry and Services”, supported by Fondation Mines-Télécom. Around 100 participants attended this particularly productive event, which enabled the chair’s academic team, consisting of 21 Télécom Paris professors, PhD students and Master’s interns to meet five of our corporate partners’ operations teams: Airbus, ENGIE, IDEMIA, Safran and Valeo. Students of data science, big data and artificial intelligence joined the public to find out about the key industrial AI issues.
The researchers chose to first review the work undertaken as part of the “Machine Learning for Big Data” Chair, which was active from 2013 to 2018. Stephan Clémençon, a full professor at Télécom Paris and former holder of this chair, gave a full report of the activities undertaken, which included close to 400 publications, ten doctorates and around 40 internships. He then spoke of the scientific challenges that big data poses. Claire Vernade, one of the first PhD students to have their thesis funded by the chair and who now works for Deepmind in London, was next. She made a presentation on the applications of reinforcement learning, which represents one element of her research. Olivier Fercoq, a lecturer at Télécom Paris, ended the first part of the proceedings with a presentation of the research on optimization algorithms carried out by Eugène N’Diaye, another PhD student associated with the chair.
The five partners of the new chair then took turns to talk about the AI and data science issues they were having to address. With machine learning-based solutions becoming more complex, being used in critical decision-making situations and automation becoming the norm, it becomes critical to introduce guarantees and mechanisms to ensure these solutions are trusted. The chair truly comes into its own as researchers and companies work collaboratively, and also as it reveals those themes which are common to the different partners.
Florence d’Alché-Buc, chairholder and full professor at Télécom Paris, then presented the chair’s scientific program, as well as the projects she will be leading for the coming five years, covering four research directions. Some of the many different activities that partners will be engaging in include a range of workshops, as well as thesis supervision. Florence d’Alché-Buc then concluded the event by providing details of the new artificial intelligence challenges that researchers will need to take up: robustness, reliability and interaction.
Official signing ceremony, with the Chair’s partners (from left to right): Yves Poilane, Télécom ParisTech Director; Véronique Deborde, Fondation Mines-Télécom Director; Marc Bertin, IDEMIA Deputy CTO; Florence Dufrasnes, Airbus Head of Defence & Space Technical Office; Gérard Guinamand, ENGIE Group Chief Data Officer; Jean-François Tarabbia, Valeo Group R&D – Product & Marketing Senior VP; Florence d’Alché-Buc, Télécom ParisTech Professor and Chairholder; Benoît Guyon, Safran R&T Partnership Director.