The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing (NLP).

Computational linguistics is the scientific study of language from a computational perspective. Computational linguists are interested in providing computational models of various kinds of linguistic phenomena. These models may be “knowledge-based” (“hand-crafted”) or “data-driven” (“statistical” or “empirical”).

Activities of the ACL include the holding of an annual meeting each summer and the sponsoring of the journal Computational Linguistics, published by MIT Press; this conference and journal are the leading publications of the field.

This year was the 60th Annual Meeting, it took place May 22-27, 2022 as a hybrid event, in Dublin and online. The DSAIDIS researchers presented four papers:

  • Chadi Helwe, Chloé Clavel, Fabian Suchanek. LogiTorch: A PyTorch-based library for logical reasoning on natural language. The 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Dec 2022, Abu Dhabi, United Arab Emirates. ⟨hal-03870592⟩
  • Cyril Chhun, Pierre Colombo, Fabian M Suchanek, Chloé Clavel. Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generation. 29th International Conference on Computational Linguistics (COLING 2022), Oct 2022, Gyeongju, South Korea. ⟨hal-03801053⟩
  • Aina Garí Soler, Matthieu Labeau, Chloé Clavel. One Word, Two Sides: Traces of Stance in Contextualized Word Representations. 29th International Conference on Computational Linguistics (COLING 2022), Oct 2022, Gyeongju, South Korea. ⟨hal-03860830⟩
  • Yann Raphalen, Chloé Clavel, Justine Cassell. “You might think about slightly revising the title”: Identifying Hedges in Peer-tutoring Interactions. ACL 2022 – 60th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, May 2022, Dublin, Ireland. pp.2160-2174, ⟨10.18653/v1/2022.acl-long.153⟩. ⟨hal-03990509⟩

 

2022|The 60th Annual Meeting of the Association for Computational Linguistics

On November 22, the DSAIDIS Chair Meetup was held at Télécom Paris. The purpose of this meeting was to connect industrial partners and students of the DSAIDIS chair.

After a presentation sessions of the various industrial partners who support the chair in its activities, PhD and postdoc students had the opportunity to meet and discuss in small groups in dedicated rooms.

Program

13h45 Welcome
14H00 Presentation of industrial partners
14h15 Installation of the teams at the 5th floor – meeting partners / students
15h30 Coffee break
15h45 Meeting partners / students
17H00 End

On Tuesday, November 15, the “Optimization and neural networks” workshop of the DSAIDIS chair was held. Permanent members and PhD students presented their research work.

Olivier Fercoq

On the convergence of the ADAM algorithm

[EN] I will present the ADAM algorithm, which is a famous stochastic gradient method with adaptive learning rate. It is based on exponential moving averages of the stochastic gradients and their squares in order to estimate the first and second moments.
Then I will explain the main ideas of its convergence proof in the case of a convex objective function. The challenges are the following: 1) the estimation of the first moment is biased; 2) the learning rate is a random variable. They are solved by finding terms that telescope almost surely and by using the fact that learning rate is small when the gradient estimate is noisy.

See the slides

Maxime Lieber

Differentiable STFT with respect to the window length: optimizing STFT window length by gradient descent

[EN] In this talk, we revisit the tuning of the spectrogram window length, making the window length a continuous parameter optimizable by gradient descent instead of an empirically tuned integer-valued hyperparameter.

We first define two differentiable versions of the STFT w.r.t. the window length, in the case where local bins centers are fixed and independent of the window length parameter, and in the more difficult case where the window length affects the position and number of bins.
We then present the smooth optimization of the window length with any standard loss function. We show that this optimization can be of interest not only for any neural network-based inference system, but also for any STFT-based signal processing algorithm. We also show that the window length can not only be fixed and learned offline, but also be adaptive and optimized on the fly.
The contribution is mainly theoretical for the moment but the approach is very general and will have a large-scale application in several fields.

See the slides

Enzo Tartaglione

To the lottery ticket hypothesis and beyond: can we really make training efficient?

[EN] Recent advances in deep learning optimization showed that, with some a-posteriori information on fully-trained models, it is possible to match the same performance by simply training a subset of their parameters which, it is said, “had won at the lottery of initialization”.

Such a discovery has a potentially-high impact, from theory to applications in deep learning, especially from power consumption perspectives. However, all the “efficient” methods proposed do not match the state-of-the-art performance with high sparsity enforced, and rely on unstructured sparsely connected models, which notoriously introduce overheads when using the most common deep learning libraries.
In this talk, a background on the lottery ticket hypothesis will be provided with some of the approaches attempting to tackle the problem of efficiently identifying the parameters winning at the lottery of initialization, and two recent works will be presented.
The first, which has been presented at ICIP 2022 as oral, investigates the reasons for which an efficient algorithm in this context is hard to design, suggesting possible research directions towards efficient training. The second, which will be presented at NeurIPS 2022, implements an automatic method to assess, at training time, which sub-graph in the neural networks does not need further training (hence, no back-propagation and gradient computation is necessary for it, saving computation).

See the slides

Hicham Janati

Optimal alignments in machine learning: The case of spatio-temporal data

See the slides

The day ended with a discussion on big data and frugal AI.

Big data or fugal AI: where can optimization techniques help?

See the slides

The Neural Information Processing Systems Foundation is a non-profit corporation whose purpose is to foster the exchange of research on neural information processing systems in their biological, technological, mathematical, and theoretical aspects. Neural information processing is a field which benefits from a combined view of biological, physical, mathematical, and computational sciences.

The primary focus of the Foundation is the presentation of a continuing series of professional meetings known as the Neural Information Processing Systems Conference or NeurIPS, held over the years at various locations in the United States, Canada and Spain.

The 2022 edition will take place in New Orleans Morial Convention Center, USA.

This year, 7 papers from the DSAIDIS chair were accepted:

  • Pierre Colombo . Eduardo Dadalto . Guillaume Staerman . Nathan Noiry . Pablo Piantanida. Beyond Mahalanobis Distance for Textual OOD Detection. NeurIPS 2022.
  • Pierre Colombo, Nathan Noiry, Ekhine Irurozki, Stephan Clémençon (2022). What are the best Systems? New perspectives on NLP benchmarking. NeurIPS 2022. [Arxiv]
  • Andrea Bragagnolo, Enzo Tartaglione, Marco Grangetto (2022). To update or not to update? Neurons at equilibrium in deep models. NeurIPS 2022.
  •  Jayneel Parekh, Sanjeel Parekh, Pavlo Mozharovskyi, Florence d’Alché-Buc, Gaël Richard (2022). Listen to interpret: Post-hoc interpretability for audio networks with NMF. NeurIPS 2022. [Link]
  • Binh T. Nguyen, Bertrand Thirion, Sylvain Arlot. (2022). A conditional randomization test for sparse logistic regression in high-dimension. NeurIPS 2022. [Arxiv]
  • Thomas Morea, Mathurin Massias, Alexandre Gramfort, Pierre Ablin, Pierre-Antoine Bannier, Benjamin Charlier, Mathieu Dagréou, Tom Dupré la Tour, Ghislain Durif, Cassio F Dantas, Quentin Klopfenstein, Johan Larsson, En Lai, Tanguy Lefort, Benoit Malézieux, Badr Moufad, Binh T. Nguyen, Alain Rakotomamonjy, Zaccharie Ramzi, Joseph Salmon, Samuel Vaiter (2022). Benchopt: Reproducible, efficient and collaborative optimization benchmarks. NeurIPS 2022. [Arxiv]
  • Rémi Leluc, François Portier, Johan Segers, Aigerim Zhuman (2022). A quadrature rule combining control variates and adaptive importance sampling. NeurIPS 2022. [Arxiv]

ICML, the International Conference on Machine Learning, is the premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence known as machine learning.

ICML is globally renowned for presenting and publishing cutting-edge research on all aspects of machine learning used in closely related areas like artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, or robotics.

ICML is one of the fastest growing artificial intelligence conferences in the world. Participants at ICML span a wide range of backgrounds, from academic and industrial researchers, to entrepreneurs and engineers, to graduate students and postdocs.

This year, it was held in Baltimore, Maryland USA, from July 17 to 23. The DSAIDIS researchers presented three papers:

Functional Output Regression with Infimal Convolution: Exploring the Huber and ϵϵ-insensitive Losses [Arxiv]
Alex Lambert (KU Leuven) · Dimitri Bouche (Télécom Paris) · Zoltan Szabo (Ecole Polytechnique) · Florence d’Alché-Buc (Télécom Paris, Institut Polytechnique de Paris)

Learning to Predict Graphs with Fused Gromov-Wasserstein Barycenters [Arxiv]
Luc Brogat-Motte (Télécom Paris) · Rémi Flamary (École Polytechnique) · Celine Brouard (INRAE) · Juho Rousu (Aalto University) · Florence d’Alché-Buc (Télécom Paris, Institut Polytechnique de Paris)

Mitigating Gender Bias in Face Recognition using the von Mises-Fisher Mixture Model [Arxiv]

Jean-Rémy Conti (Télécom Paris Idemia) · Nathan NOIRY (Telecom Paris) · Vincent Despiegel (Idemia) · Stéphane Gentric (IDEMIA) · Stephan Clemencon (Telecom ParisTech)

2022|The Thirty-ninth International Conference on Machine Learning.

The DSAIDIS Annual Day 2022 took place on Wednesday 15 June. This annual event is an opportunity for the Télécom Paris team to meet the operational teams of the five partner companies: Airbus, ENGIE, IDEMIA, Safran and Valeo. On this occasion, various members of the academic team, from professors to PhD students, presented their works based on the four research axes of the Chair. There was also plenty of time for socializing and exchanging ideas.

Programme

9h – 9h30 – Welcome coffee and Introduction

> Find the materials of the introduction presentation

9h30 – 10h50 – Axis 2 : Exploiting large scale, heterogeneous, partially labeled data

> Find the materials and the video replay of the presentations of the Axis 2

10h50 – 11h10 Coffee break

11h10 – 12h35 – Axis 1 : Building predictive analytics on time series and data streams

> Find the materials and the video replay of the presentations of the Axis 1

12h35 – 14h Lunch

14h – 15h20 – Axis 4 : Learning through interactions with environment

> Find the materials and the video replay of the presentations of the Axis 4

15h20 – 15h40 Coffee break

15h40 – 17h – Axis 3 : Machine Learning for trusted and robust decision

> Find the materials and the video replay of the presentations of the Axis 3

17h-17h15 – Discussion and conclusion

Since its inception in 1985, AISTATS has been an interdisciplinary gathering of researchers at the intersection of artificial intelligence, machine learning, statistics and related areas.

For this 25th edition, which was held online this year again, from the 28th to the 30th of March, the academic team of the DSAIDIS chair presented two papers:

More info on the AISTATS conference

Pierre Colombo received the Outstanding Student Paper award at the 36th AAAI Conference on Artificial Intelligence for his publication “InfoLM: A New Metric to Evaluate Summarization & Data2Text Generation”. This new category, added in 2022, recognizes his work as a post-doctoral fellow at the DSAIDIS Chair, under the supervision of Chloé Clavel.

P. Colombo, C. Clavel, and P. Piantanida, « InfoLM: A New Metric to Evaluate Summarization & Data2Text Generation. », AAAI (2022).