Oct 2023 – Jul 2025 · 1 yr 9 mos

• I am part of the GenAI team, working on LLaMa and fine-tuning Large Language Models.
• Currently:
• As the technical lead of the Memory project, I focus on enhancing LLMs' ability to process large-scale contextual data and novel information sources.
• Previous projects:
• Developed zero-shot tools for LLaMa3
• MoE post-training & expert duplication

May 2020 – Oct 2023 · 3 yrs 5 mos

• Pursuing a Phd. at Meta AI (prev. Facebook AI Research) and INRIA (MIND, prev. Parietal team), supervised by Alexandre Gramfort and Jean-Remi King
• I study language processing in deep neural networks and the brain. I mostly focus on Transformer-based language models and neuro-imaging techniques (fMRI, MEG, EEG).
• Selected publications:
• Long-range and hierarchical language predictions in brains and algorithms, Caucheteux, Gramfort and King, in prep. for Nature Human Behaviour, 2022
• Towards a realistic model of speech processing in the brain, Millet*, Caucheteux* et al., NeurIPS, 2022
• Brains and algorithms partially converge in Natural Language Processing, Nature Communications Biology, 2022
• Disentangling syntax and semantics in the brain with deep networks, Caucheteux, Gramfort and King, ICML 2021
• Model-based analysis of brain activity reveals the hierarchy of language in 305 subjects, Caucheteux, Gramfort and King, Findings of EMNLP, 2021
• GPT-2’s activations predict the degree of semantic comprehension in the human brain, Caucheteux, Gramfort and King, Nature Scientific Reports, 2022
• Decoding speech from non invasive brain recordings, Defossez, Caucheteux, Rapin, Kabeli and King, Under Review, 2022
• Google scholar: https://scholar.google.com/citations?hl=en&user=qu7KObMAAAAJ

Mar 2020 – Aug 2020 · 5 mos · Paris, Île-de-France, France

• Worked on the Covidom project. Covidom is a web application for home management of patients with mild-to-moderate symptoms of COVID-19 (55 000 patients).
• helped in alert prioritisation
• analysed the patient database to build a targeting strategy for PCR testing
• analysed the effect of smoking on the COVID-19 disease (infection and severity signs)

Jun 2019 – Sep 2019 · 3 mos · Région de Paris, France

• Under the supervision of Jean-Remi King and Marco Baroni. Working at the interplay of machine learning and cognitive neuroscience, focusing on two main subjects :
• The links between brains and neural networks’ representations of words. More precisely, I study the predictability of different word/sentence embeddings (word2vec, fasttext, elmo …) given the brain signals (MEG) of persons reading a text.
• The links between brains and neural networks in the processing of compositional tasks. Humans are thought to learn and process information in a ‘compositional’ way, i.e they split tasks into sub-tasks and entities into sub-entities. The idea is to design a task, tested on both humans and algorithms, in order to highlight their differences in compositional processing.

Dec 2018 – Jul 2025 · 6 yrs 7 mos · Région de Paris, France

• NoArtist is an art collective of GANs generated artworks (www.noartist.fr/)
• Creating artworks on different themes : portrait, cubism and marina.
• Launched an exhibition with ~1000 visitors in March at the Salon du Pantheon.
• Teaching is a major value in the team and we regularly organise talks on AI to present and explain our project to a non-expert public.

Sep 2018 – Apr 2019 · 7 mos · Région de Paris, France

• Under the supervision of Emmanuel Dupoux in the Cognitive Machine Learning (CoML) team.
• Working on multimodal speech recognition models using weakly supervised deep neural networks and attention mechanisms. The goal is to simulate lip reading in children’s phoneme acquisition. Framework : PyTorch.

Apr 2018 – Jul 2018 · 3 mos · Stockholm, Suède / Paris, France

• BCG Gamma is the advanced analytics and data science consulting team of the Boston Consulting Group.
• Improved a demand forecast model based on machine learning algorithms (in R, +3% accuracy achieved). Different techniques were used : feature engineering, model selection, time series analysis, granularity analysis.
• Preprocessed the database using Spark (3 TB).
• Applied unsupervised learning methods to stratified sampling.

Jun 2017 – Aug 2017 · 2 mos · Région de New York City, États-Unis

• Okwin is a unicorn (>1B valuation) applying AI to drug discovery.
• Analyzed the ClinicalTrial.gov data base (250 000 clinical trials) implementing Machine Learning algorithms (Python).
• The goal was to predict the success of a trial (accuracy obtained : 90%).
• Preprocessed the data base using NLP.
• Started to implement Deep Learning algorithms using Keras.

Oct 2016 – Mar 2017 · 5 mos · Palaiseau

• Worked on the HELIX (Human Early Life Exposome) project, an UE epidemiological research crossing Biology and Statistics, coordinated by the INSERM institute.
• Implemented omics-based approaches to characterise the impact of environmental exposures on obesity.
• Worked on two databases (400 000 methylation sites and 200 exposomic factors for 1 200 children)

Oct 2015 – Apr 2016 · 6 mos · Arcueil, France

• Conducted the "drone SDAM" project (interviewed operational soldiers, led simulations of the drone performances on Matlab, led presentations of the SDAM drone)
• Participated in the AWACS project.

Jun 2015 – Jul 2015 · 1 mo · Région de Versailles, France