Publications – Dane Malenfant

Accepted Code available

RLC 2026 · Montréal, Canada · August 2026

The Challenge of Hidden Gifts in Multi-Agent Reinforcement Learning

Dane Malenfant and Blake Aaron Richards

Question: How can agents learn reciprocal behaviour when helpful actions produce delayed or difficult-to-observe benefits?
Contribution: We introduce a shared-key grid-world that makes helpful actions unobservable, show that several state-of-the-art multi-agent reinforcement learning algorithms fail to obtain the collective reward, and derive a learning-aware policy-gradient correction that reduces variance and improves reliable convergence to collective success.

Malenfant, D., & Richards, B. A. (2026). The challenge of hidden gifts in multi-agent reinforcement learning. Reinforcement Learning Journal, 7.

@article{malenfant2026the,
  title={The challenge of hidden gifts in multi-agent reinforcement learning},
  author={Dane Malenfant and Blake Aaron Richards},
  journal={Reinforcement Learning Journal},
  volume={7},
  year={2026}
}

Variance in collective success across training for four policy-gradient variants — Figure 5b. Variance in collective success across training for vanilla policy-gradient, maximum-entropy, correction, and self-correction agents. Self-correction produces the lowest variance, making cooperative learning more reliable.

Preprint Revised v7 Code available

arXiv v7 · Revised September 2026 · Social Simulation with LLMs @ COLM 2026

Moral Hazard in Multi-Agent Language Models

Dane Malenfant

Question: How do language agents cooperate when gathering useful information is costly, hidden, and primarily benefits another agent?
Contribution: I introduce the Dialogue Moral Hazard Game and evaluate fourteen open-weight and four frontier models across acquisition, communication, downstream use, and team success. I also introduce CREDIT, a counterfactual mechanism-aligned prompt optimizer that rewards robust causal contribution and preserves query-mediated behaviour across changing information structures.

Malenfant, D. (2026). Moral hazard in multi-agent language models. arXiv preprint arXiv:2607.23982. https://doi.org/10.48550/arXiv.2607.23982

@article{malenfant2026moral,
  title={Moral Hazard in Multi-Agent Language Models},
  author={Malenfant, Dane},
  journal={arXiv preprint arXiv:2607.23982},
  year={2026},
  doi={10.48550/arXiv.2607.23982},
  url={https://arxiv.org/abs/2607.23982}
}

10.48550/arXiv.2607.23982

GPT-5.6 Sol query, team-success, and local-reward rates across query costs — Figure 2A. GPT-5.6 Sol's query-cost sweep. As querying becomes more expensive, querying and team success decline while preservation of local reward increases.

Accepted Revised v2 Code available

Accepted to Continual World Models Workshop · NeurIPS 2026 · arXiv v2, revised August 2026 · Previously presented at World Modeling Workshop 2026

Reinforcing the World's Edge: Continual Learning at the Multi-Agent–World Boundary

Dane Malenfant

Question: How long can success-conditioned reusable structure remain valid when peer learning changes the focal agent's induced rewards and dynamics?
Contribution: I recast decentralized peer learning as an agent-centric continual-RL problem and define an invariant core of high-coverage abstract patterns in successful trajectories. A worst-case-tight stability theorem yields Ω(1/η) survival and conditional Θ(1/η) first-exit laws; registered 64-stream continual-control and cue-MNIST studies show that core erosion predicts failure and enables near-oracle intervention.

Malenfant, D. (2026). Reinforcing the world's edge: Continual learning at the multi-agent–world boundary. arXiv preprint arXiv:2603.06813. https://doi.org/10.48550/arXiv.2603.06813

@article{malenfant2026reinforcing,
  title={Reinforcing the World's Edge: Continual Learning at the Multi-Agent--World Boundary},
  author={Malenfant, Dane},
  journal={arXiv preprint arXiv:2603.06813},
  year={2026},
  doi={10.48550/arXiv.2603.06813},
  url={https://arxiv.org/abs/2603.06813}
}

10.48550/arXiv.2603.06813

Mean reward comparison for core-triggered, oracle, random-matched, never-switch, and return-trigger control — Figure 2c. In the registered 64-stream continual-control confirmation, core-triggered control achieves mean reward 0.994, matching the oracle and outperforming random-matched (0.536), never-switch (0.460), and return-trigger (0.458) controls.

Preprint arXiv v1

arXiv v1 · Submitted September 2026

KV-streams for Efficient Compaction in Agentic Reinforcement Learning

Emiliano Penaloza, Dane Malenfant, Dheeraj Vattikonda, Roger Creus Castanyer, Siddarth Venkatraman, Abhay Puri, Jonathan Light, Matthew James Sargent, Augustine N. Mavor-Parker, Massimo Caccia, Lucas Caccia, Glen Berseth, Esmeralda S. Whitammer, Alessandro Sordoni, Minseon Kim, Marc-Alexandre Côté, Laurent Charlin, and Guillaume Lajoie

Question: How can agentic language models compact increasingly long interaction traces without repeatedly prefilling the full context or losing information needed later?
Contribution: We introduce KV-streams, a plug-and-play mechanism that streams the KV cache across compaction events instead of flushing it. Across three compaction strategies, it delivers 2.6–5× wall-clock training speedups without evidence of performance loss, while the streamed cache can learn through reinforcement learning alone to act as recurrent state carrying information no longer present in context.

Penaloza, E., Malenfant, D., Vattikonda, D., Creus Castanyer, R., Venkatraman, S., Puri, A., Light, J., Sargent, M. J., Mavor-Parker, A. N., Caccia, M., Caccia, L., Berseth, G., Whitammer, E. S., Sordoni, A., Kim, M., Côté, M.-A., Charlin, L., & Lajoie, G. (2026). KV-streams for efficient compaction in agentic reinforcement learning. arXiv preprint arXiv:2609.35750. https://doi.org/10.48550/arXiv.2609.35750

@article{penaloza2026kvstreams,
  title={KV-streams for Efficient Compaction in Agentic Reinforcement Learning},
  author={Penaloza, Emiliano and Malenfant, Dane and Vattikonda, Dheeraj and Creus Castanyer, Roger and Venkatraman, Siddarth and Puri, Abhay and Light, Jonathan and Sargent, Matthew James and Mavor-Parker, Augustine N. and Caccia, Massimo and Caccia, Lucas and Berseth, Glen and Whitammer, Esmeralda S. and Sordoni, Alessandro and Kim, Minseon and Côté, Marc-Alexandre and Charlin, Laurent and Lajoie, Guillaume},
  journal={arXiv preprint arXiv:2609.35750},
  year={2026},
  doi={10.48550/arXiv.2609.35750},
  url={https://arxiv.org/abs/2609.35750}
}

10.48550/arXiv.2609.35750

Five actor-name candidates and the actor recalled after context eviction — Detail from Figure 8B. After the assigned actor name leaves the visible context, the model must recover it from the candidates using information carried by retained KV-cache entries. This tests whether KV-streams trained on fruit assignments generalizes to actor names.

Published Code available

NeurIPS 2023 · New Orleans, USA · December 2023

Contrastive Retrospection: honing in on critical steps for rapid learning and generalization in RL

Chen Sun, Wannan Yang, Thomas Jiralerspong, Dane Malenfant, Benjamin Alsbury-Nealy, Yoshua Bengio, and Blake Aaron Richards

Question: How can reinforcement-learning agents identify the few temporally distant decisions that determine eventual success?
Contribution: We introduce Contrastive Retrospection (ConSpec), a plug-in method that uses offline contrastive learning to discover interpretable prototypes of critical states and turn them into intrinsic rewards. ConSpec improves learning across diverse tasks and supports out-of-distribution generalization when sensory features change.

Sun, C., Yang, W., Jiralerspong, T., Malenfant, D., Alsbury-Nealy, B., Bengio, Y., & Richards, B. A. (2023). Contrastive retrospection: Honing in on critical steps for rapid learning and generalization in RL. Advances in Neural Information Processing Systems, 36. https://doi.org/10.52202/075280-1356

@inproceedings{sun2023contrastive,
  title={Contrastive Retrospection: Honing in on Critical Steps for Rapid Learning and Generalization in RL},
  author={Sun, Chen and Yang, Wannan and Jiralerspong, Thomas and Malenfant, Dane and Alsbury-Nealy, Benjamin and Bengio, Yoshua and Richards, Blake Aaron},
  booktitle={Advances in Neural Information Processing Systems},
  volume={36},
  year={2023},
  doi={10.52202/075280-1356},
  url={https://proceedings.neurips.cc/paper_files/paper/2023/hash/6357d6d068622c962391081d296bed69-Abstract-Conference.html}
}

10.52202/075280-1356

Reward curves for ConSpec with PPO and three reinforcement-learning baselines — Figure 2b. Reward on the 3D OrangeTree task. ConSpec with PPO approaches the maximum reward, while PPO, SynthRs with PPO, and CURL with PPO remain substantially lower.

Spotlight Workshop paper

FoGen @ ICML 2026 · Seoul, South Korea · July 2026

Compositional Flow Matching with Factored Velocity Fields

Avery Hee-Woon Ryoo, Dane Malenfant, Matthew G. Perich, and Guillaume Lajoie

Question: How can conditional generative models generalize to novel combinations of factors that were absent from training?
Contribution: We propose a factored conditional flow-matching architecture with a shared base velocity and factor-specific heads. It matches or outperforms a monolithic baseline on zero-shot combinations, substantially lowers held-out FID on the hardest Shapes3D splits, and supports adding a new factor without retraining existing components.

Ryoo, A. H.-W., Malenfant, D., Perich, M. G., & Lajoie, G. (2026). Compositional flow matching with factored velocity fields. 1st Workshop on Foundation Models (FoGen), ICML 2026.

@inproceedings{ryoo2026compositional,
  title={Compositional Flow Matching with Factored Velocity Fields},
  author={Ryoo, Avery Hee-Woon and Malenfant, Dane and Perich, Matthew G. and Lajoie, Guillaume},
  booktitle={1st Workshop on Foundation Models (FoGen), ICML},
  year={2026},
  url={https://openreview.net/forum?id=77sT6Xaji7}
}

Shared encoder-decoder architecture with factor-specific velocity heads — Figure 1. Factored velocity field for zero-shot compositional generalization. A shared encoder-decoder base velocity is augmented by factor-specific conditioned heads whose outputs are combined at the bottleneck.

Workshop paper Poster available

Harms and Risks of AI in the Military 2024 · Montréal, Canada · December 2024

Inconsistencies in Artificial Intelligence Strategy Alignment of NATO Member States

Itai Epstein, Dane Malenfant, Sara Parker, and Cella Wardrop

Question: How consistently have NATO member states publicly articulated policies for military uses of artificial intelligence?
Contribution: We build a comparative dataset from official national strategies and public statements. Although 88% of NATO members had a national AI strategy, only 34% had published military-specific AI policy, revealing a substantial transparency and alignment gap.

Epstein, I., Malenfant, D., Parker, S., & Wardrop, C. (2024). Inconsistencies in artificial intelligence strategy alignment of NATO member states. Harms and Risks of AI in the Military.

@inproceedings{epstein2024inconsistencies,
  title={Inconsistencies in Artificial Intelligence Strategy Alignment of NATO Member States},
  author={Epstein, Itai and Malenfant, Dane and Parker, Sara and Wardrop, Cella},
  booktitle={Harms and Risks of AI in the Military},
  year={2024},
  url={https://openreview.net/forum?id=apB72N0nxF}
}

Stacked policy-strategy coverage bars for eleven NATO member states — Derived visualization of public national AI, digital, national security, military-AI, and cybersecurity strategies among the NATO members shown. The complete analysis covers all 32 members, finding national AI strategies for 28 members but military-AI-specific policies for only 11.

Published

Text2Story @ ECIR 2022 · Stavanger, Norway · April 2022

Causality Mining in Fiction

Margaret Meehan, Dane Malenfant, and Andrew Piper

Question: Do causal-relation models developed on standard NLP benchmarks transfer reliably to literary fiction?
Contribution: We introduce a labeled dataset containing 548 positive causal event pairs from 141 contemporary literary passages, together with matched negative examples. Comparing feature-based and BERT models against SemEval data reveals linguistic and domain differences that complicate causal-relation extraction from fiction.

Meehan, M., Malenfant, D., & Piper, A. (2022). Causality mining in fiction. In Proceedings of the 5th Workshop on Narrative Extraction from Texts (Text2Story 2022) (pp. 25–34).

@inproceedings{MalenfantP22,
  author={Meehan, Margaret and Malenfant, Dane and Piper, Andrew},
  title={Causality Mining in Fiction},
  booktitle={Proceedings of the 5th Workshop on Narrative Extraction from Texts (Text2Story 2022)},
  pages={25--34},
  year={2022},
  url={https://ceur-ws.org/Vol-3117/paper3.pdf}
}

Random Forest feature weights for literary and SemEval causal-relation data — Figure 2. Feature weights from Random Forest models trained on annotated literary and SemEval data. Semantic similarity is the strongest indicator of causal relations in both datasets, with verb-related features also contributing.

Publications.

Questions that connect the papers

Reciprocity & Multi-Agent Learning