This repository contains resources from the accepted survey paper on "A Survey on Machine Learning Approaches for Modelling Intuitive Physics" for the IJCAI-ECAI 2022 (Survey Track) with 18% Acceptance Rate.
With inspiration from, the awesome lists and and awesome-cogsci.
Research in cognitive science has provided extensive evidence of human cognitive ability in performing physical reasoning of objects from noisy perceptual inputs. Such a cognitive ability is commonly known as intuitive physics. With advancements in deep learning, there is an increasing interest in building intelligent systems that are capable of performing physical reasoning from a given scene for the purpose of building better AI systems. As a result, many contemporary approaches in modelling intuitive physics for machine cognition have been inspired by literature from cognitive science. Despite the wide range of work in physical reasoning for machine cognition, there is a scarcity of reviews that organize and group these deep learning approaches. Especially at the intersection of intuitive physics and artificial intelligence, there is a need to make sense of the diverse range of ideas and approaches. Therefore, this paper presents a comprehensive survey of recent advances and techniques in intuitive physics-inspired deep learning approaches for physical reasoning. The survey will first categorize existing deep learning approaches into three facets of physical reasoning before organizing them into three general technical approaches and propose six categorical tasks of the field. Finally, we highlight the challenges of the current field and present some future research directions.
Any question contact Jiafei Duan. If you see papers missing from the list, please send me an email or a pull request (format see below).
- Contributing
- Cognitive Science Literatures
- Facets of Physical Reasoning for AI
- Revalent Surveys
- Physics Engine
- Cite
When sending PRs, please put the new paper at the correct chronological position as the following format:
* **Paper Title** <br>
*Author(s)* <br>
Conference, Year. [[Paper]](link) [[Code]](link) [[Website]](link)
Summary of the six physical reasoning tasks
Summary of the work for intuitive physics for machine cognition.
-
Intuitive Physics
Michael McCloskey
Scientific American, 1983. [Paper] -
Intuitive physics: the straight-down belief and its origin
McCloskey, Michael, Allyson Washburn, and Linda Felch
Journal of Experimental Psychology: Learning, Memory, and Cognition 9, 1983. [Paper] -
Mind games: Game engines as an architecture for intuitive physics
Ullman, Tomer D., Elizabeth Spelke, Peter Battaglia, and Joshua B. Tenenbaum
Trends in cognitive sciences, 2017. [Paper] -
Simulation as an engine of physical scene understanding
Battaglia, Peter W., Jessica B. Hamrick, and Joshua B. Tenenbaum
Proceedings of the National Academy of Sciences, 2013. [Paper]
-
Newtonian image understanding: Unfolding the dyanmics of object in static images
Roozbeh Mottaghi, Hessam Bagherinezhad, Mohammad Rastegari, Ali Farhadi
CVPR, 2016. [Paper] [Code] -
To fall or not to fall: A visual approach to physical stability prediction
Wenbin Li, Seyedmajid Azimi, Aleš Leonardis, Mario Fritz
AAAI, 2017. [Paper] -
Learning physical intuition of block towers by example
Adam Lerer, Sam Gross, Rob Fergus
PMLR, 2016. [Paper] -
Shapestacks: Learning vision-based physical intuition for generalised object stacking
Oliver Groth, Fabian B Fuchs, Ingmar Posner, and Andrea Vedaldi
ECCV, 2018. [Paper] [Code] -
Reasoning about physical interactionswith object-oriented prediction and planning
Michael Janner, Sergey Levine, William T Freeman, Joshua B Tenenbaum, Chelsea Finn, and Jiajun Wu
ICLR, 2019. [Paper] [Code] -
"What happens if..." Learning to Predict the Effect of Forces in Images
Roozbeh Mottaghi, Mohammad Rastegari, Abhinav Kumar Gupta, and Ali Farhadi
ECCV, 2016. [Paper] [Code] -
Unsupervised intuitive physics from past experiences
Sebastien Ehrhardt, Aron Monszpart, Niloy Jyoti Mitra, and Andrea Vedaldi
ArXiv, 2019. [Paper] -
Visual interaction networks: Learning a physics simulator from video
Nicholas Watters, Daniel Zoran, Theophane Weber, Peter Battaglia, Razvan Pascanu, and Andrea Tacchetti
NeurIPS, 2017. [Paper] [Code] -
Learning to see physics via visual de-animation
Jiajun Wu, Erika Lu, Pushmeet Kohli, Bill Freeman, and Josh Tenenbaum
NeurIPS, 2017. [Paper] -
PIP:Physical Interaction Prediction via Mental Simulation with Span Selection
Jiafei Duan, Samson Yu, Soujanya Poria, Bihan Wen, Cheston Tan
Preprint, 2021. [Paper]
-
Galileo: Perceiving Physical Object Properties by Integrating a Physics Engine with Deep Learning
Jiajun Wu, Ilker Yildirim, Joseph J. Lim, Bill Freeman, Josh Tenenbaum
NeurIPS, 2015. [Paper] -
Physics 101: Learning physical object properties from unlabeled videos
Jiajun Wu, Joseph J. Lim, Hongyi Zhang, Joshua B. Tenenbaum, and William T. Freeman
BMVC, 2016. [Paper] [Code] -
Interaction network for learning about objects, relations and physics
Peter W. Battaglia, Razvan Pascanu, Matthew Lai, Danilo Rezende, Koray Kavukcuoglu
NeurIPS, 2016. [Paper] [Code] -
A compositional object-based approach to learning physical dynamics
Michael B Chang, Tomer Ullman, Antonio Torralba, and Joshua B Tenenbaum
ICLR, 2017. [Paper] [Code] -
Perceiving physical equation by observing visual scenarios
Siyu Huang, Zhi-Qi Cheng, Xi Li, Xiao Wu, Zhongfei Zhang, and Alexander Hauptmann
NeurIPS Workshop, 2018. [Paper] -
A bayesian-symbolic approach to reasoning and learning in intuitive physics
Kai Xu, Akash Srivastava, Dan Gutfreund, Felix Sosa, Tomer Ullman, Josh Tenenbaum, and Charles Sutton
NeurIPS, 2021. [Paper] [Code] -
Learning visual predictive models of physics for playing billiards
Katerina Fragkiadaki, Pulkit Agrawal, Sergey Levine, and Jitendra Malik
NeurIPS, 2021. [Paper] -
Interpretable Intuitive Physics Model
Ye, Tian, Xiaolong Wang, James Davidson, and Abhinav Gupta
ArXiv, 2015. [Paper] -
Unsupervised learning of latent physical properties using perception-prediction networks
David Zheng, Vinson Luo, Jiajun Wu, Joshua B. Tenenbaum
IJCAI, 2018. [Paper] -
End-to-End Differentiable Physics for Learning and Control
Filipe de Avila Belbute-Peres, Kevin Smith, Kelsey Allen, Josh Tenenbaum, J. Zico Kolter
NeurIPS, 2018. [Paper] -
Physics-as-inverse-graphics: Unsupervised physical parameter estimation from video
Miguel Jaques, Michael Burke, and Timothy Hospedales
ICLR, 2020. [Paper] [Code] -
Physics-as-inverse-graphics: Unsupervised physical parameter estimation from video
Miguel Jaques, Michael Burke, and Timothy Hospedales
ICLR, 2020. [Paper] [Code] -
Physical representation learning and parameter identification from video using differentiable physics
Rama Krishna Kandukuri, Jan Achterhold, Michael Moeller, and Joerg Stueckler
IJCV, 2022. [Paper] [Code]
-
Probing physics knowledge using tools from developmental psychology
Luis Piloto, Ari Weinstein, Dhruva TB, Arun Ahuja, Mehdi Mirza, Greg Wayne, David Amos, Chia-chun Hung, and Matt Botvinick
ArXiv, 2018. [Paper] -
Modeling expectation violation in intuitive physics with coarse probabilistic object representations
Kevin Smith, Lingjie Mei, Shunyu Yao, Jiajun Wu, Elizabeth Spelke, Joshua Tenenbaum, and Tomer Ullman
NeruIPS, 2019. [Paper] [Code] -
CoPhy: Counterfactual Learning of Physical Dynamics
Fabien Baradel, Natalia Neverova, Julien Mille, Greg Mori, Christian Wolf
ICLR, 2020. [Paper] [Code] -
Causal world models by unsupervised deconfounding of physical dynamics
Minne Li, Mengyue Yang, Furui Liu, Xu Chen, Zhitang Chen, and Jun Wang
ArXiv, 2020. [Paper] -
A Benchmark for Modeling Violation-of-Expectation in Physical Reasoning Across Event Categories
Arijit Dasgupta, Jiafei Duan, Marcelo H Ang Jr, Yi Lin, Su-hua Wang, Renée Baillargeon, Cheston Tan
ArXiv, 2021. [Paper] -
Intphys: A framework and benchmark for visual intuitive physics reasoning
Ronan Riochet, Mario Ynocente Castro, Mathieu Bernard, Adam Lerer, Rob Fergus, Veronique Izard, and Emmanuel Dupoux
IEEE PAMI. [Paper] [Code] -
Craft:A benchmark for causal reasoning about forces and interactions
Tayfun Ates, Muhammed Samil Atesoglu, Cagatay Yigit, Ilker Kesen, Mert Kobas, Erkut Erdem, Aykut Erdem, Tilbe Goksun, and Deniz Yuret
ACL, 2022. [Paper] [Code] -
CLEVRER: collision events for video representation and reasoning
Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B.Tenenbaum
ICLR, 2020. [Paper] [Code]
-
Intuitive physics: Current research and controversies
Kubricht, James R., Keith J. Holyoak, and Hongjing Lu
Trends in cognitive sciences, 2017. [Paper] -
Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense
Yixin Zhua,Tao Gao, Lifeng Fan, Siyuan Huang, Mark Edmonds, Hangxin Liu, Feng Gao, Chi Zhang, Siyuan Qia,Ying Nian Wua, Joshua B. Tenenbaum, Song-Chun Zhu
Engineering, 2020. [Paper]
- Bullet (Blender)
- Newton Game Dynamics
- PhysX (Unreal & Omniverse)
- Box2D
- Tokamak Game Physics
- Havok Physics
- Phyz
- Unity Physics Engine (Unity)
Please cite the following paper.
@article{duan2022survey,
title={A Survey on Machine Learning Approaches for Modelling Intuitive Physics},
author={Duan, Jiafei and Dasgupta, Arijit and Fischer, Jason and Tan, Cheston},
journal={arXiv preprint arXiv:2202.06481},
year={2022}
}