Recent strides in machine learning have shown that computation can be performed by practically any controllable physical system that responds to physical stimuli encoding data [1]. This perspective opens new frontiers for computational approaches using Physical Neural Networks (PNNs) [2, 3, 4] and provides a framework to deepen our understanding of their biological counterparts—neural circuits in living organisms.