The AG Schwarz developed a closed loop, deep learning-based real time posture detection tool allowing to autonomously conduct behavioral experiments.

The paper is in peer-review. A preprint and the code are already available online:

Preprint BioRxiv:



Jens F. Schweihoff, Matvey Loshakov, Irina Pavlova, Laura Kück, Laura A. Ewell, Martin K. Schwarz (2019) DeepLabStream: Closing the loop using deep learning-based markerless, real time posture detection, bioRxiv doi:

Abstract: In general, animal behavior can be described as the neuronal-driven sequence of reoccurring postures through time. Current technologies enable offline pose estimation with high spatio-temporal resolution, however to understand complex behaviors, it is necessary to correlate the behavior with neuronal activity in real-time. Here we present DeepLabStream, a highly versatile, closed-loop solution for freely moving mice that can autonomously conduct behavioral experiments ranging from behavior-based learning tasks to posture-dependent optogenetic stimulation. DeepLabStream has a temporal resolution in the millisecond range, can operate with multiple devices and can be easily tailored to a wide range of species and experimental designs. We employ DeepLabStream to autonomously run a second-order olfactory conditioning task for freely moving mice and to deliver optogenetic stimuli based on mouse head-direction.

For more info on the please klick here.