3D behavior phenotyping and multi-object tracking system of mice based on deep learning - Lei He; Xuezhen Jia; Chen Li; Yan Liu; Jicheng Yu; Zhen-Xia Chen; Wanneng Yang; Xiuying Liang (Smart Agricultural Technology)
To extract multiple-mice behaviors automatically, nondestructively, and long-termly, a 3D behavior extraction system was developed for extracting individual behaviors and social behaviors of mice. In this study, two depth cameras were used to acquire 3D information of mice. 3D point clouds model of mice was established and the algorithms of clipping, statistical filter, matching, plane segmenting, and rotating were used to preprocess the 3D point clouds. In this study, YOLOv7 was used to detect mice and 3D Kalman Filter, Hungarian algorithm, and improved matching algorithm were used to track mouse identity. Stratified Transformer was used to segment individual mouse from the environment and segment individual mouse into 3 parts, and 8 feature points were extracted. In individual mouse experiment, the whole individual mouse was segmented to 4 parts and 11 feature points were extracted. Individual and social behaviors of mice were extracted combined feature points and mice trajectories. The results showed the accuracy and average mIoU of individual mouse segmentation were 95.80 % and 91.90 %, respectively, and the accuracy and average mIoU of parts segmentation were 93.60 % and 85.9 %, respectively. Compared with manual measurement of mice weight, the determination coefficient (R2) of mouse volume was 0.9726. The maximum absolute error of the automatic measurements versus manual measurements of mouse tail length was 0.4 mm. The precision, recall, and mAP of YOLOv7 were 99.76 %, 99.53 %, and 99.75 %, respectively. The analysis of behaviors also showed that there were significant differences between dexamethasone-induced muscle atrophy and normal mice.