A Hierarchical Computer Vision Framework for Continuous Dairy Cow Behavior Monitoring in an Obstructed Free-Stall Barns

Information from the abstract

Continuous monitoring of dairy cows in free-stall barns remains challenging because of occlusion, variable illumination, and identity switches. This study developed a hierarchical computer vision framework integrating behavior-specific object detection, detector-assisted SAM2 segmentation, and multi-object tracking for CCTV-based monitoring. Video data were collected from three CCTV cameras in free-stall barns, covering two obstructed zone-specific views, and one wide-angle facility view. YOLOv11m and RT-DETR-L were compared across five behaviors (drinking, feeding, resting, standing, and walking) using a 214,187-image annotated detection dataset, while 645 daytime and 283 nighttime images evaluated detector-assisted SAM2 segmentation. Tracking performance was assessed using ByteTrack and SAM2-assisted tracking under variable illumination and camera-view conditions. RT-DETR-L yielded a slightly higher aggregate F1-score than YOLOv11m (66.0% vs. 63.1%), though both detectors achieved comparable localization accuracy, with identical mAP50 (65.4%) and mAP50-95 (50.9%). Feeding, resting, and standing were detected more reliably than walking and drinking, with drinking showing the lowest AP50 (2.0%).For segmentation, RT-DETR-L-SAM2 achieved a higher and more stable mean IoU (60.1% and 59.4%) than YOLOv11m-SAM2 (44.9% and 50.5%, respectively) across daytime and nighttime conditions. RT-DETR-L combined with ByteTrack achieved the highest MOTA (53.0%) and IDF1 (74.6%). SAM2-assisted pipeline exhibited substantially fewer identity switches than ByteTrack (YOLOv11: 50 vs. 8; RT-DETR-L: 75 vs. 9), suggesting improved short-term identity continuity. However, since SAM2 utilized a custom cross-chunk association strategy rather than ByteTrack’s frame-by-frame matching, this reduction cannot be attributed to SAM2 segmentation alone. This improvement did not consistently raise aggregate MOTA or IDF1 but markedly enhanced identity continuity, revealing a trade-off between tracking accuracy and within-camera short-term identity continuity. These findings demonstrate that integrating detection, SAM2-based segmentation, and tracking provides a promising within-farm-proof-of-concept framework for CCTV-based dairy cow monitoring, particularly for maintaining individual identities in complex free-stall barns.

Why this record is monitored

This record has an Impact Signal of 77/100 based on recency, source, collaboration, and bibliographic signals. It prioritizes monitoring and is not a judgment of research quality.

Related topics: Smart Agriculture and AI · Animal Behavior and Welfare Studies · Water Quality Monitoring Technologies