Search

The capacity to detect rapeseed with precision and to track the enlargement of its canopy coverage stands among the most telling phenotypic signals of how the crop is developing. When both the individual rapeseed plant and the zone it occupies can be accurately recognized, the resulting data stream provides a powerful foundation for downstream phenotypic exploration and breeding programs. The difficulty, however, is that in genuine field conditions, detection performance continues to face serious obstacles because RGB data alone provides only a limited vocabulary of representable features. Seeking a way around this limitation, the work described here advances a dual-modal instance segmentation framework, MSNet, which builds on YOLOv11n-seg and combines RGB and Near-Infrared (NIR) data streams. The chief innovations embedded in this framework comprise three strategies governing where fusion occurs (at the front end, at an intermediate stage, or at the back end), together with a newly crafted Hierarchical Attention Fusion Block (HAFB) dedicated to blending multi-modal features. Experiments designed to compare these placement choices reveal that conducting fusion at the intermediate stage delivers the strongest compromise between how accurately the model detects and how economical it remains in terms of parameter count. Relative to the baseline, the improvement in mAP50:95 can reach 3.5%. Once the HAFB unit is incorporated, the MSNet-H-HAFB configuration posts a 6.5% lift in mAP50:95 over that baseline, while requiring fewer than 38% extra parameters. A point that deserves emphasis is that mid-stage fusion consistently outperformed the alternatives in every experimental run, supplying a clear blueprint for where future multi-modal architectures should position their fusion operations. In further testing, side-by-side comparisons against a suite of RGB-only instance segmentation alternatives confirmed that all MSNet-HAFB fused variants decisively beat single-modal counterparts on rapeseed count detection, reinforcing the case that multi-spectral fusion holds genuine promise for recognizing agricultural targets. As a concluding step, MSNet was applied in a practical agricultural scenario that included vegetation index analysis and grading of frost-related damage. The data pointed to ZN6–2836 and ZS11 as likely candidate elite lines, while the EVI2 vegetation index registered the strongest classification scores for frost damage in rapeseed.