Similar Accuracy, Different Explanations in YOLO-Based Palm Fruit Ripeness Detection

Abstract

Modern palm-fruit detectors can achieve similarly high benchmark accuracy, yet class activation mapping (CAM) methods are often applied without establishing whether their explanatory behavior transfers across detector configurations. This study is a controlled exploratory within-dataset comparison: model scale and target layer were selected on the same 416-image test partition used for final CAM evaluation, so the reported values are not independent estimates of explanation generalization. We evaluated 15 YOLOv8, YOLO11, and YOLO26 variants under a matched training protocol and assessed representative medium models across three random seeds. Backbone representations were then audited on the 416 test images using class-agnostic EigenCAM, backbone-adapted Grad-CAM++ (bGrad-CAM++), and backbone-adapted Score-CAM (bScore-CAM). Explanation behavior was quantified with CAM-box intersection over union (IoU) and Pointing Game for spatial localization and with Mean Absolute Confidence Drop (MACD) and Increase in Confidence as image-level confidence-retention diagnostics. Repeated-run mean mAP50–95 was narrowly separated at 0.903 ± 0.002, 0.899 ± 0.002, and 0.901 ± 0.003 for YOLOv8m, YOLO11m, and YOLO26m, respectively. In contrast, Pointing Game varied by 68.03 percentage points and MACD by 48.62 percentage points across model–explainer configurations. EigenCAM showed the strongest observed profile on YOLOv8m, whereas bScore-CAM achieved the highest localization on YOLO11m; no explainer dominated across all evaluated backbones. The ranking reversal was also present within the difficult underripe, ripe, and overripe subset, and the recorded layer search showed that layer 8 was the strongest compatible backbone target across the layer 6–10 window in all three detectors. Similar detection accuracy therefore did not imply transferable explanation behavior. CAM explanations should be validated jointly with the explanatory target, layer, and detector rather than treated as interchangeable post-hoc visualizations.

ABSTRAK: Pengesan buah sawit moden boleh mencapai ketepatan penanda aras tinggi dan hampir setara, namun kaedah pemetaan pengaktifan kelas (CAM) sering diguna tanpa memastikan sama ada tingkah laku penjelasannya boleh dipindah merentas konfigurasi pengesan. Kajian ini merupakan perbandingan penerokaan terkawal dalam set data: skala model dan lapisan sasaran dipilih menggunakan pecahan ujian 416 imej sama seperti penilaian CAM akhir, di mana dapatan yang dilaporkan bukan anggaran bebas bagi generalisasi penjelasan. Sebanyak 15 varian YOLOv8, YOLO11 dan YOLO26 dinilai menggunakan protokol latihan sepadan, manakala model sederhana yang mewakili setiap keluarga dinilai menggunakan tiga benih rawak. Perwakilan tulang belakang kemudiannya diaudit pada 416 imej ujian menggunakan EigenCAM yang bebas kelas, Grad-CAM++ teradaptasi tulang belakang (bGrad-CAM++), dan Score-CAM teradaptasi tulang belakang (bScore-CAM). Tingkah laku penjelasan dinilai menggunakan CAM-box IoU dan Pengarahan Perburuan bagi menempat spatial, serta Purata Mutlak Keyakinan Menurun (MACD) dan Keyakinan Bertambah sebagai diagnostik pengekalan keyakinan pada aras imej. Purata mAP50–95 bagi larian berulang masing-masing adalah 0.903 ± 0.002, 0.899 ± 0.002 dan 0.901 ± 0.003 untuk YOLOv8m, YOLO11m dan YOLO26m. Sebaliknya, Pengarahan Pemburuan berbeza sebanyak 68.03 mata peratusan dan MACD sebanyak 48.62 mata merentas konfigurasi model–penerang. EigenCAM menunjukkan profil pemerhatian terkuat pada YOLOv8m, manakala bScore-CAM mencapai penyetempatan tertinggi pada YOLO11m; tiada penerang yang mendominasi semua tulang belakang yang dinilai. Pembalikan kedudukan ini turut hadir dalam subset sukar seperti kurang masak, masak dan terlebih masak, manakala carian lapisan yang direkod menunjukkan bahawa lapisan 8 ialah sasaran tulang belakang serasi terkuat dalam tetingkap lapisan 6–10 bagi ketiga-tiga pengesan. Dapatan ini menunjukkan bahawa ketepatan pengesanan yang hampir setara tidak semestinya menghasilkan tingkah laku penjelasan yang boleh dipindahkan. Oleh itu, penjelasan CAM perlu disahkan secara bersama dengan sasaran penjelasan, lapisan dan pengesan.