Visualising Multi-Sensor Predictions from a Rice Disease Classifier

Thumbnail Image

Files

Brian Muhia - Microsoft Rice Disease Classification Challenge - Data Analysis Report v0.3.pdf (2.22 MB)

Date

2022-12-03

Authors

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

The Microsoft Rice Disease Classification Challenge introduced a dataset comprising RGB and RGNiR (RG-Near-infra-Red) images. This second image type increased the difficulty of the challenge such that all of the winning models worked with RGB only. In this challenge we applied a res2next50 encoder that was first pre-trained with self-supervised learning through the SwAV algorithm, to represent each RGB and their corresponding RGNIR images with the same weights. The encoder was then fine-tuned and self-distilled to classify the images which produced a public test set score of 0.228678639, and a private score of 0.183386940. K-fold cross-validation was not used for this challenge result. To better understand the impact of self-supervised pre-training on the problem of classifying each image type, we apply t-distributed Stochastic Neighbour Embedding (t-SNE) on the logits (predictions before applying softmax). We show how this method graphically provides some of the value of a confusion matrix, by locating some incorrect predictions. We then render the visualisation by overlaying the raw images in each data point, and note that to this model, the RGNIR images do not appear to be inherently more difficult to categorise. We make no comparisons through sweeps, RGB-only models or RGNIR-only models. This is left to future work.

Description

Keywords

Artificial Intelligence and Image Processing, Crop and Pasture Nutrition, visualisation

Citation

URI

https://arxiv.org/abs/2006.09882
 https://arxiv.org/abs/1905.08094
 https://lvdmaaten.github.io/tsne/
 https://doi.org/10.1007/978-3-030-83356-5_5
 https://distill.pub/2016/misread-tsne/
 https://cs.stanford.edu/people/karpathy/cnnembed/
 https://arxiv.org/abs/1610.09204
 https://keremturgutlu.github.io/self_supervised
 https://walkwithfastai.com
 https://fast.ai
 https://arxiv.org/abs/2108.01661
 https://www.microsoft.com/en-us/research/blog/three-mysteries-in-deep-learning-ensemble-knowledge-distillation-and-self-distillation/
 https://www.jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf
 https://africarxiv.ubuntunet.net/handle/1/1201
 https://doi.org/10.60763/africarxiv/1153
 https://doi.org/10.60763/africarxiv/1153
 https://doi.org/10.60763/africarxiv/1153

Collections

Research Studies and Reports
AI Applicability in Industry
AI for Social Good
Machine Learning and Algorithms