Visualising Multi-Sensor Predictions from a Rice Disease Classifier
Loading...
Date
2022-12-03
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
The Microsoft Rice Disease Classification Challenge introduced a dataset comprising RGB and RGNiR (RG-Near-infra-Red) images. This second image type increased the difficulty of the challenge such that all of the winning models worked with RGB only. In this challenge we applied a res2next50 encoder that was first pre-trained with self-supervised learning through the SwAV algorithm, to represent each RGB and their corresponding RGNIR images with the same weights. The encoder was then fine-tuned and self-distilled to classify the images which produced a public test set score of 0.228678639, and a private score of 0.183386940. K-fold cross-validation was not used for this challenge result. To better understand the impact of self-supervised pre-training on the problem of classifying each image type, we apply t-distributed Stochastic Neighbour Embedding (t-SNE) on the logits (predictions before applying softmax). We show how this method graphically provides some of the value of a confusion matrix, by locating some incorrect predictions. We then render the visualisation by overlaying the raw images in each data point, and note that to this model, the RGNIR images do not appear to be inherently more difficult to categorise. We make no comparisons through sweeps, RGB-only models or RGNIR-only models. This is left to future work.
Description
Keywords
Artificial Intelligence and Image Processing, Crop and Pasture Nutrition, visualisation
Citation
URI
https://arxiv.org/abs/2006.09882
https://arxiv.org/abs/1905.08094
https://lvdmaaten.github.io/tsne/
https://doi.org/10.1007/978-3-030-83356-5_5
https://distill.pub/2016/misread-tsne/
https://cs.stanford.edu/people/karpathy/cnnembed/
https://arxiv.org/abs/1610.09204
https://keremturgutlu.github.io/self_supervised
https://walkwithfastai.com
https://fast.ai
https://arxiv.org/abs/2108.01661
https://www.microsoft.com/en-us/research/blog/three-mysteries-in-deep-learning-ensemble-knowledge-distillation-and-self-distillation/
https://www.jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf
https://africarxiv.ubuntunet.net/handle/1/1201
https://doi.org/10.60763/africarxiv/1153
https://doi.org/10.60763/africarxiv/1153
https://doi.org/10.60763/africarxiv/1153
https://arxiv.org/abs/1905.08094
https://lvdmaaten.github.io/tsne/
https://doi.org/10.1007/978-3-030-83356-5_5
https://distill.pub/2016/misread-tsne/
https://cs.stanford.edu/people/karpathy/cnnembed/
https://arxiv.org/abs/1610.09204
https://keremturgutlu.github.io/self_supervised
https://walkwithfastai.com
https://fast.ai
https://arxiv.org/abs/2108.01661
https://www.microsoft.com/en-us/research/blog/three-mysteries-in-deep-learning-ensemble-knowledge-distillation-and-self-distillation/
https://www.jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf
https://africarxiv.ubuntunet.net/handle/1/1201
https://doi.org/10.60763/africarxiv/1153
https://doi.org/10.60763/africarxiv/1153
https://doi.org/10.60763/africarxiv/1153