HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY
NGUYEN THI THANH NHAN
INTERACTIVE AND MULTI-ORGAN BASED
PLANT SPECIES IDENTIFICATION
Major: Computer science
Code: 9480101
THESIS ABSTRACT
COMPUTER SCIENCE
Hanoi −2020
This dissertation is completed at:
Hanoi University of science and technology
Supervisors:
1. Assoc. Prof. Dr. Le Thi Lan
2. Assoc. Prof. Dr. Hoang Van Sam
Reviewer 1: Assoc. Prof. Dr. Nguyen Thi Thuy
Reviewer 2: Assoc. Prof. Dr. Tran Quang Bao
Reviewer 3: Assoc. Prof. Dr. Pham Van Cuong
The dissertation will be defended before approval committee
at Hanoi University of Science and Technology:
Time...., date.......month......year.....
The dissertation can be found at
1. Ta Quang Buu Library
2. Vietnam National Library
INTRODUCTION
Motivation
Plant identification that aims at matching a given specimen plant to a known taxon
is considered as important key to assess flora knowledge. Nowadays, the availability
of relevant technologies (e.g. digital cameras and mobile devices), images datasets and
advance techniques in image processing and pattern recognition let the idea of auto-
mated plants/species identification become reality. The automatic plant identification
can be defined as a process of determining the name of species based on their observed
images.
In recent years, we have witnessed a significant improvement of automatic plant
identification performance in term of both accuracy and the number of species classes
[1, 2]. The use of plant identification in reality still has to overcome the following
limitations. First, the number of covered plant species (e.g., 10,000 in LifeCLEF [2])
is relatively small in comparison with the number of plant species on the earth (e.g.,
400,000 [3]). Second, the accuracy of the automatic plant identification still need to be
improved.
Objective
The main aim of this thesis is to overcome the second limitation of the automatic
plant identification (low recognition accuracy) by proposing novel and robust methods
for plant recognition. For this, we first focus on improving the recognition accuracy
of plant identification based on images of one sole organ. Among different organs
of the plant, we select leaf as this organ is the most widely in the literature. Second,
taking into consideration that using one sole organ for plant identification is not always
relevant because one organ cannot fully reflect all information of a plant due to the
large inter-class similarity and large intra-class variation. Therefore, multi-organ plant
identification is also studied in this dissertation. Finally, one more objective of the
dissertation is to contribute in spread the knowledge of one specific kind of plants
(medicinal plants) in Vietnam by developing an application for Vietnamese medicinal
plant retrieval based on plant identification. To this end, the concrete objectives are:
❼
Develop a new method for leaf-based plant identification that is able to recognize
the plants of interest even in complex background images;
❼
Propose a fusion scheme in multiple organ plant identification;
❼
Develop image-based plant search module in Vietnamese medicinal plant retrieval
application.
1
Contributions
The dissertation has three main contributions as follows:
❼
Contribution 1: A complex background leaf-based plant identification method
has been proposed. The proposed method combines the advantages of inter-
active segmentation that helps to determine the leaf region with very few user
interactions and the representative power of Kernel Descriptor (KDES).
❼
Contribution 2: One fusion scheme for two-organ based plant identification
has been introduced. The fusion is an integration between a product rule and a
classification-based approach.
❼
Contribution 3: Finally, an image-based plant search module has been devel-
oped and deployed in Vietnamese medicinal plant retrieval application named
VnMed.
Dissertation outline
❼
Introduction: This section describes the main motivations and objectives of the
study. We also present critical points the research’s context, constraints and
challenges. Additionally, the general frame-work and main contributions of the
dissertation are also presented.
❼
Chapter 1: A Literature Review: This chapter mainly surveys existing works and
approaches proposed for automatic plant identification.
❼
Chapter 2: In this chapter, a method for plant identification based on leaf image
is proposed. In the proposed method, to extract leaf region from images, we
proposed to apply interactive segmentation. Then, the improved KDES (Kernel
DEScriptor) is employ to extract leaf characteristic.
❼
Chapter 3: This chapter focuses on multi-organ plant identification. We have
proposed different strategy for determining the result of multi-organ identification
based on those of single-organ ones.
❼
Chapter 4: In this chapter, we propose a method for organ detection and the
use of this method for developing an application for Vietnamese medicinal plant
retrieval system.
❼
Conclusion: We give some conclusions and discuss the limitations of the proposed
method. Research directions are also described for future works.
2
CHAPTER 1
LITERATURE REVIEW
1.1 Plant identification from images of single organ
There are a large number of automatic plant identification methods. Among
different organs of the plant, leaf is the most widely used [4] because leaf usually exists
in a whole year. The identification results on leafscan often give the best results when
compared with other organs [5]. The popular organ is flower because its appearances
(e.g., color, shape, texture) are highly distinguishing [6]. In addition, other organs are
used to identify plant such as fruit, stem and branch. There are two main approaches
for the plant identification based on image of the plant organs. The first one uses the
hand-designed feature-based while the second one employs the deep learning method.
Hand-designed feature-based method consists of main stages: training and test-
ing. Each stage consists of four main components: image acquisition, preprocessing,
feature extraction and classification [7]. Feature extraction can be considered the most
important component in system. The purpose of extracting features is reducing the
dimensionality of the data and good representation of that data. Features include
global (color, texture, shape) and local features (organ-specific). For example, leaf has
an organ-specific feature such as leaf vein structure, leaf margin, tooth. Shape of leaf
plays the most important role [4]. Shape and color are important features for a flower.
Previous studies often combine two or more feature types for each organ because there
is no single feature strong enough to separate all categories.
The second employs deep learning methods. Recently, learning feature represen-
tations using a Convolutional Neural Networks (CNN) show a number of successes in
different topics in the field of computer vision such as object detection, segmentation,
and image classification [8]. CNN can automatically learn the features. Each layer
extracts features from the output of the previous layer. The first layers in the network
are very simple to extract for lines, curves, or blobs in the input image. This infor-
mation will be used as input for the next layer, with the task more difficult to extract
the components of the object in the image. Finally, the highest classes in the training
network will receive the task of classifying objects in the image. Typically CNNs are
AlexNet, VGG, GoogLeNet and ResNet. The teams utilizing deep learning techniques
are top winners in LifeCLEF competition.
3
