Request for Changes-7: Regression mode in OTB and OTBApp

From OTBWiki
Jump to: navigation, search

[Request for Changes - 7] Regression mode in OTB and OTBApp

Status

Summary

The purpose of this RFC is to :

  • add regression support in machine learning models (classes deriving otb::MachineLearningModel)
  • support regression mode in existing classification filters
  • add 2 applications dedicated to regression training and prediction.
  • test regression models with arbitrary functions


Rationale

Most of classifiers used in OTB (coming from OpenCV or LibSVM) also support a regression mode : predict a numeric output value instead of a class label. However this mode is not supported yet in OTB (see [1]).

Some work has already been done by Jordi after the Hackfest 2015 (see d413ed56da238425cbee022dfbadf2238a0d4b19 ).

Implementation details

Classes and files

Modifications in module OTBSupervised

M       Modules/Learning/Supervised/include/otbBoostMachineLearningModel.h
M       Modules/Learning/Supervised/include/otbBoostMachineLearningModel.txx
M       Modules/Learning/Supervised/include/otbDecisionTreeMachineLearningModel.h
M       Modules/Learning/Supervised/include/otbDecisionTreeMachineLearningModel.txx
M       Modules/Learning/Supervised/include/otbGradientBoostedTreeMachineLearningModel.h
M       Modules/Learning/Supervised/include/otbGradientBoostedTreeMachineLearningModel.txx
M       Modules/Learning/Supervised/include/otbImageClassificationFilter.txx
M       Modules/Learning/Supervised/include/otbKNearestNeighborsMachineLearningModel.h
M       Modules/Learning/Supervised/include/otbKNearestNeighborsMachineLearningModel.txx
M       Modules/Learning/Supervised/include/otbLibSVMMachineLearningModel.h
M       Modules/Learning/Supervised/include/otbLibSVMMachineLearningModel.txx
M       Modules/Learning/Supervised/include/otbMachineLearningModel.h
M       Modules/Learning/Supervised/include/otbMachineLearningModel.txx
M       Modules/Learning/Supervised/include/otbNeuralNetworkMachineLearningModel.h
M       Modules/Learning/Supervised/include/otbNeuralNetworkMachineLearningModel.txx
M       Modules/Learning/Supervised/include/otbNormalBayesMachineLearningModel.h
M       Modules/Learning/Supervised/include/otbNormalBayesMachineLearningModel.txx
M       Modules/Learning/Supervised/include/otbRandomForestsMachineLearningModel.h
M       Modules/Learning/Supervised/include/otbRandomForestsMachineLearningModel.txx
M       Modules/Learning/Supervised/include/otbSVMMachineLearningModel.h
M       Modules/Learning/Supervised/include/otbSVMMachineLearningModel.txx

In a nutshell :

  • otb::MachineLearningModel : go back to simplified interface with only Train() and Predict() methods, an exception is raised in SetRegressionMode if one tries to enable regression on a model that doesn't support it (see 1b0ea6d498f3ca16ec28633afe855eafe5d07e71). The only change in the public interface of this class is the getter/setter for the regression flag.
  • Implement regression mode for all models except NormalBayes and Boost
  • otb::ImageClassificationFilter : disable confidence index when regression is ON.
  • otb::KNearestNeighborsMachineLearningModel : for regression mode, supports two decision rules : mean and median.

Applications

Dedicated learning application base class
 
D       Modules/Applications/AppClassification/app/otbTrainBoost.cxx
D       Modules/Applications/AppClassification/app/otbTrainDecisionTree.cxx
D       Modules/Applications/AppClassification/app/otbTrainGradientBoostedTree.cxx
D       Modules/Applications/AppClassification/app/otbTrainImagesClassifier.h
D       Modules/Applications/AppClassification/app/otbTrainKNN.cxx
D       Modules/Applications/AppClassification/app/otbTrainLibSVM.cxx
D       Modules/Applications/AppClassification/app/otbTrainNeuralNetwork.cxx
D       Modules/Applications/AppClassification/app/otbTrainNormalBayes.cxx
D       Modules/Applications/AppClassification/app/otbTrainRandomForests.cxx
D       Modules/Applications/AppClassification/app/otbTrainSVM.cxx
A       Modules/Applications/AppClassification/include/otbLearningApplicationBase.h
A       Modules/Applications/AppClassification/include/otbLearningApplicationBase.txx
A       Modules/Applications/AppClassification/include/otbTrainBoost.txx
A       Modules/Applications/AppClassification/include/otbTrainDecisionTree.txx
A       Modules/Applications/AppClassification/include/otbTrainGradientBoostedTree.txx
A       Modules/Applications/AppClassification/include/otbTrainKNN.txx
A       Modules/Applications/AppClassification/include/otbTrainLibSVM.txx
A       Modules/Applications/AppClassification/include/otbTrainNeuralNetwork.txx
A       Modules/Applications/AppClassification/include/otbTrainNormalBayes.txx
A       Modules/Applications/AppClassification/include/otbTrainRandomForests.txx
A       Modules/Applications/AppClassification/include/otbTrainSVM.txx

The main modifications are :

  • Create a base application class : otb::LearningApplicationBase (for TrainImagesClassifier and TrainRegression)
    • this class handles the creation of all classification parameters ( -classifier.*)
    • this class provides generic Train() and Classify() methods that work on list samples.
    • the class is templated over input and output values
  • The classifier specific routines from app/otbTrain*.cxx are moved to include/otbTrain*.txx and they implement parts of the otb::LearningApplicationBase (see c6d666a1dc6c1fdcb4fd7469b16ed36e68c579b9 and 9d2ba18add8377c32bae973a8415d6712cec0a87).
  • Additional parameters exposed (only in regression mode) to tune the models
Adapt existing application to new framework
M       Modules/Applications/AppClassification/app/otbTrainImagesClassifier.cxx

Modifications :

New applications for regression
M       Modules/Applications/AppClassification/app/CMakeLists.txt
A       Modules/Applications/AppClassification/app/otbPredictRegression.cxx
A       Modules/Applications/AppClassification/app/otbTrainRegression.cxx

Modifications :

Tests

In module OTBSupervised
M       Modules/Learning/Supervised/test/CMakeLists.txt
A       Modules/Learning/Supervised/test/otbMachineLearningRegressionTests.cxx
M       Modules/Learning/Supervised/test/otbSupervisedTestDriver.cxx
M       Modules/Learning/Supervised/test/otbTrainMachineLearningModel.cxx
M       Modules/Learning/Supervised/test/tests-libsvm.cmake
M       Modules/Learning/Supervised/test/tests-opencv.cmake

Tests have been added for validate the machine learning models. They are tested against predifined functions (see otbMachineLearningRegressionTests.cxx):

  • monovariate affine function
  • bivariate linear function
  • monovariate polynomial function
In module OTBAppClassification
M       Modules/Applications/AppClassification/test/CMakeLists.txt

Two basic tests have been added for applications TrainRegression and PredictRegression.

See dashboard page : http://dash.orfeo-toolbox.org/buildSummary.php?buildid=201285

Documentation

The base application class otb::LearningApplicationBase has been documented to explain the new architecture : see c88bcd17e6caadf235dd72068e0f5c0c4d54debe.

A CookBook section has been added in Recipes/Classification : ac6e6b716be21530d9803841aa6573eaf1b6b3d9

Additional notes