Request for Comments-21: Optimised feature extraction on sparse entities

From OTBWiki
Jump to: navigation, search

Status

  • Author: Jordi Inglada
  • Submitted on 2016/01/22
  • Open for comments

Content

What changes will be made and why would they make a better Orfeo ToolBox?

This RFC is about the computation of features (radiometric, local statistics, textures, etc.) over sparse subsets of images (individual pixels, polygons, lines, etc.).

A use case for this functionality is for instance computing features only for the extracted samples using the New sampling module for the classification framework . This may be very useful for benchmarking classifiers with very big image data. In these cases, training and validation are performed on a small subset of the image data and computing image features on all pixels can be very expensive.

We could imagine something like the itk::UnaryFunctorImageFilter which instead of iterating over all the pixels of the image, just iterates over the selected samples (and writes samples instead of images). Then, it would be up to the user to write the appropriate functor to compute the features he wants.

Julien points out that we already have objects that are capable of doing this: they are called ImageFunction and are used to compute heavy features in the object detection framework.

We should pay attention to deriving a smart design to avoid code duplication between ImageFunction and Functor/Filter implementing the same feature (and we need the latter when you decide to actually process the whole image). For the record, we currently have features that are only available as ImageFunction and features that are only available as Functor/Filter.

If we take this one step further, we would also need to be able to compute some features for non ponctual geometries (polygons, lines) which would bring OBIA in the game.

The ImageFunction approach has efficiency issues: Evaluate and EvaluateAtIndex have to set the location of the iterator and then get the pixel, which is inefficient when scanning an image.

Also, having to write an ImageFunction class is a lot of boilerplate compared to a simple functor which implements operator(PixelType p).

Avoiding code duplication

Maybe we could use a FunctorImageFunction class which contains all the infrastructure (Evaluate, EvaluateAtIndex, EvaluateAtContinuousIndex, etc.) of an ImageFunction and which is templated over the FunctorType. Then, code duplication could be avoided by implementing the functor only, and the same functor could be used by an ImageFilter and by an ImageFunction.

A simple feature could be written inside a functor:

struct FeatureFunctor{
  PixelType operator()(PixelType p)
  {
   // compute feature and return its value
  }
}

This functor can be used by an UnaryFunctorImageFilter. We could also use it as a template parameter to a generic ImageFunction class:

typedef otb::FunctorImageFunction< ImageType, FeatureFunctor, double>  ImageFunctionType;


The Evaluate methods of the image function could be implemented like this:

template< typename TInputImage, typename TFunctor, typename TCoordRep >
typename FunctorImageFunction< TInputImage, TFunctor, TCoordRep >
::PixelType
FunctorImageFunction< TInputImage, TFunctor, TCoordRep >
::EvaluateAtIndex(const IndexType & index) const
{
 // Set the iterator at the desired location
 it.SetLocation(index);
 return m_Functor( it.GetPixel() );
 }

Some features need a local neighborhood for computation (textures), therefore a small set of classes for dealing for different needs may be necessary.

Classes
  • FunctorImageFunction to decorate functors with the image function interface
  • SparseSampleIterator to iterate over a sparse set of positions in an image
  • SparseSampleFeatureComputation to generate a sample list (same format as in the sampling framework) with the computed features
Applications

New applications to be developed:


Perspectives

When will those changes be available (target release or date)?

Target release is after 5.4.

Who will be developing the proposed changes?

TBD

Community

Comments

Support

Corresponding Requests for Changes