Connected component segmentation module

From OTBWiki
Jump to: navigation, search

The Connected Component Segmentation Module Framework

The aim of this module is to perform connected component based segmentation with user defined segmentation criteria.

The Processing chain can be divided in following steps :

  • Binary Mask Creation. This mask is used as support of Connected components segmentation (CC) . i.e zeros pixels are not taken into account by CC algorithm. Binarization criteria is defined by user, via muparser . This step is optional, if no Mask is given, entire image will be processed.
  • Connected Component Segmentation Connected Component segmentation driven by user defined criteria.
  • Labelization and Object Characterization Connected Component Label Image is used to create label map. each Label object is characterized by shape and statistical criteria.
  • Object Analysis post processing. This step consists in post processing on each detected area.
  • Output vectorization. Finally the results are exported in VectorData shape format.

Binary Mask Creation

A Binary mask can be used as support mask for connected component segmentation. Each pixel, which value is different from 0 is taken into account in segmentation step. If no mask given, the whole image is processed. Support Mask can be given by different ways.

Mu parser based filter : MaskMuParserFilter is dedicated to the mask creation.

A standalone application named MaskGenerator is available :

otbMaskGenerator -in InputImageFileName -out OutputMaskFileName -expression Mu Parser formula


In CC Application, Mu Parser formula has to be used ( see command line examples for further details .)

MuParser Formula, available Variables and Functions

The equation is applied for each pixel image taken individually.

  • variables :
    • each band is denoted b1,b2...,bn , which correspond to band1,band2,...,bandn.
    • intensity of each pixel is accessible by intensity (arithmetic mean of pixel band).
  • fonctions (... means variable length list of variables):
    • ndvi(r,niri)
    • abs()
    • acos()
    • acosh()
    • asin()
    • asinh()
    • atan()
    • atanh()
    • avg(...)
    • cos()
    • cosh()
    • exp()
    • if(cond,true,false)
    • ln()
    • log()
    • log10()
    • log2()
    • max(...)
    • min(...)
    • ndvi(r,niri)
    • rint()
    • sign()
    • sin()
    • sinh()
    • sqrt()
    • sum(...)
    • tan()
    • tanh()

Connected Component Segmentation

Connected component segmentation is based on itk class ConnectedComponentImageFilter. This filter needs a functor to define adjacent connected condition. A muParser test expression is used.

Available variables

  • distance : Euclidian intensity pixel distance between two adjacent pixels.
  • intensity_p1 (intensity_p2) : mean intensity of first pixel (resp. second pixel).
  • pXbY : Band Y value of pixel X.
  • spectralAngle : Spectral angle between adjacent pixels (first pixel is set as reference).

Labelization and Label Object Characterization

Connected Component output is is then relabeled, discarding too small objects. Eliminating too small objects at this step is needed to lighten further computation. minsize parameter determines the minimum object area. This new label image image is then transfomed into labelmap and each label object is characterized by shape and statistical attributes.

Computed attributes

Labeled image is, then, transformed to labelmap (label map adjacency in order to include fusion step in further developments). Each object is characterized by shape and radiometric attributes.

  • shape attributes :
    • SHAPE_Elongation
    • SHAPE_EquivalentEllipsoidRadius0
    • SHAPE_EquivalentEllipsoidRadius1
    • SHAPE_EquivalentPerimeter
    • SHAPE_EquivalentRadius
    • SHAPE_Flusser01
    • SHAPE_Flusser02
    • SHAPE_Flusser03
    • SHAPE_Flusser04
    • SHAPE_Flusser05
    • SHAPE_Flusser06
    • SHAPE_Flusser07
    • SHAPE_Flusser08
    • SHAPE_Flusser09
    • SHAPE_Flusser10
    • SHAPE_Flusser11
    • SHAPE_PhysicalCentroid0
    • SHAPE_PhysicalCentroid1
    • SHAPE_PhysicalSize
    • SHAPE_PhysicalSizeOnBorder
    • SHAPE_PrincipalAxis00
    • SHAPE_PrincipalAxis01
    • SHAPE_PrincipalAxis10
    • SHAPE_PrincipalAxis11
    • SHAPE_PrincipalMoments0
    • SHAPE_PrincipalMoments1
    • SHAPE_RegionElongation
    • SHAPE_RegionIndex0
    • SHAPE_RegionIndex1
    • SHAPE_RegionRatio
    • SHAPE_RegionSize0
    • SHAPE_RegionSize1
    • SHAPE_Size
    • SHAPE_SizeOnBorder
  • radiometric attributes (by band '*' should be replaced by band number )
    • STATS_b*_CenterOfGravity0
    • STATS_b*_CenterOfGravity1
    • STATS_b*_FirstMaximumIndex0
    • STATS_b*_FirstMaximumIndex1
    • STATS_b*_FirstMinimumIndex0
    • STATS_b*_FirstMinimumIndex1
    • STATS_b*_Kurtosis
    • STATS_b*_Maximum
    • STATS_b*_Mean
    • STATS_b*_Minimum
    • STATS_b*_PrincipalAxis00
    • STATS_b*_PrincipalAxis01
    • STATS_b*_PrincipalAxis10
    • STATS_b*_PrincipalAxis11
    • STATS_b*_PrincipalMoments0
    • STATS_b*_PrincipalMoments1
    • STATS_b*_Sigma
    • STATS_b*_Skewness
    • STATS_b*_Sum
    • STATS_b*_Variance

attributes in italics are available only if reduced set of attributes is set to off (by default small set of attributes is selected to enhance post processing delay). A complete description is available in ITK software guide. Below a description of these parameters :

  • Shape attributes
    • Size is the size of the object in number of pixels.
    • PhysicalSize is the size of the object in physical unit. It is equal to the Size multiplicated by the physical pixel size.
    • Centroid is the position of the center of the shape in physical coordinates. It is not constrained to be in the object, and thus can be outside if the object is not convex.
    • Region is the bounding box of the object given in the pixel coordinates. The physical coordinate can easily be computed from it.
    • RegionElongation is the ratio of the longest physical size of the region on one dimension and its smallest physical size. This descriptor is not robust, and in particular is sensitive to rotation.
    • SizeRegionRatio is the ratio of the size of the object region (the bounding box) and the real size of the object.
    • SizeOnBorder is the number of pixels in the objects which are on the border of the image. A pixel on several borders (a pixel in a corner) is counted only one time, so the size on border can’t be greater than the size of the object. This attribute is particulary useful to remove the objects which are touching too much the border.
    • PhysicalSizeOnBorder is the physical size of the objects which are on the border of the image. In 2D, it is a distance, in 3D, a surface, etc. Contrary to the PhysicalSize attribute which is directly linked to the Size, this attribute is not directly linked to the SizeOnBorder attribute. This attribute is particulary useful to remove the objects which are touching too much the border.
    • FeretDiameter is the diameter in physical units of the sphere which include all the object. The feret diameter is not computed by default, because of its high computation.
    • BinaryPrincipalMoments contains the principal moments.
    • BinaryPrincipalAxes contains the principal axes of the object.
    • BinaryElongation is the elongation of the shape, computed as the ratio of the largest principal moment by the smallest principal moment. Its value is greater or equal to 1. Its type is double.
    • EquivalentRadius is the equivalent radius of the hypersphere of the same size than the label object. The value depends on the image spacing.
    • EquivalentPerimeter is the equivalent perimeter of the hypersphere of the same size than the label object. The value depends on the image spacing.
    • EquivalentEllipsoidPerimeter is the size of the ellipsoid of the same size and the same ratio on all the axes than the label object. The value depends on the image spacing. Its type is
  • Statistics attributes
    • Minimum is the minimum value in the feature image for the object.
    • MinimumIndex is the index position in the image where the first minimum was found.
    • Maximum is the maximum value in the feature image for the object.
    • MaximumIndex is the index position in the image where the first maximum was found.
    • Mean is the mean of the pixel values in the object.
    • Sum is the sum of all the pixel values in the objects.
    • Sigma is the standard deviation of the pixels values in the objects.
    • Variance is the variance of the pixels values in the objects.
    • Median is the median of the pixels values in the obejct.
    • CenterOfGravity is the center of gravity of the object.
    • Kurtosis is the kurtosis of the pixel values in the objects.
    • Skewness is the skewness of the pixel values in the objects.
    • PrincipalMoments contains the principal moments.
    • PrincipalAxes contains the principal axes of the object.
    • Elongation is the elongation of the shape, computed as the ratio of the largest principal moment by the smallest principal moment. Its value is greater or equal to 1.
    • Histogram is the histogram of the pixels covered by the label object in the feature image.

OBIA Post Processing

In OBIA post processing step, object attributes are exploited to construct acception criteria based on MuParser functor for example "SHAPE_Elongation>3" for rejecting object with elongation below '3'

Command Line examples

  • Connected Component segmentation with acception criteria : distance between adjacent pixels is less than 40 . A mask is given (we keep only the pixels with value > 100 on band 1 and > 120 on band 2). After connected component segmentation, labeled areas with size in pixel below 15 are discarded. Then, OBIA step keeps only objects with Elongation greater than 8.
-in ${INPUTDATA}/ROI_QB_MUL_4.tif  -out test.tif -outshape out.shp -expression 'distance<40' -minsize 15 -OBIAexpression 'SHAPE_Elongation>8'
-maskexpression '(b1>100) *(b2>120)'
  • Same as previous example, except that the whole input image is taken into consideration.
-in ${INPUTDATA}/ROI_QB_MUL_4.tif  -out test.tif -outshape out.shp -expression 'distance<40' -dbg -minsize 15 -OBIAexpression 'SHAPE_Elongation>8'

TODO List and Open issues

Please note that this framework is still in development. We would be glad to receive feedbacks on these tools.

TODO List

  • make the variables and expression more user friendly.
  • Tests are needed to determines interesting variable and attributes, and to test if attributes normalisation is needed.
  • Add spectral angle attribute in Mask and Object Analysis functor (choice of reference pixel ?)
TODO List Monteverdi
  • add label object and variables infos.
  • allow the use of external mask image.
  • process the whole image at the end of process. (tile process)
  • parse OBIA formula to compute only appropriate object attributes
  • add pixel description window in the application.
  • add helper with command line examples
  • add a progress reporting label near the update button.
  • change label of output "segmentation after small object c . Need a coherent name with the applications and with formula's labels
  • Improve labels quality-> not OBIA but "Object based analysis" or Geospatial analysis...etc
  • add the possibility to click on a labelObject in the visualization window and display values of features (elongation, stats...) to facilitate the choice of thresholds
  • Relabel color mapper problem.
  • It would be interesting to add customize object attributes calculation, for example using a config file.

Open Issues

  • It should be interesting to add an optionnal smothing filter (mean,gaussian ...) before the segmentation step.
  • An adjacency fusion step seems to be interesting before OBIA post filtering.
  • Add an option "keepNobjects" instead of binary post processing.
  • lot's of mu parser warnings in the console in the monteverdi module
  • full resolution window is not updated if I modify the formula and click
  • How to use spectral angle in the 2nd formula? -> possible to click on a pixel reference in the viewer? add reference pixel to compute spectral angle and euclidian distance
  • Use external Mask image
  • unable to perform process on the entire image if the input of the CC module is the output of the ExtractROI module -> monteverdi crash (abandon)
  • Sometimes input image is displayed with wrong colormap.
  • Viewer like visualization of each band.
  • View histogram bands.

Monteverdi Module

The monteverdi module available from "Filtering" menu.

Screenshot GUI v5.jpg

First user had to enter formulas :

  • Mask Formula : To use mask support in connected component segmentation. If this field is left blank no mask is used.
  • Connected Component Formula : functor test expression to accept or reject adjacent pixels connected condition
  • Object Analysis formula : use of shape and statistics object attributes to post process labeled objects. If this field is left blank the whole set of labeled objects is accepted.

Each formula is check in live, if formula is correct the background is colored in green, in red otherwise. In order to help user, all available variables are displayed on the right.

Some intermediate output are available using display choice box :

  • Input image : input image.
  • Mask Output : mask image created using formula.
  • Masked image : input image multiplied by mask image.
  • Segmentation Output : output of Connected component segmentation filter.
  • Segmentation after small object rejection : output of connected component segmentation after relabeling and small object rejection.
  • Filter Output : final output after OBIA opening post processing.

When a first pass have been done, Specific label object properties can be displayed. Select the "Filter Output" visualization mode, Update the visualization. Then use right click on selected object in image to display object properties.

Examples

Capture Boats.png
Capture Boats CC.png

Step by step example : Round area detection

In this example we want to detect plain round area like silo in multi band image.

The input image :

Round example input.jpg

We don't use mask (left the formula blank). Connected Component criteria is pixel intensity distance :

distance < 8

Then we reject small objects (pixel size below 90).

The temporary output after filtering and small objects rejection is presented below :

Round example relabeling.jpg

Finally we want to keep only round objects :

We use this object analysis formula :

(SHAPE_RegionRatio > 0.65) and ( SHAPE_SizeOnBorder==0)  

The first term characterize the roundness of the object, the second eliminate object on the border of image, which have bad characteristics.

Round example output.jpg

The Vectorized data projected on image is depicted in blue :

Round example VD1.jpg
Round example VD2.jpg