This page contain some documentation about the use of the MPI support, available since the OTB 5.6 release.
How does it works?
This section describes how the magic operates.
Message Passing Interface (MPI) in OTB
MPI is a communication standard for nodes composing a cluster. Basically, instead of using one node (that is, a computer) to write an image, an entire cluster can be used.
Writing data in parallel
Currently, OTB uses MPI to write an image using multiples nodes, including writing data and workload balancing, for every pipeline supporting streaming. Currently, depending of the way OTB has been built, there is two options to write one image using multiple nodes:
- Writing multiples tiles (using output image file extension .vrt) and a VRT file (OTB_USE_MPI must be set to ON),
- Writing one GeoTiff (using output image file extension .tif) in parallel (OTB_USE_SPTW must be set to ON) using the SPTW library.
Performances and limits
The speedup is defined by the ratio between the elapsed time of a process on one processing node, and the elapsed time fo the same process parallelized over N processing nodes. For one MPI Writer, the speedup provided by the MPI support is maximized if the entire upstream process supports streaming. Theoretically, the speedup can be linear. However, the I/O burden limits considerably the approach when the processing time is epsilon VS the writing time. But, hey, we are not using the MPI support to compute NDVI here... See this paper for more details.
Which OTB applications benefit from it?
This section presents a list of applications that currently benefit from the MPI support . Here is a list of the OTB 6.0 applications that supports natively the parallel approach:
otbcli_BandMath otbcli_BinaryMorphologicalOperation otbcli_BlockMatching otbcli_BundleToPerfectSensor otbcli_ComputeModulusAndPhase otbcli_ConcatenateImages otbcli_Convert otbcli_Despeckle otbcli_DisparityMapToElevationMap otbcli_EdgeExtraction otbcli_ExtractROI otbcli_FineRegistration otbcli_FusionOfClassifications otbcli_GrayScaleMorphologicalOperation otbcli_GridBasedImageResampling otbcli_HaralickTextureExtraction otbcli_ImageClassifier otbcli_LocalStatisticExtraction otbcli_MeanShiftSmoothing otbcli_Mosaic otbcli_MultiResolutionPyramid otbcli_MultivariateAlterationDetector otbcli_OpticalCalibration otbcli_OrthoRectification otbcli_Pansharpening otbcli_PredictRegression otbcli_Quicklook otbcli_RadiometricIndices otbcli_Rescale otbcli_RigidTransformResample otbcli_SARCalibration otbcli_SARDecompositions otbcli_SARPolarMatrixConvert otbcli_SARPolarSynth otbcli_SFSTextureExtraction otbcli_SarRadiometricCalibration otbcli_Smoothing otbcli_SplitImage otbcli_StereoFramework otbcli_StereoRectificationGridGenerator otbcli_Superimpose otbcli_TileFusion