Request for Changes-26: MPI Image File Writer
[Request for Changes - 24] MPI Image writing
- Author: Rémi Cresson
- Additional contributors: Julien Michel, Emmanuelle Sarazin
- Submitted on 11.02.2016
- Proposed target release 5.6
- Branch: https://git.orfeo-toolbox.org/otb.git/shortlog/refs/heads/rfc-26-mpi_writer
Provide a set of parallel image file writers:
- MPITiffWriter, that use MPI-IO to write one single Geotiff image simultaneously
- MPIVrtWriter, that writes multiple files simultaneously then produce a VRT
Seamlessly use those writers instead of the standard one when calling an application with mpirun command.
Use OTB on HPC architectures
Classes and files
New third party modules
- MPI (Message Passing Interface)
- SPTW (Simple Parallel Tiff Writer) developed by David Matthew Mattli at USGC (current maintainer is Kornelijus Survila). Pull request submitted on 15.02.2016 (to make SPTW writing stripped images,only tiled writing was implemented). Currently used is my fork of SPTW.
New group of modules: MPI
A Modules/MPI/MPIConfig/test/CMakeLists.txt A Modules/MPI/MPIConfig/test/otbMPIConfigTestDriver.cxx A Modules/MPI/MPIConfig/test/otbMPIConfigTest.cxx A Modules/MPI/MPIConfig/include/otbMPIConfig.h A Modules/MPI/MPIConfig/otb-module.cmake A Modules/MPI/MPIConfig/CMakeLists.txt A Modules/MPI/MPIConfig/src/CMakeLists.txt A Modules/MPI/MPIConfig/src/otbMPIConfig.cxx A Modules/MPI/MPIConfig/README A Modules/MPI/MPIVrtWriter/test/CMakeLists.txt A Modules/MPI/MPIVrtWriter/test/otbMPIVrtWriterTestDriver.cxx A Modules/MPI/MPIVrtWriter/test/otbMPIReadWriteTest.cxx A Modules/MPI/MPIVrtWriter/include/otbMPIVrtWriter.h A Modules/MPI/MPIVrtWriter/otb-module.cmake A Modules/MPI/MPIVrtWriter/CMakeLists.txt A Modules/MPI/MPITiffWriter/test/otbMPISPTWReadWriteTest.cxx A Modules/MPI/MPITiffWriter/test/CMakeLists.txt A Modules/MPI/MPITiffWriter/test/otbMPITiffWriterTestDriver.cxx A Modules/MPI/MPITiffWriter/include/otbSimpleParallelTiffWriter.txx A Modules/MPI/MPITiffWriter/include/otbSimpleParallelTiffWriter.h A Modules/MPI/MPITiffWriter/otb-module.cmake A Modules/MPI/MPITiffWriter/CMakeLists.txt
- MPIConfig module encapsulates basic MPI operations (initialization, errors, etc).
- MPITiffWriter module writes one GeoTiff file with multiple processes.
- MPIVrtWriter module writes multiple files, then creates a VRT.
Changes in Wrapper module
Changes are made to seamlessly parallelize all image files writing of existing applications either with MPITiffWriter or MPIVrtWriter, depending on the extension of the output image file. This means that any output image parameter can be written in parallel with MPI, wihtout changing the code of the application.
Tests added for all three new modules in group MPI.
Tests consists in comparing one image written with the original otb::ImageFileWriter, VS images produced with the MPI image file writer using different tiling scheme.
Git branch is added to tested feature branches. Among dashboard platforms, pc-christophe and hulk have the MPI libraries installed and MPI modules enabled, while the others do not.
- What kind of documentation do we need for this feature ? Section in Software Guide ? Wiki page ? A short section in Cookbook could be useful.
Remaining issues are:
- Copyright of MPIVrtWriter should be IRSTEA (Remi Cresson). Copyright notice should be updated accordingly.
- Licence of SPTW is said "Public Domain" on github. Is this valid for code provenance review and/or debian packaging ?
- Both MPIVrtWriter and MPITiffWriter include gdal headers in their headers. Which means that binary packages built with MPI ON will depend on gdal headers. Probably not very important since MPI is mainly intended for cluster arch, where people will rebuild OTB themselves anyway.
- Shall we add OpenMPI to Superbuild ? Probably not.
Not addressed by this RFC:
- MPI-based persistent filters, for global statistics computation
- MPI-based segmentation