Request for Comments-18: Factorize CMake configuration
- 1 Status
- 2 Content
- 2.1 What changes will be made and why they would make a better Orfeo ToolBox?
- 2.2 When will those changes be available (target release or date)?
- 2.3 Who will be developing the proposed changes?
- 3 Community
- 4 Corresponding Requests for Changes
- Author: Victor Poughon
- Submitted on 04.12.2015
- Open for comments
What changes will be made and why they would make a better Orfeo ToolBox?
This RFC concerns otb-devutils only.
Currently, we are using cmake and ctest for nightly integration and testing of multiple projects accross multiples machines. Each build server uses its own configuration and its own builds, with some shared code in otb_common.cmake. With M build configurations and N build servers, our present setup requires N*M cmake configuration files. The main objective of this RFC is to reduce that number to N+M by separating machine specific and build specific configuration.
Problems identified with the current setup:
- Script maintenance takes a lot of time
- Adding or editing a build configuration requires editing as many files as there are build servers, making maintenance costly and error prone.
- Dashboard breaks often
- Dependency upgrade is troublesome (file paths break, etc.)
- A lot of configuration is copy-pasted across builds and test sites, and should be factorized whenever possible
Summary of tasks
- Reorganize otb-devutils/Config into otb-devutils/Config/Sites/ and otb-devutils/Config/Builds/
- Modify otb_common to be the new ctest entry point (done in proof of concept branch)
- Migrate all builds
- Enable more tests by easily composing site and builds (i.e. we can have each server run all available builds, with exceptions)
- Remove the need for platform specific bash scripts with build lists
A build: A configuration specifying which project is compiled and tested (otb, ice, etc.) and all its configuration (compiler, flags, dependencies, Release or Debug, trunk or stable, etc.). Examples of possible builds:
A test site: Configuration of a specific server or machine running builds. This configuration can include dashboard root location, dependencies install location, etc. Examples:
A build list: A list of builds indicating which build a test site will run. Ideally, all test sites should run all builds, but this may not be possible for various reasons (dependency not available, windows vs linux, etc). This enables:
- Selecting which builds to run at a given site by composing site config and build config
- Making multiple lists. For example, nightly, weekly, continuous, etc.
Configuration files structure
With the modifications proposed by this RFC, otb-devutils/Config would contain:
Lists/ nightly-default.txt weekly-default.txt nightly-raoul.txt nightly-pc-christophe.txt Sites/ default.cmake pc-christophe.cmake hulk.cmake raoul.cmake ... Builds/ OTB/ stable.cmake trunk.cmake clang.cmake gcc-x.x.cmake c++11.cmake ThirdPartyTrunk.cmake ... Ice/ stable.cmake clang.cmake trunk.cmake ... ITK/ stable.cmake trunk.cmake OpenJPEG/ Muparserx/
Standard dashboard layout
This RFC requires a standard dashboard layout common to build servers, located at the configurable CTEST_DASHBOARD_ROOT.
sources/ otb otb-devutils otb-documents ice monteverdi itk opencv openjpeg ... build/ otb/ stable trunk c++11 ... muparserx/ stable trunk ... ... install/ otb/ stable trunk c++11 ... muparserx/ stable trunk trunk-c++98 ... ... logs/ 2015-11-30/ otb/ stable/ ... itk/ ...
Composing site and build config is achieved by using otb_common.cmake as the entry point to ctest. Two variables containing paths to the site and build configuration files are given to otb_common with ctest -D flag:
ctest -S otb_common.cmake -DCTEST_OTB_SITE_CONFIG:PATH=Sites/site.cmake -DCTEST_OTB_BUILD_CONFIG:PATH=Builds/build.cmake
These files define macros (dashboard_hook_init_site and dashboard_hook_init_build) which are then called at the appropriate time in otb_common. Build scripts sizes are minimized by factorizing what can be into otb_common (with possibly defaults), otherwise into site specific configuration (e.g. Sites/raoul.cmake, Sites/pc-christophe.cmake, etc.).
Proof of concept branch
I pushed a proof of concept in otb-devutils branch called factor_config_proof_of_concept.
Using this branch, running the following command will build OTB/nightly.cmake with Sites/pc-christophe.cmake configuration, and submit to dashboard.
ctest -S otb_common.cmake -V -DCTEST_OTB_SITE_CONFIG:PATH=Sites/pc-christophe.cmake -DCTEST_OTB_BUILD_CONFIG:PATH=Builds/OTB/nightly.cmake
When will those changes be available (target release or date)?
To be determined.
Who will be developing the proposed changes?
Me, but help is welcome.
List here important comments (possibly links) received.
List here community members that support this RFComments.
Corresponding Requests for Changes
List here links to corresponding Requests for Changes if any.