ITKv4 Migration Plan

From OTBWiki
Jump to: navigation, search

This page is intended to present the migration plan of OTB to the next major release of ITK (v4)

Planning

//TODO

Roadmap

  • Framework newStatistics
    • [done] Compile with the new ITK Statistics framework. There is a branch (newStatistics) in the SandBox http://hg.orfeo-toolbox.org/OTB-SandBox/shortlog/10926216ce6b
    • Refactoring of some classes using ListSample as input/ouput. Those classes inherits itk::ProcessObject in order to not break the pipeline
    • Learning
      • [OK] otbListSampleSource
      • [OK] otbConcatenateSampleListFilter
      • [OK] otbGaussianAdditiveNoiseSampleListFilter
      • [OK] otbListSampleGenerator
      • [OK] otbListSampleToBalancedListSampleFilter
      • [OK] otbListSampleToListSampleFilter
      • [OK] otbShiftScaleSampleListFilter
      • [OK] otbSVMMarginSampler --> Not tested (added Tu)
      • [OK] otbSVMSampleListModelEstimator
    • [OK]ObjectDetection[OK]
      • [OK] otbDescriptorsListSampleGenerator.h
      • [OK] otbLabeledSampleLocalizationGenerator.h
    • OBIA
      • [OK] otbLabelMapToSampleListFilter (method Compute() removed , replaced by GenerateData())
    • Refactoring some classes having itk::Histogram as input/output
    • BasicFilters
    • [OK] BasicFilters/otbListSampleToVariableDimensionHistogramGenerator
    • [OK] BasicFilters/otbListSampleToHistogramListGenerator
    • Common
    • [OK] Common/otbPathListToHistogramGenerator --> inherits ProcessObject DONE (Histogram Source needed, for MiniPipeline GraftOutput())
    • Common/otbHistogramStatisticsFunction --> /Utilities/ITK/Code/Algorithms/itkHistogramAlgorithmBase.h (problem not inheriting ProcessObject, use histogram an input)
    • Tests failing with difference about 1e-4 with baseline (http://dash.orfeo-toolbox.org/testDetails.php?test=2800273&build=40723)
    • Merge newStatistics branch in OTB-SandBox default branch.


OTB ITK patches

  • 3 patches left, kept in their branches :
  • OTB with ITKv4 support
    • managed on bitbucket : https://bitbucket.org/julienmalik/otb-itkv4
    • regularly rebased onto OTB master
    • when a rebase happen, a new head is pushed, and the bitbucket strip functionnality is used to strip the older branch. this gives the same functionnality as "git push -f"

Dashboard


Steps to set up a platform testing otb-itkv4 with ITKv4 patched (deprecated)

Description of the workflow with the corresponding commands

  • On the ITK side:
    • MAJ regularly ITK and rebase OTB patches :
      • git pull origin master (pull do the merge automatically)
      • git fetch github_julienmalik
      • git checkout OTB_ITKv4 (or checkout every otb branches wi=hich contains a patch -> OTB_ITKV4 is a working branch which containall the branches)
      • rebase on the master : git rebase master
      • Push to jkulmienmalik github : git push -f github_julienmalik OTB_ITKv4
  • On OTB Side
    • Rebase regularly the otb-itkv4 clone on julien malik bitbucket account with the OTB hg trunk
    • Push regularly to bitbucket -> push 2 heads
      • hg push -f
      • Need after to strip one head with the bitbucket interface -> use the hg in command to see the revision we have to script, which is the first commit in the terminal (risky operation! DOUBLE CHECK)

Set up ITK platform with OTB patches

ITK + patch for OTB are managed on github : git://github.com/julienmalik/ITK.git 

 + git clone git://itk.org/ITK.git 
 + git remote add github git@github.com:julienmalik/ITK.git 
 + git remote -vv 
 + git branch -vv 
   master e2705ed [origin/master] KWSys Nightly Date Stamp
 + socks git fetch github 
   remote: Counting objects: 148, done.
   remote: Compressing objects: 100% (38/38), done.
   remote: Total 107 (delta 72), reused 97 (delta 69)
   Receiving objects: 100% (107/107), 43.90 KiB, done.
   Resolving deltas: 100% (72/72), completed with 40 local objects.
   From github.com:julienmalik/ITK
    * [new branch]      DiffeomorphicDemonsRegistrationWithMaskFilter -> github/DiffeomorphicDemonsRegistrationWithMaskFilter
    * [new branch]      GDALImageIO -> github/GDALImageIO
    * [new branch]      OTBMissingHeaderForitkSize -> github/OTBMissingHeaderForitkSize
    * [new branch]      OTBMulticomponentsComplexForImageIO -> github/OTBMulticomponentsComplexForImageIO
    * [new branch]      OTBNumericTraitsForComplexCharAndShort -> github/OTBNumericTraitsForComplexCharAndShort
    * [new branch]      OTBWarpImageFilterVectorImage -> github/OTBWarpImageFilterVectorImage
    * [new branch]      OTB_ITKv4  -> github/OTB_ITKv4
    * [new branch]      master     -> github/master
    * [new branch]      nightly-master -> github/nightly-master
    * [new branch]      release    -> github/release
 + git branch -vv  -a 
   master
     remotes/github/DiffeomorphicDemonsRegistrationWithMaskFilter
     remotes/github/GDALImageIO
     remotes/github/OTBMissingHeaderForitkSize
     remotes/github/OTBMulticomponentsComplexForImageIO
     remotes/github/OTBNumericTraitsForComplexCharAndShort
     remotes/github/OTBWarpImageFilterVectorImage
     remotes/github/OTB_ITKv4
     remotes/github/master
     remotes/github/nightly-master
     remotes/github/release
     remotes/origin/HEAD -> origin/master
     remotes/origin/dashboard
     remotes/origin/hooks
     remotes/origin/master
     remotes/origin/nightly-master
     remotes/origin/release
 + create local branch to track otb_itkv4 remote branch 
   git checkout -b OTB_ITKv4 github/OTB_ITKv4  (or git checkout OTB_ITKv4 which also works as if)
   Branch OTB_ITKv4 set up to track remote branch OTB_ITKv4 from github.
   Switched to a new branch 'OTB_ITKv4'
 + git branch -a -vv 
   /* OTB_ITKv4                                                    7690a1a [github/OTB_ITKv4] Merge branch 'OTBMissingHeaderForitkSize' into OTB_ITKv4
     master                                                       e2705ed [origin/master] KWSys Nightly Date Stamp
     remotes/github/DiffeomorphicDemonsRegistrationWithMaskFilter 6251fdd ENH: add DiffeomorphicDemonsRegistrationWithMask
     remotes/github/GDALImageIO                                   1019d6b ENH: add ImageIO based on GDAL
     remotes/github/OTBMissingHeaderForitkSize                    487c315 COMP: Missing <cstring> include for memcpy in itkSize.h
     remotes/github/OTBMulticomponentsComplexForImageIO           abcdea2 ENH: Add support for multicomponent complex images in IO
     remotes/github/OTBNumericTraitsForComplexCharAndShort        79309e7 ENH: add missing NumericTraits for complex
     remotes/github/OTBWarpImageFilterVectorImage                 b54a7e1 ENH: Add support for VectorImage in WarpImageFilter
     remotes/github/OTB_ITKv4                                     7690a1a Merge branch 'OTBMissingHeaderForitkSize' into OTB_ITKv4
     remotes/github/master                                        49eb46c KWSys Nightly Date Stamp
     remotes/github/nightly-master                                e865347 Merge topic 'i18n-typo'
     remotes/github/release                                       64e398d Merge branch 'FixSettingMetricNumberOfThreads' into release
     remotes/origin/HEAD                                          -> origin/master
     remotes/origin/dashboard                                     e1253c4 DOC: Add LICENSE, NOTICE, and source header notices
     remotes/origin/hooks                                         5ebbe2d Set GIT_DIR if not already set
     remotes/origin/master                                        e2705ed KWSys Nightly Date Stamp
     remotes/origin/nightly-master                                fc927b6 KWSys Nightly Date Stamp
     remotes/origin/release                                       0041f5f Merge branch 'FixNrrddExport' into release
 + git checkout -b OTBWarpImageFilterVectorImage       github/OTBWarpImageFilterVectorImage 
   do this for all otb branches!!!
 + update master : git checkout master; git pull origin master 
 + rebase local otb branches on master : git checkout GDALIMageIO; git rebase master 
   do this for all orb branches
 + remove local OTB_ITKv4 : git branch -D OTB_ITKv4 
 + recreate branch : git checkout master; git checkout -b OTB_ITKv4 
 + merge all obt branches in OTB_ITKv4 : git merge --no-ff OTBWarpImageFilterVectorImage 
 + side note : delete a branch from github : git push github :name_of_branch 
 + push each OTB branches independantly 
   git push -f github OTBWarpImageFilterVectorImage
 + push OTB_ITKv4 branch now : 
   git push -f github OTB_ITKv4
 + How to update your local git when your local copy is outdated 
   - remove all local branches (OTB_ITKv4 included) : OTB git branch -D OTBWarpImageFilterVectorImage
   - git fetch github
 + Add new branch or rebase -> you need first to delete OTB_ITKv4 (integration branch) 
   - git branch -D OTB_ITKv4
 + add new branch 
   - git checkout master
   - git checkout -b OTBInverseDeformationField
   - apply your patch
   - git commit -a


Wrapping project

This section is intended to compile informations of development of ITK regarding bindings and future actions concerning the migration of the OTB-Wrapping project to the new ITK wrapping framework.

Open issues

  • external ITKv4 : error: itkPixelBuilder.h: No such file or directory
  • OBIA Library
    • LabelMap/LabelObject have been strongly refactored
    • LabelMap does not expose its internal container. Have to use iterators
    • LabelObject does not expose its internal LineContainer. Have to use iterators
    • no more PerimeterCalculator class. Code has been partially ported from ShapeAttributesLabelObject
  • Some OTB classes relies on ITKReview classes (otbStreamingResampleImageFilter for instance).
    • We have to handle the case ITK_USE_REVIEW set to OFF (and use the ITK_USE_REVIEW in OTB when necessary), or always use external ITK with ITK_USE_REVIEW=ON and throw an error in the configuration step if it is not.... ???
  • No patended code anymore in ITKV4, 2 examples cannot be compiled : FuzzyConnectednessImageFilter and HybridSegmentationVoronoi
  • Problem with OTB_USE_EXTERNAL_ITK : If OTB is first configured without giving a proper ITK_DIR, the internal ITK is used by default. It is still used even if we put OTB_USE_EXTERNAL_ITK=ON later when configuring with cmake gui (ghost cmake variable ITK_DIR in the CMakeCache)
  • in OTB, we have a test called ResampleSLCImage, trying to resample complex data. The test pass only ecause the transform is a translation with integer spacing.
  • IO registration has been refactored completely in ITK. They are now the responsability of the external project using ITK.
    • UseITK.cmake generates a header file in the build tree of OTB, in ITKIOFactoryRegistration, which loads the ITK IO enabled in the ITK build at OTB static init time.
    • This header file is included at the end of itkImageFileReader and itkImageFileWriter
    • Without further modifications on the OTB side, the result is that the ITK IO comes before the OTB IO, leading to problem on file formats like TIFF, PNG, JPEG, for which an ITK IO exists.
    • Same mechanism reproduced in OTB : the client application will have a OTBIOFactoryRegistration directory in its build tree, geenrated by UseOTB.cmake
    • To avoid static initialization order fiasco, the OTBIOFactoryRegistration header file loads both the OTB and ITK IO (in that order), and prevents the use of the ITK registration mechanism. This should do the trick as long as itkImageFileReader is not included before an OTB file. We should ask ITK to provide more control on that...


Open issues of internal ITK patches

changeset: 15458:3b714e3bf45d user: Julien Malik <julien.malik@orfeo-toolbox.org> date: Tue Jan 29 18:00:02 2013 +0100 summary: COMP: use this-> when calling a base class member function in templated class

changeset: 15179:b7a83cf6868e user: Julien Malik <julien.malik@orfeo-toolbox.org> date: Mon Nov 19 18:34:45 2012 +0100 summary: COMP: fix compilation issue on quantal

changeset: 15159:5977d2e794dd user: Julien Michel <julien.michel@orfeo-toolbox.org> date: Fri Nov 16 11:20:21 2012 +0100 summary: BUG: #418 fixing ld error again (2/2)

changeset: 15458:3b714e3bf45d user: Julien Malik <julien.malik@orfeo-toolbox.org> date: Tue Jan 29 18:00:02 2013 +0100 summary: COMP: use this-> when calling a base class member function in templated class

changeset: 15179:b7a83cf6868e user: Julien Malik <julien.malik@orfeo-toolbox.org> date: Mon Nov 19 18:34:45 2012 +0100 summary: COMP: fix compilation issue on quantal

changeset: 15159:5977d2e794dd user: Julien Michel <julien.michel@orfeo-toolbox.org> date: Fri Nov 16 11:20:21 2012 +0100 summary: BUG: #418 fixing ld error again (2/2)

changeset: 14525:a77643c801c9 user: Julien Malik <julien.malik@orfeo-toolbox.org> date: Tue Jun 05 11:44:25 2012 +0200 summary: BUG: itkWatershedSegmenter was generating a STB_GNU_UNIQUE symbol (cf bug #418), causing the Segmentation application to assert when unloaded

-> send patches to ITK -> don't need to check ITK internal patches before 09/19/2013

References

Informations about ITK v4

Information specific to the new statistics framework:

Retrieved from "https://wiki.orfeo-toolbox.org/index.php?title=ITKv4_Migration_Plan&oldid=6647"