Ph. D. Project
Feasibility of the 3D hollow organ cartography using 2D endoscopic images
2017/10/01 - 2020/09/30
Medical context.
Endoscopy is the gold standard for the detection of cancerous or inflammatory lesions in hollow organs like
the bladder (in urology the endoscopes are cystoscopes) or the stomach (in gastroscopy the endoscopes are
gastroscopes). Endoscopes provide clinicians with small high resolution field of view (FoV) images. Since such
limited FoV images only partially visualize regions of interest, they do not facilitate lesion diagnosis and patient
follow-up. Moreover, the endoscopic video-sequences are difficult to interpret after the cystoscopy or the
gastroscopy. This fact is a barrier for data archiving and examination traceability.

Scientific context (image processing).
Image mosaicing can be used to address the above mentioned medical problems. The principle of image
mosaicing is to build images with an extended FoV (panoramic image or mosaic) by superimposing the
common parts of the small FoV images of a video-sequence. More precisely, the image mosaicing process
consists of several steps, namely i) in finding the correspondence between homologous points of image pairs,
ii) in the use of this correspondence knowledge to find the geometrical link between images (image
registration), iii) in the placement of all pixels or images in an unique and common mosaic coordinate system
(image stitching) and iv) in the correction of texture or colour discontinuities in the mosaic. However,
endoscopic data are affected by strong illumination changes (e.g. depending on the viewpoint), specular
reflexions and blur, and the epithelium of the inner bladder or stomach walls have only weakly contrasted
textures. These scene characteristics explain why endoscopic image mosaicing, and especially the point
correspondence establishment step, are challenging. The CRAN laboratory has a recognized experience in
endoscopic image mosaicing. Homologous point correspondence establishment was notably proposed with
global [1, 2] and local [4] optical flow methods and a graph-cut [3] method.

Thesis work
Although the bi-dimensional (2D) mosaics built with the algorithms proposed at the CRAN laboratory
represent a real advance in terms of diagnosis, patient follow and data archiving, the potential of image
mosaicing techniques can still be better exploited. Indeed, due to the geometrical distortion occurring when
placing images in 2D mosaics, large or complete organ parts cannot be entirely represented by an unique
panoramic image. Moreover, only the first image of a mosaic has the original image resolution, the resolution
of the other images placed in the mosaic depends on the endoscope trajectory in the hollow organ. Such
problems can be avoided by building tri-dimensional (3D) mosaics (surfaces superimposed by the textures of
the 2D images). 3D image mosaicing is the aim of this thesis. In this thesis, the feasibility of Structure from
Motion (SfM) and/or Simultaneous Localization and Mapping (SLAM) techniques will be studied in the case of
endoscopic data. SfM and SLAM techniques determine simultaneously the surface shape and the camera (here
an endoscope) trajectory using 2D image information (and intrinsic camera parameters) only. These 3D
mosaicing methods are based on optimization techniques exploiting the point correspondence between 2D

Publication of the CRAN laboratory in the field of image mosaicing
1] Sharib Ali, Christian Daul, Ernest Galbrun, François Guillemin and Walter Blondel, Anisotropic motion
estimation on edge preserving Riesz wavelets for robust video mosaicing, Pattern Recognition, vol.51, pages
425-442, March 2016.
[2] Sharib Ali, Christian Daul, Ernest Galbrun and Walter Blondel, Illumination invariant optical flow using
neighborhood descriptors, Computer Vision and Image Understanding, Accepted - to be published in 2016.
[3] Thomas Weibel, Christian Daul, Didier Wolf, Ronald Rösch, François Guillemin. Graph based construction of
textured large field of view mosaics for bladder cancer diagnosis, Pattern Recognition, vol. 45, issue 12, pages
4138-4150, 2012.
[4] Yahir Hernandez-Mier, Walter Blondel, Christian Daul Didier Wolf, François Guillemin, Fast construction of
panoramic images for cystoscopic exploration, Computerized Medical Imaging and Graphics, vol. 34, issue 7,
pages 579-592, 2010.
Image processing, 3D mosaicing, Structure from Motion,SLAM, optical flow,registration, endoscopy
Duration ; 3 years.
Host laboratory :CRAN (Centre de recherche en automatique de Nancy)
Address : CRAN-ENSEM-UL,2 avenue de la Forêt de Haye, 54518 VANDOEUVRE-LES-NANCY CEDEX

Candidate profile
The candidate should ideally have a master in image processing/computer vision or a master in applied
Biology, Signals and Systems in Cancer and Neuroscience
Grant of CONACYT