opentl::modalities::TemplateMap Class Reference

Processing functions related to the template visual feature (cvdata::TemplateMap). More...

Inherits opentl::modalities::Modality.

List of all members.

Public Types
Public Member Functions
virtual modalities::Modality *	clone () const
	Clone (it calls the Deep Copy Constructor).
void	init ()
	Memory allocation and initialization (common to both constructors).
virtual int	matchFeatLevel (const TargetPtrVector &targets, T_MEAS_FEATPtrVector &outputMeas, std::size_t partitionIdx)
	matchFeatLevel. Match template gray level values with underlying image values (multi-resolution) The overall similarity function can be: Sum of Squared Differences (SSD), Normalized Cross-Correlation (NCC), Mutual Information (MI).
virtual int	matchObjLevel (const TargetPtrVector &targets, T_MEAS_OBJPtrVector &outputMeas, std::size_t partitionIdx)
	matchObjLevel. Find local maximum of feature-level likelihood (i.e. the template similarity function)
virtual int	matchPixLevel (const TargetPtrVector &targets, T_MEAS_PIXPtrVector &outputMeas, std::size_t partitionIdx)
	Matching on pixel level compare pixel-level image data with rendered texture image, by projecting model data under a predicted state.
virtual int	preProcess (const opentl::core::cvdata::Image &image, const std::vector< std::vector< int > > &preProcessROIs)
	Get the multi-resolution image pyramid.
void	resetOnlineTemplate (const TargetPtrVector &targets)
	Reset the on-line template to the off-line values.
virtual int	sampleModelFeatures (const TargetPtrVector &targets)
	Compute visible points from the template map at pose p (occlusion query).
void	showTemplateMap (const TargetPtrVector &targets, bool bOnline=false)
	Debug: show the template map, re-projected on the screen at all resolutions, with keyboard input (roto-translation).
	TemplateMap (const TemplateMap &tmap)
	Deep Copy Constructor.
	TemplateMap (opentl::modelprojection::WarpBack *warpBack, int camIdx, opentl::modelprojection::GLRenderer *renderer)
	Constructor.
virtual int	updateModelFeatures (const TargetPtrVector &targets)
	Update the on-line observed gray values (= feature) from the multi-resolution image pyramid.
virtual	~TemplateMap ()
	Destructor.
Static Public Attributes
static std::vector < std::vector < boost::shared_ptr < opentl::core::cvdata::Image > > >	mTemplateImageData
	Pre-processed data (D) = multi-resolution image pyramid 1st index: camera idx 2nd index: resolution.
static std::vector < std::vector < boost::shared_ptr < opentl::core::cvdata::Image > > >	mTemplateImageXGrad
	Pre-processed data (D) = Sobel filter (image gradients), computed only if Jacobians are needed.
static std::vector < std::vector < boost::shared_ptr < opentl::core::cvdata::Image > > >	mTemplateImageYGrad

---

Detailed Description

Processing functions related to the template visual feature (cvdata::TemplateMap).

---

Member Enumeration Documentation

enum opentl::modalities::TemplateMap::OfflineParameters

Enumerator:

param_nBins	(int) In order to generate a maximum number of template points. (default = -1 -> Unlimited number)
OFFLINE_COUNT

Reimplemented from opentl::modalities::Modality.

enum opentl::modalities::TemplateMap::OnlineParameters

Enumerator:

param_pixLevelCov	(double) Pixel-level variance (gray-values).
param_matchPixUsingResidualImage	(bool) Whether to compute the residual image or a scalar residual (default: false)
param_featLevelCov	(double) Feature-level covariance (gray-values).
param_simFunc	(SimFuncType) Similarity function for template matching.
param_useOnlineTemplate	(bool) Whether to use (and update) on-line template from image data.
param_smoothGauss	(std::vector(int) ) Gauss filter size (for multi-resolution).
param_sampleSize	(std::vector(int) ) Desired sub-sample size per resolution. Default: -1 = all of the sample is used.
param_noisySample	(bool) Switch the subsampling index (noisy optimization) or not.
param_minVisPoints	(int) Minimum number of visible points (on-screen) before declaring a track loss (default: 100).
param_validateResiduals	(bool) Whether to apply a validation gate (recommended for SSD matching). Default = false.
param_debugShowMatchImage	(bool) Whether to show the re-projected template at each call of match feature-level. Default = false.
param_computeVectorRes	(bool) Whether to compute scalar or vector residual (scalar can also be mutual information or NCC, vector is only E = z-h).
param_normalizeVectorRes	(bool) Whether remove the mean from z and h while computing the difference.
ONLINE_COUNT

Reimplemented from opentl::modalities::Modality.

enum opentl::modalities::TemplateMap::SimFuncType

Enumerator:

MI
SSD
NCC

---

Constructor & Destructor Documentation

opentl::modalities::TemplateMap::TemplateMap	(	opentl::modelprojection::WarpBack *	warpBack,
		int	camIdx,
		opentl::modelprojection::GLRenderer *	renderer
	)

Constructor.

opentl::modalities::TemplateMap::TemplateMap

(

const TemplateMap &

tmap

)

Deep Copy Constructor.

virtual opentl::modalities::TemplateMap::~TemplateMap

(

)

[virtual]

Destructor.

---

Member Function Documentation

virtual modalities::Modality* opentl::modalities::TemplateMap::clone

(

)

const [virtual]

Clone (it calls the Deep Copy Constructor).

Implements opentl::modalities::Modality.

void opentl::modalities::TemplateMap::init

(

)

[virtual]

Memory allocation and initialization (common to both constructors).

Reimplemented from opentl::modalities::Modality.

virtual int opentl::modalities::TemplateMap::matchFeatLevel	(	const TargetPtrVector &	targets,
		T_MEAS_FEATPtrVector &	outputMeas,
		std::size_t	partitionIdx
	)			[virtual]

matchFeatLevel. Match template gray level values with underlying image values (multi-resolution) The overall similarity function can be: Sum of Squared Differences (SSD), Normalized Cross-Correlation (NCC), Mutual Information (MI).

Parameters:

	targets	Input vector of targets
	outputMeas	Output feature-level measurements (gray-level pairs, Jacobians and UNIQUE residual = overall template similarity) NOTE: The second index of outputMeas (state inside the target) is not used, and always set to 0.

Reimplemented from opentl::modalities::Modality.

virtual int opentl::modalities::TemplateMap::matchObjLevel	(	const TargetPtrVector &	targets,
		T_MEAS_OBJPtrVector &	outputMeas,
		std::size_t	partitionIdx
	)			[virtual]

matchObjLevel. Find local maximum of feature-level likelihood (i.e. the template similarity function)

Parameters:

	targets	Input vector of targets
	outputMeas	Output object-level measurements (predicted and estimated pose, and residual = incremental pose) NOTE: The second index of outputMeas (state inside the target) is not used, and always set to 0.

Reimplemented from opentl::modalities::Modality.

virtual int opentl::modalities::TemplateMap::matchPixLevel	(	const TargetPtrVector &	targets,
		T_MEAS_PIXPtrVector &	outputMeas,
		std::size_t	partitionIdx
	)			[virtual]

Matching on pixel level

• compare pixel-level image data with rendered texture image, by projecting model data under a predicted state.

Parameters:

	targets	Vector of targets
	outputMeas	Pixel-space measurement (can be a residual image, or a unique residual value such as SSD, NCC, MI)

Reimplemented from opentl::modalities::Modality.

virtual int opentl::modalities::TemplateMap::preProcess	(	const opentl::core::cvdata::Image &	image,
		const std::vector< std::vector< int > > &	preProcessROIs
	)			[virtual]

Get the multi-resolution image pyramid.

Parameters:

	image	Input sensor data (e.g. camera image in RGB)
	preProcessROIs	Regions of interest (x0,y0,width,height), per target

Reimplemented from opentl::modalities::Modality.

void opentl::modalities::TemplateMap::resetOnlineTemplate

(

const TargetPtrVector &

targets

)

Reset the on-line template to the off-line values.

Parameters:

targets

Input target vector

virtual int opentl::modalities::TemplateMap::sampleModelFeatures

(

const TargetPtrVector &

targets

)

[virtual]

Compute visible points from the template map at pose p (occlusion query).

Parameters:

targets

Input target vector

Reimplemented from opentl::modalities::Modality.

void opentl::modalities::TemplateMap::showTemplateMap	(	const TargetPtrVector &	targets,
		bool	bOnline = false
	)

Debug: show the template map, re-projected on the screen at all resolutions, with keyboard input (roto-translation).

Parameters:

	targets	Input target vector
	bOnline	whether to show the off-line (default) or the on-line map

virtual int opentl::modalities::TemplateMap::updateModelFeatures

(

const TargetPtrVector &

targets

)

[virtual]

Update the on-line observed gray values (= feature) from the multi-resolution image pyramid.

Parameters:

targets

Input target vector

Reimplemented from opentl::modalities::Modality.

---

Member Data Documentation

std::vector<std::vector<boost::shared_ptr<opentl::core::cvdata::Image> > > opentl::modalities::TemplateMap::mTemplateImageData [static]

Pre-processed data (D) = multi-resolution image pyramid 1st index: camera idx 2nd index: resolution.

std::vector<std::vector<boost::shared_ptr<opentl::core::cvdata::Image> > > opentl::modalities::TemplateMap::mTemplateImageXGrad [static]

Pre-processed data (D) = Sobel filter (image gradients), computed only if Jacobians are needed.

std::vector<std::vector<boost::shared_ptr<opentl::core::cvdata::Image> > > opentl::modalities::TemplateMap::mTemplateImageYGrad [static]