opentl::modalities::HarrisKeypoints Class Reference

Inherits opentl::modalities::Modality.

List of all members.

Classes
class	HarrisKeyframe
	A key-frame contains: an image, and its filtered version (for NCC matching) the annotated pose of the visible target (NOTE: only one target per keyframe!) the detected (and back-projected) Harris keypoints. More...
Public Types
Public Member Functions
void	addKeyFrame (const boost::shared_ptr< opentl::models::Target > &target, opentl::core::cvdata::Image *img=NULL)
	Add key-frame.
virtual HarrisKeypoints *	clone () const
	Clone: call the copy constructor.
int	countOfflineMatchings (int targetId)
void	drawCurrKeypoints (opentl::core::cvdata::Image *img)
	Draw current detected features (red crosses).
void	drawKeyframe (opentl::core::cvdata::Image *img, int keyInd)
	Superimpose keyframe on the image.
void	drawNearestKeyframe (opentl::core::cvdata::Image *img, int targetId)
	Superimpose nearest keyframe to a given target on the image.
void	drawOfflineKeypoints (opentl::core::cvdata::Image *img, int targetId=0)
	Draw off-line (previously matched) features (purple crosses).
void	drawOfflineMatchings (opentl::core::cvdata::Image *img, int targetId=0)
	Draw keyframe (= off-line) matching features (blue dots and lines).
void	drawOnlineKeypoints (opentl::core::cvdata::Image *img, int targetId=0)
	Draw on-line (previously matched) features (purple crosses).
void	drawOnlineMatchings (opentl::core::cvdata::Image *img, int targetId=0)
	Draw on-line matching features (purple dots and lines).
int	getNKeyFrames ()
	Number of available keyframes.
void	getOfflineMatchings (std::vector< math::Vector3 > &bodyPoints, std::vector< math::Vector2 > &screenPoints, int targetId=0)
	Obtain keyframe (= off-line) matchings.
void	getOnlineMatchings (std::vector< math::Vector3 > &bodyPoints, std::vector< math::Vector2 > &screenPoints, int targetId=0)
	Obtain on-line matchings.
	HarrisKeypoints (const HarrisKeypoints &c)
	Copy constructor.
	HarrisKeypoints (opentl::modelprojection::WarpBack *warpBack, int camIdx, modelprojection::GLRenderer *renderer=NULL)
	Constructor.
virtual void	init ()
	Allocate memory and set parameters.
virtual int	matchFeatLevel (const TargetPtrVector &targets, T_MEAS_FEATPtrVector &outputMeas, std::size_t partitionIdx)
	matchFeatLevel. Match detected keypoints on short-baseline (previous frame) and wide-baseline (nearest key-frame)
int	matchKeypointsNCC (opentl::core::cvdata::HarrisKeypoints *feat1, opentl::core::cvdata::HarrisKeypoints *feat2)
	Match two feature sets with normalized cross-correlation (NCC).
virtual int	matchObjLevel (const TargetPtrVector &targets, T_MEAS_OBJPtrVector &outputMeas, std::size_t partitionIdx)
	matchObjLevel. Estimate pose from point correspondences (it calls matchFeatLevel first)
virtual int	preProcess (const opentl::core::cvdata::Image &image, const std::vector< std::vector< int > > &preProcessROIs)
	Detect Harris keypoints in the current image.
void	removeAllKeyframes ()
	Remove all key-frames.
virtual int	sampleModelFeatures (const TargetPtrVector &targets)
	Sample visible model features M_off, from the off-line given ShapeAppearance model (i.e. off-line).
void	shortBaselineWarp (opentl::core::cvdata::HarrisKeypoints *onlineFeatures, const opentl::core::State &predState)
	Re-project keypoint positions from the previous frame to the current, predicted pose.
virtual int	updateModelFeatures (const TargetPtrVector &targets)
	Update model features M_on, from the on-line image stream.
void	wideBaselineWarp (HarrisKeyframe *keyFrame, opentl::core::cvdata::HarrisKeypoints *offlineFeatures, const opentl::core::State &predState)
	Warp keypoint descriptors from the nearest keyframe (pre-computed) to the current position in the screen. For each patch (point and normal) in object-space, compute the invert homography from the current (predicted) view, back to the keyframe and warp the keyframe descriptor.
virtual	~HarrisKeypoints ()
	Destructor.

---

Member Enumeration Documentation

enum opentl::modalities::HarrisKeypoints::OfflineParameters

Enumerator:

maxFeatures	(int) Maximum number of detectable features on any image (keyframes or current frames)
corrWin	(int) Size of the correlation window
OFFLINE_COUNT

Reimplemented from opentl::modalities::Modality.

enum opentl::modalities::HarrisKeypoints::OnlineParameters

Enumerator:

computeJacobians	<bool> Flag for computing Jacobians of re-projected keypoints (for example, needed by DataFusion::upgradeToObjLevel) Default: false
minDetectDist	(double) Minimum distance between detected features
qualityLevel	(double) Minimum acceptable "quality" for each feature
maxMatchDist	(double) Maximum search distance for matching (~ std of feature-level measurement noise, R)
nccThresh	(double) Minimum NCC value to accept a match
measStdFeatOffline	(double) Standard deviation of measurement noise (feature-level), for the off-line sampled keypoints
measStdFeatOnline	(double) Standard deviation of measurement noise (feature-level), for the on-line updated keypoints
measStdObj	(double) Standard deviation of measurement noise (object-level)
poseElementWeights	(math::Transform) Weights (Mahalanobis distance) for the finite pose parameters, in order to evaluate "distances" in pose-space
usePoseElementWeights	(bool) Whether to use pose element weights
maxKeyframes	(int) Maximum number of possible key-frames
renderKeyframes	(bool) If true the key-frames are rendered using the texture model. Otherwise, the current image is used. NOTE the latter may introduce errors, if the pose is not well aligned!
createKeyframeDist	(double) If minimum distance between current frame and the nearest keyframe is higher than this, then a new keyframe is created default: -1 (no keyframe is created, apart from the first)
removeKeyframeDist	(double) If distance between current frame and a keyframe is higher than this, then the keyframe is removed default: -1 (no keyframe is removed)
ONLINE_COUNT

Reimplemented from opentl::modalities::Modality.

---

Constructor & Destructor Documentation

opentl::modalities::HarrisKeypoints::HarrisKeypoints	(	opentl::modelprojection::WarpBack *	warpBack,
		int	camIdx,
		modelprojection::GLRenderer *	renderer = NULL
	)

Constructor.

opentl::modalities::HarrisKeypoints::HarrisKeypoints

(

const HarrisKeypoints &

c

)

Copy constructor.

virtual opentl::modalities::HarrisKeypoints::~HarrisKeypoints

(

)

[virtual]

Destructor.

---

Member Function Documentation

void opentl::modalities::HarrisKeypoints::addKeyFrame	(	const boost::shared_ptr< opentl::models::Target > &	target,
		opentl::core::cvdata::Image *	img = NULL
	)

Add key-frame.

Parameters:

	target	Input target
	img	Input (optional) input image; if absent, the key-frame will be rendered from the target model

Returns:

Number of matches found

virtual HarrisKeypoints* opentl::modalities::HarrisKeypoints::clone

(

)

const [inline, virtual]

Clone: call the copy constructor.

Implements opentl::modalities::Modality.

int opentl::modalities::HarrisKeypoints::countOfflineMatchings

(

int

targetId

)

void opentl::modalities::HarrisKeypoints::drawCurrKeypoints

(

opentl::core::cvdata::Image *

img

)

Draw current detected features (red crosses).

void opentl::modalities::HarrisKeypoints::drawKeyframe	(	opentl::core::cvdata::Image *	img,
		int	keyInd
	)

Superimpose keyframe on the image.

void opentl::modalities::HarrisKeypoints::drawNearestKeyframe	(	opentl::core::cvdata::Image *	img,
		int	targetId
	)

Superimpose nearest keyframe to a given target on the image.

void opentl::modalities::HarrisKeypoints::drawOfflineKeypoints	(	opentl::core::cvdata::Image *	img,
		int	targetId = 0
	)

Draw off-line (previously matched) features (purple crosses).

void opentl::modalities::HarrisKeypoints::drawOfflineMatchings	(	opentl::core::cvdata::Image *	img,
		int	targetId = 0
	)

Draw keyframe (= off-line) matching features (blue dots and lines).

void opentl::modalities::HarrisKeypoints::drawOnlineKeypoints	(	opentl::core::cvdata::Image *	img,
		int	targetId = 0
	)

Draw on-line (previously matched) features (purple crosses).

void opentl::modalities::HarrisKeypoints::drawOnlineMatchings	(	opentl::core::cvdata::Image *	img,
		int	targetId = 0
	)

Draw on-line matching features (purple dots and lines).

int opentl::modalities::HarrisKeypoints::getNKeyFrames

(

)

[inline]

Number of available keyframes.

void opentl::modalities::HarrisKeypoints::getOfflineMatchings	(	std::vector< math::Vector3 > &	bodyPoints,
		std::vector< math::Vector2 > &	screenPoints,
		int	targetId = 0
	)

Obtain keyframe (= off-line) matchings.

void opentl::modalities::HarrisKeypoints::getOnlineMatchings	(	std::vector< math::Vector3 > &	bodyPoints,
		std::vector< math::Vector2 > &	screenPoints,
		int	targetId = 0
	)

Obtain on-line matchings.

virtual void opentl::modalities::HarrisKeypoints::init

(

)

[virtual]

Allocate memory and set parameters.

Reimplemented from opentl::modalities::Modality.

virtual int opentl::modalities::HarrisKeypoints::matchFeatLevel	(	const TargetPtrVector &	targets,
		T_MEAS_FEATPtrVector &	outputMeas,
		std::size_t	partitionIdx
	)			[virtual]

matchFeatLevel. Match detected keypoints on short-baseline (previous frame) and wide-baseline (nearest key-frame)

Parameters:

	targets	Input vector of targets
	outputMeas	Output feature-level measurements (matched keypoints, residuals and evtl. Jacobians) NOTE: The second index of outputMeas (state inside the target) is not used, and always set to 0.

Reimplemented from opentl::modalities::Modality.

int opentl::modalities::HarrisKeypoints::matchKeypointsNCC	(	opentl::core::cvdata::HarrisKeypoints *	feat1,
		opentl::core::cvdata::HarrisKeypoints *	feat2
	)

Match two feature sets with normalized cross-correlation (NCC).

Parameters:

	feat1	Input feature set 1; found correspondences with feat2 are stored inside
	feat2	Input feature set 2; found correspondences with feat1 are stored inside

Returns:

Number of matches found

virtual int opentl::modalities::HarrisKeypoints::matchObjLevel	(	const TargetPtrVector &	targets,
		T_MEAS_OBJPtrVector &	outputMeas,
		std::size_t	partitionIdx
	)			[virtual]

matchObjLevel. Estimate pose from point correspondences (it calls matchFeatLevel first)

Parameters:

	targets	Input vector of targets
	outputMeas	Output object-level measurements (state-space residuals) NOTE: The second index of outputMeas (state inside the target) is not used, and always set to 0.

Reimplemented from opentl::modalities::Modality.

virtual int opentl::modalities::HarrisKeypoints::preProcess	(	const opentl::core::cvdata::Image &	image,
		const std::vector< std::vector< int > > &	preProcessROIs
	)			[virtual]

Detect Harris keypoints in the current image.

Parameters:

	image	Input sensor data (e.g. camera image in RGB)
	preProcessROIs	Regions of interest (x0,y0,width,height), per target

Reimplemented from opentl::modalities::Modality.

void opentl::modalities::HarrisKeypoints::removeAllKeyframes

(

)

Remove all key-frames.

virtual int opentl::modalities::HarrisKeypoints::sampleModelFeatures

(

const TargetPtrVector &

targets

)

[virtual]

Sample visible model features M_off, from the off-line given ShapeAppearance model (i.e. off-line).

Get the nearest key-frame (considering all targets.

Reimplemented from opentl::modalities::Modality.

void opentl::modalities::HarrisKeypoints::shortBaselineWarp	(	opentl::core::cvdata::HarrisKeypoints *	onlineFeatures,
		const opentl::core::State &	predState
	)

Re-project keypoint positions from the previous frame to the current, predicted pose.

Parameters:

	onlineFeatures	Previous frame (back-projected in space) keypoints
	predState	Predicted state for this target

virtual int opentl::modalities::HarrisKeypoints::updateModelFeatures

(

const TargetPtrVector &

targets

)

[virtual]

Update model features M_on, from the on-line image stream.

1 - Update on-line features with the detected ones (from the pre-processing step) 2 - If no nearby key-frame is present, then create a new one under the estimated pose, and back-project the detected features

Parameters:

targets

Vector of targets

Reimplemented from opentl::modalities::Modality.

void opentl::modalities::HarrisKeypoints::wideBaselineWarp	(	HarrisKeyframe *	keyFrame,
		opentl::core::cvdata::HarrisKeypoints *	offlineFeatures,
		const opentl::core::State &	predState
	)

Warp keypoint descriptors from the nearest keyframe (pre-computed) to the current position in the screen. For each patch (point and normal) in object-space, compute the invert homography from the current (predicted) view, back to the keyframe and warp the keyframe descriptor.

Parameters:

	keyFrame	Key-frame information (image and keypoints)
	predState	Predicted state for this target