p3_ort_base

namespace Ort

class P3OrtBase : public Ort::OrtBase

#include <p3_ort_base.hpp>

An Precision-Level 3 (P3) ONNXRuntime (Ort) Base class object. This class object instantiates a Precision Level 3 Ort Session which takes a typical ONNX model used for solely object detection and runs an inference engine.

Public Functions

P3OrtBase(float ratio, int newW, int newH, int paddedW, int paddedH, const uint16_t numClasses, const std::string &modelPath, const boost::optional<size_t> &gpuIdx = boost::none, const boost::optional<std::vector<std::vector<int64_t>>> &inputShapes = boost::none)

A Constructor function.

~P3OrtBase()

A Destructor function.

EPD::EPDObjectDetection infer(const cv::Mat &inputImg)

A auxillary Mutator function that calls the internal overloading infer function.

EPD::EPDObjectLocalization infer(const cv::Mat &inputImg, const cv::Mat &depthImg, sensor_msgs::msg::CameraInfo camera_info, double camera_to_plane_distance_mm)

A auxillary Mutator function that calls the internal overloading infer function.

EPD::EPDObjectTracking infer(const cv::Mat &inputImg, const cv::Mat &depthImg, sensor_msgs::msg::CameraInfo camera_info, double camera_to_plane_distance_mm, const std::string tracker_type, std::vector<cv::Ptr<cv::Tracker>> &trackers, std::vector<int> &tracker_logs, std::vector<EPD::LabelledRect2d> &tracker_results)

A auxillary Mutator function that calls the internal overloading infer function.

inline uint16_t getNumClasses() const

A Getter function that gets the number of object names used for an ongoing session.

inline const std::vector<std::string> &getClassNames() const

A Getter function that gets the list of object text labels which will be used for outputting visualized inference result or for specific use-case filters.

void initClassNames(const std::vector<std::string> &classNames)

A Mutator function that sets the list of object text labels to be used for the P1 Ort Session.

Public Static Attributes

static constexpr int64_t MIN_IMAGE_SIZE = 800

A fixed minimal image size needed for a lower bound requirement for image classification of adequate result.

Private Functions

void preprocess(float *dst, const cv::Mat &imgSrc, const int64_t targetImgWidth, const int64_t targetImgHeight, const int numChannels) const

A Mutator function that converts a 3-layered 2D RGB input image into a 1D input data tensor to be passed to the Ort

Session for processing.

This variant takes a generic input image represented by a conventional opencv Matrix.

EPD::EPDObjectDetection infer(const cv::Mat &inputImg, int newW, int newH, int paddedW, int paddedH, float ratio, float *dst, float confThresh, const cv::Scalar &meanVal)

A Mutator function that runs a P3 Ort Session and gets P3 inference result for use by external agents.

EPD::EPDObjectLocalization infer(const cv::Mat &inputImg, const cv::Mat &depthImg, sensor_msgs::msg::CameraInfo camera_info, double camera_to_plane_distance_mm, int newW, int newH, int paddedW, int paddedH, float ratio, float *dst, float confThresh, const cv::Scalar &meanVal)

A Mutator function that runs a P3 Ort Session and gets P3 inference result with Localization results for use by external agents.

EPD::EPDObjectTracking infer(const cv::Mat &inputImg, const cv::Mat &depthImg, sensor_msgs::msg::CameraInfo camera_info, double camera_to_plane_distance_mm, const std::string tracker_type, std::vector<cv::Ptr<cv::Tracker>> &trackers, std::vector<int> &tracker_logs, std::vector<EPD::LabelledRect2d> &tracker_results, int newW, int newH, int paddedW, int paddedH, float ratio, float *dst, float confThresh, const cv::Scalar &meanVal)

A Mutator function that runs a P3 Ort Session and gets P3 inference result with Tracking results for use by external agents.

cv::Mat visualize(const cv::Mat &img, const std::vector<std::array<float, 4>> &bboxes, const std::vector<uint64_t> &classIndices, const std::vector<cv::Mat> &masks, const std::vector<std::string> &allClassNames, const float maskThreshold)

A Mutator function that takes P2 inference outputs and illustrates derived bounding boxes with corresponding object labels for visualization purposes.

double findMedian(cv::Mat depthImg)

A Getter function that returns the median z-value of the scene.

double findMin(cv::Mat depthImg)

A Getter function that returns the largest z-value of the scene.

double getIOU(cv::Rect2d detected_box, cv::Rect2d tracked_box) const

A Getter function that returns the Intersection-over-Union (IoU) area between two Rect2d objects.

cv::Ptr<cv::Tracker> create_tracker(std::string tracker_type)

A Mutator function that add a user-defined OpenCV tracker. The tracker can be KCF, MedianFlow or CSRT.

void create_tracker_tag(std::vector<int> &tracker_logs)

A Mutator function that create an integer tracker tag and add it to a session-persistent log of tracker tags.

void tracking_evaluate(const std::vector<std::array<float, 4>> &bboxes, const cv::Mat &img, const std::string tracker_type, std::vector<cv::Ptr<cv::Tracker>> &trackers, std::vector<int> &tracker_logs, std::vector<EPD::LabelledRect2d> &tracker_results)

A Mutator function that updates, adds or removes tracked objects by comparing trackers to new detections.

cv::Mat localize_visualize(const cv::Mat &img, const cv::Mat &depthImg, sensor_msgs::msg::CameraInfo camera_info, const std::vector<std::array<float, 4>> &bboxes, const std::vector<uint64_t> &classIndices, const std::vector<cv::Mat> &masks, const std::vector<std::string> &allClassNames, const float maskThreshold)

A Mutator function that takes P3 inference outputs and illustrates derived bounding boxes with corresponding object labels for visualization purposes and localization.

Private Members

const uint16_t m_numClasses

The number of object text labels given an input label list.

float m_ratio

The aspect ratio calculated from the dimension of an input image frame, which is provided when the first input image is received by EasyPerceptionDeployment.

int m_newW

The new padded frame dimensions of an input image. This is used for P2 and P3 object detection.

int m_newH

int m_paddedW

int m_paddedH

std::vector<std::string> m_classNames

A vector of object text labels given an input label list.