diff --git a/README.md b/README.md index d77adfcf..d21c26e9 100644 --- a/README.md +++ b/README.md @@ -68,17 +68,17 @@ Currently, the inference feature list is supported: - Face Detection: ```/ros_openvino_toolkit/face_detection```([object_msgs::ObjectsInBoxes](https://github.com/intel/object_msgs/blob/master/msg/ObjectsInBoxes.msg)) - Emotion Recognition: -```/ros_openvino_toolkit/emotions_recognition```([people_msgs::EmotionsStamped](https://github.com/intel/ros_openvino_toolkit/blob/master/people_msgs/msg/EmotionsStamped.msg)) +```/ros_openvino_toolkit/emotions_recognition```([vino_people_msgs::EmotionsStamped](https://github.com/intel/ros_openvino_toolkit/blob/master/vino_people_msgs/msg/EmotionsStamped.msg)) - Age and Gender Recognition: -```/ros_openvino_toolkit/age_genders_Recognition```([people_msgs::AgeGenderStamped](https://github.com/intel/ros_openvino_toolkit/blob/master/people_msgs/msg/AgeGenderStamped.msg)) +```/ros_openvino_toolkit/age_genders_Recognition```([vino_people_msgs::AgeGenderStamped](https://github.com/intel/ros_openvino_toolkit/blob/master/vino_people_msgs/msg/AgeGenderStamped.msg)) - Head Pose Estimation: -```/ros_openvino_toolkit/headposes_estimation```([people_msgs::HeadPoseStamped](https://github.com/intel/ros_openvino_toolkit/blob/master/people_msgs/msg/HeadPoseStamped.msg)) +```/ros_openvino_toolkit/headposes_estimation```([vino_people_msgs::HeadPoseStamped](https://github.com/intel/ros_openvino_toolkit/blob/master/vino_people_msgs/msg/HeadPoseStamped.msg)) - Object Detection: ```/ros_openvino_toolkit/detected_objects```([object_msgs::ObjectsInBoxes](https://github.com/intel/object_msgs/blob/master/msg/ObjectsInBoxes.msg)) - Object Segmentation: -```/ros_openvino_toolkit/segmented_obejcts```([people_msgs::ObjectsInMasks](https://github.com/intel/ros_openvino_toolkit/blob/devel/people_msgs/msg/ObjectsInMasks.msg)) +```/ros_openvino_toolkit/segmented_obejcts```([vino_people_msgs::ObjectsInMasks](https://github.com/intel/ros_openvino_toolkit/blob/devel/vino_people_msgs/msg/ObjectsInMasks.msg)) - Person Reidentification: -```/ros_openvino_toolkit/reidentified_persons```([people_msgs::ReidentificationStamped](https://github.com/intel/ros_openvino_toolkit/blob/devel/people_msgs/msg/ReidentificationStamped.msg)) +```/ros_openvino_toolkit/reidentified_persons```([vino_people_msgs::ReidentificationStamped](https://github.com/intel/ros_openvino_toolkit/blob/devel/vino_people_msgs/msg/ReidentificationStamped.msg)) - Rviz Output: ```/ros_openvino_toolkit/image_rviz```([sensor_msgs::Image](http://docs.ros.org/melodic/api/sensor_msgs/html/msg/Image.html)) @@ -88,11 +88,11 @@ Currently, the inference feature list is supported: - Face Detection Service: ```/detect_face``` ([object_msgs::DetectObject](https://github.com/intel/object_msgs/blob/master/srv/DetectObject.srv)) - Age & Gender Detection Service: -```/detect_age_gender``` ([people_msgs::AgeGender](https://github.com/intel/ros_openvino_toolkit/blob/master/people_msgs/srv/AgeGenderSrv.srv)) +```/detect_age_gender``` ([vino_people_msgs::AgeGender](https://github.com/intel/ros_openvino_toolkit/blob/master/vino_people_msgs/srv/AgeGenderSrv.srv)) - Headpose Detection Service: -```/detect_head_pose``` ([people_msgs::HeadPose](https://github.com/intel/ros_openvino_toolkit/blob/master/people_msgs/srv/HeadPoseSrv.srv)) +```/detect_head_pose``` ([vino_people_msgs::HeadPose](https://github.com/intel/ros_openvino_toolkit/blob/master/vino_people_msgs/srv/HeadPoseSrv.srv)) - Emotion Detection Service: -```/detect_emotion``` ([people_msgs::Emotion](https://github.com/intel/ros_openvino_toolkit/blob/master/people_msgs/srv/EmotionSrv.srv)) +```/detect_emotion``` ([vino_people_msgs::Emotion](https://github.com/intel/ros_openvino_toolkit/blob/master/vino_people_msgs/srv/EmotionSrv.srv)) ### RViz @@ -249,7 +249,7 @@ One-step installation scripts are provided for the dependencies' installation. P ``` Run example application with an absolute path of an image on another console: ```bash - rosrun dynamic_vino_sample image_object_client ~/catkin_ws/src/ros_openvino_toolkit/data/images/car.png + rosrun vino_sample image_object_client ~/catkin_ws/src/ros_openvino_toolkit/data/images/car.png ``` * run face detection service sample code input from Image Run image processing service: @@ -258,7 +258,7 @@ One-step installation scripts are provided for the dependencies' installation. P ``` Run example application with an absolute path of an image on another console: ```bash - rosrun dynamic_vino_sample image_people_client ~/catkin_ws/src/ros_openvino_toolkit/data/images/team.jpg + rosrun vino_sample image_people_client ~/catkin_ws/src/ros_openvino_toolkit/data/images/team.jpg ``` # TODO Features * Support **result filtering** for inference process, so that the inference results can be filtered to different subsidiary inference. For example, given an image, firstly we do Object Detection on it, secondly we pass cars to vehicle brand recognition and pass license plate to license number recognition. diff --git a/doc/BINARY_VERSION_README.md b/doc/BINARY_VERSION_README.md index f4d731c8..2bb115e1 100644 --- a/doc/BINARY_VERSION_README.md +++ b/doc/BINARY_VERSION_README.md @@ -177,7 +177,7 @@ sudo ln -s ~/catkin_ws/src/ros_openvino_toolkit /opt/openvino_toolkit/ros_openvi ```bash roslaunch vino_launch pipeline_image.launch ``` -* run object detection sample code input from RealSenseCamera.(connect IntelĀ® Neural Compute Stick 2) +* run object detection sample code input from RealSenseCamera. ```bash roslaunch vino_launch pipeline_object.launch ``` @@ -185,7 +185,7 @@ sudo ln -s ~/catkin_ws/src/ros_openvino_toolkit /opt/openvino_toolkit/ros_openvi ```bash roslaunch vino_launch pipeline_object_topic.launch ``` -* run object segmentation sample code input from RealSenseCameraTopic.(connect IntelĀ® Neural Compute Stick 2) +* run object segmentation sample code input from RealSenseCameraTopic. ```bash roslaunch vino_launch pipeline_segmentation.launch ``` @@ -204,7 +204,7 @@ sudo ln -s ~/catkin_ws/src/ros_openvino_toolkit /opt/openvino_toolkit/ros_openvi ``` Run example application with an absolute path of an image on another console: ```bash - rosrun dynamic_vino_sample image_object_client ~/catkin_ws/src/ros_openvino_toolkit/data/images/car.png + rosrun vino_sample image_object_client ~/catkin_ws/src/ros_openvino_toolkit/data/images/car.png ``` * run people detection service sample code input from Image Run image processing service: @@ -213,7 +213,7 @@ sudo ln -s ~/catkin_ws/src/ros_openvino_toolkit /opt/openvino_toolkit/ros_openvi ``` Run example application with an absolute path of an image on another console: ```bash - rosrun dynamic_vino_sample image_people_client ~/catkin_ws/src/ros_openvino_toolkit/data/images/team.jpg + rosrun vino_sample image_people_client ~/catkin_ws/src/ros_openvino_toolkit/data/images/team.jpg ``` ## 6. Known Issues * Possible problems diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/outputs/ros_service_output.h b/dynamic_vino_lib/include/dynamic_vino_lib/outputs/ros_service_output.h deleted file mode 100644 index 3ce30e74..00000000 --- a/dynamic_vino_lib/include/dynamic_vino_lib/outputs/ros_service_output.h +++ /dev/null @@ -1,86 +0,0 @@ -// Copyright (c) 2018 Intel Corporation -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -/** - * @brief A header file with declaration for RosTopicOutput Class - * @file ros_topic_output.h - */ - -#ifndef DYNAMIC_VINO_LIB__OUTPUTS__ROS_SERVICE_OUTPUT_HPP_ -#define DYNAMIC_VINO_LIB__OUTPUTS__ROS_SERVICE_OUTPUT_HPP_ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include -#include -#include -#include -#include -#include -#include - - -#include - -#include -#include -#include - -#include "dynamic_vino_lib/inferences/face_detection.h" -#include "dynamic_vino_lib/outputs/ros_topic_output.h" - -namespace Outputs -{ -/** - * @class RosServiceOutput - * @brief This class handles and publish the detection result for service calling. - */ -class RosServiceOutput : public RosTopicOutput -{ -public: - RosServiceOutput() {} - - /** - * @brief Publish all the detected infomations generated by the accept - * functions with ros topic. - */ - void handleOutput() override {} - void clearData(); - - void setServiceResponse(boost::shared_ptr response); - void setResponseForFace(boost::shared_ptr response); - void setServiceResponse(boost::shared_ptr response); - void setServiceResponse(boost::shared_ptr response); - void setServiceResponse(boost::shared_ptr response); - void setServiceResponse(boost::shared_ptr response); - void setServiceResponse(boost::shared_ptr response); - void setServiceResponse(boost::shared_ptr response); - -private: - const std::string service_name_; -}; -} // namespace Outputs -#endif // DYNAMIC_VINO_LIB__OUTPUTS__ROS_SERVICE_OUTPUT_HPP_ diff --git a/script/environment_setup.sh b/script/environment_setup.sh index bcd0808a..903e7fb0 100755 --- a/script/environment_setup.sh +++ b/script/environment_setup.sh @@ -63,14 +63,13 @@ echo "Set OTHER_DEPENDENCY to $OTHER_DEPENDENCY" # Clean Existing Directories if [ "$CLEAN" == "1" ]; then read -n1 -p "The clean operation will delete some manually created directories, - including ~/code, ~/catkin_ws, /opt/intel, /opt/openvino_toolkit, and OpenVINO tar ball. + including ~/code, /opt/intel, /opt/openvino_toolkit, and OpenVINO tar ball. Do you want to clean existing directories[Y/N]?" answer case $answer in Y|y) echo echo "===================Cleaning...====================================" echo $ROOT_PASSWD | sudo -S rm -rf ~/code echo $ROOT_PASSWD | sudo -S rm -rf /opt/intel - rm -rf ~/catkin_ws echo $ROOT_PASSWD | sudo -S rm -rf /opt/openvino_toolkit if [[ $system_ver = "16.04" && -L "/usr/lib/x86_64-linux-gnu/libboost_python3.so" ]]; then echo $ROOT_PASSWD | sudo -S rm /usr/lib/x86_64-linux-gnu/libboost_python3.so diff --git a/script/environment_setup_binary.sh b/script/environment_setup_binary.sh index d5d6e2cb..32d54e48 100755 --- a/script/environment_setup_binary.sh +++ b/script/environment_setup_binary.sh @@ -59,13 +59,12 @@ if [ "$CLEAN" == "1" ]; then echo "===================Clean Existing Directories...====================================" read -n1 -p "The clean operation will delete some manually created directories, - including ~/code, ~/catkin_ws, /opt/intel, /opt/openvino_toolkit, and OpenVINO tar ball. + including ~/code, /opt/intel, /opt/openvino_toolkit, and OpenVINO tar ball. Do you want to clean existing directories[Y/N]?" answer case $answer in Y|y) echo echo "===================Cleaning...====================================" echo $ROOT_PASSWD | sudo -S rm -rf ~/code - rm -rf ~/catkin_ws echo $ROOT_PASSWD | sudo -S rm -rf /opt/intel echo $ROOT_PASSWD | sudo -S rm -rf /opt/openvino_toolkit if [[ $system_ver = "16.04" && -L "/usr/lib/x86_64-linux-gnu/libboost_python3.so" ]]; then diff --git a/skel-track-notes.md b/skel-track-notes.md new file mode 100644 index 00000000..46a1b036 --- /dev/null +++ b/skel-track-notes.md @@ -0,0 +1,34 @@ +# Notes for Skeleton Tracking Package + +* Model URL: https://docs.openvinotoolkit.org/latest/_models_intel_human_pose_estimation_0001_description_human_pose_estimation_0001.html + +## Installation + +```bash +$ cd /opt/openvino_toolkit/open_model_zoo/tools/downloader +$ python3 downloader.py --name human-pose-estimation-0001 +``` + +## Keypoints + +| Id | Keypoint Name | +| -- | -- | +| 0 | nose | +| 1 | neck | +| 2 | right shoulder | +| 3 | right elbow | +| 4 | right wrist | +| 5 | left shoulder | +| 6 | left elbow | +| 7 | left wrist | +| 8 | right hip | +| 9 | right knee | +| 10 | right ankle | +| 11 | left hip | +| 12 | left knee | +| 13 | left ankle | +| 14 | right eye | +| 15 | left eye | +| 16 | right ear | +| 17 | left ear | + diff --git a/dynamic_vino_lib/CMakeLists.txt b/vino_core_lib/CMakeLists.txt similarity index 97% rename from dynamic_vino_lib/CMakeLists.txt rename to vino_core_lib/CMakeLists.txt index 86c0679f..eac9c900 100644 --- a/dynamic_vino_lib/CMakeLists.txt +++ b/vino_core_lib/CMakeLists.txt @@ -14,7 +14,7 @@ cmake_minimum_required (VERSION 2.8.3) -project(dynamic_vino_lib) +project(vino_core_lib) message(STATUS "Looking for inference engine configuration file at: ${CMAKE_PREFIX_PATH}") find_package(InferenceEngine 1.1) @@ -29,7 +29,7 @@ find_package( std_msgs sensor_msgs object_msgs - people_msgs + vino_people_msgs image_transport cv_bridge InferenceEngine @@ -147,6 +147,8 @@ add_library(${PROJECT_NAME} SHARED src/inferences/object_detection.cpp src/inferences/object_segmentation.cpp src/inferences/person_reidentification.cpp + src/inferences/human_pose_estimation.cpp + src/inferences/peak.cpp src/inputs/realsense_camera.cpp src/inputs/realsense_camera_topic.cpp src/inputs/standard_camera.cpp @@ -160,6 +162,7 @@ add_library(${PROJECT_NAME} SHARED src/models/object_detection_model.cpp src/models/object_segmentation_model.cpp src/models/person_reidentification_model.cpp + src/models/human_pose_estimation_model.cpp src/outputs/image_window_output.cpp src/outputs/ros_topic_output.cpp src/outputs/rviz_output.cpp diff --git a/dynamic_vino_lib/Doxyfile b/vino_core_lib/Doxyfile similarity index 100% rename from dynamic_vino_lib/Doxyfile rename to vino_core_lib/Doxyfile diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/args_helper.h b/vino_core_lib/include/vino_core_lib/args_helper.h similarity index 100% rename from dynamic_vino_lib/include/dynamic_vino_lib/args_helper.h rename to vino_core_lib/include/vino_core_lib/args_helper.h diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/common.h b/vino_core_lib/include/vino_core_lib/common.h similarity index 100% rename from dynamic_vino_lib/include/dynamic_vino_lib/common.h rename to vino_core_lib/include/vino_core_lib/common.h diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/engines/engine.h b/vino_core_lib/include/vino_core_lib/engines/engine.h similarity index 90% rename from dynamic_vino_lib/include/dynamic_vino_lib/engines/engine.h rename to vino_core_lib/include/vino_core_lib/engines/engine.h index 5eb284a2..b548c395 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/engines/engine.h +++ b/vino_core_lib/include/vino_core_lib/engines/engine.h @@ -18,12 +18,12 @@ * @brief A header file with declaration for NetworkEngine class * @file engine.h */ -#ifndef DYNAMIC_VINO_LIB_ENGINES_ENGINE_H -#define DYNAMIC_VINO_LIB_ENGINES_ENGINE_H +#ifndef VINO_CORE_LIB_ENGINES_ENGINE_H +#define VINO_CORE_LIB_ENGINES_ENGINE_H #pragma once -#include "dynamic_vino_lib/models/base_model.h" +#include "vino_core_lib/models/base_model.h" #include "inference_engine.hpp" /** @@ -65,4 +65,4 @@ class Engine }; } // namespace Engines -#endif // DYNAMIC_VINO_LIB_ENGINES_ENGINE_H +#endif // VINO_CORE_LIB_ENGINES_ENGINE_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/factory.h b/vino_core_lib/include/vino_core_lib/factory.h similarity index 92% rename from dynamic_vino_lib/include/dynamic_vino_lib/factory.h rename to vino_core_lib/include/vino_core_lib/factory.h index 98210efb..cc76fba4 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/factory.h +++ b/vino_core_lib/include/vino_core_lib/factory.h @@ -19,16 +19,16 @@ * @file factory.h */ -#ifndef DYNAMIC_VINO_LIB_FACTORY_H -#define DYNAMIC_VINO_LIB_FACTORY_H +#ifndef VINO_CORE_LIB_FACTORY_H +#define VINO_CORE_LIB_FACTORY_H #include #include #include -#include "dynamic_vino_lib/common.h" -#include "dynamic_vino_lib/inputs/base_input.h" +#include "vino_core_lib/common.h" +#include "vino_core_lib/inputs/base_input.h" #include "extension/ext_list.hpp" /** @@ -67,4 +67,4 @@ class Factory const std::string& custom_cldnn_message, bool performance_message); }; -#endif // DYNAMIC_VINO_LIB_FACTORY_H +#endif // VINO_CORE_LIB_FACTORY_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/inferences/age_gender_detection.h b/vino_core_lib/include/vino_core_lib/inferences/age_gender_detection.h similarity index 86% rename from dynamic_vino_lib/include/dynamic_vino_lib/inferences/age_gender_detection.h rename to vino_core_lib/include/vino_core_lib/inferences/age_gender_detection.h index 6c1b2749..9b07496a 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/inferences/age_gender_detection.h +++ b/vino_core_lib/include/vino_core_lib/inferences/age_gender_detection.h @@ -18,16 +18,16 @@ * @brief A header file with declaration for AgeGenderDetection Class * @file age_gender_recignition.h */ -#ifndef DYNAMIC_VINO_LIB_INFERENCES_AGE_GENDER_DETECTION_H -#define DYNAMIC_VINO_LIB_INFERENCES_AGE_GENDER_DETECTION_H +#ifndef VINO_CORE_LIB_INFERENCES_AGE_GENDER_DETECTION_H +#define VINO_CORE_LIB_INFERENCES_AGE_GENDER_DETECTION_H #include #include #include -#include "dynamic_vino_lib/engines/engine.h" -#include "dynamic_vino_lib/inferences/base_inference.h" -#include "dynamic_vino_lib/models/age_gender_detection_model.h" +#include "vino_core_lib/engines/engine.h" +#include "vino_core_lib/inferences/base_inference.h" +#include "vino_core_lib/models/age_gender_detection_model.h" #include "inference_engine.hpp" #include "opencv2/opencv.hpp" @@ -36,7 +36,7 @@ namespace Outputs class BaseOuput; } -namespace dynamic_vino_lib +namespace vino_core_lib { /** * @class AgeGenderResult @@ -76,7 +76,7 @@ class AgeGenderResult : public Result class AgeGenderDetection : public BaseInference { public: - using Result = dynamic_vino_lib::AgeGenderResult; + using Result = vino_core_lib::AgeGenderResult; AgeGenderDetection(); ~AgeGenderDetection() override; /** @@ -114,7 +114,7 @@ class AgeGenderDetection : public BaseInference * to the frame generated by the input device. * @param[in] idx The index of the result. */ - const dynamic_vino_lib::Result* getLocationResult(int idx) const override; + const vino_core_lib::Result* getLocationResult(int idx) const override; /** * @brief Get the name of the Inference instance. * @return The name of the Inference instance. @@ -131,6 +131,6 @@ class AgeGenderDetection : public BaseInference std::shared_ptr valid_model_; std::vector results_; }; -} // namespace dynamic_vino_lib +} // namespace vino_core_lib -#endif // DYNAMIC_VINO_LIB_INFERENCES_AGE_GENDER_DETECTION_H +#endif // VINO_CORE_LIB_INFERENCES_AGE_GENDER_DETECTION_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/inferences/base_inference.h b/vino_core_lib/include/vino_core_lib/inferences/base_inference.h similarity index 92% rename from dynamic_vino_lib/include/dynamic_vino_lib/inferences/base_inference.h rename to vino_core_lib/include/vino_core_lib/inferences/base_inference.h index e2b569d9..2260de98 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/inferences/base_inference.h +++ b/vino_core_lib/include/vino_core_lib/inferences/base_inference.h @@ -18,14 +18,14 @@ * @brief A header file with declaration for BaseInference Class * @file base_inference.h */ -#ifndef DYNAMIC_VINO_LIB_INFERENCES_BASE_INFERENCE_H -#define DYNAMIC_VINO_LIB_INFERENCES_BASE_INFERENCE_H +#ifndef VINO_CORE_LIB_INFERENCES_BASE_INFERENCE_H +#define VINO_CORE_LIB_INFERENCES_BASE_INFERENCE_H #include #include -#include "dynamic_vino_lib/engines/engine.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/engines/engine.h" +#include "vino_core_lib/slog.h" #include "inference_engine.hpp" #include "opencv2/opencv.hpp" @@ -73,7 +73,7 @@ void matU8ToBlob(const cv::Mat& orig_image, InferenceEngine::Blob::Ptr& blob, } } -namespace dynamic_vino_lib +namespace vino_core_lib { /** * @class Result @@ -85,6 +85,7 @@ class Result friend class BaseInference; explicit Result(const cv::Rect& location); inline const cv::Rect getLocation() const { return location_; } + inline void setLocation(const cv::Rect& location) { location_ = location; } private: cv::Rect location_; @@ -155,7 +156,7 @@ class BaseInference * to the frame generated by the input device. * @param[in] idx The index of the result. */ - virtual const dynamic_vino_lib::Result* getLocationResult(int idx) const = 0; + virtual const vino_core_lib::Result* getLocationResult(int idx) const = 0; /** * @brief Get the name of the Inference instance. * @return The name of the Inference instance. @@ -198,6 +199,6 @@ class BaseInference int enqueued_frames = 0; bool results_fetched_ = false; }; -} // namespace dynamic_vino_lib +} // namespace vino_core_lib -#endif // DYNAMIC_VINO_LIB_INFERENCES_BASE_INFERENCE_H +#endif // VINO_CORE_LIB_INFERENCES_BASE_INFERENCE_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/inferences/emotions_detection.h b/vino_core_lib/include/vino_core_lib/inferences/emotions_detection.h similarity index 86% rename from dynamic_vino_lib/include/dynamic_vino_lib/inferences/emotions_detection.h rename to vino_core_lib/include/vino_core_lib/inferences/emotions_detection.h index bb4e1f29..f774dab5 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/inferences/emotions_detection.h +++ b/vino_core_lib/include/vino_core_lib/inferences/emotions_detection.h @@ -18,16 +18,16 @@ * @brief A header file with declaration for EmotionsDetection Class * @file emotions_detection.h */ -#ifndef DYNAMIC_VINO_LIB_INFERENCES_EMOTIONS_DETECTION_H -#define DYNAMIC_VINO_LIB_INFERENCES_EMOTIONS_DETECTION_H +#ifndef VINO_CORE_LIB_INFERENCES_EMOTIONS_DETECTION_H +#define VINO_CORE_LIB_INFERENCES_EMOTIONS_DETECTION_H #include #include #include -#include "dynamic_vino_lib/engines/engine.h" -#include "dynamic_vino_lib/inferences/base_inference.h" -#include "dynamic_vino_lib/models/emotion_detection_model.h" +#include "vino_core_lib/engines/engine.h" +#include "vino_core_lib/inferences/base_inference.h" +#include "vino_core_lib/models/emotion_detection_model.h" #include "inference_engine.hpp" #include "opencv2/opencv.hpp" @@ -35,7 +35,7 @@ namespace Outputs { class BaseOuput; } -namespace dynamic_vino_lib +namespace vino_core_lib { /** * @class EmotionResult @@ -67,7 +67,7 @@ class EmotionsResult : public Result class EmotionsDetection : public BaseInference { public: - using Result = dynamic_vino_lib::EmotionsResult; + using Result = vino_core_lib::EmotionsResult; EmotionsDetection(); ~EmotionsDetection() override; /** @@ -105,7 +105,7 @@ class EmotionsDetection : public BaseInference * to the frame generated by the input device. * @param[in] idx The index of the result. */ - const dynamic_vino_lib::Result* getLocationResult(int idx) const override; + const vino_core_lib::Result* getLocationResult(int idx) const override; /** * @brief Get the name of the Inference instance. * @return The name of the Inference instance. @@ -122,6 +122,6 @@ class EmotionsDetection : public BaseInference std::shared_ptr valid_model_; std::vector results_; }; -} // namespace dynamic_vino_lib +} // namespace vino_core_lib -#endif // DYNAMIC_VINO_LIB_INFERENCES_EMOTIONS_DETECTION_H +#endif // VINO_CORE_LIB_INFERENCES_EMOTIONS_DETECTION_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/inferences/face_detection.h b/vino_core_lib/include/vino_core_lib/inferences/face_detection.h similarity index 87% rename from dynamic_vino_lib/include/dynamic_vino_lib/inferences/face_detection.h rename to vino_core_lib/include/vino_core_lib/inferences/face_detection.h index 7addb3a5..ef692f56 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/inferences/face_detection.h +++ b/vino_core_lib/include/vino_core_lib/inferences/face_detection.h @@ -18,8 +18,8 @@ * @brief A header file with declaration for FaceDetection Class * @file face_detection.h */ -#ifndef DYNAMIC_VINO_LIB_INFERENCES_FACE_DETECTION_H -#define DYNAMIC_VINO_LIB_INFERENCES_FACE_DETECTION_H +#ifndef VINO_CORE_LIB_INFERENCES_FACE_DETECTION_H +#define VINO_CORE_LIB_INFERENCES_FACE_DETECTION_H #include #include @@ -30,14 +30,14 @@ #include #include -#include "dynamic_vino_lib/engines/engine.h" -#include "dynamic_vino_lib/inferences/base_inference.h" -#include "dynamic_vino_lib/models/face_detection_model.h" +#include "vino_core_lib/engines/engine.h" +#include "vino_core_lib/inferences/base_inference.h" +#include "vino_core_lib/models/face_detection_model.h" #include "inference_engine.hpp" #include "opencv2/opencv.hpp" // namespace -namespace dynamic_vino_lib +namespace vino_core_lib { /** * @class FaceDetectionResult @@ -73,7 +73,7 @@ class FaceDetectionResult : public Result class FaceDetection : public BaseInference { public: - using Result = dynamic_vino_lib::FaceDetectionResult; + using Result = vino_core_lib::FaceDetectionResult; explicit FaceDetection(double); ~FaceDetection() override; /** @@ -111,7 +111,7 @@ class FaceDetection : public BaseInference * to the frame generated by the input device. * @param[in] idx The index of the result. */ - const dynamic_vino_lib::Result* getLocationResult(int idx) const override; + const vino_core_lib::Result* getLocationResult(int idx) const override; /** * @brief Show the observed detection result either through image window or ROS topic. @@ -132,5 +132,5 @@ class FaceDetection : public BaseInference int object_size_; double show_output_thresh_ = 0; }; -} // namespace dynamic_vino_lib -#endif // DYNAMIC_VINO_LIB_INFERENCES_FACE_DETECTION_H +} // namespace vino_core_lib +#endif // VINO_CORE_LIB_INFERENCES_FACE_DETECTION_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/inferences/head_pose_detection.h b/vino_core_lib/include/vino_core_lib/inferences/head_pose_detection.h similarity index 87% rename from dynamic_vino_lib/include/dynamic_vino_lib/inferences/head_pose_detection.h rename to vino_core_lib/include/vino_core_lib/inferences/head_pose_detection.h index 125d6207..87996a63 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/inferences/head_pose_detection.h +++ b/vino_core_lib/include/vino_core_lib/inferences/head_pose_detection.h @@ -18,20 +18,20 @@ * @brief A header file with declaration for FaceDetection Class * @file head_pose_detection.h */ -#ifndef DYNAMIC_VINO_LIB_INFERENCES_HEAD_POSE_DETECTION_H -#define DYNAMIC_VINO_LIB_INFERENCES_HEAD_POSE_DETECTION_H +#ifndef VINO_CORE_LIB_INFERENCES_HEAD_POSE_DETECTION_H +#define VINO_CORE_LIB_INFERENCES_HEAD_POSE_DETECTION_H #include #include #include -#include "dynamic_vino_lib/engines/engine.h" -#include "dynamic_vino_lib/inferences/base_inference.h" -#include "dynamic_vino_lib/models/head_pose_detection_model.h" +#include "vino_core_lib/engines/engine.h" +#include "vino_core_lib/inferences/base_inference.h" +#include "vino_core_lib/models/head_pose_detection_model.h" #include "inference_engine.hpp" #include "opencv2/opencv.hpp" -namespace dynamic_vino_lib +namespace vino_core_lib { /** * @class HeadPoseResult @@ -80,7 +80,7 @@ class HeadPoseResult : public Result class HeadPoseDetection : public BaseInference { public: - using Result = dynamic_vino_lib::HeadPoseResult; + using Result = vino_core_lib::HeadPoseResult; HeadPoseDetection(); ~HeadPoseDetection() override; /** @@ -118,7 +118,7 @@ class HeadPoseDetection : public BaseInference * to the frame generated by the input device. * @param[in] idx The index of the result. */ - const dynamic_vino_lib::Result* getLocationResult(int idx) const override; + const vino_core_lib::Result* getLocationResult(int idx) const override; /** * @brief Get the name of the Inference instance. * @return The name of the Inference instance. @@ -135,5 +135,5 @@ class HeadPoseDetection : public BaseInference std::shared_ptr valid_model_; std::vector results_; }; -} // namespace dynamic_vino_lib -#endif // DYNAMIC_VINO_LIB_INFERENCES_HEAD_POSE_DETECTION_H +} // namespace vino_core_lib +#endif // VINO_CORE_LIB_INFERENCES_HEAD_POSE_DETECTION_H diff --git a/vino_core_lib/include/vino_core_lib/inferences/human_pose_estimation.h b/vino_core_lib/include/vino_core_lib/inferences/human_pose_estimation.h new file mode 100644 index 00000000..b1bd74e6 --- /dev/null +++ b/vino_core_lib/include/vino_core_lib/inferences/human_pose_estimation.h @@ -0,0 +1,215 @@ +// TODO add license + +/** + * @brief A header file with declaration for AgeGenderDetection Class + * @file human_pose_estimation.h + */ +#ifndef VINO_CORE_LIB_INFERENCES_HUMAN_POSE_ESTIMATION_H +#define VINO_CORE_LIB_INFERENCES_HUMAN_POSE_ESTIMATION_H + +#include +#include +#include + +#include "vino_core_lib/engines/engine.h" +#include "vino_core_lib/inferences/base_inference.h" +#include "vino_core_lib/models/human_pose_estimation_model.h" +#include "inference_engine.hpp" +#include "opencv2/opencv.hpp" + +namespace vino_core_lib +{ + +class HumanPoseKeypoint : public cv::Point2f +{ +public: + HumanPoseKeypoint(float x, float y, float score) + : HumanPoseKeypoint::Point_(x, y), + score(score) {}; + HumanPoseKeypoint(cv::Point2f point) + : HumanPoseKeypoint::Point_(point), + score(score) {}; + float score; +}; + +/** + * @class HumanPoseResult + * @brief Class for storing and processing age and gender detection result. + */ +class HumanPoseResult : public Result +{ + public: + explicit HumanPoseResult(const cv::Rect& location); + HumanPoseResult( + const cv::Rect& location, + const std::vector& keypoints, // = std::vector(), + const float& score); // = 0); + + // Following similar structure of vino_core_lib/inferences/object_detection.h + // and human_pose_estimation_demo/src/human_pose_estimator.h + + /** + * @brief Get the age keypoints of the estimated pose from the result. + * @return The estimated keypoints. + */ + std::vector getKeypoints() const + { + return keypoints; + } + + /** + * @brief Get the score of the estimated pose from the result. + * @return The score of the estimation. + */ + float getScore() const + { + return score; + } + + std::vector keypointsScores; + std::vector keypoints; + float score = -1; +}; + + +/** + * @class HumanPoseEstimation + * @brief Class to load the human pose estimation model and perform + human pose estimation. + */ +class HumanPoseEstimation : public BaseInference +{ + public: + using Result = vino_core_lib::HumanPoseResult; + HumanPoseEstimation(float minPeaksDistance, + float midPointsScoreThreshold, + float foundMidPointsRatioThreshold, + int minJointsNumber, + float minSubsetScore); + ~HumanPoseEstimation() override; + /** + * @brief Load the age gender detection model. + */ + void loadNetwork(std::shared_ptr); + /** + * @brief Enqueue a frame to this class. + * The frame will be buffered but not inferred yet. + * @param[in] frame The frame to be enqueued. + * @param[in] input_frame_loc The location of the enqueued frame with respect + * to the frame generated by the input device. + * @return Whether this operation is successful. + */ + bool enqueue(const cv::Mat& frame, const cv::Rect&) override; + /** + * @brief Start inference for all buffered frames. + * @return Whether this operation is successful. + */ + bool submitRequest() override; + /** + * @brief This function will fetch the results of the previous inference and + * stores the results in a result buffer array. All buffered frames will be + * cleared. + * @return Whether the Inference object fetches a result this time + */ + bool fetchResults() override; + /** + * @brief Get the length of the buffer result array. + * @return The length of the buffer result array. + */ + const int getResultsLength() const override; + /** + * @brief Get the location of result with respect + * to the frame generated by the input device. + * @param[in] idx The index of the result. + */ + const vino_core_lib::Result* getLocationResult(int idx) const override; + /** + * @brief Get the name of the Inference instance. + * @return The name of the Inference instance. + */ + const std::string getName() const override; + /** + * @brief Show the observed detection result either through image window + * or ROS topic. + */ + const void observeOutput( + const std::shared_ptr& output) override; + + private: + /** + * @brief Processess the network's outputs to extract the valid poses. + * + * Copied from: https://github.com/opencv/open_model_zoo/blob/master/demos/human_pose_estimation_demo/src/human_pose_estimator.cpp + * + * @param heatMapsData Data outputted by the heatmap network. + * @param heatMapOffset Size of each heatmap result. + * @param nHeatMaps Number of keypoints. + * @param pafsData Data outputted by the PAF network. + * @param pafOffset Size of each PAF result. + * @param nPafs Numver of PAFs. + * @param featureMapWidth Width of the heatmap. + * @param featureMapHeight Height of the heatmap. + * @param imageSize Size of the input image. + * @return std::vector A vector with the detected poses. + */ + std::vector postprocess( + const float* heatMapsData, const int heatMapOffset, const int nHeatMaps, + const float* pafsData, const int pafOffset, const int nPafs, + const int featureMapWidth, const int featureMapHeight, + const cv::Size& imageSize) const; + + /** + * @brief Resizes the heatmap by upSampleRatio. + * + * @param featureMaps A vector with the heatmaps to resize. + */ + void resizeFeatureMaps(std::vector& featureMaps) const; + + /** + * @brief Extracts the poses from the given heatmaps and PAFs. + * + * @param heatMaps Postprocessed heatmaps. + * @param pafs Postprocessed PAFs. + * @return std::vector The detected poses. + */ + std::vector extractPoses( + const std::vector& heatMaps, + const std::vector& pafs) const; + + /** + * @brief Aligns the poses' keypoints to the input image. + * + * @param poses Poses (extracted from the heatmaps and PAFs). + * @param featureMapsSize The size of the heatmaps. + * @param imageSize The input image size. + */ + void correctCoordinates(std::vector& poses, + const cv::Size& featureMapsSize, + const cv::Size& imageSize) const; + + /** + * @brief Correct the bonding boxes based on the poses' keypoints. + * + * @param poses Poses (with corrected keypoints). + */ + void correctROI(std::vector& poses) const; + + std::shared_ptr valid_model_; + std::vector results_; + int width_ = 0; + int height_ = 0; + + const size_t keypointsNumber_ = 18; + int upsampleRatio_; + float minPeaksDistance_; + float midPointsScoreThreshold_; + float foundMidPointsRatioThreshold_; + int minJointsNumber_; + float minSubsetScore_; + int stride_; + cv::Vec4i pad_; +}; + +} // namespace vino_core_lib + +#endif // VINO_CORE_LIB_INFERENCES_HUMAN_POSE_ESTIMATION_H \ No newline at end of file diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/inferences/object_detection.h b/vino_core_lib/include/vino_core_lib/inferences/object_detection.h similarity index 87% rename from dynamic_vino_lib/include/dynamic_vino_lib/inferences/object_detection.h rename to vino_core_lib/include/vino_core_lib/inferences/object_detection.h index 6eb50efa..fa4c314d 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/inferences/object_detection.h +++ b/vino_core_lib/include/vino_core_lib/inferences/object_detection.h @@ -17,21 +17,21 @@ * @brief A header file with declaration for ObjectDetection Class * @file object_detection.hpp */ -#ifndef DYNAMIC_VINO_LIB_INFERENCES_OBJECT_DETECTION_H -#define DYNAMIC_VINO_LIB_INFERENCES_OBJECT_DETECTION_H +#ifndef VINO_CORE_LIB_INFERENCES_OBJECT_DETECTION_H +#define VINO_CORE_LIB_INFERENCES_OBJECT_DETECTION_H #include #include #include #include #include #include -#include "dynamic_vino_lib/models/object_detection_model.h" -#include "dynamic_vino_lib/engines/engine.h" -#include "dynamic_vino_lib/inferences/base_inference.h" +#include "vino_core_lib/models/object_detection_model.h" +#include "vino_core_lib/engines/engine.h" +#include "vino_core_lib/inferences/base_inference.h" #include "inference_engine.hpp" #include "opencv2/opencv.hpp" // namespace -namespace dynamic_vino_lib { +namespace vino_core_lib { /** * @class ObjectDetectionResult * @brief Class for storing and processing face detection result. @@ -56,7 +56,7 @@ class ObjectDetectionResult : public Result { */ class ObjectDetection : public BaseInference { public: - using Result = dynamic_vino_lib::ObjectDetectionResult; + using Result = vino_core_lib::ObjectDetectionResult; explicit ObjectDetection(bool,double); ~ObjectDetection() override; /** @@ -94,7 +94,7 @@ class ObjectDetection : public BaseInference { * to the frame generated by the input device. * @param[in] idx The index of the result. */ - const dynamic_vino_lib::Result* getLocationResult(int idx) const override; + const vino_core_lib::Result* getLocationResult(int idx) const override; /** * @brief Show the observed detection result either through image window or ROS topic. @@ -114,5 +114,5 @@ class ObjectDetection : public BaseInference { int object_size_; double show_output_thresh_ = 0; }; -} // namespace dynamic_vino_lib -#endif // DYNAMIC_VINO_LIB_INFERENCES_OBJECT_DETECTION_H +} // namespace vino_core_lib +#endif // VINO_CORE_LIB_INFERENCES_OBJECT_DETECTION_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/inferences/object_segmentation.h b/vino_core_lib/include/vino_core_lib/inferences/object_segmentation.h similarity index 86% rename from dynamic_vino_lib/include/dynamic_vino_lib/inferences/object_segmentation.h rename to vino_core_lib/include/vino_core_lib/inferences/object_segmentation.h index e6100e1a..29bfe805 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/inferences/object_segmentation.h +++ b/vino_core_lib/include/vino_core_lib/inferences/object_segmentation.h @@ -16,21 +16,21 @@ * @brief A header file with declaration for ObjectSegmentation Class * @file object_detection.hpp */ -#ifndef DYNAMIC_VINO_LIB__INFERENCES__OBJECT_SEGMENTATION_HPP_ -#define DYNAMIC_VINO_LIB__INFERENCES__OBJECT_SEGMENTATION_HPP_ +#ifndef VINO_CORE_LIB__INFERENCES__OBJECT_SEGMENTATION_HPP_ +#define VINO_CORE_LIB__INFERENCES__OBJECT_SEGMENTATION_HPP_ #include #include #include #include #include #include -#include "dynamic_vino_lib/models/object_segmentation_model.h" -#include "dynamic_vino_lib/engines/engine.h" -#include "dynamic_vino_lib/inferences/base_inference.h" +#include "vino_core_lib/models/object_segmentation_model.h" +#include "vino_core_lib/engines/engine.h" +#include "vino_core_lib/inferences/base_inference.h" #include "inference_engine.hpp" #include "opencv2/opencv.hpp" // namespace -namespace dynamic_vino_lib +namespace vino_core_lib { /** * @class ObjectSegmentationResult @@ -70,7 +70,7 @@ class ObjectSegmentationResult : public Result class ObjectSegmentation : public BaseInference { public: - using Result = dynamic_vino_lib::ObjectSegmentationResult; + using Result = vino_core_lib::ObjectSegmentationResult; explicit ObjectSegmentation(double); ~ObjectSegmentation() override; /** @@ -108,7 +108,7 @@ class ObjectSegmentation : public BaseInference * to the frame generated by the input device. * @param[in] idx The index of the result. */ - const dynamic_vino_lib::Result * getLocationResult(int idx) const override; + const vino_core_lib::Result * getLocationResult(int idx) const override; /** * @brief Show the observed detection result either through image window or ROS topic. @@ -127,5 +127,5 @@ class ObjectSegmentation : public BaseInference int height_ = 0; double show_output_thresh_ = 0; }; -} // namespace dynamic_vino_lib -#endif // DYNAMIC_VINO_LIB__INFERENCES__OBJECT_SEGMENTATION_HPP_ +} // namespace vino_core_lib +#endif // VINO_CORE_LIB__INFERENCES__OBJECT_SEGMENTATION_HPP_ diff --git a/vino_core_lib/include/vino_core_lib/inferences/peak.h b/vino_core_lib/include/vino_core_lib/inferences/peak.h new file mode 100644 index 00000000..e0310ce8 --- /dev/null +++ b/vino_core_lib/include/vino_core_lib/inferences/peak.h @@ -0,0 +1,87 @@ +// Copyright (C) 2018-2019 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#pragma once + +#include + +#include + +// #include "human_pose.hpp" +#include "vino_core_lib/inferences/human_pose_estimation.h" + +namespace human_pose_estimation +{ + +using Result = vino_core_lib::HumanPoseResult; + +struct Peak +{ + Peak(const int id = -1, + const cv::Point2f &pos = cv::Point2f(), + const float score = 0.0f); + + int id; + cv::Point2f pos; + float score; +}; + +struct HumanPoseByPeaksIndices +{ + explicit HumanPoseByPeaksIndices(const int keypointsNumber); + + std::vector peaksIndices; + int nJoints; + float score; +}; + +struct TwoJointsConnection +{ + TwoJointsConnection(const int firstJointIdx, + const int secondJointIdx, + const float score); + + int firstJointIdx; + int secondJointIdx; + float score; +}; + +void findPeaks(const std::vector &heatMaps, + const float minPeaksDistance, + std::vector> &allPeaks, + int heatMapId); + +std::vector groupPeaksToPoses( + const std::vector> &allPeaks, + const std::vector &pafs, + const size_t keypointsNumber, + const float midPointsScoreThreshold, + const float foundMidPointsRatioThreshold, + const int minJointsNumber, + const float minSubsetScore); + +class FindPeaksBody : public cv::ParallelLoopBody +{ +public: + FindPeaksBody(const std::vector &heatMaps, float minPeaksDistance, + std::vector> &peaksFromHeatMap) + : heatMaps(heatMaps), + minPeaksDistance(minPeaksDistance), + peaksFromHeatMap(peaksFromHeatMap) {} + + virtual void operator()(const cv::Range &range) const + { + for (int i = range.start; i < range.end; i++) + { + findPeaks(heatMaps, minPeaksDistance, peaksFromHeatMap, i); + } + } + +private: + const std::vector &heatMaps; + float minPeaksDistance; + std::vector> &peaksFromHeatMap; +}; + +} // namespace human_pose_estimation diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/inferences/person_reidentification.h b/vino_core_lib/include/vino_core_lib/inferences/person_reidentification.h similarity index 86% rename from dynamic_vino_lib/include/dynamic_vino_lib/inferences/person_reidentification.h rename to vino_core_lib/include/vino_core_lib/inferences/person_reidentification.h index 72b9b7a0..7605b1ff 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/inferences/person_reidentification.h +++ b/vino_core_lib/include/vino_core_lib/inferences/person_reidentification.h @@ -16,18 +16,18 @@ * @brief A header file with declaration for PersonReidentification Class * @file person_reidentification.hpp */ -#ifndef DYNAMIC_VINO_LIB__INFERENCES__PERSON_REIDENTIFICATION_HPP_ -#define DYNAMIC_VINO_LIB__INFERENCES__PERSON_REIDENTIFICATION_HPP_ +#ifndef VINO_CORE_LIB__INFERENCES__PERSON_REIDENTIFICATION_HPP_ +#define VINO_CORE_LIB__INFERENCES__PERSON_REIDENTIFICATION_HPP_ #include #include #include -#include "dynamic_vino_lib/models/person_reidentification_model.h" -#include "dynamic_vino_lib/engines/engine.h" -#include "dynamic_vino_lib/inferences/base_inference.h" +#include "vino_core_lib/models/person_reidentification_model.h" +#include "vino_core_lib/engines/engine.h" +#include "vino_core_lib/inferences/base_inference.h" #include "inference_engine.hpp" #include "opencv2/opencv.hpp" // namespace -namespace dynamic_vino_lib +namespace vino_core_lib { /** * @class PersonReidentificationResult @@ -50,7 +50,7 @@ class PersonReidentificationResult : public Result class PersonReidentification : public BaseInference { public: - using Result = dynamic_vino_lib::PersonReidentificationResult; + using Result = vino_core_lib::PersonReidentificationResult; explicit PersonReidentification(double); ~PersonReidentification() override; /** @@ -88,7 +88,7 @@ class PersonReidentification : public BaseInference * to the frame generated by the input device. * @param[in] idx The index of the result. */ - const dynamic_vino_lib::Result * getLocationResult(int idx) const override; + const vino_core_lib::Result * getLocationResult(int idx) const override; /** * @brief Show the observed detection result either through image window or ROS topic. @@ -118,5 +118,5 @@ class PersonReidentification : public BaseInference std::vector> recorded_persons_; double match_thresh_ = 0; }; -} // namespace dynamic_vino_lib -#endif // DYNAMIC_VINO_LIB__INFERENCES__PERSON_REIDENTIFICATION_HPP_ +} // namespace vino_core_lib +#endif // VINO_CORE_LIB__INFERENCES__PERSON_REIDENTIFICATION_HPP_ diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/inputs/base_input.h b/vino_core_lib/include/vino_core_lib/inputs/base_input.h similarity index 95% rename from dynamic_vino_lib/include/dynamic_vino_lib/inputs/base_input.h rename to vino_core_lib/include/vino_core_lib/inputs/base_input.h index e300fdee..54fab55f 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/inputs/base_input.h +++ b/vino_core_lib/include/vino_core_lib/inputs/base_input.h @@ -18,11 +18,11 @@ * @brief A header file with declaration for BaseInput Class * @file base_input.h */ -#ifndef DYNAMIC_VINO_LIB_INPUTS_BASE_INPUT_H -#define DYNAMIC_VINO_LIB_INPUTS_BASE_INPUT_H +#ifndef VINO_CORE_LIB_INPUTS_BASE_INPUT_H +#define VINO_CORE_LIB_INPUTS_BASE_INPUT_H #include -#include "dynamic_vino_lib/inputs/ros_handler.h" +#include "vino_core_lib/inputs/ros_handler.h" /** * @class BaseInputDevice @@ -131,4 +131,4 @@ class BaseInputDevice : public Ros2Handler std::string frame_id_; }; } // namespace Input -#endif // DYNAMIC_VINO_LIB_INPUTS_BASE_INPUT_H +#endif // VINO_CORE_LIB_INPUTS_BASE_INPUT_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/inputs/image_input.h b/vino_core_lib/include/vino_core_lib/inputs/image_input.h similarity index 90% rename from dynamic_vino_lib/include/dynamic_vino_lib/inputs/image_input.h rename to vino_core_lib/include/vino_core_lib/inputs/image_input.h index afdbd619..536511e3 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/inputs/image_input.h +++ b/vino_core_lib/include/vino_core_lib/inputs/image_input.h @@ -18,12 +18,12 @@ * @brief A header file with declaration for Image class * @file file_input.h */ -#ifndef DYNAMIC_VINO_LIB_INPUTS_IMAGE_INPUT_H -#define DYNAMIC_VINO_LIB_INPUTS_IMAGE_INPUT_H +#ifndef VINO_CORE_LIB_INPUTS_IMAGE_INPUT_H +#define VINO_CORE_LIB_INPUTS_IMAGE_INPUT_H #include #include -#include "dynamic_vino_lib/inputs/base_input.h" +#include "vino_core_lib/inputs/base_input.h" namespace Input { @@ -72,4 +72,4 @@ class Image : public BaseInputDevice }; } // namespace Input -#endif // DYNAMIC_VINO_LIB_INPUTS_IMAGE_INPUT_H +#endif // VINO_CORE_LIB_INPUTS_IMAGE_INPUT_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/inputs/realsense_camera.h b/vino_core_lib/include/vino_core_lib/inputs/realsense_camera.h similarity index 90% rename from dynamic_vino_lib/include/dynamic_vino_lib/inputs/realsense_camera.h rename to vino_core_lib/include/vino_core_lib/inputs/realsense_camera.h index f4a87d0f..3e51690b 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/inputs/realsense_camera.h +++ b/vino_core_lib/include/vino_core_lib/inputs/realsense_camera.h @@ -19,12 +19,12 @@ * @file realsense_camera.h */ -#ifndef DYNAMIC_VINO_LIB_INPUTS_REALSENSE_CAMERA_H -#define DYNAMIC_VINO_LIB_INPUTS_REALSENSE_CAMERA_H +#ifndef VINO_CORE_LIB_INPUTS_REALSENSE_CAMERA_H +#define VINO_CORE_LIB_INPUTS_REALSENSE_CAMERA_H #include #include -#include "dynamic_vino_lib/inputs/base_input.h" +#include "vino_core_lib/inputs/base_input.h" namespace Input { @@ -69,4 +69,4 @@ class RealSenseCamera : public BaseInputDevice }; } // namespace Input -#endif // DYNAMIC_VINO_LIB_INPUTS_REALSENSE_CAMERA_H +#endif // VINO_CORE_LIB_INPUTS_REALSENSE_CAMERA_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/inputs/realsense_camera_topic.h b/vino_core_lib/include/vino_core_lib/inputs/realsense_camera_topic.h similarity index 87% rename from dynamic_vino_lib/include/dynamic_vino_lib/inputs/realsense_camera_topic.h rename to vino_core_lib/include/vino_core_lib/inputs/realsense_camera_topic.h index 8d5fbf9d..a03f83d0 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/inputs/realsense_camera_topic.h +++ b/vino_core_lib/include/vino_core_lib/inputs/realsense_camera_topic.h @@ -19,8 +19,8 @@ * @file realsense_camera.h */ -#ifndef DYNAMIC_VINO_LIB_INPUTS_REALSENSE_CAMERA_TOPIC_H -#define DYNAMIC_VINO_LIB_INPUTS_REALSENSE_CAMERA_TOPIC_H +#ifndef VINO_CORE_LIB_INPUTS_REALSENSE_CAMERA_TOPIC_H +#define VINO_CORE_LIB_INPUTS_REALSENSE_CAMERA_TOPIC_H #include #include @@ -30,7 +30,7 @@ #include -#include "dynamic_vino_lib/inputs/base_input.h" +#include "vino_core_lib/inputs/base_input.h" namespace Input { @@ -64,4 +64,4 @@ class RealSenseCameraTopic : public BaseInputDevice }; } // namespace Input -#endif // DYNAMIC_VINO_LIB_INPUTS_REALSENSE_CAMERA_TOPIC_H_ +#endif // VINO_CORE_LIB_INPUTS_REALSENSE_CAMERA_TOPIC_H_ diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/inputs/ros_handler.h b/vino_core_lib/include/vino_core_lib/inputs/ros_handler.h similarity index 100% rename from dynamic_vino_lib/include/dynamic_vino_lib/inputs/ros_handler.h rename to vino_core_lib/include/vino_core_lib/inputs/ros_handler.h diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/inputs/standard_camera.h b/vino_core_lib/include/vino_core_lib/inputs/standard_camera.h similarity index 90% rename from dynamic_vino_lib/include/dynamic_vino_lib/inputs/standard_camera.h rename to vino_core_lib/include/vino_core_lib/inputs/standard_camera.h index 8928a741..b6581625 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/inputs/standard_camera.h +++ b/vino_core_lib/include/vino_core_lib/inputs/standard_camera.h @@ -19,11 +19,11 @@ * @file standard_camera.h */ -#ifndef DYNAMIC_VINO_LIB_INPUTS_STANDARD_CAMERA_H -#define DYNAMIC_VINO_LIB_INPUTS_STANDARD_CAMERA_H +#ifndef VINO_CORE_LIB_INPUTS_STANDARD_CAMERA_H +#define VINO_CORE_LIB_INPUTS_STANDARD_CAMERA_H #include -#include "dynamic_vino_lib/inputs/base_input.h" +#include "vino_core_lib/inputs/base_input.h" namespace Input { @@ -63,4 +63,4 @@ class StandardCamera : public BaseInputDevice cv::VideoCapture cap; }; } // namespace Input -#endif // DYNAMIC_VINO_LIB_INPUTS_STANDARD_CAMERA_H +#endif // VINO_CORE_LIB_INPUTS_STANDARD_CAMERA_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/inputs/video_input.h b/vino_core_lib/include/vino_core_lib/inputs/video_input.h similarity index 90% rename from dynamic_vino_lib/include/dynamic_vino_lib/inputs/video_input.h rename to vino_core_lib/include/vino_core_lib/inputs/video_input.h index d1ae6f47..05875cc0 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/inputs/video_input.h +++ b/vino_core_lib/include/vino_core_lib/inputs/video_input.h @@ -18,12 +18,12 @@ * @brief A header file with declaration for Video class * @file video_input.h */ -#ifndef DYNAMIC_VINO_LIB_INPUTS_VIDEO_INPUT_H -#define DYNAMIC_VINO_LIB_INPUTS_VIDEO_INPUT_H +#ifndef VINO_CORE_LIB_INPUTS_VIDEO_INPUT_H +#define VINO_CORE_LIB_INPUTS_VIDEO_INPUT_H #include #include -#include "dynamic_vino_lib/inputs/base_input.h" +#include "vino_core_lib/inputs/base_input.h" namespace Input { @@ -69,4 +69,4 @@ class Video : public BaseInputDevice }; } // namespace Input -#endif // DYNAMIC_VINO_LIB_INPUTS_VIDEO_INPUT_H +#endif // VINO_CORE_LIB_INPUTS_VIDEO_INPUT_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/models/age_gender_detection_model.h b/vino_core_lib/include/vino_core_lib/models/age_gender_detection_model.h similarity index 89% rename from dynamic_vino_lib/include/dynamic_vino_lib/models/age_gender_detection_model.h rename to vino_core_lib/include/vino_core_lib/models/age_gender_detection_model.h index 586b0e86..c24a0977 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/models/age_gender_detection_model.h +++ b/vino_core_lib/include/vino_core_lib/models/age_gender_detection_model.h @@ -19,11 +19,11 @@ * @file age_gender_detection_model.h */ -#ifndef DYNAMIC_VINO_LIB_MODELS_AGE_GENDER_DETECTION_MODEL_H -#define DYNAMIC_VINO_LIB_MODELS_AGE_GENDER_DETECTION_MODEL_H +#ifndef VINO_CORE_LIB_MODELS_AGE_GENDER_DETECTION_MODEL_H +#define VINO_CORE_LIB_MODELS_AGE_GENDER_DETECTION_MODEL_H #include -#include "dynamic_vino_lib/models/base_model.h" +#include "vino_core_lib/models/base_model.h" namespace Models { @@ -76,4 +76,4 @@ class AgeGenderDetectionModel : public BaseModel }; } // namespace Models -#endif // DYNAMIC_VINO_LIB_MODELS_AGE_GENDER_DETECTION_MODEL_H +#endif // VINO_CORE_LIB_MODELS_AGE_GENDER_DETECTION_MODEL_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/models/base_model.h b/vino_core_lib/include/vino_core_lib/models/base_model.h similarity index 100% rename from dynamic_vino_lib/include/dynamic_vino_lib/models/base_model.h rename to vino_core_lib/include/vino_core_lib/models/base_model.h diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/models/emotion_detection_model.h b/vino_core_lib/include/vino_core_lib/models/emotion_detection_model.h similarity index 87% rename from dynamic_vino_lib/include/dynamic_vino_lib/models/emotion_detection_model.h rename to vino_core_lib/include/vino_core_lib/models/emotion_detection_model.h index 65a1e490..6856a943 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/models/emotion_detection_model.h +++ b/vino_core_lib/include/vino_core_lib/models/emotion_detection_model.h @@ -19,11 +19,11 @@ * @file emotion_detection_model.h */ -#ifndef DYNAMIC_VINO_LIB_MODELS_EMOTION_DETECTION_MODEL_H -#define DYNAMIC_VINO_LIB_MODELS_EMOTION_DETECTION_MODEL_H +#ifndef VINO_CORE_LIB_MODELS_EMOTION_DETECTION_MODEL_H +#define VINO_CORE_LIB_MODELS_EMOTION_DETECTION_MODEL_H #include -#include "dynamic_vino_lib/models/base_model.h" +#include "vino_core_lib/models/base_model.h" namespace Models { @@ -59,4 +59,4 @@ class EmotionDetectionModel : public BaseModel }; } // namespace Models -#endif // DYNAMIC_VINO_LIB_MODELS_EMOTION_DETECTION_MODEL_H +#endif // VINO_CORE_LIB_MODELS_EMOTION_DETECTION_MODEL_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/models/face_detection_model.h b/vino_core_lib/include/vino_core_lib/models/face_detection_model.h similarity index 88% rename from dynamic_vino_lib/include/dynamic_vino_lib/models/face_detection_model.h rename to vino_core_lib/include/vino_core_lib/models/face_detection_model.h index 778c1db3..284cfc58 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/models/face_detection_model.h +++ b/vino_core_lib/include/vino_core_lib/models/face_detection_model.h @@ -19,11 +19,11 @@ * @file face_detection_model.h */ -#ifndef DYNAMIC_VINO_LIB_MODELS_FACE_DETECTION_MODEL_H -#define DYNAMIC_VINO_LIB_MODELS_FACE_DETECTION_MODEL_H +#ifndef VINO_CORE_LIB_MODELS_FACE_DETECTION_MODEL_H +#define VINO_CORE_LIB_MODELS_FACE_DETECTION_MODEL_H #include -#include "dynamic_vino_lib/models/base_model.h" +#include "vino_core_lib/models/base_model.h" namespace Models { @@ -69,4 +69,4 @@ class FaceDetectionModel : public BaseModel }; } // namespace Models -#endif // DYNAMIC_VINO_LIB_MODELS_FACE_DETECTION_MODEL_H +#endif // VINO_CORE_LIB_MODELS_FACE_DETECTION_MODEL_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/models/head_pose_detection_model.h b/vino_core_lib/include/vino_core_lib/models/head_pose_detection_model.h similarity index 89% rename from dynamic_vino_lib/include/dynamic_vino_lib/models/head_pose_detection_model.h rename to vino_core_lib/include/vino_core_lib/models/head_pose_detection_model.h index db3eb498..9816d6b1 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/models/head_pose_detection_model.h +++ b/vino_core_lib/include/vino_core_lib/models/head_pose_detection_model.h @@ -19,11 +19,11 @@ * @file head_pose_detection_model.h */ -#ifndef DYNAMIC_VINO_LIB_MODELS_HEAD_POSE_DETECTION_MODEL_H -#define DYNAMIC_VINO_LIB_MODELS_HEAD_POSE_DETECTION_MODEL_H +#ifndef VINO_CORE_LIB_MODELS_HEAD_POSE_DETECTION_MODEL_H +#define VINO_CORE_LIB_MODELS_HEAD_POSE_DETECTION_MODEL_H #include -#include "dynamic_vino_lib/models/base_model.h" +#include "vino_core_lib/models/base_model.h" namespace Models { @@ -81,4 +81,4 @@ class HeadPoseDetectionModel : public BaseModel }; } // namespace Models -#endif // DYNAMIC_VINO_LIB_MODELS_HEAD_POSE_DETECTION_MODEL_H +#endif // VINO_CORE_LIB_MODELS_HEAD_POSE_DETECTION_MODEL_H diff --git a/vino_core_lib/include/vino_core_lib/models/human_pose_estimation_model.h b/vino_core_lib/include/vino_core_lib/models/human_pose_estimation_model.h new file mode 100644 index 00000000..f96ddeb3 --- /dev/null +++ b/vino_core_lib/include/vino_core_lib/models/human_pose_estimation_model.h @@ -0,0 +1,67 @@ +// TODO: Add license + +/** + * @brief A header file with declaration for FaceDetectionModel Class + * @file human_pose_estimation_model.h + */ + +#ifndef VINO_CORE_LIB_MODELS_HUMAN_POSE_ESTIMATION_MODEL_H +#define VINO_CORE_LIB_MODELS_HUMAN_POSE_ESTIMATION_MODEL_H + +#include +#include "vino_core_lib/models/base_model.h" + +namespace Models +{ +/** + * @class HumanPoseEstimationModel + * @brief This class generates the human pose estimation model. + */ +class HumanPoseEstimationModel : public BaseModel +{ +public: + HumanPoseEstimationModel(const std::string&, int, int, int); + + /** + * @brief Get the input name. + * @return Input name. + */ + inline const std::string getInputName() const + { + return input_; + } + /** + * @brief Get the age from the detection reuslt. + * @return Detected age. + */ + inline const std::string getOutputKeypointsName() const + { + return output_keypoints_; + } + /** + * @brief Get the gender from the detection reuslt. + * @return Detected gender. + */ + inline const std::string getOutputHeatmapName() const + { + return output_heatmap_; + } + /** + * @brief Get the name of this detection model. + * @return Name of the model. + */ + const std::string getModelName() const override; + +protected: + void checkLayerProperty(const InferenceEngine::CNNNetReader::Ptr&) override; + void setLayerProperty(InferenceEngine::CNNNetReader::Ptr) override; + + private: + std::string input_; + std::string output_keypoints_; + std::string output_heatmap_; +}; + +} // namespace Models + +#endif // VINO_CORE_LIB_MODELS_HUMAN_POSE_ESTIMATION_MODEL_H \ No newline at end of file diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/models/object_detection_model.h b/vino_core_lib/include/vino_core_lib/models/object_detection_model.h similarity index 88% rename from dynamic_vino_lib/include/dynamic_vino_lib/models/object_detection_model.h rename to vino_core_lib/include/vino_core_lib/models/object_detection_model.h index b0fe6c9f..23a23613 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/models/object_detection_model.h +++ b/vino_core_lib/include/vino_core_lib/models/object_detection_model.h @@ -17,10 +17,10 @@ * @brief A header file with declaration for ObjectDetectionModel Class * @file face_detection_model.h */ -#ifndef DYNAMIC_VINO_LIB_MODELS_OBJECT_DETECTION_MODEL_H -#define DYNAMIC_VINO_LIB_MODELS_OBJECT_DETECTION_MODEL_H +#ifndef VINO_CORE_LIB_MODELS_OBJECT_DETECTION_MODEL_H +#define VINO_CORE_LIB_MODELS_OBJECT_DETECTION_MODEL_H #include -#include "dynamic_vino_lib/models/base_model.h" +#include "vino_core_lib/models/base_model.h" namespace Models { /** * @class ObjectDetectionModel @@ -48,4 +48,4 @@ class ObjectDetectionModel : public BaseModel { std::string output_; }; } // namespace Models -#endif // DYNAMIC_VINO_LIB_MODELS_OBJECT_DETECTION_MODEL_H +#endif // VINO_CORE_LIB_MODELS_OBJECT_DETECTION_MODEL_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/models/object_segmentation_model.h b/vino_core_lib/include/vino_core_lib/models/object_segmentation_model.h similarity index 88% rename from dynamic_vino_lib/include/dynamic_vino_lib/models/object_segmentation_model.h rename to vino_core_lib/include/vino_core_lib/models/object_segmentation_model.h index 38418824..85eeab91 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/models/object_segmentation_model.h +++ b/vino_core_lib/include/vino_core_lib/models/object_segmentation_model.h @@ -15,10 +15,10 @@ * @brief A header file with declaration for ObjectSegmentationModel Class * @file face_detection_model.h */ -#ifndef DYNAMIC_VINO_LIB__MODELS__OBJECT_SEGMENTATION_MODEL_HPP_ -#define DYNAMIC_VINO_LIB__MODELS__OBJECT_SEGMENTATION_MODEL_HPP_ +#ifndef VINO_CORE_LIB__MODELS__OBJECT_SEGMENTATION_MODEL_HPP_ +#define VINO_CORE_LIB__MODELS__OBJECT_SEGMENTATION_MODEL_HPP_ #include -#include "dynamic_vino_lib/models/base_model.h" +#include "vino_core_lib/models/base_model.h" namespace Models { /** @@ -69,4 +69,4 @@ class ObjectSegmentationModel : public BaseModel std::string detection_output_; }; } // namespace Models -#endif // DYNAMIC_VINO_LIB__MODELS__OBJECT_SEGMENTATION_MODEL_HPP_ +#endif // VINO_CORE_LIB__MODELS__OBJECT_SEGMENTATION_MODEL_HPP_ diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/models/person_reidentification_model.h b/vino_core_lib/include/vino_core_lib/models/person_reidentification_model.h similarity index 85% rename from dynamic_vino_lib/include/dynamic_vino_lib/models/person_reidentification_model.h rename to vino_core_lib/include/vino_core_lib/models/person_reidentification_model.h index a9cff01f..f3dba956 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/models/person_reidentification_model.h +++ b/vino_core_lib/include/vino_core_lib/models/person_reidentification_model.h @@ -16,10 +16,10 @@ * @brief A header file with declaration for PersonReidentificationModel Class * @file face_detection_model.h */ -#ifndef DYNAMIC_VINO_LIB__MODELS__PERSON_REIDENTIFICATION_MODEL_HPP_ -#define DYNAMIC_VINO_LIB__MODELS__PERSON_REIDENTIFICATION_MODEL_HPP_ +#ifndef VINO_CORE_LIB__MODELS__PERSON_REIDENTIFICATION_MODEL_HPP_ +#define VINO_CORE_LIB__MODELS__PERSON_REIDENTIFICATION_MODEL_HPP_ #include -#include "dynamic_vino_lib/models/base_model.h" +#include "vino_core_lib/models/base_model.h" namespace Models { /** @@ -45,4 +45,4 @@ class PersonReidentificationModel : public BaseModel std::string output_; }; } // namespace Models -#endif // DYNAMIC_VINO_LIB__MODELS__PERSON_REIDENTIFICATION_MODEL_HPP_ +#endif // VINO_CORE_LIB__MODELS__PERSON_REIDENTIFICATION_MODEL_HPP_ diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/outputs/base_output.h b/vino_core_lib/include/vino_core_lib/outputs/base_output.h similarity index 59% rename from dynamic_vino_lib/include/dynamic_vino_lib/outputs/base_output.h rename to vino_core_lib/include/vino_core_lib/outputs/base_output.h index 00cafc1d..0f0a0401 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/outputs/base_output.h +++ b/vino_core_lib/include/vino_core_lib/outputs/base_output.h @@ -19,21 +19,22 @@ * @file head_pose_detection_model.h */ -#ifndef DYNAMIC_VINO_LIB_OUTPUTS_BASE_OUTPUT_H -#define DYNAMIC_VINO_LIB_OUTPUTS_BASE_OUTPUT_H +#ifndef VINO_CORE_LIB_OUTPUTS_BASE_OUTPUT_H +#define VINO_CORE_LIB_OUTPUTS_BASE_OUTPUT_H #include #include -#include "dynamic_vino_lib/inferences/age_gender_detection.h" -#include "dynamic_vino_lib/inferences/base_inference.h" -#include "dynamic_vino_lib/inferences/emotions_detection.h" -#include "dynamic_vino_lib/inferences/face_detection.h" -#include "dynamic_vino_lib/inferences/head_pose_detection.h" -#include "dynamic_vino_lib/inferences/object_detection.h" -#include "dynamic_vino_lib/inferences/object_segmentation.h" -#include "dynamic_vino_lib/inferences/person_reidentification.h" -#include "dynamic_vino_lib/services/frame_processing_server.h" +#include "vino_core_lib/inferences/age_gender_detection.h" +#include "vino_core_lib/inferences/base_inference.h" +#include "vino_core_lib/inferences/emotions_detection.h" +#include "vino_core_lib/inferences/face_detection.h" +#include "vino_core_lib/inferences/head_pose_detection.h" +#include "vino_core_lib/inferences/object_detection.h" +#include "vino_core_lib/inferences/object_segmentation.h" +#include "vino_core_lib/inferences/person_reidentification.h" +#include "vino_core_lib/inferences/human_pose_estimation.h" +#include "vino_core_lib/services/frame_processing_server.h" #include "opencv2/opencv.hpp" class Pipeline; @@ -56,45 +57,51 @@ class BaseOutput * @brief Generate output content according to the face detection result. */ virtual void accept( - const std::vector&) + const std::vector&) { } /** * @brief Generate output content according to the face detection result. */ virtual void accept( - const std::vector&) + const std::vector&) { } /** * @brief Generate output content according to the emotion detection result. */ - virtual void accept(const std::vector&) + virtual void accept(const std::vector&) { } /** * @brief Generate output content according to the age and gender detection * result. */ - virtual void accept(const std::vector&) + virtual void accept(const std::vector&) { } /** * @brief Generate output content according to the headpose detection result. */ - virtual void accept(const std::vector&) + virtual void accept(const std::vector&) { } /** * @brief Generate output content according to the object segmentation result. */ - virtual void accept(const std::vector&) + virtual void accept(const std::vector&) { } /** * @brief Generate output content according to the person reidentification result. */ - virtual void accept(const std::vector &) + virtual void accept(const std::vector &) + { + } + /** + * @brief Generate output content according to the human pose estimation result. + */ + virtual void accept(const std::vector &) { } /** @@ -115,17 +122,19 @@ class BaseOutput virtual void setServiceResponseForFace( boost::shared_ptr response) {} virtual void setServiceResponse( - boost::shared_ptr response) {} + boost::shared_ptr response) {} + virtual void setServiceResponse( + boost::shared_ptr response) {} virtual void setServiceResponse( - boost::shared_ptr response) {} + boost::shared_ptr response) {} virtual void setServiceResponse( - boost::shared_ptr response) {} + boost::shared_ptr response) {} virtual void setServiceResponse( - boost::shared_ptr response) {} + boost::shared_ptr response) {} virtual void setServiceResponse( - boost::shared_ptr response) {} + boost::shared_ptr response) {} virtual void setServiceResponse( - boost::shared_ptr response) {} + boost::shared_ptr response) {} Pipeline* getPipeline() const; cv::Mat getFrame() const; virtual void clearData() {} @@ -135,4 +144,4 @@ class BaseOutput Pipeline* pipeline_; }; } // namespace Outputs -#endif // DYNAMIC_VINO_LIB_OUTPUTS_BASE_OUTPUT_H +#endif // VINO_CORE_LIB_OUTPUTS_BASE_OUTPUT_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/outputs/image_window_output.h b/vino_core_lib/include/vino_core_lib/outputs/image_window_output.h similarity index 78% rename from dynamic_vino_lib/include/dynamic_vino_lib/outputs/image_window_output.h rename to vino_core_lib/include/vino_core_lib/outputs/image_window_output.h index 2bdca7c2..4fb41274 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/outputs/image_window_output.h +++ b/vino_core_lib/include/vino_core_lib/outputs/image_window_output.h @@ -19,12 +19,12 @@ * @file image_window_output.h */ -#ifndef DYNAMIC_VINO_LIB_OUTPUTS_IMAGE_WINDOW_OUTPUT_H -#define DYNAMIC_VINO_LIB_OUTPUTS_IMAGE_WINDOW_OUTPUT_H +#ifndef VINO_CORE_LIB_OUTPUTS_IMAGE_WINDOW_OUTPUT_H +#define VINO_CORE_LIB_OUTPUTS_IMAGE_WINDOW_OUTPUT_H #include #include -#include "dynamic_vino_lib/outputs/base_output.h" +#include "vino_core_lib/outputs/base_output.h" namespace Outputs { @@ -58,49 +58,57 @@ class ImageWindowOutput : public BaseOutput * @param[in] A face detection result objetc. */ void accept( - const std::vector&) override; + const std::vector&) override; /** * @brief Generate image window output content according to * the emotion detection result. * @param[in] A emotion detection result objetc. */ - void accept(const std::vector&) override; + void accept(const std::vector&) override; /** * @brief Generate image window output content according to * the age and gender detection result. * @param[in] A head pose detection result objetc. */ - void accept(const std::vector&) override; + void accept(const std::vector&) override; /** * @brief Generate image window output content according to * the headpose detection result. * @param[in] An age gender detection result objetc. */ - void accept(const std::vector&) override; + void accept(const std::vector&) override; /** * @brief Generate image window output content according to * the object detection result. * @param[in] An object detection result objetc. */ - void accept(const std::vector&) override; + void accept(const std::vector&) override; /** * @brief Generate image window output content according to * the object segmentation result. * @param[in] An object segmentation result objetc. */ - void accept(const std::vector&) override; + void accept(const std::vector&) override; /** * @brief Generate image window output content according to * the person re-ID result. * @param[in] An object segmentation result objetc. */ - void accept(const std::vector &) override; + void accept(const std::vector &) override; + + /** + * @brief Generate image window output content according to + * the human pose estimation result. + * @param[in] An human pose estimation result objetc. + */ + void accept(const std::vector &) override; + /** * @brief Merge mask for image window ouput * the object segmentation result. * @param[in] An object segmentation result objetc. */ - void mergeMask(const std::vector &); + void mergeMask(const std::vector &); private: unsigned findOutput(const cv::Rect &); @@ -128,6 +136,7 @@ class ImageWindowOutput : public BaseOutput cv::Point hp_y; // for headpose, end point of yAxis cv::Point hp_zs; // for headpose, start point of zAxis cv::Point hp_ze; // for headpose, end point of zAxis + std::vector kp; // for humanpose, keypoints }; std::vector outputs_; @@ -143,4 +152,4 @@ class ImageWindowOutput : public BaseOutput }; }; } // namespace Outputs -#endif // DYNAMIC_VINO_LIB_OUTPUTS_IMAGE_WINDOW_OUTPUT_H +#endif // VINO_CORE_LIB_OUTPUTS_IMAGE_WINDOW_OUTPUT_H diff --git a/vino_core_lib/include/vino_core_lib/outputs/ros_service_output.h b/vino_core_lib/include/vino_core_lib/outputs/ros_service_output.h new file mode 100644 index 00000000..9615c7c3 --- /dev/null +++ b/vino_core_lib/include/vino_core_lib/outputs/ros_service_output.h @@ -0,0 +1,89 @@ +// Copyright (c) 2018 Intel Corporation +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +/** + * @brief A header file with declaration for RosTopicOutput Class + * @file ros_topic_output.h + */ + +#ifndef VINO_CORE_LIB__OUTPUTS__ROS_SERVICE_OUTPUT_HPP_ +#define VINO_CORE_LIB__OUTPUTS__ROS_SERVICE_OUTPUT_HPP_ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include +#include +#include + +#include "vino_core_lib/inferences/face_detection.h" +#include "vino_core_lib/outputs/ros_topic_output.h" + +namespace Outputs +{ +/** + * @class RosServiceOutput + * @brief This class handles and publish the detection result for service calling. + */ +class RosServiceOutput : public RosTopicOutput +{ +public: + RosServiceOutput() {} + + /** + * @brief Publish all the detected infomations generated by the accept + * functions with ros topic. + */ + void handleOutput() override {} + void clearData(); + + void setServiceResponse(boost::shared_ptr response); + void setResponseForFace(boost::shared_ptr response); + void setServiceResponse(boost::shared_ptr response); + void setServiceResponse(boost::shared_ptr response); + void setServiceResponse(boost::shared_ptr response); + void setServiceResponse(boost::shared_ptr response); + void setServiceResponse(boost::shared_ptr response); + void setServiceResponse(boost::shared_ptr response); + void setServiceResponse(boost::shared_ptr response); + +private: + const std::string service_name_; +}; +} // namespace Outputs +#endif // VINO_CORE_LIB__OUTPUTS__ROS_SERVICE_OUTPUT_HPP_ diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/outputs/ros_topic_output.h b/vino_core_lib/include/vino_core_lib/outputs/ros_topic_output.h similarity index 60% rename from dynamic_vino_lib/include/dynamic_vino_lib/outputs/ros_topic_output.h rename to vino_core_lib/include/vino_core_lib/outputs/ros_topic_output.h index 583c9afd..adfc563d 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/outputs/ros_topic_output.h +++ b/vino_core_lib/include/vino_core_lib/outputs/ros_topic_output.h @@ -19,31 +19,33 @@ * @file ros_topic_output.h */ -#ifndef DYNAMIC_VINO_LIB_OUTPUTS_ROS_TOPIC_OUTPUT_H -#define DYNAMIC_VINO_LIB_OUTPUTS_ROS_TOPIC_OUTPUT_H +#ifndef VINO_CORE_LIB_OUTPUTS_ROS_TOPIC_OUTPUT_H +#define VINO_CORE_LIB_OUTPUTS_ROS_TOPIC_OUTPUT_H #include #include #include #include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include #include #include #include #include -#include "dynamic_vino_lib/inferences/face_detection.h" -#include "dynamic_vino_lib/outputs/base_output.h" +#include "vino_core_lib/inferences/face_detection.h" +#include "vino_core_lib/outputs/base_output.h" namespace Outputs { @@ -70,44 +72,51 @@ class RosTopicOutput : public BaseOutput * the person reidentification result. * @param[in] results a bundle of person reidentification results. */ - void accept(const std::vector &) override; + void accept(const std::vector &) override; /** * @brief Generate ros topic infomation according to * the object segmentation result. * @param[in] results a bundle of object segmentation results. */ - void accept(const std::vector &) override; + void accept(const std::vector &) override; /** * @brief Generate ros topic infomation according to * the face detection result. * @param[in] An face detection result objetc. */ void accept( - const std::vector&) override; + const std::vector&) override; /** * @brief Generate ros topic infomation according to * the emotion detection result. * @param[in] An emotion detection result objetc. */ - void accept(const std::vector&) override; + void accept(const std::vector&) override; /** * @brief Generate ros topic infomation according to * the age gender detection result. * @param[in] An age gender detection result objetc. */ - void accept(const std::vector &) override; + void accept(const std::vector &) override; /**detected_objects_topic_ * @brief Generate ros topic infomation according to * the headpose detection result. * @param[in] An head pose detection result objetc. */ - void accept(const std::vector&) override; + void accept(const std::vector&) override; /** * @brief Generate ros topic infomation according to * the headpose detection result. * @param[in] An head pose detection result objetc. */ - void accept(const std::vector&) override; + void accept(const std::vector&) override; + + /** + * @brief Generate ros topic infomation according to + * the human pose estimation result. + * @param[in] An human pose estimation result objetc. + */ + void accept(const std::vector&) override; private: @@ -119,18 +128,20 @@ class RosTopicOutput : public BaseOutput ros::Publisher pub_face_; std::shared_ptr faces_msg_ptr_; ros::Publisher pub_emotion_; - std::shared_ptr emotions_msg_ptr_; + std::shared_ptr emotions_msg_ptr_; ros::Publisher pub_age_gender_; - std::shared_ptr age_gender_msg_ptr_; + std::shared_ptr age_gender_msg_ptr_; ros::Publisher pub_headpose_; - std::shared_ptr headpose_msg_ptr_; + std::shared_ptr headpose_msg_ptr_; ros::Publisher pub_object_; std::shared_ptr object_msg_ptr_; ros::Publisher pub_person_reid_; - std::shared_ptr person_reid_msg_ptr_; + std::shared_ptr person_reid_msg_ptr_; ros::Publisher pub_segmented_object_; - std::shared_ptr segmented_object_msg_ptr_; + std::shared_ptr segmented_object_msg_ptr_; + ros::Publisher pub_human_pose_; + std::shared_ptr human_pose_msg_ptr_; }; } // namespace Outputs -#endif // DYNAMIC_VINO_LIB_OUTPUTS_ROS_TOPIC_OUTPUT_H +#endif // VINO_CORE_LIB_OUTPUTS_ROS_TOPIC_OUTPUT_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/outputs/rviz_output.h b/vino_core_lib/include/vino_core_lib/outputs/rviz_output.h similarity index 71% rename from dynamic_vino_lib/include/dynamic_vino_lib/outputs/rviz_output.h rename to vino_core_lib/include/vino_core_lib/outputs/rviz_output.h index 595e1864..82a5c6b0 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/outputs/rviz_output.h +++ b/vino_core_lib/include/vino_core_lib/outputs/rviz_output.h @@ -17,8 +17,8 @@ * @file image_window_output.h */ -#ifndef DYNAMIC_VINO_LIB__OUTPUTS__RVIZ_OUTPUT_HPP_ -#define DYNAMIC_VINO_LIB__OUTPUTS__RVIZ_OUTPUT_HPP_ +#ifndef VINO_CORE_LIB__OUTPUTS__RVIZ_OUTPUT_HPP_ +#define VINO_CORE_LIB__OUTPUTS__RVIZ_OUTPUT_HPP_ @@ -26,8 +26,8 @@ #include #include -#include "dynamic_vino_lib/outputs/base_output.h" -#include "dynamic_vino_lib/outputs/image_window_output.h" +#include "vino_core_lib/outputs/base_output.h" +#include "vino_core_lib/outputs/image_window_output.h" namespace Outputs { @@ -54,44 +54,53 @@ class RvizOutput : public BaseOutput * the face detection result. * @param[in] A face detection result objetc. */ - void accept(const std::vector &) override; + void accept(const std::vector &) override; /** * @brief Generate rviz output content according to * the object detection result. * @param[in] results A bundle of object detection results. */ - void accept(const std::vector &) override; + void accept(const std::vector &) override; /** * @brief Generate rviz output content according to * the object segmentation result. * @param[in] results A bundle of object segmentation results. */ - void accept(const std::vector &) override; + void accept(const std::vector &) override; /** * @brief Generate rviz output content according to * the age and gender detection result. * @param[in] A head pose detection result objetc. */ - void accept(const std::vector &) override; + void accept(const std::vector &) override; /** * @brief Generate rviz output content according to * the headpose detection result. * @param[in] An age gender detection result objetc. */ - void accept(const std::vector &) override; + void accept(const std::vector &) override; /** * @brief Generate rviz output content according to * the object segmentation result. * @param[in] An object segmentation result objetc. */ - void accept(const std::vector&) override; + void accept(const std::vector&) override; /** * @brief Generate rviz output content according to * the person re-ID result. * @param[in] An object segmentation result objetc. */ - void accept(const std::vector &) override; + void accept(const std::vector &) override; + + /** + * @brief Generate rviz output content according to + * the human pose estimation result. + * @param[in] An object segmentation result objetc. + */ + void accept(const std::vector &) override; + + // TODO remove comment. /** * @brief Merge mask for image window ouput * the object segmentation result. @@ -106,5 +115,5 @@ class RvizOutput : public BaseOutput std::shared_ptr image_window_output_; }; } // namespace Outputs -#endif // DYNAMIC_VINO_LIB__OUTPUTS__RVIZ_OUTPUT_HPP_ +#endif // VINO_CORE_LIB__OUTPUTS__RVIZ_OUTPUT_HPP_ diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/pipeline.h b/vino_core_lib/include/vino_core_lib/pipeline.h similarity index 90% rename from dynamic_vino_lib/include/dynamic_vino_lib/pipeline.h rename to vino_core_lib/include/vino_core_lib/pipeline.h index bf7394f2..db8d7ddc 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/pipeline.h +++ b/vino_core_lib/include/vino_core_lib/pipeline.h @@ -18,8 +18,8 @@ * @brief a header file with declaration of Pipeline class * @file pipeline.h */ -#ifndef DYNAMIC_VINO_LIB_PIPELINE_H -#define DYNAMIC_VINO_LIB_PIPELINE_H +#ifndef VINO_CORE_LIB_PIPELINE_H +#define VINO_CORE_LIB_PIPELINE_H #include #include @@ -29,10 +29,10 @@ #include #include -#include "dynamic_vino_lib/inferences/base_inference.h" -#include "dynamic_vino_lib/inputs/standard_camera.h" -#include "dynamic_vino_lib/outputs/base_output.h" -#include "dynamic_vino_lib/pipeline_params.h" +#include "vino_core_lib/inferences/base_inference.h" +#include "vino_core_lib/inputs/standard_camera.h" +#include "vino_core_lib/outputs/base_output.h" +#include "vino_core_lib/pipeline_params.h" #include "opencv2/opencv.hpp" /** @@ -61,7 +61,7 @@ class Pipeline * @return whether the add operation is successful */ bool add(const std::string& parent, const std::string& name, - std::shared_ptr inference); + std::shared_ptr inference); /** * @brief Add inference network to the pipeline. * @param[in] name name of the current inference network. @@ -69,7 +69,7 @@ class Pipeline * @return whether the add operation is successful */ bool add(const std::string & name, - std::shared_ptr inference); + std::shared_ptr inference); /** * @brief Add output device to the pipeline. * @param[in] parent name of the parent inference. @@ -162,7 +162,7 @@ class Pipeline std::shared_ptr input_device_; std::string input_device_name_; std::multimap next_; - std::map> + std::map> name_to_detection_map_; std::map> name_to_output_map_; @@ -178,4 +178,4 @@ class Pipeline int fps_ = 0; }; -#endif // DYNAMIC_VINO_LIB_PIPELINE_H_ +#endif // vino_core_lib_PIPELINE_H_ diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/pipeline_manager.h b/vino_core_lib/include/vino_core_lib/pipeline_manager.h similarity index 77% rename from dynamic_vino_lib/include/dynamic_vino_lib/pipeline_manager.h rename to vino_core_lib/include/vino_core_lib/pipeline_manager.h index 3deb2aa9..3a6d81db 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/pipeline_manager.h +++ b/vino_core_lib/include/vino_core_lib/pipeline_manager.h @@ -18,8 +18,8 @@ * @brief a header file with declaration of Pipeline Manager class * @file pipeline_manager.hpp */ -#ifndef DYNAMIC_VINO_LIB__PIPELINE_MANAGER_HPP_ -#define DYNAMIC_VINO_LIB__PIPELINE_MANAGER_HPP_ +#ifndef VINO_CORE_LIB__PIPELINE_MANAGER_HPP_ +#define VINO_CORE_LIB__PIPELINE_MANAGER_HPP_ #include #include @@ -30,7 +30,7 @@ #include #include -#include "dynamic_vino_lib/pipeline.h" +#include "vino_core_lib/pipeline.h" /** * @class PipelineManager @@ -88,24 +88,26 @@ class PipelineManager { parseInputDevice(const Params::ParamManager::PipelineParams& params); std::map> parseOutput( const Params::ParamManager::PipelineParams& params); - std::map> + std::map> parseInference(const Params::ParamManager::PipelineParams& params); - std::shared_ptr createFaceDetection( + std::shared_ptr createFaceDetection( const Params::ParamManager::InferenceParams& infer); - std::shared_ptr createAgeGenderRecognition( + std::shared_ptr createAgeGenderRecognition( const Params::ParamManager::InferenceParams& infer); - std::shared_ptr createEmotionRecognition( + std::shared_ptr createEmotionRecognition( const Params::ParamManager::InferenceParams& infer); - std::shared_ptr createHeadPoseEstimation( + std::shared_ptr createHeadPoseEstimation( const Params::ParamManager::InferenceParams& infer); - std::shared_ptr createObjectDetection( + std::shared_ptr createObjectDetection( const Params::ParamManager::InferenceParams& infer); - std::shared_ptr createObjectSegmentation( + std::shared_ptr createObjectSegmentation( const Params::ParamManager::InferenceParams& infer); - std::shared_ptr createPersonReidentification( + std::shared_ptr createPersonReidentification( + const Params::ParamManager::InferenceParams& infer); + std::shared_ptr createHumanPoseEstimation( const Params::ParamManager::InferenceParams& infer); std::map pipelines_; std::map plugins_for_devices_; }; -#endif // DYNAMIC_VINO_LIB__PIPELINE_MANAGER_HPP_ \ No newline at end of file +#endif // VINO_CORE_LIB__PIPELINE_MANAGER_HPP_ diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/pipeline_params.h b/vino_core_lib/include/vino_core_lib/pipeline_params.h similarity index 88% rename from dynamic_vino_lib/include/dynamic_vino_lib/pipeline_params.h rename to vino_core_lib/include/vino_core_lib/pipeline_params.h index 4344a48c..2a934d42 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/pipeline_params.h +++ b/vino_core_lib/include/vino_core_lib/pipeline_params.h @@ -18,8 +18,8 @@ * @brief a header file with declaration of Pipeline class * @file pipeline_param.h */ -#ifndef DYNAMIC_VINO_LIB_PIPELINE_PARAMS_H -#define DYNAMIC_VINO_LIB_PIPELINE_PARAMS_H +#ifndef VINO_CORE_LIB_PIPELINE_PARAMS_H +#define VINO_CORE_LIB_PIPELINE_PARAMS_H #include #include @@ -29,9 +29,9 @@ #include #include -#include "dynamic_vino_lib/inferences/base_inference.h" -#include "dynamic_vino_lib/inputs/standard_camera.h" -#include "dynamic_vino_lib/outputs/base_output.h" +#include "vino_core_lib/inferences/base_inference.h" +#include "vino_core_lib/inputs/standard_camera.h" +#include "vino_core_lib/outputs/base_output.h" #include "opencv2/opencv.hpp" #include "vino_param_lib/param_manager.h" @@ -55,6 +55,7 @@ const char kInferTpye_HeadPoseEstimation[] = "HeadPoseEstimation"; const char kInferTpye_ObjectDetection[] = "ObjectDetection"; const char kInferTpye_ObjectSegmentation[] = "ObjectSegmentation"; const char kInferTpye_PersonReidentification[] = "PersonReidentification"; +const char kInferTpye_HumanPoseEstimation[] = "HumanPoseEstimation"; /** * @class PipelineParams @@ -81,4 +82,4 @@ class PipelineParams Params::ParamManager::PipelineParams params_; }; -#endif // DYNAMIC_VINO_LIB_PIPELINE_PARAMS_H +#endif // VINO_CORE_LIB_PIPELINE_PARAMS_H diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/services/frame_processing_server.h b/vino_core_lib/include/vino_core_lib/services/frame_processing_server.h similarity index 76% rename from dynamic_vino_lib/include/dynamic_vino_lib/services/frame_processing_server.h rename to vino_core_lib/include/vino_core_lib/services/frame_processing_server.h index 0284a47c..7d1cf431 100644 --- a/dynamic_vino_lib/include/dynamic_vino_lib/services/frame_processing_server.h +++ b/vino_core_lib/include/vino_core_lib/services/frame_processing_server.h @@ -17,20 +17,21 @@ #include #include #include -#include -#include -#include -#include -#include -#include +#include +#include +#include +#include +#include +#include -#include -#include -#include -#include +#include +#include +#include +#include #include -#include -#include +#include +#include +#include #include #include @@ -67,4 +68,4 @@ class FrameProcessingServer //: public ros::NodeHandle bool add3(ros::ServiceEvent &); }; } // namespace vino_service -#endif // DYNAMIC_VINO_LIB__SERVICES__FRAME_PROCESSING_SERVER_HPP_ \ No newline at end of file +#endif // DYNAMIC_VINO_LIB__SERVICES__FRAME_PROCESSING_SERVER_HPP_ diff --git a/dynamic_vino_lib/include/dynamic_vino_lib/slog.h b/vino_core_lib/include/vino_core_lib/slog.h similarity index 100% rename from dynamic_vino_lib/include/dynamic_vino_lib/slog.h rename to vino_core_lib/include/vino_core_lib/slog.h diff --git a/dynamic_vino_lib/package.xml b/vino_core_lib/package.xml similarity index 93% rename from dynamic_vino_lib/package.xml rename to vino_core_lib/package.xml index fbab9c42..d9a5b213 100644 --- a/dynamic_vino_lib/package.xml +++ b/vino_core_lib/package.xml @@ -14,7 +14,7 @@ limitations under the License. --> - dynamic_vino_lib + vino_core_lib 0.3.0 a ROS wrapper package for Intel OpenVINO Weizhi Liu @@ -32,7 +32,7 @@ limitations under the License. image_transport cv_bridge object_msgs - people_msgs + vino_people_msgs vino_param_lib roscpp @@ -42,6 +42,6 @@ limitations under the License. image_transport cv_bridge object_msgs - people_msgs + vino_people_msgs vino_param_lib diff --git a/dynamic_vino_lib/src/engines/engine.cpp b/vino_core_lib/src/engines/engine.cpp similarity index 95% rename from dynamic_vino_lib/src/engines/engine.cpp rename to vino_core_lib/src/engines/engine.cpp index db008926..45ddb5d4 100644 --- a/dynamic_vino_lib/src/engines/engine.cpp +++ b/vino_core_lib/src/engines/engine.cpp @@ -18,7 +18,7 @@ * @brief a header file with definition of Engine class * @file engine.cpp */ -#include "dynamic_vino_lib/engines/engine.h" +#include "vino_core_lib/engines/engine.h" Engines::Engine::Engine(InferenceEngine::InferencePlugin plg, const Models::BaseModel::Ptr base_model) diff --git a/dynamic_vino_lib/src/factory.cpp b/vino_core_lib/src/factory.cpp similarity index 91% rename from dynamic_vino_lib/src/factory.cpp rename to vino_core_lib/src/factory.cpp index d886124c..cedd83c5 100644 --- a/dynamic_vino_lib/src/factory.cpp +++ b/vino_core_lib/src/factory.cpp @@ -22,12 +22,12 @@ #include #include -#include "dynamic_vino_lib/factory.h" -#include "dynamic_vino_lib/inputs/image_input.h" -#include "dynamic_vino_lib/inputs/realsense_camera.h" -#include "dynamic_vino_lib/inputs/realsense_camera_topic.h" -#include "dynamic_vino_lib/inputs/standard_camera.h" -#include "dynamic_vino_lib/inputs/video_input.h" +#include "vino_core_lib/factory.h" +#include "vino_core_lib/inputs/image_input.h" +#include "vino_core_lib/inputs/realsense_camera.h" +#include "vino_core_lib/inputs/realsense_camera_topic.h" +#include "vino_core_lib/inputs/standard_camera.h" +#include "vino_core_lib/inputs/video_input.h" std::shared_ptr Factory::makeInputDeviceByName( const std::string& input_device_name, const std::string& input_file_path) diff --git a/dynamic_vino_lib/src/inferences/age_gender_detection.cpp b/vino_core_lib/src/inferences/age_gender_detection.cpp similarity index 65% rename from dynamic_vino_lib/src/inferences/age_gender_detection.cpp rename to vino_core_lib/src/inferences/age_gender_detection.cpp index b087ab2a..f654d85a 100644 --- a/dynamic_vino_lib/src/inferences/age_gender_detection.cpp +++ b/vino_core_lib/src/inferences/age_gender_detection.cpp @@ -22,38 +22,38 @@ #include #include -#include "dynamic_vino_lib/inferences/age_gender_detection.h" -#include "dynamic_vino_lib/outputs/base_output.h" +#include "vino_core_lib/inferences/age_gender_detection.h" +#include "vino_core_lib/outputs/base_output.h" // AgeGenderResult -dynamic_vino_lib::AgeGenderResult::AgeGenderResult(const cv::Rect& location) +vino_core_lib::AgeGenderResult::AgeGenderResult(const cv::Rect& location) : Result(location) { } // AgeGender Detection -dynamic_vino_lib::AgeGenderDetection::AgeGenderDetection() - : dynamic_vino_lib::BaseInference() +vino_core_lib::AgeGenderDetection::AgeGenderDetection() + : vino_core_lib::BaseInference() { } -dynamic_vino_lib::AgeGenderDetection::~AgeGenderDetection() = default; +vino_core_lib::AgeGenderDetection::~AgeGenderDetection() = default; -void dynamic_vino_lib::AgeGenderDetection::loadNetwork( +void vino_core_lib::AgeGenderDetection::loadNetwork( std::shared_ptr network) { valid_model_ = network; setMaxBatchSize(network->getMaxBatchSize()); } -bool dynamic_vino_lib::AgeGenderDetection::enqueue( +bool vino_core_lib::AgeGenderDetection::enqueue( const cv::Mat& frame, const cv::Rect& input_frame_loc) { if (getEnqueuedNum() == 0) { results_.clear(); } - bool succeed = dynamic_vino_lib::BaseInference::enqueue( + bool succeed = vino_core_lib::BaseInference::enqueue( frame, input_frame_loc, 1, getResultsLength(), valid_model_->getInputName()); if (!succeed) return false; @@ -62,14 +62,14 @@ bool dynamic_vino_lib::AgeGenderDetection::enqueue( return true; } -bool dynamic_vino_lib::AgeGenderDetection::submitRequest() +bool vino_core_lib::AgeGenderDetection::submitRequest() { - return dynamic_vino_lib::BaseInference::submitRequest(); + return vino_core_lib::BaseInference::submitRequest(); } -bool dynamic_vino_lib::AgeGenderDetection::fetchResults() +bool vino_core_lib::AgeGenderDetection::fetchResults() { - bool can_fetch = dynamic_vino_lib::BaseInference::fetchResults(); + bool can_fetch = vino_core_lib::BaseInference::fetchResults(); if (!can_fetch) return false; auto request = getEngine()->getRequest(); InferenceEngine::Blob::Ptr genderBlob = @@ -85,23 +85,23 @@ bool dynamic_vino_lib::AgeGenderDetection::fetchResults() return true; } -const int dynamic_vino_lib::AgeGenderDetection::getResultsLength() const +const int vino_core_lib::AgeGenderDetection::getResultsLength() const { return static_cast(results_.size()); } -const dynamic_vino_lib::Result* -dynamic_vino_lib::AgeGenderDetection::getLocationResult(int idx) const +const vino_core_lib::Result* +vino_core_lib::AgeGenderDetection::getLocationResult(int idx) const { return &(results_[idx]); } -const std::string dynamic_vino_lib::AgeGenderDetection::getName() const +const std::string vino_core_lib::AgeGenderDetection::getName() const { return valid_model_->getModelName(); } -const void dynamic_vino_lib::AgeGenderDetection::observeOutput( +const void vino_core_lib::AgeGenderDetection::observeOutput( const std::shared_ptr& output) { if (output != nullptr) diff --git a/dynamic_vino_lib/src/inferences/base_inference.cpp b/vino_core_lib/src/inferences/base_inference.cpp similarity index 74% rename from dynamic_vino_lib/src/inferences/base_inference.cpp rename to vino_core_lib/src/inferences/base_inference.cpp index 4ca7d251..253a64c0 100644 --- a/dynamic_vino_lib/src/inferences/base_inference.cpp +++ b/vino_core_lib/src/inferences/base_inference.cpp @@ -21,26 +21,26 @@ #include -#include "dynamic_vino_lib/inferences/base_inference.h" +#include "vino_core_lib/inferences/base_inference.h" // Result -dynamic_vino_lib::Result::Result(const cv::Rect& location) +vino_core_lib::Result::Result(const cv::Rect& location) { location_ = location; } // BaseInference -dynamic_vino_lib::BaseInference::BaseInference() = default; +vino_core_lib::BaseInference::BaseInference() = default; -dynamic_vino_lib::BaseInference::~BaseInference() = default; +vino_core_lib::BaseInference::~BaseInference() = default; -void dynamic_vino_lib::BaseInference::loadEngine( +void vino_core_lib::BaseInference::loadEngine( const std::shared_ptr engine) { engine_ = engine; } -bool dynamic_vino_lib::BaseInference::submitRequest() +bool vino_core_lib::BaseInference::submitRequest() { if (engine_->getRequest() == nullptr) return false; if (!enqueued_frames) return false; @@ -50,7 +50,7 @@ bool dynamic_vino_lib::BaseInference::submitRequest() return true; } -bool dynamic_vino_lib::BaseInference::fetchResults() +bool vino_core_lib::BaseInference::fetchResults() { if (results_fetched_) return false; results_fetched_ = true; diff --git a/dynamic_vino_lib/src/inferences/emotions_detection.cpp b/vino_core_lib/src/inferences/emotions_detection.cpp similarity index 74% rename from dynamic_vino_lib/src/inferences/emotions_detection.cpp rename to vino_core_lib/src/inferences/emotions_detection.cpp index 9e10e3d3..3aee9eff 100644 --- a/dynamic_vino_lib/src/inferences/emotions_detection.cpp +++ b/vino_core_lib/src/inferences/emotions_detection.cpp @@ -23,39 +23,39 @@ #include #include -#include "dynamic_vino_lib/inferences/emotions_detection.h" -#include "dynamic_vino_lib/outputs/base_output.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/inferences/emotions_detection.h" +#include "vino_core_lib/outputs/base_output.h" +#include "vino_core_lib/slog.h" // EmotionsResult -dynamic_vino_lib::EmotionsResult::EmotionsResult(const cv::Rect& location) +vino_core_lib::EmotionsResult::EmotionsResult(const cv::Rect& location) : Result(location) { } // Emotions Detection -dynamic_vino_lib::EmotionsDetection::EmotionsDetection() - : dynamic_vino_lib::BaseInference() +vino_core_lib::EmotionsDetection::EmotionsDetection() + : vino_core_lib::BaseInference() { } -dynamic_vino_lib::EmotionsDetection::~EmotionsDetection() = default; +vino_core_lib::EmotionsDetection::~EmotionsDetection() = default; -void dynamic_vino_lib::EmotionsDetection::loadNetwork( +void vino_core_lib::EmotionsDetection::loadNetwork( const std::shared_ptr network) { valid_model_ = network; setMaxBatchSize(network->getMaxBatchSize()); } -bool dynamic_vino_lib::EmotionsDetection::enqueue( +bool vino_core_lib::EmotionsDetection::enqueue( const cv::Mat& frame, const cv::Rect& input_frame_loc) { if (getEnqueuedNum() == 0) { results_.clear(); } - bool succeed = dynamic_vino_lib::BaseInference::enqueue( + bool succeed = vino_core_lib::BaseInference::enqueue( frame, input_frame_loc, 1, getResultsLength(), valid_model_->getInputName()); if (!succeed) @@ -69,14 +69,14 @@ bool dynamic_vino_lib::EmotionsDetection::enqueue( return true; } -bool dynamic_vino_lib::EmotionsDetection::submitRequest() +bool vino_core_lib::EmotionsDetection::submitRequest() { - return dynamic_vino_lib::BaseInference::submitRequest(); + return vino_core_lib::BaseInference::submitRequest(); } -bool dynamic_vino_lib::EmotionsDetection::fetchResults() +bool vino_core_lib::EmotionsDetection::fetchResults() { - bool can_fetch = dynamic_vino_lib::BaseInference::fetchResults(); + bool can_fetch = vino_core_lib::BaseInference::fetchResults(); if (!can_fetch) return false; int label_length = static_cast(valid_model_->getLabels().size()); std::string output_name = valid_model_->getOutputName(); @@ -113,23 +113,23 @@ bool dynamic_vino_lib::EmotionsDetection::fetchResults() return true; } -const int dynamic_vino_lib::EmotionsDetection::getResultsLength() const +const int vino_core_lib::EmotionsDetection::getResultsLength() const { return static_cast(results_.size()); } -const dynamic_vino_lib::Result* -dynamic_vino_lib::EmotionsDetection::getLocationResult(int idx) const +const vino_core_lib::Result* +vino_core_lib::EmotionsDetection::getLocationResult(int idx) const { return &(results_[idx]); } -const std::string dynamic_vino_lib::EmotionsDetection::getName() const +const std::string vino_core_lib::EmotionsDetection::getName() const { return valid_model_->getModelName(); } -const void dynamic_vino_lib::EmotionsDetection::observeOutput( +const void vino_core_lib::EmotionsDetection::observeOutput( const std::shared_ptr& output) { if (output != nullptr) diff --git a/dynamic_vino_lib/src/inferences/face_detection.cpp b/vino_core_lib/src/inferences/face_detection.cpp similarity index 75% rename from dynamic_vino_lib/src/inferences/face_detection.cpp rename to vino_core_lib/src/inferences/face_detection.cpp index e7aa3c36..356875b4 100644 --- a/dynamic_vino_lib/src/inferences/face_detection.cpp +++ b/vino_core_lib/src/inferences/face_detection.cpp @@ -24,27 +24,27 @@ #include #include -#include "dynamic_vino_lib/inferences/face_detection.h" -#include "dynamic_vino_lib/outputs/base_output.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/inferences/face_detection.h" +#include "vino_core_lib/outputs/base_output.h" +#include "vino_core_lib/slog.h" // FaceDetectionResult -dynamic_vino_lib::FaceDetectionResult::FaceDetectionResult( +vino_core_lib::FaceDetectionResult::FaceDetectionResult( const cv::Rect& location) : Result(location) { } // FaceDetection -dynamic_vino_lib::FaceDetection::FaceDetection(double show_output_thresh) - : dynamic_vino_lib::BaseInference(), +vino_core_lib::FaceDetection::FaceDetection(double show_output_thresh) + : vino_core_lib::BaseInference(), show_output_thresh_(show_output_thresh) { } -dynamic_vino_lib::FaceDetection::~FaceDetection() = default; +vino_core_lib::FaceDetection::~FaceDetection() = default; -void dynamic_vino_lib::FaceDetection::loadNetwork( +void vino_core_lib::FaceDetection::loadNetwork( const std::shared_ptr network) { valid_model_ = network; @@ -53,7 +53,7 @@ void dynamic_vino_lib::FaceDetection::loadNetwork( setMaxBatchSize(network->getMaxBatchSize()); } -bool dynamic_vino_lib::FaceDetection::enqueue(const cv::Mat& frame, +bool vino_core_lib::FaceDetection::enqueue(const cv::Mat& frame, const cv::Rect& input_frame_loc) { // slog::info << "Face-enqueue" << slog::endl; @@ -62,7 +62,7 @@ bool dynamic_vino_lib::FaceDetection::enqueue(const cv::Mat& frame, width_ = frame.cols; height_ = frame.rows; } - if (!dynamic_vino_lib::BaseInference::enqueue( + if (!vino_core_lib::BaseInference::enqueue( frame, input_frame_loc, 1, 0, valid_model_->getInputName())) { return false; @@ -73,15 +73,15 @@ bool dynamic_vino_lib::FaceDetection::enqueue(const cv::Mat& frame, return true; } -bool dynamic_vino_lib::FaceDetection::submitRequest() +bool vino_core_lib::FaceDetection::submitRequest() { // slog::info << "Face-submitRequest" << slog::endl; - return dynamic_vino_lib::BaseInference::submitRequest(); + return vino_core_lib::BaseInference::submitRequest(); } -bool dynamic_vino_lib::FaceDetection::fetchResults() +bool vino_core_lib::FaceDetection::fetchResults() { - bool can_fetch = dynamic_vino_lib::BaseInference::fetchResults(); + bool can_fetch = vino_core_lib::BaseInference::fetchResults(); if (!can_fetch) return false; bool found_result = false; results_.clear(); @@ -122,23 +122,23 @@ bool dynamic_vino_lib::FaceDetection::fetchResults() return true; } -const int dynamic_vino_lib::FaceDetection::getResultsLength() const +const int vino_core_lib::FaceDetection::getResultsLength() const { return static_cast(results_.size()); } -const dynamic_vino_lib::Result* -dynamic_vino_lib::FaceDetection::getLocationResult(int idx) const +const vino_core_lib::Result* +vino_core_lib::FaceDetection::getLocationResult(int idx) const { return &(results_[idx]); } -const std::string dynamic_vino_lib::FaceDetection::getName() const +const std::string vino_core_lib::FaceDetection::getName() const { return valid_model_->getModelName(); } -const void dynamic_vino_lib::FaceDetection::observeOutput( +const void vino_core_lib::FaceDetection::observeOutput( const std::shared_ptr& output) { if (output != nullptr) diff --git a/dynamic_vino_lib/src/inferences/head_pose_detection.cpp b/vino_core_lib/src/inferences/head_pose_detection.cpp similarity index 67% rename from dynamic_vino_lib/src/inferences/head_pose_detection.cpp rename to vino_core_lib/src/inferences/head_pose_detection.cpp index 490936cf..b6de5827 100644 --- a/dynamic_vino_lib/src/inferences/head_pose_detection.cpp +++ b/vino_core_lib/src/inferences/head_pose_detection.cpp @@ -20,40 +20,40 @@ * @file head_pose_recognition.cpp */ -#include "dynamic_vino_lib/inferences/head_pose_detection.h" +#include "vino_core_lib/inferences/head_pose_detection.h" #include #include -#include "dynamic_vino_lib/outputs/base_output.h" +#include "vino_core_lib/outputs/base_output.h" // HeadPoseResult -dynamic_vino_lib::HeadPoseResult::HeadPoseResult(const cv::Rect& location) +vino_core_lib::HeadPoseResult::HeadPoseResult(const cv::Rect& location) : Result(location) { } // Head Pose Detection -dynamic_vino_lib::HeadPoseDetection::HeadPoseDetection() - : dynamic_vino_lib::BaseInference() +vino_core_lib::HeadPoseDetection::HeadPoseDetection() + : vino_core_lib::BaseInference() { } -dynamic_vino_lib::HeadPoseDetection::~HeadPoseDetection() = default; +vino_core_lib::HeadPoseDetection::~HeadPoseDetection() = default; -void dynamic_vino_lib::HeadPoseDetection::loadNetwork( +void vino_core_lib::HeadPoseDetection::loadNetwork( std::shared_ptr network) { valid_model_ = network; setMaxBatchSize(network->getMaxBatchSize()); } -bool dynamic_vino_lib::HeadPoseDetection::enqueue( +bool vino_core_lib::HeadPoseDetection::enqueue( const cv::Mat& frame, const cv::Rect& input_frame_loc) { if (getEnqueuedNum() == 0) { results_.clear(); } - bool succeed = dynamic_vino_lib::BaseInference::enqueue( + bool succeed = vino_core_lib::BaseInference::enqueue( frame, input_frame_loc, 1, getResultsLength(), valid_model_->getInputName()); if (!succeed) return false; @@ -62,14 +62,14 @@ bool dynamic_vino_lib::HeadPoseDetection::enqueue( return true; } -bool dynamic_vino_lib::HeadPoseDetection::submitRequest() +bool vino_core_lib::HeadPoseDetection::submitRequest() { - return dynamic_vino_lib::BaseInference::submitRequest(); + return vino_core_lib::BaseInference::submitRequest(); } -bool dynamic_vino_lib::HeadPoseDetection::fetchResults() +bool vino_core_lib::HeadPoseDetection::fetchResults() { - bool can_fetch = dynamic_vino_lib::BaseInference::fetchResults(); + bool can_fetch = vino_core_lib::BaseInference::fetchResults(); if (!can_fetch) return false; auto request = getEngine()->getRequest(); InferenceEngine::Blob::Ptr angle_r = @@ -88,23 +88,23 @@ bool dynamic_vino_lib::HeadPoseDetection::fetchResults() return true; } -const int dynamic_vino_lib::HeadPoseDetection::getResultsLength() const +const int vino_core_lib::HeadPoseDetection::getResultsLength() const { return static_cast(results_.size()); } -const dynamic_vino_lib::Result* -dynamic_vino_lib::HeadPoseDetection::getLocationResult(int idx) const +const vino_core_lib::Result* +vino_core_lib::HeadPoseDetection::getLocationResult(int idx) const { return &(results_[idx]); } -const std::string dynamic_vino_lib::HeadPoseDetection::getName() const +const std::string vino_core_lib::HeadPoseDetection::getName() const { return valid_model_->getModelName(); } -const void dynamic_vino_lib::HeadPoseDetection::observeOutput( +const void vino_core_lib::HeadPoseDetection::observeOutput( const std::shared_ptr& output) { if (output != nullptr) diff --git a/vino_core_lib/src/inferences/human_pose_estimation.cpp b/vino_core_lib/src/inferences/human_pose_estimation.cpp new file mode 100644 index 00000000..f1f9bb8a --- /dev/null +++ b/vino_core_lib/src/inferences/human_pose_estimation.cpp @@ -0,0 +1,262 @@ +// TODO add license + +/** + * @brief a header file with declaration of HumanPoseResult class //TODO update + * @file human_pose_estimation.cpp + * + * This file was mostly based on age_gender_detection.cpp. It also uses the code + * from object_detection.cpp. + */ + +#include +#include + +#include "vino_core_lib/inferences/human_pose_estimation.h" +#include "vino_core_lib/outputs/base_output.h" +#include "vino_core_lib/inferences/peak.h" + +// HumanPoseResult +vino_core_lib::HumanPoseResult::HumanPoseResult(const cv::Rect& location) + : Result(location) +{ +} + +vino_core_lib::HumanPoseResult::HumanPoseResult( + const cv::Rect& location, + const std::vector& keypoints, + const float& score) + : keypoints(keypoints), + score(score), + Result(location) +{ +} + +// HumanPoseEstimation +vino_core_lib::HumanPoseEstimation::HumanPoseEstimation( + float minPeaksDistance, + float midPointsScoreThreshold, + float foundMidPointsRatioThreshold, + int minJointsNumber, + float minSubsetScore) + : vino_core_lib::BaseInference(), + upsampleRatio_(4), + minPeaksDistance_(minPeaksDistance), + midPointsScoreThreshold_(midPointsScoreThreshold), + foundMidPointsRatioThreshold_(foundMidPointsRatioThreshold), + minJointsNumber_(minJointsNumber), + minSubsetScore_(minSubsetScore), + stride_(8), + pad_(cv::Vec4i::all(0)) +{ +} + +vino_core_lib::HumanPoseEstimation::~HumanPoseEstimation() = default; + +void vino_core_lib::HumanPoseEstimation::loadNetwork( + std::shared_ptr network) +{ + valid_model_ = network; + setMaxBatchSize(network->getMaxBatchSize()); +} + +bool vino_core_lib::HumanPoseEstimation::enqueue( + const cv::Mat& frame, const cv::Rect& input_frame_loc) +{ + // object_detection.cpp + if (width_ == 0 && height_ == 0) + { + width_ = frame.cols; + height_ = frame.rows; + } + + if (!vino_core_lib::BaseInference::enqueue( + frame, input_frame_loc, 1, 0, valid_model_->getInputName())) + { + return false; + } + Result r(input_frame_loc); + results_.clear(); + results_.emplace_back(r); + return true; +} + +bool vino_core_lib::HumanPoseEstimation::submitRequest() +{ + return vino_core_lib::BaseInference::submitRequest(); +} + +bool vino_core_lib::HumanPoseEstimation::fetchResults() +{ + bool can_fetch = vino_core_lib::BaseInference::fetchResults(); + if (!can_fetch) return false; + auto request = getEngine()->getRequest(); + InferenceEngine::Blob::Ptr keypointsBlob = + request->GetBlob(valid_model_->getOutputKeypointsName()); + InferenceEngine::Blob::Ptr heatmapBlob = + request->GetBlob(valid_model_->getOutputHeatmapName()); + + results_.clear(); + CV_Assert(heatmapBlob->getTensorDesc().getDims()[1] == keypointsNumber_ + 1); + InferenceEngine::SizeVector heatMapDims = + heatmapBlob->getTensorDesc().getDims(); + //std::vector poses = postprocess( + results_ = postprocess( + heatmapBlob->buffer(), + heatMapDims[2] * heatMapDims[3], + keypointsNumber_, + keypointsBlob->buffer(), + heatMapDims[2] * heatMapDims[3], + keypointsBlob->getTensorDesc().getDims()[1], + heatMapDims[3], heatMapDims[2], cv::Size(width_, height_)); + return true; +} + +const int vino_core_lib::HumanPoseEstimation::getResultsLength() const +{ + return static_cast(results_.size()); +} + +const vino_core_lib::Result* +vino_core_lib::HumanPoseEstimation::getLocationResult(int idx) const +{ + return &(results_[idx]); +} + +const std::string vino_core_lib::HumanPoseEstimation::getName() const +{ + return valid_model_->getModelName(); +} + +const void vino_core_lib::HumanPoseEstimation::observeOutput( + const std::shared_ptr& output) +{ + if (output != nullptr) + { + output->accept(results_); + } +} + +using Result = vino_core_lib::HumanPoseResult; + +std::vector vino_core_lib::HumanPoseEstimation::postprocess( + const float* heatMapsData, const int heatMapOffset, const int nHeatMaps, + const float* pafsData, const int pafOffset, const int nPafs, + const int featureMapWidth, const int featureMapHeight, + const cv::Size& imageSize) const +{ + std::vector heatMaps(nHeatMaps); + for (size_t i = 0; i < heatMaps.size(); i++) + { + heatMaps[i] = cv::Mat(featureMapHeight, featureMapWidth, CV_32FC1, + reinterpret_cast( + const_cast( + heatMapsData + i * heatMapOffset))); + } + + resizeFeatureMaps(heatMaps); + + std::vector pafs(nPafs); + for (size_t i = 0; i < pafs.size(); i++) + { + pafs[i] = cv::Mat(featureMapHeight, featureMapWidth, CV_32FC1, + reinterpret_cast( + const_cast( + pafsData + i * pafOffset))); + } + resizeFeatureMaps(pafs); + + std::vector poses = extractPoses(heatMaps, pafs); + correctCoordinates(poses, heatMaps[0].size(), imageSize); + correctROI(poses); + return poses; +} + +std::vector vino_core_lib::HumanPoseEstimation::extractPoses( + const std::vector& heatMaps, + const std::vector& pafs) const +{ + std::vector> peaksFromHeatMap(heatMaps.size()); + human_pose_estimation::FindPeaksBody findPeaksBody(heatMaps, minPeaksDistance_, peaksFromHeatMap); + cv::parallel_for_(cv::Range(0, static_cast(heatMaps.size())), findPeaksBody); + int peaksBefore = 0; + for (size_t heatmapId = 1; heatmapId < heatMaps.size(); heatmapId++) + { + peaksBefore += static_cast(peaksFromHeatMap[heatmapId - 1].size()); + for (auto& peak : peaksFromHeatMap[heatmapId]) + { + peak.id += peaksBefore; + } + } + std::vector poses = groupPeaksToPoses( + peaksFromHeatMap, pafs, keypointsNumber_, midPointsScoreThreshold_, + foundMidPointsRatioThreshold_, minJointsNumber_, minSubsetScore_); + return poses; +} + +void vino_core_lib::HumanPoseEstimation::resizeFeatureMaps( + std::vector& featureMaps) const +{ + for (auto& featureMap : featureMaps) + { + cv::resize(featureMap, featureMap, cv::Size(), + upsampleRatio_, upsampleRatio_, cv::INTER_CUBIC); + } +} + +void vino_core_lib::HumanPoseEstimation::correctCoordinates(std::vector& poses, + const cv::Size& featureMapsSize, + const cv::Size& imageSize) const +{ + CV_Assert(stride_ % upsampleRatio_ == 0); + + cv::Size fullFeatureMapSize = featureMapsSize * stride_ / upsampleRatio_; + + float scaleX = imageSize.width / + static_cast(fullFeatureMapSize.width - pad_(1) - pad_(3)); + float scaleY = imageSize.height / + static_cast(fullFeatureMapSize.height - pad_(0) - pad_(2)); + for (auto& pose : poses) + { + for (auto& keypoint : pose.keypoints) + { + if (keypoint != cv::Point2f(-1, -1)) + { + keypoint.x *= stride_ / upsampleRatio_; + keypoint.x -= pad_(1); + keypoint.x *= scaleX; + + keypoint.y *= stride_ / upsampleRatio_; + keypoint.y -= pad_(0); + keypoint.y *= scaleY; + } + } + } +} + +void vino_core_lib::HumanPoseEstimation::correctROI( + std::vector& poses) const +{ + for (auto& pose : poses) + { + int xMin = width_; + int xMax = 0; + int yMin = height_; + int yMax = 0; + for (auto& kp: pose.keypoints) + { + if (kp.x < 0) continue; + + int x = static_cast(kp.x); + int y = static_cast(kp.y); + + if (x > xMax) xMax = x; + if (x < xMin) xMin = x; + + if (y > yMax) yMax = y; + if (y < yMin) yMin = y; + } + // slog::info << "rect at: (" << xMin << ", " << yMin << "), (" << xMax << ", " << yMax << ")" << slog::endl; + cv::Rect newLocation = cv::Rect(xMin, yMin, xMax - xMin, yMax - yMin); + pose.setLocation(newLocation); + } +} \ No newline at end of file diff --git a/dynamic_vino_lib/src/inferences/object_detection.cpp b/vino_core_lib/src/inferences/object_detection.cpp similarity index 74% rename from dynamic_vino_lib/src/inferences/object_detection.cpp rename to vino_core_lib/src/inferences/object_detection.cpp index f25259bc..4f5f3273 100644 --- a/dynamic_vino_lib/src/inferences/object_detection.cpp +++ b/vino_core_lib/src/inferences/object_detection.cpp @@ -21,33 +21,33 @@ #include #include #include -#include "dynamic_vino_lib/inferences/object_detection.h" -#include "dynamic_vino_lib/outputs/base_output.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/inferences/object_detection.h" +#include "vino_core_lib/outputs/base_output.h" +#include "vino_core_lib/slog.h" // ObjectDetectionResult -dynamic_vino_lib::ObjectDetectionResult::ObjectDetectionResult( +vino_core_lib::ObjectDetectionResult::ObjectDetectionResult( const cv::Rect& location) : Result(location){} // ObjectDetection -dynamic_vino_lib::ObjectDetection::ObjectDetection(bool enable_roi_constraint, +vino_core_lib::ObjectDetection::ObjectDetection(bool enable_roi_constraint, double show_output_thresh) - : dynamic_vino_lib::BaseInference(), + : vino_core_lib::BaseInference(), show_output_thresh_(show_output_thresh){} -dynamic_vino_lib::ObjectDetection::~ObjectDetection() = default; -void dynamic_vino_lib::ObjectDetection::loadNetwork( +vino_core_lib::ObjectDetection::~ObjectDetection() = default; +void vino_core_lib::ObjectDetection::loadNetwork( const std::shared_ptr network) { valid_model_ = network; max_proposal_count_ = network->getMaxProposalCount(); object_size_ = network->getObjectSize(); setMaxBatchSize(network->getMaxBatchSize()); } -bool dynamic_vino_lib::ObjectDetection::enqueue(const cv::Mat& frame, +bool vino_core_lib::ObjectDetection::enqueue(const cv::Mat& frame, const cv::Rect& input_frame_loc) { if (width_ == 0 && height_ == 0) { width_ = frame.cols; height_ = frame.rows; } - if (!dynamic_vino_lib::BaseInference::enqueue( + if (!vino_core_lib::BaseInference::enqueue( frame, input_frame_loc, 1, 0, valid_model_->getInputName())) { return false; } @@ -56,11 +56,11 @@ bool dynamic_vino_lib::ObjectDetection::enqueue(const cv::Mat& frame, results_.emplace_back(r); return true; } -bool dynamic_vino_lib::ObjectDetection::submitRequest() { - return dynamic_vino_lib::BaseInference::submitRequest(); +bool vino_core_lib::ObjectDetection::submitRequest() { + return vino_core_lib::BaseInference::submitRequest(); } -bool dynamic_vino_lib::ObjectDetection::fetchResults() { - bool can_fetch = dynamic_vino_lib::BaseInference::fetchResults(); +bool vino_core_lib::ObjectDetection::fetchResults() { + bool can_fetch = vino_core_lib::BaseInference::fetchResults(); if (!can_fetch) return false; bool found_result = false; results_.clear(); @@ -94,17 +94,17 @@ bool dynamic_vino_lib::ObjectDetection::fetchResults() { if (!found_result) results_.clear(); return true; } -const int dynamic_vino_lib::ObjectDetection::getResultsLength() const { +const int vino_core_lib::ObjectDetection::getResultsLength() const { return static_cast(results_.size()); } -const dynamic_vino_lib::Result* -dynamic_vino_lib::ObjectDetection::getLocationResult(int idx) const { +const vino_core_lib::Result* +vino_core_lib::ObjectDetection::getLocationResult(int idx) const { return &(results_[idx]); } -const std::string dynamic_vino_lib::ObjectDetection::getName() const { +const std::string vino_core_lib::ObjectDetection::getName() const { return valid_model_->getModelName(); } -const void dynamic_vino_lib::ObjectDetection::observeOutput( +const void vino_core_lib::ObjectDetection::observeOutput( const std::shared_ptr& output) { if (output != nullptr) { output->accept(results_); diff --git a/dynamic_vino_lib/src/inferences/object_segmentation.cpp b/vino_core_lib/src/inferences/object_segmentation.cpp similarity index 77% rename from dynamic_vino_lib/src/inferences/object_segmentation.cpp rename to vino_core_lib/src/inferences/object_segmentation.cpp index 69b7fd23..62a060bc 100644 --- a/dynamic_vino_lib/src/inferences/object_segmentation.cpp +++ b/vino_core_lib/src/inferences/object_segmentation.cpp @@ -22,32 +22,32 @@ #include #include -#include "dynamic_vino_lib/inferences/object_segmentation.h" -#include "dynamic_vino_lib/outputs/base_output.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/inferences/object_segmentation.h" +#include "vino_core_lib/outputs/base_output.h" +#include "vino_core_lib/slog.h" // ObjectSegmentationResult -dynamic_vino_lib::ObjectSegmentationResult::ObjectSegmentationResult(const cv::Rect & location) +vino_core_lib::ObjectSegmentationResult::ObjectSegmentationResult(const cv::Rect & location) : Result(location) { } // ObjectSegmentation -dynamic_vino_lib::ObjectSegmentation::ObjectSegmentation(double show_output_thresh) -: show_output_thresh_(show_output_thresh), dynamic_vino_lib::BaseInference() +vino_core_lib::ObjectSegmentation::ObjectSegmentation(double show_output_thresh) +: show_output_thresh_(show_output_thresh), vino_core_lib::BaseInference() { } -dynamic_vino_lib::ObjectSegmentation::~ObjectSegmentation() = default; +vino_core_lib::ObjectSegmentation::~ObjectSegmentation() = default; -void dynamic_vino_lib::ObjectSegmentation::loadNetwork( +void vino_core_lib::ObjectSegmentation::loadNetwork( const std::shared_ptr network) { valid_model_ = network; setMaxBatchSize(network->getMaxBatchSize()); } -bool dynamic_vino_lib::ObjectSegmentation::enqueue( +bool vino_core_lib::ObjectSegmentation::enqueue( const cv::Mat & frame, const cv::Rect & input_frame_loc) { @@ -55,7 +55,7 @@ bool dynamic_vino_lib::ObjectSegmentation::enqueue( width_ = frame.cols; height_ = frame.rows; } - if (!dynamic_vino_lib::BaseInference::enqueue(frame, input_frame_loc, 1, 0, + if (!vino_core_lib::BaseInference::enqueue(frame, input_frame_loc, 1, 0, valid_model_->getInputName())) { return false; @@ -66,14 +66,14 @@ bool dynamic_vino_lib::ObjectSegmentation::enqueue( return true; } -bool dynamic_vino_lib::ObjectSegmentation::submitRequest() +bool vino_core_lib::ObjectSegmentation::submitRequest() { - return dynamic_vino_lib::BaseInference::submitRequest(); + return vino_core_lib::BaseInference::submitRequest(); } -bool dynamic_vino_lib::ObjectSegmentation::fetchResults() +bool vino_core_lib::ObjectSegmentation::fetchResults() { - bool can_fetch = dynamic_vino_lib::BaseInference::fetchResults(); + bool can_fetch = vino_core_lib::BaseInference::fetchResults(); if (!can_fetch) { return false; } @@ -129,23 +129,23 @@ bool dynamic_vino_lib::ObjectSegmentation::fetchResults() return true; } -const int dynamic_vino_lib::ObjectSegmentation::getResultsLength() const +const int vino_core_lib::ObjectSegmentation::getResultsLength() const { return static_cast(results_.size()); } -const dynamic_vino_lib::Result * -dynamic_vino_lib::ObjectSegmentation::getLocationResult(int idx) const +const vino_core_lib::Result * +vino_core_lib::ObjectSegmentation::getLocationResult(int idx) const { return &(results_[idx]); } -const std::string dynamic_vino_lib::ObjectSegmentation::getName() const +const std::string vino_core_lib::ObjectSegmentation::getName() const { return valid_model_->getModelName(); } -const void dynamic_vino_lib::ObjectSegmentation::observeOutput( +const void vino_core_lib::ObjectSegmentation::observeOutput( const std::shared_ptr & output) { if (output != nullptr) { diff --git a/vino_core_lib/src/inferences/peak.cpp b/vino_core_lib/src/inferences/peak.cpp new file mode 100644 index 00000000..532fb6ae --- /dev/null +++ b/vino_core_lib/src/inferences/peak.cpp @@ -0,0 +1,370 @@ +// Copyright (C) 2018-2019 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include +#include +#include + +#include "vino_core_lib/inferences/peak.h" + +namespace human_pose_estimation +{ +Peak::Peak(const int id, const cv::Point2f &pos, const float score) + : id(id), + pos(pos), + score(score) {} + +HumanPoseByPeaksIndices::HumanPoseByPeaksIndices(const int keypointsNumber) + : peaksIndices(std::vector(keypointsNumber, -1)), + nJoints(0), + score(0.0f) {} + +TwoJointsConnection::TwoJointsConnection(const int firstJointIdx, + const int secondJointIdx, + const float score) + : firstJointIdx(firstJointIdx), + secondJointIdx(secondJointIdx), + score(score) {} + +void findPeaks(const std::vector &heatMaps, + const float minPeaksDistance, + std::vector> &allPeaks, + int heatMapId) +{ + const float threshold = 0.1f; + std::vector peaks; + const cv::Mat &heatMap = heatMaps[heatMapId]; + const float *heatMapData = heatMap.ptr(); + size_t heatMapStep = heatMap.step1(); + for (int y = -1; y < heatMap.rows + 1; y++) + { + for (int x = -1; x < heatMap.cols + 1; x++) + { + float val = 0; + if (x >= 0 && y >= 0 && x < heatMap.cols && y < heatMap.rows) + { + val = heatMapData[y * heatMapStep + x]; + val = val >= threshold ? val : 0; + } + + float left_val = 0; + if (y >= 0 && x < (heatMap.cols - 1) && y < heatMap.rows) + { + left_val = heatMapData[y * heatMapStep + x + 1]; + left_val = left_val >= threshold ? left_val : 0; + } + + float right_val = 0; + if (x > 0 && y >= 0 && y < heatMap.rows) + { + right_val = heatMapData[y * heatMapStep + x - 1]; + right_val = right_val >= threshold ? right_val : 0; + } + + float top_val = 0; + if (x >= 0 && x < heatMap.cols && y < (heatMap.rows - 1)) + { + top_val = heatMapData[(y + 1) * heatMapStep + x]; + top_val = top_val >= threshold ? top_val : 0; + } + + float bottom_val = 0; + if (x >= 0 && y > 0 && x < heatMap.cols) + { + bottom_val = heatMapData[(y - 1) * heatMapStep + x]; + bottom_val = bottom_val >= threshold ? bottom_val : 0; + } + + if ((val > left_val) && (val > right_val) && (val > top_val) && (val > bottom_val)) + { + peaks.push_back(cv::Point(x, y)); + } + } + } + std::sort(peaks.begin(), peaks.end(), [](const cv::Point &a, const cv::Point &b) { + return a.x < b.x; + }); + std::vector isActualPeak(peaks.size(), true); + int peakCounter = 0; + std::vector &peaksWithScoreAndID = allPeaks[heatMapId]; + for (size_t i = 0; i < peaks.size(); i++) + { + if (isActualPeak[i]) + { + for (size_t j = i + 1; j < peaks.size(); j++) + { + if (sqrt((peaks[i].x - peaks[j].x) * (peaks[i].x - peaks[j].x) + + (peaks[i].y - peaks[j].y) * (peaks[i].y - peaks[j].y)) < minPeaksDistance) + { + isActualPeak[j] = false; + } + } + peaksWithScoreAndID.push_back(Peak(peakCounter++, peaks[i], heatMap.at(peaks[i]))); + } + } +} + +std::vector groupPeaksToPoses( + const std::vector> &allPeaks, + const std::vector &pafs, + const size_t keypointsNumber, + const float midPointsScoreThreshold, + const float foundMidPointsRatioThreshold, + const int minJointsNumber, + const float minSubsetScore) +{ + const std::vector> limbIdsHeatmap = { + {2, 3}, {2, 6}, {3, 4}, {4, 5}, {6, 7}, {7, 8}, {2, 9}, + {9, 10}, {10, 11}, {2, 12}, {12, 13}, {13, 14}, {2, 1}, + {1, 15}, {15, 17}, {1, 16}, {16, 18}, {3, 17}, {6, 18}}; + const std::vector> limbIdsPaf = { + {31, 32}, {39, 40}, {33, 34}, {35, 36}, {41, 42}, {43, 44}, {19, 20}, + {21, 22}, {23, 24}, {25, 26}, {27, 28}, {29, 30}, {47, 48}, {49, 50}, + {53, 54}, {51, 52}, {55, 56}, {37, 38}, {45, 46}}; + + std::vector candidates; + for (const auto &peaks : allPeaks) + { + candidates.insert(candidates.end(), peaks.begin(), peaks.end()); + } + std::vector subset(0, HumanPoseByPeaksIndices(keypointsNumber)); + for (size_t k = 0; k < limbIdsPaf.size(); k++) + { + std::vector connections; + const int mapIdxOffset = keypointsNumber + 1; + std::pair scoreMid = {pafs[limbIdsPaf[k].first - mapIdxOffset], + pafs[limbIdsPaf[k].second - mapIdxOffset]}; + const int idxJointA = limbIdsHeatmap[k].first - 1; + const int idxJointB = limbIdsHeatmap[k].second - 1; + const std::vector &candA = allPeaks[idxJointA]; + const std::vector &candB = allPeaks[idxJointB]; + const size_t nJointsA = candA.size(); + const size_t nJointsB = candB.size(); + if (nJointsA == 0 && nJointsB == 0) + { + continue; + } + else if (nJointsA == 0) + { + for (size_t i = 0; i < nJointsB; i++) + { + int num = 0; + for (size_t j = 0; j < subset.size(); j++) + { + if (subset[j].peaksIndices[idxJointB] == candB[i].id) + { + num++; + continue; + } + } + if (num == 0) + { + HumanPoseByPeaksIndices personKeypoints(keypointsNumber); + personKeypoints.peaksIndices[idxJointB] = candB[i].id; + personKeypoints.nJoints = 1; + personKeypoints.score = candB[i].score; + subset.push_back(personKeypoints); + } + } + continue; + } + else if (nJointsB == 0) + { + for (size_t i = 0; i < nJointsA; i++) + { + int num = 0; + for (size_t j = 0; j < subset.size(); j++) + { + if (subset[j].peaksIndices[idxJointA] == candA[i].id) + { + num++; + continue; + } + } + if (num == 0) + { + HumanPoseByPeaksIndices personKeypoints(keypointsNumber); + personKeypoints.peaksIndices[idxJointA] = candA[i].id; + personKeypoints.nJoints = 1; + personKeypoints.score = candA[i].score; + subset.push_back(personKeypoints); + } + } + continue; + } + + std::vector tempJointConnections; + for (size_t i = 0; i < nJointsA; i++) + { + for (size_t j = 0; j < nJointsB; j++) + { + cv::Point2f pt = candA[i].pos * 0.5 + candB[j].pos * 0.5; + cv::Point mid = cv::Point(cvRound(pt.x), cvRound(pt.y)); + cv::Point2f vec = candB[j].pos - candA[i].pos; + double norm_vec = cv::norm(vec); + if (norm_vec == 0) + { + continue; + } + vec /= norm_vec; + float score = vec.x * scoreMid.first.at(mid) + vec.y * scoreMid.second.at(mid); + int height_n = pafs[0].rows / 2; + float suc_ratio = 0.0f; + float mid_score = 0.0f; + const int mid_num = 10; + const float scoreThreshold = -100.0f; + if (score > scoreThreshold) + { + float p_sum = 0; + int p_count = 0; + cv::Size2f step((candB[j].pos.x - candA[i].pos.x) / (mid_num - 1), + (candB[j].pos.y - candA[i].pos.y) / (mid_num - 1)); + for (int n = 0; n < mid_num; n++) + { + cv::Point midPoint(cvRound(candA[i].pos.x + n * step.width), + cvRound(candA[i].pos.y + n * step.height)); + cv::Point2f pred(scoreMid.first.at(midPoint), + scoreMid.second.at(midPoint)); + score = vec.x * pred.x + vec.y * pred.y; + if (score > midPointsScoreThreshold) + { + p_sum += score; + p_count++; + } + } + suc_ratio = static_cast(p_count / mid_num); + float ratio = p_count > 0 ? p_sum / p_count : 0.0f; + mid_score = ratio + static_cast(std::min(height_n / norm_vec - 1, 0.0)); + } + if (mid_score > 0 && suc_ratio > foundMidPointsRatioThreshold) + { + tempJointConnections.push_back(TwoJointsConnection(i, j, mid_score)); + } + } + } + if (!tempJointConnections.empty()) + { + std::sort(tempJointConnections.begin(), tempJointConnections.end(), + [](const TwoJointsConnection &a, + const TwoJointsConnection &b) { + return (a.score > b.score); + }); + } + size_t num_limbs = std::min(nJointsA, nJointsB); + size_t cnt = 0; + std::vector occurA(nJointsA, 0); + std::vector occurB(nJointsB, 0); + for (size_t row = 0; row < tempJointConnections.size(); row++) + { + if (cnt == num_limbs) + { + break; + } + const int &indexA = tempJointConnections[row].firstJointIdx; + const int &indexB = tempJointConnections[row].secondJointIdx; + const float &score = tempJointConnections[row].score; + if (occurA[indexA] == 0 && occurB[indexB] == 0) + { + connections.push_back(TwoJointsConnection(candA[indexA].id, candB[indexB].id, score)); + cnt++; + occurA[indexA] = 1; + occurB[indexB] = 1; + } + } + if (connections.empty()) + { + continue; + } + + bool extraJointConnections = (k == 17 || k == 18); + if (k == 0) + { + subset = std::vector( + connections.size(), HumanPoseByPeaksIndices(keypointsNumber)); + for (size_t i = 0; i < connections.size(); i++) + { + const int &indexA = connections[i].firstJointIdx; + const int &indexB = connections[i].secondJointIdx; + subset[i].peaksIndices[idxJointA] = indexA; + subset[i].peaksIndices[idxJointB] = indexB; + subset[i].nJoints = 2; + subset[i].score = candidates[indexA].score + candidates[indexB].score + connections[i].score; + } + } + else if (extraJointConnections) + { + for (size_t i = 0; i < connections.size(); i++) + { + const int &indexA = connections[i].firstJointIdx; + const int &indexB = connections[i].secondJointIdx; + for (size_t j = 0; j < subset.size(); j++) + { + if (subset[j].peaksIndices[idxJointA] == indexA && subset[j].peaksIndices[idxJointB] == -1) + { + subset[j].peaksIndices[idxJointB] = indexB; + } + else if (subset[j].peaksIndices[idxJointB] == indexB && subset[j].peaksIndices[idxJointA] == -1) + { + subset[j].peaksIndices[idxJointA] = indexA; + } + } + } + continue; + } + else + { + for (size_t i = 0; i < connections.size(); i++) + { + const int &indexA = connections[i].firstJointIdx; + const int &indexB = connections[i].secondJointIdx; + bool num = false; + for (size_t j = 0; j < subset.size(); j++) + { + if (subset[j].peaksIndices[idxJointA] == indexA) + { + subset[j].peaksIndices[idxJointB] = indexB; + subset[j].nJoints++; + subset[j].score += candidates[indexB].score + connections[i].score; + num = true; + } + } + if (!num) + { + HumanPoseByPeaksIndices hpWithScore(keypointsNumber); + hpWithScore.peaksIndices[idxJointA] = indexA; + hpWithScore.peaksIndices[idxJointB] = indexB; + hpWithScore.nJoints = 2; + hpWithScore.score = candidates[indexA].score + candidates[indexB].score + connections[i].score; + subset.push_back(hpWithScore); + } + } + } + } + std::vector poses; + for (const auto &subsetI : subset) + { + if (subsetI.nJoints < minJointsNumber || subsetI.score / subsetI.nJoints < minSubsetScore) + { + continue; + } + int position = -1; + Result pose(cv::Rect(), + std::vector(keypointsNumber, cv::Point2f(-1.0f, -1.0f)), + subsetI.score * std::max(0, subsetI.nJoints - 1)); + for (const auto &peakIdx : subsetI.peaksIndices) + { + position++; + if (peakIdx >= 0) + { + pose.keypoints[position] = candidates[peakIdx].pos; + pose.keypoints[position].x += 0.5; + pose.keypoints[position].y += 0.5; + pose.keypoints[position].score = candidates[peakIdx].score; + } + } + poses.push_back(pose); + } + return poses; +} +} // namespace human_pose_estimation diff --git a/dynamic_vino_lib/src/inferences/person_reidentification.cpp b/vino_core_lib/src/inferences/person_reidentification.cpp similarity index 73% rename from dynamic_vino_lib/src/inferences/person_reidentification.cpp rename to vino_core_lib/src/inferences/person_reidentification.cpp index 005a128d..ef192590 100644 --- a/dynamic_vino_lib/src/inferences/person_reidentification.cpp +++ b/vino_core_lib/src/inferences/person_reidentification.cpp @@ -20,34 +20,34 @@ #include #include #include -#include "dynamic_vino_lib/inferences/person_reidentification.h" -#include "dynamic_vino_lib/outputs/base_output.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/inferences/person_reidentification.h" +#include "vino_core_lib/outputs/base_output.h" +#include "vino_core_lib/slog.h" // PersonReidentificationResult -dynamic_vino_lib::PersonReidentificationResult::PersonReidentificationResult( +vino_core_lib::PersonReidentificationResult::PersonReidentificationResult( const cv::Rect & location) : Result(location) {} // PersonReidentification -dynamic_vino_lib::PersonReidentification::PersonReidentification(double match_thresh) -: match_thresh_(match_thresh), dynamic_vino_lib::BaseInference() {} +vino_core_lib::PersonReidentification::PersonReidentification(double match_thresh) +: match_thresh_(match_thresh), vino_core_lib::BaseInference() {} -dynamic_vino_lib::PersonReidentification::~PersonReidentification() = default; -void dynamic_vino_lib::PersonReidentification::loadNetwork( +vino_core_lib::PersonReidentification::~PersonReidentification() = default; +void vino_core_lib::PersonReidentification::loadNetwork( const std::shared_ptr network) { valid_model_ = network; setMaxBatchSize(network->getMaxBatchSize()); } -bool dynamic_vino_lib::PersonReidentification::enqueue( +bool vino_core_lib::PersonReidentification::enqueue( const cv::Mat & frame, const cv::Rect & input_frame_loc) { if (getEnqueuedNum() == 0) { results_.clear(); } - if (!dynamic_vino_lib::BaseInference::enqueue( + if (!vino_core_lib::BaseInference::enqueue( frame, input_frame_loc, 1, 0, valid_model_->getInputName())) { return false; @@ -57,14 +57,14 @@ bool dynamic_vino_lib::PersonReidentification::enqueue( return true; } -bool dynamic_vino_lib::PersonReidentification::submitRequest() +bool vino_core_lib::PersonReidentification::submitRequest() { - return dynamic_vino_lib::BaseInference::submitRequest(); + return vino_core_lib::BaseInference::submitRequest(); } -bool dynamic_vino_lib::PersonReidentification::fetchResults() +bool vino_core_lib::PersonReidentification::fetchResults() { - bool can_fetch = dynamic_vino_lib::BaseInference::fetchResults(); + bool can_fetch = vino_core_lib::BaseInference::fetchResults(); if (!can_fetch) {return false;} bool found_result = false; InferenceEngine::InferRequest::Ptr request = getEngine()->getRequest(); @@ -81,7 +81,7 @@ bool dynamic_vino_lib::PersonReidentification::fetchResults() return true; } -float dynamic_vino_lib::PersonReidentification::calcSimilarity( +float vino_core_lib::PersonReidentification::calcSimilarity( const std::vector & person_a, const std::vector & person_b) { if (person_a.size() != person_b.size()) { @@ -105,7 +105,7 @@ float dynamic_vino_lib::PersonReidentification::calcSimilarity( return mul_sum / (sqrt(denom_a) * sqrt(denom_b)); } -std::string dynamic_vino_lib::PersonReidentification::findMatchPerson( +std::string vino_core_lib::PersonReidentification::findMatchPerson( const std::vector & new_person) { auto size = recorded_persons_.size(); @@ -128,23 +128,23 @@ std::string dynamic_vino_lib::PersonReidentification::findMatchPerson( } } -const int dynamic_vino_lib::PersonReidentification::getResultsLength() const +const int vino_core_lib::PersonReidentification::getResultsLength() const { return static_cast(results_.size()); } -const dynamic_vino_lib::Result * -dynamic_vino_lib::PersonReidentification::getLocationResult(int idx) const +const vino_core_lib::Result * +vino_core_lib::PersonReidentification::getLocationResult(int idx) const { return &(results_[idx]); } -const std::string dynamic_vino_lib::PersonReidentification::getName() const +const std::string vino_core_lib::PersonReidentification::getName() const { return valid_model_->getModelName(); } -const void dynamic_vino_lib::PersonReidentification::observeOutput( +const void vino_core_lib::PersonReidentification::observeOutput( const std::shared_ptr & output) { if (output != nullptr) { diff --git a/dynamic_vino_lib/src/inputs/image_input.cpp b/vino_core_lib/src/inputs/image_input.cpp similarity index 96% rename from dynamic_vino_lib/src/inputs/image_input.cpp rename to vino_core_lib/src/inputs/image_input.cpp index c2790f0e..a9ef8be3 100644 --- a/dynamic_vino_lib/src/inputs/image_input.cpp +++ b/vino_core_lib/src/inputs/image_input.cpp @@ -19,7 +19,7 @@ * @file image_input.cpp */ -#include "dynamic_vino_lib/inputs/image_input.h" +#include "vino_core_lib/inputs/image_input.h" #include // Image diff --git a/dynamic_vino_lib/src/inputs/realsense_camera.cpp b/vino_core_lib/src/inputs/realsense_camera.cpp similarity index 96% rename from dynamic_vino_lib/src/inputs/realsense_camera.cpp rename to vino_core_lib/src/inputs/realsense_camera.cpp index 8ebd273c..e014e412 100644 --- a/dynamic_vino_lib/src/inputs/realsense_camera.cpp +++ b/vino_core_lib/src/inputs/realsense_camera.cpp @@ -18,8 +18,8 @@ * @brief a header file with declaration of RealSenseCamera class * @file realsense_camera.cpp */ -#include "dynamic_vino_lib/inputs/realsense_camera.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/inputs/realsense_camera.h" +#include "vino_core_lib/slog.h" // RealSenseCamera bool Input::RealSenseCamera::initialize() diff --git a/dynamic_vino_lib/src/inputs/realsense_camera_topic.cpp b/vino_core_lib/src/inputs/realsense_camera_topic.cpp similarity index 95% rename from dynamic_vino_lib/src/inputs/realsense_camera_topic.cpp rename to vino_core_lib/src/inputs/realsense_camera_topic.cpp index 900ffa07..ebd941c5 100644 --- a/dynamic_vino_lib/src/inputs/realsense_camera_topic.cpp +++ b/vino_core_lib/src/inputs/realsense_camera_topic.cpp @@ -19,9 +19,9 @@ * @file realsense_camera_topic.cpp */ -#include "dynamic_vino_lib/inputs/realsense_camera_topic.h" +#include "vino_core_lib/inputs/realsense_camera_topic.h" #include -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/slog.h" #include diff --git a/dynamic_vino_lib/src/inputs/standard_camera.cpp b/vino_core_lib/src/inputs/standard_camera.cpp similarity index 96% rename from dynamic_vino_lib/src/inputs/standard_camera.cpp rename to vino_core_lib/src/inputs/standard_camera.cpp index d8649144..46a883f4 100644 --- a/dynamic_vino_lib/src/inputs/standard_camera.cpp +++ b/vino_core_lib/src/inputs/standard_camera.cpp @@ -18,7 +18,7 @@ * @brief a header file with declaration of StandardCamera class * @file standard_camera.cpp */ -#include "dynamic_vino_lib/inputs/standard_camera.h" +#include "vino_core_lib/inputs/standard_camera.h" // StandardCamera bool Input::StandardCamera::initialize() diff --git a/dynamic_vino_lib/src/inputs/video_input.cpp b/vino_core_lib/src/inputs/video_input.cpp similarity index 96% rename from dynamic_vino_lib/src/inputs/video_input.cpp rename to vino_core_lib/src/inputs/video_input.cpp index d493731b..dfba5c19 100644 --- a/dynamic_vino_lib/src/inputs/video_input.cpp +++ b/vino_core_lib/src/inputs/video_input.cpp @@ -21,7 +21,7 @@ #include -#include "dynamic_vino_lib/inputs/video_input.h" +#include "vino_core_lib/inputs/video_input.h" // Video Input::Video::Video(const std::string& video) diff --git a/dynamic_vino_lib/src/models/age_gender_detection_model.cpp b/vino_core_lib/src/models/age_gender_detection_model.cpp similarity index 97% rename from dynamic_vino_lib/src/models/age_gender_detection_model.cpp rename to vino_core_lib/src/models/age_gender_detection_model.cpp index 2a1122ad..c47351b8 100644 --- a/dynamic_vino_lib/src/models/age_gender_detection_model.cpp +++ b/vino_core_lib/src/models/age_gender_detection_model.cpp @@ -22,8 +22,8 @@ #include #include -#include "dynamic_vino_lib/models/age_gender_detection_model.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/models/age_gender_detection_model.h" +#include "vino_core_lib/slog.h" // Validated Age Gender Classification Network Models::AgeGenderDetectionModel::AgeGenderDetectionModel( diff --git a/dynamic_vino_lib/src/models/base_model.cpp b/vino_core_lib/src/models/base_model.cpp similarity index 97% rename from dynamic_vino_lib/src/models/base_model.cpp rename to vino_core_lib/src/models/base_model.cpp index 08153777..11c81c5a 100644 --- a/dynamic_vino_lib/src/models/base_model.cpp +++ b/vino_core_lib/src/models/base_model.cpp @@ -24,8 +24,8 @@ #include #include -#include "dynamic_vino_lib/models/base_model.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/models/base_model.h" +#include "vino_core_lib/slog.h" // Validated Base Network Models::BaseModel::BaseModel(const std::string& model_loc, int input_num, diff --git a/dynamic_vino_lib/src/models/emotion_detection_model.cpp b/vino_core_lib/src/models/emotion_detection_model.cpp similarity index 96% rename from dynamic_vino_lib/src/models/emotion_detection_model.cpp rename to vino_core_lib/src/models/emotion_detection_model.cpp index d1e6150d..e4700b52 100644 --- a/dynamic_vino_lib/src/models/emotion_detection_model.cpp +++ b/vino_core_lib/src/models/emotion_detection_model.cpp @@ -20,8 +20,8 @@ */ #include -#include "dynamic_vino_lib/models/emotion_detection_model.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/models/emotion_detection_model.h" +#include "vino_core_lib/slog.h" // Validated Emotions Detection Network Models::EmotionDetectionModel::EmotionDetectionModel( diff --git a/dynamic_vino_lib/src/models/face_detection_model.cpp b/vino_core_lib/src/models/face_detection_model.cpp similarity index 97% rename from dynamic_vino_lib/src/models/face_detection_model.cpp rename to vino_core_lib/src/models/face_detection_model.cpp index caadb0e2..1b0c74d4 100644 --- a/dynamic_vino_lib/src/models/face_detection_model.cpp +++ b/vino_core_lib/src/models/face_detection_model.cpp @@ -22,8 +22,8 @@ #include -#include "dynamic_vino_lib/models/face_detection_model.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/models/face_detection_model.h" +#include "vino_core_lib/slog.h" // Validated Face Detection Network Models::FaceDetectionModel::FaceDetectionModel(const std::string& model_loc, diff --git a/dynamic_vino_lib/src/models/head_pose_detection_model.cpp b/vino_core_lib/src/models/head_pose_detection_model.cpp similarity index 97% rename from dynamic_vino_lib/src/models/head_pose_detection_model.cpp rename to vino_core_lib/src/models/head_pose_detection_model.cpp index 2012deb2..ad67c3e3 100644 --- a/dynamic_vino_lib/src/models/head_pose_detection_model.cpp +++ b/vino_core_lib/src/models/head_pose_detection_model.cpp @@ -23,8 +23,8 @@ #include #include -#include "dynamic_vino_lib/models/head_pose_detection_model.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/models/head_pose_detection_model.h" +#include "vino_core_lib/slog.h" // Validated Head Pose Network Models::HeadPoseDetectionModel::HeadPoseDetectionModel( diff --git a/vino_core_lib/src/models/human_pose_estimation_model.cpp b/vino_core_lib/src/models/human_pose_estimation_model.cpp new file mode 100644 index 00000000..0875086a --- /dev/null +++ b/vino_core_lib/src/models/human_pose_estimation_model.cpp @@ -0,0 +1,140 @@ +// TODO add license + +/* +Model URL: https://docs.openvinotoolkit.org/2019_R3.1/_models_intel_human_pose_estimation_0001_description_human_pose_estimation_0001.html +*/ + +#include +#include + +#include "vino_core_lib/models/human_pose_estimation_model.h" +#include "vino_core_lib/slog.h" + +Models::HumanPoseEstimationModel::HumanPoseEstimationModel( + const std::string& model_loc, + int input_num, int output_num, + int max_batch_size) + : BaseModel(model_loc, input_num, output_num, max_batch_size) +{ +} + +void Models::HumanPoseEstimationModel::setLayerProperty( + InferenceEngine::CNNNetReader::Ptr net_reader) +{ + // set input property + InferenceEngine::InputsDataMap input_info_map( + net_reader->getNetwork().getInputsInfo()); + InferenceEngine::InputInfo::Ptr input_info = input_info_map.begin()->second; + input_info->setPrecision(InferenceEngine::Precision::U8); // It's an image + input_info->setLayout(InferenceEngine::Layout::NCHW); // No idea. + // set output property + InferenceEngine::OutputsDataMap output_info_map( + net_reader->getNetwork().getOutputsInfo()); + auto it = output_info_map.begin(); + InferenceEngine::DataPtr keypoints_output_ptr = (it++)->second; + InferenceEngine::DataPtr heatmap_output_ptr = (it++)->second; + keypoints_output_ptr->setPrecision(InferenceEngine::Precision::FP32); + keypoints_output_ptr->setLayout(InferenceEngine::Layout::NCHW); + heatmap_output_ptr->setPrecision(InferenceEngine::Precision::FP32); + heatmap_output_ptr->setLayout(InferenceEngine::Layout::NCHW); + // set input and output layer name + input_ = input_info_map.begin()->first; + output_keypoints_ = keypoints_output_ptr->name; + output_heatmap_ = heatmap_output_ptr->name; +} + +void Models::HumanPoseEstimationModel::checkLayerProperty( + const InferenceEngine::CNNNetReader::Ptr& net_reader) +{ + slog::info << "Checking Human Pose Estimation outputs" << slog::endl; + InferenceEngine::OutputsDataMap output_info( + net_reader->getNetwork().getOutputsInfo()); + auto it = output_info.begin(); + InferenceEngine::DataPtr paf_output_ptr = (it++)->second; + InferenceEngine::DataPtr heatmap_output_ptr = (it++)->second; + + if (paf_output_ptr->getCreatorLayer().lock()->type != "Convolution") + { + throw std::logic_error("In Human Pose Estimation network, PAF layer (" + + paf_output_ptr->getCreatorLayer().lock()->name + + ") should be a Convolution, but was: " + + paf_output_ptr->getCreatorLayer().lock()->type); + } + if (heatmap_output_ptr->getCreatorLayer().lock()->type != "Convolution") + { + throw std::logic_error("In Human Pose Estimation network, heatmap layer (" + + heatmap_output_ptr->getCreatorLayer().lock()->name + + ") should be a Convolution, but was: " + + heatmap_output_ptr->getCreatorLayer().lock()->type); + } + + if (paf_output_ptr->getCreatorLayer().lock()->outData.size() != 1) + { + throw std::logic_error( + "In Human Pose Estimation network, PAF layer (" + + paf_output_ptr->getCreatorLayer().lock()->name + + ") should have 1 output, but had: " + + std::to_string(paf_output_ptr->getCreatorLayer().lock()->outData.size())); + } + if (heatmap_output_ptr->getCreatorLayer().lock()->outData.size() != 1) + { + throw std::logic_error( + "In Human Pose Estimation network, Heatmap layer (" + + heatmap_output_ptr->getCreatorLayer().lock()->name + + ") should have 1 output, but had: " + + std::to_string(heatmap_output_ptr->getCreatorLayer().lock()->outData.size())); + } + + if (paf_output_ptr->getCreatorLayer().lock()->outData[0]->dims.size() == 4 && + paf_output_ptr->getCreatorLayer().lock()->outData[0]->dims[2] == 19) + { + std::swap(paf_output_ptr, heatmap_output_ptr); + } + + auto pafDims = paf_output_ptr->getCreatorLayer().lock()->outData[0]->dims; + auto heatDims = heatmap_output_ptr->getCreatorLayer().lock()->outData[0]->dims; + + if (pafDims.size() != 4 || pafDims[0] != 57 || pafDims[1] != 32 + || pafDims[2] != 38 || pafDims[3] != 1) + { + std::ostringstream size; + size << "[ "; + for (size_t s : pafDims) + { + size << s << " "; + } + size << "]"; + throw std::logic_error( + "In Human Pose Estimation network, PAF layer (" + + paf_output_ptr->getCreatorLayer().lock()->name + + ") should have output size of [ 57 32 38 1 ], but had: " + size.str()); + } + + if (heatDims.size() != 4 || heatDims[0] != 57 || heatDims[1] != 32 + || heatDims[2] != 19 || heatDims[3] != 1) + { + std::ostringstream size; + size << "[ "; + for (size_t s : heatDims) + { + size << s << " "; + } + size << "]"; + throw std::logic_error( + "In Human Pose Estimation network, Heatmap layer (" + + heatmap_output_ptr->getCreatorLayer().lock()->name + + ") should have output size of [ 57 32 19 1 ], but had: " + size.str()); + } + + slog::info << "PAF layer: " + << paf_output_ptr->getCreatorLayer().lock()->name + << slog::endl; + slog::info << "Heatmap layer: " + << heatmap_output_ptr->getCreatorLayer().lock()->name + << slog::endl; +} + +const std::string Models::HumanPoseEstimationModel::getModelName() const +{ + return "Human Pose Estimation"; +} \ No newline at end of file diff --git a/dynamic_vino_lib/src/models/object_detection_model.cpp b/vino_core_lib/src/models/object_detection_model.cpp similarity index 98% rename from dynamic_vino_lib/src/models/object_detection_model.cpp rename to vino_core_lib/src/models/object_detection_model.cpp index e16ddc20..0a25c8e6 100644 --- a/dynamic_vino_lib/src/models/object_detection_model.cpp +++ b/vino_core_lib/src/models/object_detection_model.cpp @@ -18,8 +18,8 @@ * @file object_detection_model.cpp */ #include -#include "dynamic_vino_lib/models/object_detection_model.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/models/object_detection_model.h" +#include "vino_core_lib/slog.h" // Validated Object Detection Network Models::ObjectDetectionModel::ObjectDetectionModel(const std::string& model_loc, int input_num, int output_num, diff --git a/dynamic_vino_lib/src/models/object_segmentation_model.cpp b/vino_core_lib/src/models/object_segmentation_model.cpp similarity index 96% rename from dynamic_vino_lib/src/models/object_segmentation_model.cpp rename to vino_core_lib/src/models/object_segmentation_model.cpp index 2feb29c7..20dbdb24 100644 --- a/dynamic_vino_lib/src/models/object_segmentation_model.cpp +++ b/vino_core_lib/src/models/object_segmentation_model.cpp @@ -17,8 +17,8 @@ * @file object_detection_model.cpp */ #include -#include "dynamic_vino_lib/models/object_segmentation_model.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/models/object_segmentation_model.h" +#include "vino_core_lib/slog.h" // Validated Object Detection Network Models::ObjectSegmentationModel::ObjectSegmentationModel( const std::string & model_loc, diff --git a/dynamic_vino_lib/src/models/person_reidentification_model.cpp b/vino_core_lib/src/models/person_reidentification_model.cpp similarity index 94% rename from dynamic_vino_lib/src/models/person_reidentification_model.cpp rename to vino_core_lib/src/models/person_reidentification_model.cpp index 7367e212..3a147bdf 100644 --- a/dynamic_vino_lib/src/models/person_reidentification_model.cpp +++ b/vino_core_lib/src/models/person_reidentification_model.cpp @@ -17,8 +17,8 @@ * @file person_reidentification_model.cpp */ #include -#include "dynamic_vino_lib/models/person_reidentification_model.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/models/person_reidentification_model.h" +#include "vino_core_lib/slog.h" // Validated Object Detection Network Models::PersonReidentificationModel::PersonReidentificationModel( const std::string & model_loc, int input_num, int output_num, int max_batch_size) diff --git a/dynamic_vino_lib/src/outputs/base_output.cpp b/vino_core_lib/src/outputs/base_output.cpp similarity index 90% rename from dynamic_vino_lib/src/outputs/base_output.cpp rename to vino_core_lib/src/outputs/base_output.cpp index a3248c8d..b22192c9 100644 --- a/dynamic_vino_lib/src/outputs/base_output.cpp +++ b/vino_core_lib/src/outputs/base_output.cpp @@ -14,8 +14,8 @@ * limitations under the License. */ -#include "dynamic_vino_lib/outputs/base_output.h" -#include "dynamic_vino_lib/pipeline.h" +#include "vino_core_lib/outputs/base_output.h" +#include "vino_core_lib/pipeline.h" void Outputs::BaseOutput::setPipeline(Pipeline* const pipeline) { diff --git a/dynamic_vino_lib/src/outputs/image_window_output.cpp b/vino_core_lib/src/outputs/image_window_output.cpp similarity index 87% rename from dynamic_vino_lib/src/outputs/image_window_output.cpp rename to vino_core_lib/src/outputs/image_window_output.cpp index 3a6f2df1..bcd6125d 100644 --- a/dynamic_vino_lib/src/outputs/image_window_output.cpp +++ b/vino_core_lib/src/outputs/image_window_output.cpp @@ -19,11 +19,11 @@ * @file image_window_output.cpp */ -#include "dynamic_vino_lib/outputs/image_window_output.h" +#include "vino_core_lib/outputs/image_window_output.h" #include #include #include -#include "dynamic_vino_lib/pipeline.h" +#include "vino_core_lib/pipeline.h" Outputs::ImageWindowOutput::ImageWindowOutput(const std::string& window_name, int focal_length) @@ -49,7 +49,7 @@ void Outputs::ImageWindowOutput::feedFrame(const cv::Mat& frame) } void Outputs::ImageWindowOutput::accept( - const std::vector& results) + const std::vector& results) { // std::cout<<"call face"<& results) + const std::vector& results) { if (outputs_.size() == 0) { @@ -102,7 +102,7 @@ void Outputs::ImageWindowOutput::accept( } void Outputs::ImageWindowOutput::accept( - const std::vector& results) + const std::vector& results) { if (outputs_.size() == 0) { @@ -168,7 +168,7 @@ cv::Mat Outputs::ImageWindowOutput::getRotationTransform(double yaw, } void Outputs::ImageWindowOutput::accept( - const std::vector& results) + const std::vector& results) { if (outputs_.size() == 0) { @@ -203,7 +203,7 @@ void Outputs::ImageWindowOutput::accept( } void Outputs::ImageWindowOutput::accept( - const std::vector& results) + const std::vector& results) { // std::cout<<"call"< & results) + const std::vector & results) { std::map class_color; for (unsigned i = 0; i < results.size(); i++) { @@ -266,7 +266,7 @@ void Outputs::ImageWindowOutput::mergeMask( } void Outputs::ImageWindowOutput::accept( - const std::vector & results) + const std::vector & results) { if (outputs_.size() == 0) { initOutputs(results.size()); @@ -305,7 +305,7 @@ unsigned Outputs::ImageWindowOutput::findOutput( } void Outputs::ImageWindowOutput::accept( - const std::vector & results) + const std::vector & results) { for (unsigned i = 0; i < results.size(); i++) { cv::Rect result_rect = results[i].getLocation(); @@ -315,6 +315,22 @@ void Outputs::ImageWindowOutput::accept( } } +void Outputs::ImageWindowOutput::accept( + const std::vector & results) +{ + OutputData output; + output.scalar = cv::Scalar(255, 0, 0); + for (unsigned i = 0; i < results.size(); i++) { + cv::Rect result_rect = results[i].getLocation(); + auto score = std::to_string(results[i].getScore()); + + output.rect = result_rect; + output.desc = "Pose " + std::to_string(i) + " [" + score.substr(0, score.find(".") + 2) + "]"; + output.kp = results[i].getKeypoints(); + outputs_.push_back(output); + } +} + void Outputs::ImageWindowOutput::decorateFrame() { if (getPipeline()->getParameters()->isGetFps()) @@ -336,6 +352,17 @@ void Outputs::ImageWindowOutput::decorateFrame() cv::line(frame_, o.hp_cp, o.hp_y, cv::Scalar(0, 255, 0), 2); cv::line(frame_, o.hp_zs, o.hp_ze, cv::Scalar(255, 0, 0), 2); cv::circle(frame_, o.hp_ze, 3, cv::Scalar(255, 0, 0), 2); + + for (auto kp : o.kp) + { + if (kp.x >= 0) + { + std::string score = std::to_string(kp.score).substr(0, 3); + cv::circle(frame_, kp, 3, cv::Scalar(255, 0, 0), cv::FILLED); + cv::putText(frame_, score, kp + cv::Point2f(5, 0), + cv::FONT_HERSHEY_COMPLEX_SMALL, 0.5, o.scalar); + } + } } outputs_.clear(); diff --git a/dynamic_vino_lib/src/outputs/ros_service_output.cpp b/vino_core_lib/src/outputs/ros_service_output.cpp similarity index 82% rename from dynamic_vino_lib/src/outputs/ros_service_output.cpp rename to vino_core_lib/src/outputs/ros_service_output.cpp index f05c21d7..d878434b 100644 --- a/dynamic_vino_lib/src/outputs/ros_service_output.cpp +++ b/vino_core_lib/src/outputs/ros_service_output.cpp @@ -20,7 +20,7 @@ #include #include #include -#include "dynamic_vino_lib/outputs/ros_service_output.h" +#include "vino_core_lib/outputs/ros_service_output.h" #include "cv_bridge/cv_bridge.h" #include // Outputs::RosServiceOutput::RosServiceOutput() @@ -51,7 +51,7 @@ void Outputs::RosServiceOutput::setResponseForFace( } void Outputs::RosServiceOutput::setServiceResponse( - boost::shared_ptr response) + boost::shared_ptr response) { if (age_gender_msg_ptr_ != nullptr) { response->age_gender.objects = age_gender_msg_ptr_->objects; @@ -59,7 +59,7 @@ void Outputs::RosServiceOutput::setServiceResponse( } void Outputs::RosServiceOutput::setServiceResponse( - boost::shared_ptr response) + boost::shared_ptr response) { if (emotions_msg_ptr_ != nullptr) { response->emotion.emotions = emotions_msg_ptr_->emotions; @@ -67,7 +67,7 @@ void Outputs::RosServiceOutput::setServiceResponse( } void Outputs::RosServiceOutput::setServiceResponse( - boost::shared_ptr response) + boost::shared_ptr response) { if (headpose_msg_ptr_ != nullptr) { response->headpose.headposes = headpose_msg_ptr_->headposes; @@ -75,7 +75,7 @@ void Outputs::RosServiceOutput::setServiceResponse( } void Outputs::RosServiceOutput::setServiceResponse( - boost::shared_ptr response) + boost::shared_ptr response) { slog::info << "in ObjectsInMasks service::Response ..."; if (segmented_object_msg_ptr_ != nullptr) { @@ -83,7 +83,7 @@ void Outputs::RosServiceOutput::setServiceResponse( } } void Outputs::RosServiceOutput::setServiceResponse( - boost::shared_ptr response) + boost::shared_ptr response) { slog::info << "in Reidentification service::Response ..."; if (person_reid_msg_ptr_ != nullptr) { @@ -92,7 +92,17 @@ void Outputs::RosServiceOutput::setServiceResponse( } void Outputs::RosServiceOutput::setServiceResponse( - boost::shared_ptr response) + boost::shared_ptr response) +{ + slog::info << "in Human Pose Estimation service::Response ..."; + if (human_pose_msg_ptr_ != nullptr) { + response->humanposes = human_pose_msg_ptr_->humanposes; + } + slog::info << "DONE!" << slog::endl; +} + +void Outputs::RosServiceOutput::setServiceResponse( + boost::shared_ptr response) { slog::info << "in People::Response ..."; if (faces_msg_ptr_ != nullptr) { @@ -126,4 +136,5 @@ void Outputs::RosServiceOutput::clearData() headpose_msg_ptr_ = nullptr; segmented_object_msg_ptr_ = nullptr; person_reid_msg_ptr_ = nullptr; + human_pose_msg_ptr_ = nullptr; } diff --git a/dynamic_vino_lib/src/outputs/ros_topic_output.cpp b/vino_core_lib/src/outputs/ros_topic_output.cpp similarity index 69% rename from dynamic_vino_lib/src/outputs/ros_topic_output.cpp rename to vino_core_lib/src/outputs/ros_topic_output.cpp index 3dab5174..3ef55bc3 100644 --- a/dynamic_vino_lib/src/outputs/ros_topic_output.cpp +++ b/vino_core_lib/src/outputs/ros_topic_output.cpp @@ -19,28 +19,31 @@ * @file ros_topic_output.cpp */ -#include "dynamic_vino_lib/outputs/ros_topic_output.h" +#include "vino_core_lib/outputs/ros_topic_output.h" #include #include #include #include +#include Outputs::RosTopicOutput::RosTopicOutput() { pub_face_ = nh_.advertise("/openvino_toolkit/faces", 16); - pub_emotion_ = nh_.advertise( + pub_emotion_ = nh_.advertise( "/openvino_toolkit/emotions", 16); - pub_age_gender_ = nh_.advertise( + pub_age_gender_ = nh_.advertise( "/openvino_toolkit/age_genders", 16); - pub_headpose_ = nh_.advertise( + pub_headpose_ = nh_.advertise( "/openvino_toolkit/headposes", 16); pub_object_ = nh_.advertise( "/openvino_toolkit/detected_objects", 16); - pub_person_reid_ = nh_.advertise( + pub_person_reid_ = nh_.advertise( "/openvino_toolkit/reidentified_persons", 16); - pub_segmented_object_ = nh_.advertise( + pub_segmented_object_ = nh_.advertise( "/openvino_toolkit/segmented_obejcts", 16); + pub_human_pose_ = nh_.advertise( + "/openvino_toolkit/human_poses", 16); emotions_msg_ptr_ = NULL; faces_msg_ptr_ = NULL; @@ -49,16 +52,17 @@ Outputs::RosTopicOutput::RosTopicOutput() object_msg_ptr_ = NULL; person_reid_msg_ptr_ = NULL; segmented_object_msg_ptr_ = NULL; + human_pose_msg_ptr_ = NULL; } void Outputs::RosTopicOutput::feedFrame(const cv::Mat& frame) {} void Outputs::RosTopicOutput::accept( - const std::vector & results) + const std::vector & results) { - person_reid_msg_ptr_ = std::make_shared(); - people_msgs::Reidentification person; + person_reid_msg_ptr_ = std::make_shared(); + vino_people_msgs::Reidentification person; for (auto & r : results) { // slog::info << ">"; auto loc = r.getLocation(); @@ -72,10 +76,10 @@ void Outputs::RosTopicOutput::accept( } void Outputs::RosTopicOutput::accept( - const std::vector & results) + const std::vector & results) { - segmented_object_msg_ptr_ = std::make_shared(); - people_msgs::ObjectInMask object; + segmented_object_msg_ptr_ = std::make_shared(); + vino_people_msgs::ObjectInMask object; for (auto & r : results) { // slog::info << ">"; auto loc = r.getLocation(); @@ -97,7 +101,7 @@ void Outputs::RosTopicOutput::accept( void Outputs::RosTopicOutput::accept( - const std::vector& results) + const std::vector& results) { faces_msg_ptr_ = std::make_shared(); @@ -116,11 +120,11 @@ void Outputs::RosTopicOutput::accept( } void Outputs::RosTopicOutput::accept( - const std::vector& results) + const std::vector& results) { - emotions_msg_ptr_ = std::make_shared(); + emotions_msg_ptr_ = std::make_shared(); - people_msgs::Emotion emotion; + vino_people_msgs::Emotion emotion; for (auto r : results) { auto loc = r.getLocation(); @@ -134,11 +138,11 @@ void Outputs::RosTopicOutput::accept( } void Outputs::RosTopicOutput::accept( - const std::vector& results) + const std::vector& results) { - age_gender_msg_ptr_ = std::make_shared(); + age_gender_msg_ptr_ = std::make_shared(); - people_msgs::AgeGender ag; + vino_people_msgs::AgeGender ag; for (auto r : results) { auto loc = r.getLocation(); @@ -163,11 +167,11 @@ void Outputs::RosTopicOutput::accept( } void Outputs::RosTopicOutput::accept( - const std::vector& results) + const std::vector& results) { - headpose_msg_ptr_ = std::make_shared(); + headpose_msg_ptr_ = std::make_shared(); - people_msgs::HeadPose hp; + vino_people_msgs::HeadPose hp; for (auto r : results) { auto loc = r.getLocation(); @@ -183,7 +187,7 @@ void Outputs::RosTopicOutput::accept( } void Outputs::RosTopicOutput::accept( - const std::vector& results) + const std::vector& results) { object_msg_ptr_ = std::make_shared(); @@ -201,11 +205,48 @@ void Outputs::RosTopicOutput::accept( } } +void Outputs::RosTopicOutput::accept( + const std::vector& results) +{ + human_pose_msg_ptr_ = std::make_shared(); + + for (auto r : results) + { + vino_people_msgs::HumanPose hp; + + auto loc = r.getLocation(); + hp.roi.x_offset = loc.x; + hp.roi.y_offset = loc.y; + hp.roi.width = loc.width; + hp.roi.height = loc.height; + hp.score = r.getScore(); + + auto p = vino_people_msgs::HumanPoseKeypoint();// geometry_msgs::Point(); + p.position.z = -1; + for (auto kp : r.keypoints) + { + p.position.x = kp.x; + p.position.y = kp.y; + if (kp.x >= 0) + { + p.score = kp.score; + } + else + { + p.score = 0; + } + + hp.keypoints.push_back(p); + } + human_pose_msg_ptr_->humanposes.push_back(hp); + } +} + void Outputs::RosTopicOutput::handleOutput() { std_msgs::Header header = getHeader(); if (person_reid_msg_ptr_ != nullptr) { - people_msgs::ReidentificationStamped person_reid_msg; + vino_people_msgs::ReidentificationStamped person_reid_msg; person_reid_msg.header = header; person_reid_msg.reidentified_vector.swap(person_reid_msg_ptr_->reidentified_vector); pub_person_reid_.publish(person_reid_msg); @@ -213,7 +254,7 @@ void Outputs::RosTopicOutput::handleOutput() } if (segmented_object_msg_ptr_ != nullptr) { // slog::info << "publishing faces outputs." << slog::endl; - people_msgs::ObjectsInMasks segmented_msg; + vino_people_msgs::ObjectsInMasks segmented_msg; segmented_msg.header = header; segmented_msg.objects_vector.swap(segmented_object_msg_ptr_->objects_vector); pub_segmented_object_.publish(segmented_msg); @@ -230,7 +271,7 @@ void Outputs::RosTopicOutput::handleOutput() } if (emotions_msg_ptr_ != nullptr) { - people_msgs::EmotionsStamped emotions_msg; + vino_people_msgs::EmotionsStamped emotions_msg; emotions_msg.header = header; emotions_msg.emotions.swap(emotions_msg_ptr_->emotions); @@ -239,7 +280,7 @@ void Outputs::RosTopicOutput::handleOutput() } if (age_gender_msg_ptr_ != nullptr) { - people_msgs::AgeGenderStamped age_gender_msg; + vino_people_msgs::AgeGenderStamped age_gender_msg; age_gender_msg.header = header; age_gender_msg.objects.swap(age_gender_msg_ptr_->objects); @@ -248,7 +289,7 @@ void Outputs::RosTopicOutput::handleOutput() } if (headpose_msg_ptr_ != nullptr) { - people_msgs::HeadPoseStamped headpose_msg; + vino_people_msgs::HeadPoseStamped headpose_msg; headpose_msg.header = header; headpose_msg.headposes.swap(headpose_msg_ptr_->headposes); @@ -264,7 +305,7 @@ void Outputs::RosTopicOutput::handleOutput() pub_object_.publish(object_msg); object_msg_ptr_ = nullptr; } - if (object_msg_ptr_ != nullptr) + if (object_msg_ptr_ != nullptr) // TODO: verify and remove duplicate code. { object_msgs::ObjectsInBoxes object_msg; object_msg.header = header; @@ -273,6 +314,15 @@ void Outputs::RosTopicOutput::handleOutput() pub_object_.publish(object_msg); object_msg_ptr_ = nullptr; } + if (human_pose_msg_ptr_ != nullptr) + { + vino_people_msgs::HumanPoseStamped human_pose_msg; + human_pose_msg.header = header; + human_pose_msg.humanposes.swap(human_pose_msg_ptr_->humanposes); + + pub_human_pose_.publish(human_pose_msg); + human_pose_msg_ptr_ = nullptr; + } } std_msgs::Header Outputs::RosTopicOutput::getHeader() diff --git a/dynamic_vino_lib/src/outputs/rviz_output.cpp b/vino_core_lib/src/outputs/rviz_output.cpp similarity index 73% rename from dynamic_vino_lib/src/outputs/rviz_output.cpp rename to vino_core_lib/src/outputs/rviz_output.cpp index 64f99b92..e23c38fc 100644 --- a/dynamic_vino_lib/src/outputs/rviz_output.cpp +++ b/vino_core_lib/src/outputs/rviz_output.cpp @@ -22,8 +22,8 @@ #include #include #include "cv_bridge/cv_bridge.h" -#include "dynamic_vino_lib/pipeline.h" -#include "dynamic_vino_lib/outputs/rviz_output.h" +#include "vino_core_lib/pipeline.h" +#include "vino_core_lib/outputs/rviz_output.h" Outputs::RvizOutput::RvizOutput() { @@ -37,37 +37,41 @@ void Outputs::RvizOutput::feedFrame(const cv::Mat & frame) image_window_output_->feedFrame(frame); } -void Outputs::RvizOutput::accept(const std::vector & results) +void Outputs::RvizOutput::accept(const std::vector & results) { image_window_output_->accept(results); } void Outputs::RvizOutput::accept( - const std::vector & results) + const std::vector & results) { image_window_output_->accept(results); } -void Outputs::RvizOutput::accept(const std::vector & results) +void Outputs::RvizOutput::accept(const std::vector & results) { image_window_output_->accept(results); } -void Outputs::RvizOutput::accept(const std::vector & results) +void Outputs::RvizOutput::accept(const std::vector & results) { image_window_output_->accept(results); } -void Outputs::RvizOutput::accept(const std::vector & results) +void Outputs::RvizOutput::accept(const std::vector & results) { image_window_output_->accept(results); } -void Outputs::RvizOutput::accept(const std::vector& results) +void Outputs::RvizOutput::accept(const std::vector& results) { image_window_output_->accept(results); } -void Outputs::RvizOutput::accept(const std::vector & results) +void Outputs::RvizOutput::accept(const std::vector & results) +{ + image_window_output_->accept(results); +} +void Outputs::RvizOutput::accept(const std::vector & results) { image_window_output_->accept(results); } diff --git a/dynamic_vino_lib/src/pipeline.cpp b/vino_core_lib/src/pipeline.cpp similarity index 95% rename from dynamic_vino_lib/src/pipeline.cpp rename to vino_core_lib/src/pipeline.cpp index 4da9fea4..17cb2d77 100644 --- a/dynamic_vino_lib/src/pipeline.cpp +++ b/vino_core_lib/src/pipeline.cpp @@ -24,7 +24,7 @@ #include #include -#include "dynamic_vino_lib/pipeline.h" +#include "vino_core_lib/pipeline.h" Pipeline::Pipeline(const std::string& name) { @@ -101,7 +101,7 @@ bool Pipeline::add(const std::string& name, } bool Pipeline::add(const std::string& parent, const std::string& name, - std::shared_ptr inference) + std::shared_ptr inference) { if (name_to_detection_map_.find(parent) == name_to_detection_map_.end() && input_device_name_ != parent) @@ -116,13 +116,13 @@ bool Pipeline::add(const std::string& parent, const std::string& name, } bool Pipeline::add(const std::string& name, - std::shared_ptr inference) { + std::shared_ptr inference) { if (name.empty()) { slog::err << "Item name can't be empty!" << slog::endl; return false; } - std::map>::iterator it = name_to_detection_map_.find(name); + std::map>::iterator it = name_to_detection_map_.find(name); if (it != name_to_detection_map_.end()) { slog::warn << "inferance instance for [" << name << "] already exists, update it with new instance." << slog::endl; @@ -267,7 +267,7 @@ void Pipeline::callback(const std::string& detection_name) auto next_detection_ptr = detection_ptr_iter->second; for (int i = 0; i < detection_ptr->getResultsLength(); ++i) { - const dynamic_vino_lib::Result* prev_result = + const vino_core_lib::Result* prev_result = detection_ptr->getLocationResult(i); auto clippedRect = prev_result->getLocation() & cv::Rect(0, 0, width_, height_); diff --git a/dynamic_vino_lib/src/pipeline_manager.cpp b/vino_core_lib/src/pipeline_manager.cpp similarity index 76% rename from dynamic_vino_lib/src/pipeline_manager.cpp rename to vino_core_lib/src/pipeline_manager.cpp index 0e066d8f..f4228082 100644 --- a/dynamic_vino_lib/src/pipeline_manager.cpp +++ b/vino_core_lib/src/pipeline_manager.cpp @@ -24,27 +24,27 @@ #include #include -#include "dynamic_vino_lib/factory.h" -#include "dynamic_vino_lib/inferences/age_gender_detection.h" -#include "dynamic_vino_lib/inferences/emotions_detection.h" -#include "dynamic_vino_lib/inferences/face_detection.h" -#include "dynamic_vino_lib/inferences/head_pose_detection.h" -#include "dynamic_vino_lib/inputs/image_input.h" -#include "dynamic_vino_lib/inputs/realsense_camera.h" -#include "dynamic_vino_lib/inputs/realsense_camera_topic.h" -#include "dynamic_vino_lib/inputs/standard_camera.h" -#include "dynamic_vino_lib/inputs/video_input.h" -#include "dynamic_vino_lib/models/age_gender_detection_model.h" -#include "dynamic_vino_lib/models/emotion_detection_model.h" -#include "dynamic_vino_lib/models/face_detection_model.h" -#include "dynamic_vino_lib/models/head_pose_detection_model.h" -#include "dynamic_vino_lib/outputs/image_window_output.h" -#include "dynamic_vino_lib/outputs/ros_topic_output.h" -#include "dynamic_vino_lib/outputs/rviz_output.h" -#include "dynamic_vino_lib/outputs/ros_service_output.h" -#include "dynamic_vino_lib/pipeline.h" -#include "dynamic_vino_lib/pipeline_manager.h" -#include "dynamic_vino_lib/pipeline_params.h" +#include "vino_core_lib/factory.h" +#include "vino_core_lib/inferences/age_gender_detection.h" +#include "vino_core_lib/inferences/emotions_detection.h" +#include "vino_core_lib/inferences/face_detection.h" +#include "vino_core_lib/inferences/head_pose_detection.h" +#include "vino_core_lib/inputs/image_input.h" +#include "vino_core_lib/inputs/realsense_camera.h" +#include "vino_core_lib/inputs/realsense_camera_topic.h" +#include "vino_core_lib/inputs/standard_camera.h" +#include "vino_core_lib/inputs/video_input.h" +#include "vino_core_lib/models/age_gender_detection_model.h" +#include "vino_core_lib/models/emotion_detection_model.h" +#include "vino_core_lib/models/face_detection_model.h" +#include "vino_core_lib/models/head_pose_detection_model.h" +#include "vino_core_lib/outputs/image_window_output.h" +#include "vino_core_lib/outputs/ros_topic_output.h" +#include "vino_core_lib/outputs/rviz_output.h" +#include "vino_core_lib/outputs/ros_service_output.h" +#include "vino_core_lib/pipeline.h" +#include "vino_core_lib/pipeline_manager.h" +#include "vino_core_lib/pipeline_params.h" std::shared_ptr PipelineManager::createPipeline( const Params::ParamManager::PipelineParams& params) { @@ -156,7 +156,7 @@ PipelineManager::parseOutput( return outputs; } -std::map> +std::map> PipelineManager::parseInference( const Params::ParamManager::PipelineParams& params) { /**< update plugins for devices >**/ @@ -165,14 +165,14 @@ PipelineManager::parseInference( std::string FLAGS_c = pcommon.custom_cldnn_library; bool FLAGS_pc = pcommon.enable_performance_count; - std::map> + std::map> inferences; for (auto& infer : params.infers) { if (infer.name.empty() || infer.model.empty()) { continue; } slog::info << "Parsing Inference: " << infer.name << slog::endl; - std::shared_ptr object = nullptr; + std::shared_ptr object = nullptr; if (plugins_for_devices_.find(infer.engine) == plugins_for_devices_.end()) { plugins_for_devices_[infer.engine] = *Factory::makePluginByName(infer.engine, FLAGS_l, FLAGS_c, FLAGS_pc); @@ -192,14 +192,16 @@ PipelineManager::parseInference( } else if (infer.name == kInferTpye_ObjectDetection) { object = createObjectDetection(infer); - } else if (infer.name == kInferTpye_ObjectSegmentation) { object = createObjectSegmentation(infer); } else if (infer.name == kInferTpye_PersonReidentification) { object = createPersonReidentification(infer); - } + } + else if (infer.name == kInferTpye_HumanPoseEstimation) { + object = createHumanPoseEstimation(infer); + } else { slog::err << "Invalid inference name: " << infer.name << slog::endl; } @@ -214,7 +216,7 @@ PipelineManager::parseInference( return inferences; } -std::shared_ptr +std::shared_ptr PipelineManager::createFaceDetection( const Params::ParamManager::InferenceParams& infer) { // TODO: add batch size in param_manager @@ -223,7 +225,7 @@ PipelineManager::createFaceDetection( face_detection_model->modelInit(); auto face_detection_engine = std::make_shared( plugins_for_devices_[infer.engine], face_detection_model); - auto face_inference_ptr = std::make_shared( + auto face_inference_ptr = std::make_shared( 0.5); // TODO: add output_threshold in param_manager face_inference_ptr->loadNetwork(face_detection_model); face_inference_ptr->loadEngine(face_detection_engine); @@ -231,7 +233,7 @@ PipelineManager::createFaceDetection( return face_inference_ptr; } -std::shared_ptr +std::shared_ptr PipelineManager::createAgeGenderRecognition( const Params::ParamManager::InferenceParams& param) { auto model = @@ -239,14 +241,14 @@ PipelineManager::createAgeGenderRecognition( model->modelInit(); auto engine = std::make_shared( plugins_for_devices_[param.engine], model); - auto infer = std::make_shared(); + auto infer = std::make_shared(); infer->loadNetwork(model); infer->loadEngine(engine); return infer; } -std::shared_ptr +std::shared_ptr PipelineManager::createEmotionRecognition( const Params::ParamManager::InferenceParams& param) { auto model = @@ -254,14 +256,14 @@ PipelineManager::createEmotionRecognition( model->modelInit(); auto engine = std::make_shared( plugins_for_devices_[param.engine], model); - auto infer = std::make_shared(); + auto infer = std::make_shared(); infer->loadNetwork(model); infer->loadEngine(engine); return infer; } -std::shared_ptr +std::shared_ptr PipelineManager::createHeadPoseEstimation( const Params::ParamManager::InferenceParams& param) { auto model = @@ -269,14 +271,14 @@ PipelineManager::createHeadPoseEstimation( model->modelInit(); auto engine = std::make_shared( plugins_for_devices_[param.engine], model); - auto infer = std::make_shared(); + auto infer = std::make_shared(); infer->loadNetwork(model); infer->loadEngine(engine); return infer; } -std::shared_ptr +std::shared_ptr PipelineManager::createObjectDetection( const Params::ParamManager::InferenceParams & infer) { @@ -285,7 +287,7 @@ const Params::ParamManager::InferenceParams & infer) object_detection_model->modelInit(); auto object_detection_engine = std::make_shared( plugins_for_devices_[infer.engine], object_detection_model); - auto object_inference_ptr = std::make_shared( + auto object_inference_ptr = std::make_shared( infer.enable_roi_constraint, infer.confidence_threshold); // To-do theshold configuration object_inference_ptr->loadNetwork(object_detection_model); object_inference_ptr->loadEngine(object_detection_engine); @@ -293,7 +295,7 @@ const Params::ParamManager::InferenceParams & infer) return object_inference_ptr; } -std::shared_ptr +std::shared_ptr PipelineManager::createObjectSegmentation(const Params::ParamManager::InferenceParams & infer) { auto obejct_segmentation_model = @@ -301,14 +303,14 @@ PipelineManager::createObjectSegmentation(const Params::ParamManager::InferenceP obejct_segmentation_model->modelInit(); auto obejct_segmentation_engine = std::make_shared( plugins_for_devices_[infer.engine], obejct_segmentation_model); - auto segmentation_inference_ptr = std::make_shared(0.5); + auto segmentation_inference_ptr = std::make_shared(0.5); segmentation_inference_ptr->loadNetwork(obejct_segmentation_model); segmentation_inference_ptr->loadEngine(obejct_segmentation_engine); return segmentation_inference_ptr; } -std::shared_ptr +std::shared_ptr PipelineManager::createPersonReidentification( const Params::ParamManager::InferenceParams & infer) { @@ -318,13 +320,42 @@ PipelineManager::createPersonReidentification( auto person_reidentification_engine = std::make_shared( plugins_for_devices_[infer.engine], person_reidentification_model); auto reidentification_inference_ptr = - std::make_shared(infer.confidence_threshold); + std::make_shared(infer.confidence_threshold); reidentification_inference_ptr->loadNetwork(person_reidentification_model); reidentification_inference_ptr->loadEngine(person_reidentification_engine); return reidentification_inference_ptr; } +std::shared_ptr +PipelineManager::createHumanPoseEstimation( + const Params::ParamManager::InferenceParams & infer) +{ + // Following createObjectDetection and createFaceDetection + // 1 input, 2 outputs, 1 batch + auto human_estimation_model = + std::make_shared(infer.model, 1, 2, 1); + human_estimation_model->modelInit(); + auto pose_engine = std::make_shared( + plugins_for_devices_[infer.engine], human_estimation_model); + slog::info << "params: " << infer.min_peaks_distance << ", " << + infer.mid_points_score_threshold << ", " << + infer.found_mid_points_ratio_threshold << ", " << + infer.min_joints_number << ", " << + infer.min_subset_score << slog::endl; + auto pose_inference_ptr = + std::make_shared( + infer.min_peaks_distance, + infer.mid_points_score_threshold, + infer.found_mid_points_ratio_threshold, + infer.min_joints_number, + infer.min_subset_score); + pose_inference_ptr->loadNetwork(human_estimation_model); + pose_inference_ptr->loadEngine(pose_engine); + + return pose_inference_ptr; +} + void PipelineManager::threadPipeline(const char* name) { PipelineData& p = pipelines_[name]; while (p.state == PipelineState_ThreadRunning && p.pipeline != nullptr && ros::ok()) { diff --git a/dynamic_vino_lib/src/pipeline_params.cpp b/vino_core_lib/src/pipeline_params.cpp similarity index 97% rename from dynamic_vino_lib/src/pipeline_params.cpp rename to vino_core_lib/src/pipeline_params.cpp index 947688a5..f5d3bba1 100644 --- a/dynamic_vino_lib/src/pipeline_params.cpp +++ b/vino_core_lib/src/pipeline_params.cpp @@ -24,7 +24,7 @@ #include #include -#include "dynamic_vino_lib/pipeline_params.h" +#include "vino_core_lib/pipeline_params.h" PipelineParams::PipelineParams(const std::string& name) { params_.name = name; } diff --git a/dynamic_vino_lib/src/services/frame_processing_server.cpp b/vino_core_lib/src/services/frame_processing_server.cpp similarity index 80% rename from dynamic_vino_lib/src/services/frame_processing_server.cpp rename to vino_core_lib/src/services/frame_processing_server.cpp index baaaae67..ad5d3ba4 100644 --- a/dynamic_vino_lib/src/services/frame_processing_server.cpp +++ b/vino_core_lib/src/services/frame_processing_server.cpp @@ -12,10 +12,11 @@ // See the License for the specific language governing permissions and // limitations under the License. -#include "dynamic_vino_lib/services/frame_processing_server.h" -#include -#include -#include +#include "vino_core_lib/services/frame_processing_server.h" +#include +#include +#include +#include #include #include #include @@ -25,11 +26,11 @@ #include #include -#include "dynamic_vino_lib/pipeline_manager.h" -#include "dynamic_vino_lib/pipeline.h" -#include "dynamic_vino_lib/inputs/base_input.h" -#include "dynamic_vino_lib/inputs/image_input.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/pipeline_manager.h" +#include "vino_core_lib/pipeline.h" +#include "vino_core_lib/inputs/base_input.h" +#include "vino_core_lib/inputs/image_input.h" +#include "vino_core_lib/slog.h" namespace vino_service { @@ -96,7 +97,8 @@ bool FrameProcessingServer::cbService( } template class FrameProcessingServer; -template class FrameProcessingServer; -template class FrameProcessingServer; -template class FrameProcessingServer; +template class FrameProcessingServer; +template class FrameProcessingServer; +template class FrameProcessingServer; +template class FrameProcessingServer; } // namespace vino_service diff --git a/vino_launch/CMakeLists.txt b/vino_launch/CMakeLists.txt index b463d711..1af2dc8c 100644 --- a/vino_launch/CMakeLists.txt +++ b/vino_launch/CMakeLists.txt @@ -22,9 +22,9 @@ catkin_package( LIBRARIES ${PROJECT_NAME} CATKIN_DEPENDS roslaunch - dynamic_vino_lib + vino_core_lib vino_param_lib - dynamic_vino_sample + vino_sample nodelet ) diff --git a/vino_launch/launch/image_human_pose_server.launch b/vino_launch/launch/image_human_pose_server.launch new file mode 100644 index 00000000..16c191b1 --- /dev/null +++ b/vino_launch/launch/image_human_pose_server.launch @@ -0,0 +1,8 @@ + + + + + + + diff --git a/vino_launch/launch/image_object_server.launch b/vino_launch/launch/image_object_server.launch index d8ed17cd..f1e7c428 100644 --- a/vino_launch/launch/image_object_server.launch +++ b/vino_launch/launch/image_object_server.launch @@ -2,7 +2,7 @@ - diff --git a/vino_launch/launch/image_object_server_oss.launch b/vino_launch/launch/image_object_server_oss.launch index a2698e2b..69ee5972 100644 --- a/vino_launch/launch/image_object_server_oss.launch +++ b/vino_launch/launch/image_object_server_oss.launch @@ -2,7 +2,7 @@ - diff --git a/vino_launch/launch/image_people_server.launch b/vino_launch/launch/image_people_server.launch index eacd85fa..2846803a 100644 --- a/vino_launch/launch/image_people_server.launch +++ b/vino_launch/launch/image_people_server.launch @@ -2,7 +2,7 @@ - diff --git a/vino_launch/launch/image_people_server_oss.launch b/vino_launch/launch/image_people_server_oss.launch index 0d9bcf8b..f26cf97c 100644 --- a/vino_launch/launch/image_people_server_oss.launch +++ b/vino_launch/launch/image_people_server_oss.launch @@ -2,7 +2,7 @@ - diff --git a/vino_launch/launch/image_reidentification_server_oss.launch b/vino_launch/launch/image_reidentification_server_oss.launch index 3ea12149..96576c00 100644 --- a/vino_launch/launch/image_reidentification_server_oss.launch +++ b/vino_launch/launch/image_reidentification_server_oss.launch @@ -2,7 +2,7 @@ - diff --git a/vino_launch/launch/image_segmentation_server.launch b/vino_launch/launch/image_segmentation_server.launch index 8d748249..8e97e01c 100644 --- a/vino_launch/launch/image_segmentation_server.launch +++ b/vino_launch/launch/image_segmentation_server.launch @@ -2,7 +2,7 @@ - diff --git a/vino_launch/launch/pipeline_image.launch b/vino_launch/launch/pipeline_image.launch index ac037862..f2995a25 100644 --- a/vino_launch/launch/pipeline_image.launch +++ b/vino_launch/launch/pipeline_image.launch @@ -3,7 +3,7 @@ - diff --git a/vino_launch/launch/pipeline_image_oss.launch b/vino_launch/launch/pipeline_image_oss.launch index 04d43e8f..20de794c 100644 --- a/vino_launch/launch/pipeline_image_oss.launch +++ b/vino_launch/launch/pipeline_image_oss.launch @@ -2,7 +2,7 @@ - diff --git a/vino_launch/launch/pipeline_object.launch b/vino_launch/launch/pipeline_object.launch index 021ed11b..8debc692 100644 --- a/vino_launch/launch/pipeline_object.launch +++ b/vino_launch/launch/pipeline_object.launch @@ -2,7 +2,7 @@ - diff --git a/vino_launch/launch/pipeline_object_oss.launch b/vino_launch/launch/pipeline_object_oss.launch index 8f450044..bb76808f 100644 --- a/vino_launch/launch/pipeline_object_oss.launch +++ b/vino_launch/launch/pipeline_object_oss.launch @@ -2,7 +2,7 @@ - diff --git a/vino_launch/launch/pipeline_object_oss_topic.launch b/vino_launch/launch/pipeline_object_oss_topic.launch index 5624dfea..121726f3 100644 --- a/vino_launch/launch/pipeline_object_oss_topic.launch +++ b/vino_launch/launch/pipeline_object_oss_topic.launch @@ -11,7 +11,7 @@ - diff --git a/vino_launch/launch/pipeline_object_topic.launch b/vino_launch/launch/pipeline_object_topic.launch index 3e2f0e98..8424ee13 100644 --- a/vino_launch/launch/pipeline_object_topic.launch +++ b/vino_launch/launch/pipeline_object_topic.launch @@ -11,7 +11,7 @@ - diff --git a/vino_launch/launch/pipeline_param.launch b/vino_launch/launch/pipeline_param.launch index 0e2031f8..8236b236 100644 --- a/vino_launch/launch/pipeline_param.launch +++ b/vino_launch/launch/pipeline_param.launch @@ -1,7 +1,7 @@ - diff --git a/vino_launch/launch/pipeline_people_myriad.launch b/vino_launch/launch/pipeline_people_myriad.launch index 2dbc86ca..9b49eddc 100644 --- a/vino_launch/launch/pipeline_people_myriad.launch +++ b/vino_launch/launch/pipeline_people_myriad.launch @@ -2,7 +2,7 @@ - diff --git a/vino_launch/launch/pipeline_people_oss.launch b/vino_launch/launch/pipeline_people_oss.launch index 0d7a2b13..db5e91a5 100644 --- a/vino_launch/launch/pipeline_people_oss.launch +++ b/vino_launch/launch/pipeline_people_oss.launch @@ -2,7 +2,7 @@ - diff --git a/vino_launch/launch/pipeline_pose.launch b/vino_launch/launch/pipeline_pose.launch new file mode 100644 index 00000000..879e03e8 --- /dev/null +++ b/vino_launch/launch/pipeline_pose.launch @@ -0,0 +1,19 @@ + + + + + + + + + + + + + + + + + + diff --git a/vino_launch/launch/pipeline_reidentification.launch b/vino_launch/launch/pipeline_reidentification.launch index 5461fcc9..c9fd5fed 100644 --- a/vino_launch/launch/pipeline_reidentification.launch +++ b/vino_launch/launch/pipeline_reidentification.launch @@ -2,7 +2,7 @@ - diff --git a/vino_launch/launch/pipeline_reidentification_oss.launch b/vino_launch/launch/pipeline_reidentification_oss.launch index eaa393e3..83828d2a 100644 --- a/vino_launch/launch/pipeline_reidentification_oss.launch +++ b/vino_launch/launch/pipeline_reidentification_oss.launch @@ -2,7 +2,7 @@ - diff --git a/vino_launch/launch/pipeline_segmentation.launch b/vino_launch/launch/pipeline_segmentation.launch index 4a47a9e3..5074e480 100644 --- a/vino_launch/launch/pipeline_segmentation.launch +++ b/vino_launch/launch/pipeline_segmentation.launch @@ -11,7 +11,7 @@ - diff --git a/vino_launch/launch/pipeline_video.launch b/vino_launch/launch/pipeline_video.launch index f74971b7..c0d5fc97 100644 --- a/vino_launch/launch/pipeline_video.launch +++ b/vino_launch/launch/pipeline_video.launch @@ -2,7 +2,7 @@ - diff --git a/vino_launch/package.xml b/vino_launch/package.xml index 27e5b3ab..d78cd8fe 100644 --- a/vino_launch/package.xml +++ b/vino_launch/package.xml @@ -26,16 +26,16 @@ limitations under the License. catkin roslaunch - dynamic_vino_lib + vino_core_lib vino_param_lib - dynamic_vino_sample + vino_sample usb_cam nodelet roslaunch - dynamic_vino_lib + vino_core_lib vino_param_lib - dynamic_vino_sample + vino_sample usb_cam nodelet diff --git a/vino_launch/param/image_human_pose_server.yaml b/vino_launch/param/image_human_pose_server.yaml new file mode 100644 index 00000000..6ae92aeb --- /dev/null +++ b/vino_launch/param/image_human_pose_server.yaml @@ -0,0 +1,19 @@ +Pipelines: +- name: people + inputs: [Image] + infers: + - name: HumanPoseEstimation + model: /opt/openvino_toolkit/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001/FP16/human-pose-estimation-0001.xml + engine: MYRIAD + label: to/be/set/xxx.labels + batch: 1 + outputs: [RosService] + confidence_threshold: 0.2 + connects: + - left: Image + right: [HumanPoseEstimation] + - left: HumanPoseEstimation + right: [RosService] + input_path: "/opt/openvino_toolkit/ros_openvino_toolkit/data/images/team.jpg" + +OpenvinoCommon: diff --git a/vino_launch/param/pipeline_object_oss.yaml b/vino_launch/param/pipeline_object_oss.yaml index 4fe4ca7a..9b656fc1 100644 --- a/vino_launch/param/pipeline_object_oss.yaml +++ b/vino_launch/param/pipeline_object_oss.yaml @@ -3,7 +3,7 @@ Pipelines: inputs: [RealSenseCamera] infers: - name: ObjectDetection - model: /opt/intel/computer_vision_sdk/deployment_tools/model_downloader/object_detection/common/mobilenet-ssd/caffe/output/FP16/mobilenet-ssd.xml + model: /opt/openvino_toolkit/open_model_zoo/model_downloader/object_detection/common/mobilenet-ssd/caffe/output/FP32/mobilenet-ssd.xml engine: CPU label: to/be/set/xxx.labels batch: 16 diff --git a/vino_launch/param/pipeline_pose.yaml b/vino_launch/param/pipeline_pose.yaml new file mode 100644 index 00000000..7ecaeccd --- /dev/null +++ b/vino_launch/param/pipeline_pose.yaml @@ -0,0 +1,22 @@ +Pipelines: +- name: people + inputs: [RealSenseCamera] + infers: + - name: HumanPoseEstimation + model: /opt/openvino_toolkit/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001/FP16/human-pose-estimation-0001.xml + engine: MYRIAD + label: to/be/set/xxx.labels + batch: 1 + min_peaks_distance: 3.0 + mid_points_score_threshold: 0.05 + found_mid_points_ratio_threshold: 0.8 + min_joints_number: 3 + min_subset_score: 0.2 + outputs: [ImageWindow, RosTopic, RViz] + connects: + - left: RealSenseCamera + right: [HumanPoseEstimation] + - left: HumanPoseEstimation + right: [ImageWindow, RosTopic, RViz] + +OpenvinoCommon: diff --git a/vino_param_lib/include/vino_param_lib/param_manager.h b/vino_param_lib/include/vino_param_lib/param_manager.h index 1f0a5cc3..4ab13deb 100644 --- a/vino_param_lib/include/vino_param_lib/param_manager.h +++ b/vino_param_lib/include/vino_param_lib/param_manager.h @@ -70,6 +70,13 @@ class ParamManager // singleton int batch; float confidence_threshold = 0.5; bool enable_roi_constraint = false; + + // Human Pose Estimation parameters. + float min_peaks_distance = 3.0; + float mid_points_score_threshold = 0.05; + float found_mid_points_ratio_threshold = 0.8; + int min_joints_number = 3; + float min_subset_score = 0.2; }; struct PipelineParams { diff --git a/vino_param_lib/src/param_manager.cpp b/vino_param_lib/src/param_manager.cpp index 3d5d2bb6..8f362a87 100644 --- a/vino_param_lib/src/param_manager.cpp +++ b/vino_param_lib/src/param_manager.cpp @@ -105,6 +105,13 @@ void operator>>(const YAML::Node& node, ParamManager::InferenceParams& infer) YAML_PARSE(node, "batch", infer.batch) YAML_PARSE(node, "confidence_threshold", infer.confidence_threshold) YAML_PARSE(node, "enable_roi_constraint", infer.enable_roi_constraint) + + // Human Pose Estimation parameters + YAML_PARSE(node, "min_peaks_distance", infer.min_peaks_distance) + YAML_PARSE(node, "mid_points_score_threshold", infer.mid_points_score_threshold) + YAML_PARSE(node, "found_mid_points_ratio_threshold", infer.found_mid_points_ratio_threshold) + YAML_PARSE(node, "min_joints_number", infer.min_joints_number) + YAML_PARSE(node, "min_subset_score", infer.min_subset_score) slog::info << "Inference Params:name=" << infer.name << slog::endl; } @@ -184,6 +191,11 @@ void ParamManager::print() const slog::info << "\t\tBatch: " << infer.batch << slog::endl; slog::info << "\t\tConfidence_threshold: " << infer.confidence_threshold << slog::endl; slog::info << "\t\tEnable_roi_constraint: " << infer.enable_roi_constraint << slog::endl; + slog::info << "\t\tMin_peaks_distance: " << infer.min_peaks_distance << slog::endl; + slog::info << "\t\tMid_points_score_threshold: " << infer.mid_points_score_threshold << slog::endl; + slog::info << "\t\tFound_mid_points_ratio_threshold: " << infer.found_mid_points_ratio_threshold << slog::endl; + slog::info << "\t\tMin_joints_number: " << infer.min_joints_number << slog::endl; + slog::info << "\t\tMin_subset_score: " << infer.min_subset_score << slog::endl; } slog::info << "\tConnections: " << slog::endl; diff --git a/people_msgs/CMakeLists.txt b/vino_people_msgs/CMakeLists.txt similarity index 96% rename from people_msgs/CMakeLists.txt rename to vino_people_msgs/CMakeLists.txt index fbf650ad..42d40f5d 100644 --- a/people_msgs/CMakeLists.txt +++ b/vino_people_msgs/CMakeLists.txt @@ -14,13 +14,14 @@ cmake_minimum_required(VERSION 2.8.3) -project(people_msgs) +project(vino_people_msgs) find_package(catkin REQUIRED COMPONENTS std_msgs sensor_msgs geometry_msgs message_generation + object_msgs ) add_message_files(DIRECTORY msg FILES @@ -35,6 +36,9 @@ add_message_files(DIRECTORY msg FILES ObjectsInMasks.msg Reidentification.msg ReidentificationStamped.msg + HumanPoseKeypoint.msg + HumanPose.msg + HumanPoseStamped.msg ) add_service_files(FILES @@ -44,6 +48,7 @@ add_service_files(FILES PeopleSrv.srv ReidentificationSrv.srv ObjectsInMasksSrv.srv + HumanPoseSrv.srv ) generate_messages(DEPENDENCIES std_msgs diff --git a/people_msgs/msg/AgeGender.msg b/vino_people_msgs/msg/AgeGender.msg similarity index 100% rename from people_msgs/msg/AgeGender.msg rename to vino_people_msgs/msg/AgeGender.msg diff --git a/people_msgs/msg/AgeGenderStamped.msg b/vino_people_msgs/msg/AgeGenderStamped.msg similarity index 94% rename from people_msgs/msg/AgeGenderStamped.msg rename to vino_people_msgs/msg/AgeGenderStamped.msg index efa0c724..bbcd3d5e 100644 --- a/people_msgs/msg/AgeGenderStamped.msg +++ b/vino_people_msgs/msg/AgeGenderStamped.msg @@ -13,4 +13,4 @@ # limitations under the License. std_msgs/Header header -people_msgs/AgeGender[] objects +vino_people_msgs/AgeGender[] objects diff --git a/people_msgs/msg/Emotion.msg b/vino_people_msgs/msg/Emotion.msg similarity index 100% rename from people_msgs/msg/Emotion.msg rename to vino_people_msgs/msg/Emotion.msg diff --git a/people_msgs/msg/EmotionsStamped.msg b/vino_people_msgs/msg/EmotionsStamped.msg similarity index 94% rename from people_msgs/msg/EmotionsStamped.msg rename to vino_people_msgs/msg/EmotionsStamped.msg index 1636fc02..9809a247 100644 --- a/people_msgs/msg/EmotionsStamped.msg +++ b/vino_people_msgs/msg/EmotionsStamped.msg @@ -13,4 +13,4 @@ # limitations under the License. std_msgs/Header header -people_msgs/Emotion[] emotions +vino_people_msgs/Emotion[] emotions diff --git a/people_msgs/msg/HeadPose.msg b/vino_people_msgs/msg/HeadPose.msg similarity index 100% rename from people_msgs/msg/HeadPose.msg rename to vino_people_msgs/msg/HeadPose.msg diff --git a/people_msgs/msg/HeadPoseStamped.msg b/vino_people_msgs/msg/HeadPoseStamped.msg similarity index 94% rename from people_msgs/msg/HeadPoseStamped.msg rename to vino_people_msgs/msg/HeadPoseStamped.msg index 75d97828..43006fd8 100644 --- a/people_msgs/msg/HeadPoseStamped.msg +++ b/vino_people_msgs/msg/HeadPoseStamped.msg @@ -13,4 +13,4 @@ # limitations under the License. std_msgs/Header header -people_msgs/HeadPose[] headposes \ No newline at end of file +vino_people_msgs/HeadPose[] headposes diff --git a/vino_people_msgs/msg/HumanPose.msg b/vino_people_msgs/msg/HumanPose.msg new file mode 100644 index 00000000..cf5fc2ce --- /dev/null +++ b/vino_people_msgs/msg/HumanPose.msg @@ -0,0 +1,17 @@ +# Copyright (c) 2018 Intel Corporation +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +vino_people_msgs/HumanPoseKeypoint[] keypoints # Keypoints +float32 score # score +sensor_msgs/RegionOfInterest roi # region of interest for a person's body \ No newline at end of file diff --git a/vino_people_msgs/msg/HumanPoseKeypoint.msg b/vino_people_msgs/msg/HumanPoseKeypoint.msg new file mode 100644 index 00000000..a13caa02 --- /dev/null +++ b/vino_people_msgs/msg/HumanPoseKeypoint.msg @@ -0,0 +1,2 @@ +float64 score +geometry_msgs/Point position \ No newline at end of file diff --git a/vino_people_msgs/msg/HumanPoseStamped.msg b/vino_people_msgs/msg/HumanPoseStamped.msg new file mode 100644 index 00000000..8a93b1d7 --- /dev/null +++ b/vino_people_msgs/msg/HumanPoseStamped.msg @@ -0,0 +1,16 @@ +# Copyright (c) 2018 Intel Corporation +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +std_msgs/Header header +vino_people_msgs/HumanPose[] humanposes diff --git a/people_msgs/msg/ObjectInMask.msg b/vino_people_msgs/msg/ObjectInMask.msg similarity index 100% rename from people_msgs/msg/ObjectInMask.msg rename to vino_people_msgs/msg/ObjectInMask.msg diff --git a/people_msgs/msg/ObjectsInMasks.msg b/vino_people_msgs/msg/ObjectsInMasks.msg similarity index 100% rename from people_msgs/msg/ObjectsInMasks.msg rename to vino_people_msgs/msg/ObjectsInMasks.msg diff --git a/people_msgs/msg/PersonsStamped.msg b/vino_people_msgs/msg/PersonsStamped.msg similarity index 84% rename from people_msgs/msg/PersonsStamped.msg rename to vino_people_msgs/msg/PersonsStamped.msg index 2aae1a08..2eae7eed 100644 --- a/people_msgs/msg/PersonsStamped.msg +++ b/vino_people_msgs/msg/PersonsStamped.msg @@ -14,6 +14,6 @@ std_msgs/Header header object_msgs/ObjectInBox[] faces -people_msgs/Emotion[] emotions -people_msgs/AgeGender[] agegenders -people_msgs/HeadPose[] headposes +vino_people_msgs/Emotion[] emotions +vino_people_msgs/AgeGender[] agegenders +vino_people_msgs/HeadPose[] headposes diff --git a/people_msgs/msg/Reidentification.msg b/vino_people_msgs/msg/Reidentification.msg similarity index 100% rename from people_msgs/msg/Reidentification.msg rename to vino_people_msgs/msg/Reidentification.msg diff --git a/people_msgs/msg/ReidentificationStamped.msg b/vino_people_msgs/msg/ReidentificationStamped.msg similarity index 100% rename from people_msgs/msg/ReidentificationStamped.msg rename to vino_people_msgs/msg/ReidentificationStamped.msg diff --git a/people_msgs/package.xml b/vino_people_msgs/package.xml similarity index 98% rename from people_msgs/package.xml rename to vino_people_msgs/package.xml index 3d12f136..2960f5e5 100644 --- a/people_msgs/package.xml +++ b/vino_people_msgs/package.xml @@ -14,7 +14,7 @@ limitations under the License. --> - people_msgs + vino_people_msgs 0.3.0 A package containing people message definitions. Weizhi Liu diff --git a/people_msgs/srv/AgeGenderSrv.srv b/vino_people_msgs/srv/AgeGenderSrv.srv similarity index 100% rename from people_msgs/srv/AgeGenderSrv.srv rename to vino_people_msgs/srv/AgeGenderSrv.srv diff --git a/people_msgs/srv/EmotionSrv.srv b/vino_people_msgs/srv/EmotionSrv.srv similarity index 100% rename from people_msgs/srv/EmotionSrv.srv rename to vino_people_msgs/srv/EmotionSrv.srv diff --git a/people_msgs/srv/HeadPoseSrv.srv b/vino_people_msgs/srv/HeadPoseSrv.srv similarity index 100% rename from people_msgs/srv/HeadPoseSrv.srv rename to vino_people_msgs/srv/HeadPoseSrv.srv diff --git a/vino_people_msgs/srv/HumanPoseSrv.srv b/vino_people_msgs/srv/HumanPoseSrv.srv new file mode 100644 index 00000000..2e468088 --- /dev/null +++ b/vino_people_msgs/srv/HumanPoseSrv.srv @@ -0,0 +1,17 @@ +# Copyright (c) 2017 Intel Corporation +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +string image_path # input: an image +--- +HumanPose[] humanposes # ObjectInBox array diff --git a/people_msgs/srv/ObjectsInMasksSrv.srv b/vino_people_msgs/srv/ObjectsInMasksSrv.srv similarity index 100% rename from people_msgs/srv/ObjectsInMasksSrv.srv rename to vino_people_msgs/srv/ObjectsInMasksSrv.srv diff --git a/people_msgs/srv/PeopleSrv.srv b/vino_people_msgs/srv/PeopleSrv.srv similarity index 100% rename from people_msgs/srv/PeopleSrv.srv rename to vino_people_msgs/srv/PeopleSrv.srv diff --git a/people_msgs/srv/ReidentificationSrv.srv b/vino_people_msgs/srv/ReidentificationSrv.srv similarity index 100% rename from people_msgs/srv/ReidentificationSrv.srv rename to vino_people_msgs/srv/ReidentificationSrv.srv diff --git a/sample/CMakeLists.txt b/vino_sample/CMakeLists.txt similarity index 87% rename from sample/CMakeLists.txt rename to vino_sample/CMakeLists.txt index f91efb5c..259a0ebe 100644 --- a/sample/CMakeLists.txt +++ b/vino_sample/CMakeLists.txt @@ -14,7 +14,7 @@ cmake_minimum_required(VERSION 2.8.3) -project(dynamic_vino_sample) +project(vino_sample) message(STATUS "Looking for inference engine configuration file at: ${CMAKE_PREFIX_PATH}") find_package(InferenceEngine 1.1) @@ -28,9 +28,9 @@ find_package( roslint cv_bridge object_msgs - people_msgs + vino_people_msgs vino_param_lib - dynamic_vino_lib + vino_core_lib ) find_package( @@ -47,7 +47,7 @@ catkin_package( roscpp object_msgs vino_param_lib - dynamic_vino_lib + vino_core_lib ) if (CMAKE_BUILD_TYPE EQUAL "RELEASE") @@ -66,7 +66,7 @@ include_directories ( ${InferenceEngine_INCLUDE_DIRS}/../build/samples/thirdparty/gflags/include ${InferenceEngine_INCLUDE_DIRS}/../samples ${InferenceEngine_DIR}/../src - ${dynamic_vino_lib_INCLUDE_DIRS} + ${vino_core_lib_INCLUDE_DIRS} ${vino_param_lib_INCLUDE_DIRS} ) @@ -104,7 +104,7 @@ add_executable(pipeline_with_params add_dependencies(pipeline_with_params ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS} - ${dynamic_vino_lib_TARGETS} + ${vino_core_lib_TARGETS} ) target_link_libraries(pipeline_with_params @@ -123,7 +123,7 @@ add_executable(image_people_server add_dependencies(image_people_server ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS} - ${dynamic_vino_lib_TARGETS} + ${vino_core_lib_TARGETS} ) target_link_libraries(image_people_server @@ -143,7 +143,7 @@ add_executable(image_people_client add_dependencies(image_people_client ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS} - ${dynamic_vino_lib_TARGETS} + ${vino_core_lib_TARGETS} ) target_link_libraries(image_people_client @@ -162,7 +162,7 @@ add_executable(image_object_server add_dependencies(image_object_server ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS} - ${dynamic_vino_lib_TARGETS} + ${vino_core_lib_TARGETS} ) target_link_libraries(image_object_server @@ -182,7 +182,7 @@ add_executable(image_object_client add_dependencies(image_object_client ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS} - ${dynamic_vino_lib_TARGETS} + ${vino_core_lib_TARGETS} ) target_link_libraries(image_object_client @@ -202,7 +202,7 @@ add_executable(image_segmentation_client add_dependencies(image_segmentation_client ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS} - ${dynamic_vino_lib_TARGETS} + ${vino_core_lib_TARGETS} ) target_link_libraries(image_segmentation_client @@ -221,7 +221,7 @@ add_executable(image_segmentation_server add_dependencies(image_segmentation_server ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS} - ${dynamic_vino_lib_TARGETS} + ${vino_core_lib_TARGETS} ) target_link_libraries(image_segmentation_server @@ -241,7 +241,7 @@ add_executable(image_reidentification_client add_dependencies(image_reidentification_client ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS} - ${dynamic_vino_lib_TARGETS} + ${vino_core_lib_TARGETS} ) target_link_libraries(image_reidentification_client @@ -260,7 +260,7 @@ add_executable(image_reidentification_server add_dependencies(image_reidentification_server ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS} - ${dynamic_vino_lib_TARGETS} + ${vino_core_lib_TARGETS} ) target_link_libraries(image_reidentification_server @@ -272,6 +272,43 @@ target_link_libraries(image_reidentification_server ${OpenCV_LIBRARIES} ) +add_executable(image_human_pose_client + src/image_human_pose_client.cpp +) + +add_dependencies(image_human_pose_client + ${${PROJECT_NAME}_EXPORTED_TARGETS} + ${catkin_EXPORTED_TARGETS} + ${vino_core_lib_TARGETS} +) + +target_link_libraries(image_human_pose_client + ${catkin_LIBRARIES} + cpu_extension + gflags + ${vino_param_lib_LIBRARIES} + ${InferenceEngine_LIBRARIES} + ${OpenCV_LIBRARIES} +) + +add_executable(image_human_pose_server + src/image_human_pose_server.cpp +) + +add_dependencies(image_human_pose_server + ${${PROJECT_NAME}_EXPORTED_TARGETS} + ${catkin_EXPORTED_TARGETS} + ${vino_core_lib_TARGETS} +) + +target_link_libraries(image_human_pose_server + ${catkin_LIBRARIES} + cpu_extension + gflags + ${vino_param_lib_LIBRARIES} + ${InferenceEngine_LIBRARIES} + ${OpenCV_LIBRARIES} +) if(UNIX OR APPLE) # Linker flags. diff --git a/sample/include/sample/utility.hpp b/vino_sample/include/vino_sample/utility.hpp similarity index 100% rename from sample/include/sample/utility.hpp rename to vino_sample/include/vino_sample/utility.hpp diff --git a/sample/package.xml b/vino_sample/package.xml similarity index 89% rename from sample/package.xml rename to vino_sample/package.xml index 08c81d9d..4d6d7fd2 100644 --- a/sample/package.xml +++ b/vino_sample/package.xml @@ -14,7 +14,7 @@ limitations under the License. --> - dynamic_vino_sample + vino_sample 0.3.0 a ROS2 wrapper package for Intel OpenVINO Weizhi Liu @@ -29,19 +29,19 @@ limitations under the License. roscpp roslint std_msgs - dynamic_vino_lib + vino_core_lib vino_param_lib cv_bridge object_msgs - people_msgs + vino_people_msgs roscpp roslint std_msgs sensor_msgs cv_bridge - dynamic_vino_lib + vino_core_lib vino_param_lib object_msgs - people_msgs + vino_people_msgs diff --git a/vino_sample/src/image_human_pose_client.cpp b/vino_sample/src/image_human_pose_client.cpp new file mode 100644 index 00000000..dc2f6c88 --- /dev/null +++ b/vino_sample/src/image_human_pose_client.cpp @@ -0,0 +1,69 @@ +/* + * Copyright (c) 2018 Intel Corporation + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** +* \brief A sample for this library. This sample performs human pose estimation. +* \file src/image_human_pose_client.cpp +*/ + +#include +#include +#include "opencv2/opencv.hpp" + +#include + + + +int main(int argc, char ** argv) +{ + ros::init(argc, argv, "image_human_pose_client"); + + ros::NodeHandle n; + + ros::ServiceClient client = n.serviceClient("/openvino_toolkit/service"); + + + vino_people_msgs::HumanPoseSrv srv; + + if (client.call(srv)) + { + ROS_INFO("Request service success!"); + + for (unsigned int i = 0; i < srv.response.humanposes.size(); i++) { + std::ostringstream os; + os << "Pose " << i << ", keypoints (" << srv.response.humanposes[i].keypoints.size() << "):"; + + for (auto kp : srv.response.humanposes[i].keypoints) + { + os << std::endl<< "\tx: " << kp.position.x << "\ty: " << kp.position.y << "\tscore: " << kp.score; + } + ROS_INFO(os.str().c_str()); + } + + + } + else + { + ROS_ERROR("Failed to request service \"openvino_toolkit/service\" "); return -1; + } + + + + +} + + + diff --git a/vino_sample/src/image_human_pose_server.cpp b/vino_sample/src/image_human_pose_server.cpp new file mode 100644 index 00000000..f5f9d764 --- /dev/null +++ b/vino_sample/src/image_human_pose_server.cpp @@ -0,0 +1,98 @@ +/* + * Copyright (c) 2018 Intel Corporation + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** +* \brief A sample for this library. This sample performs human pose estimation. +* \file src/image_human_pose_server.cpp +*/ + +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include "vino_core_lib/common.h" +#include "vino_core_lib/engines/engine.h" +#include "vino_core_lib/factory.h" +#include "vino_core_lib/inferences/age_gender_detection.h" +#include "vino_core_lib/inferences/base_inference.h" +#include "vino_core_lib/inferences/emotions_detection.h" +#include "vino_core_lib/inferences/face_detection.h" +#include "vino_core_lib/inferences/head_pose_detection.h" +#include "vino_core_lib/inferences/human_pose_estimation.h" +#include "vino_core_lib/inputs/realsense_camera_topic.h" +#include "vino_core_lib/outputs/image_window_output.h" +#include "vino_core_lib/outputs/ros_topic_output.h" +#include "vino_core_lib/outputs/rviz_output.h" +#include "vino_core_lib/pipeline.h" +#include "vino_core_lib/pipeline_manager.h" +#include "vino_core_lib/slog.h" +#include "inference_engine.hpp" +#include "opencv2/opencv.hpp" +#include "vino_sample/utility.hpp" +#include "vino_people_msgs/HumanPoseSrv.h" + +bool parseAndCheckCommandLine(int argc, char** argv) +{ + // -----Parsing and validation of input args--------------------------- + gflags::ParseCommandLineNonHelpFlags(&argc, &argv, true); + if (FLAGS_h) + { + showUsageForParam(); + return false; + } + + return true; +} + +int main(int argc, char** argv) +{ + ros::init(argc, argv, "human_pose_server"); + + if (!parseAndCheckCommandLine(argc, argv)) return 0; + + ros::param::param("~param_file", FLAGS_config, "/param/pipeline_pose.yaml"); + + slog::info << "FLAGS_config=" << FLAGS_config << slog::endl; + + std::string service_name = "/openvino_toolkit/service"; + slog::info << "service name = " << service_name << slog::endl; + // ----- Parsing and validation of input args----------------------- + + + auto node = std::make_shared>(service_name, FLAGS_config); + + slog::info << "Waiting for service request..." << slog::endl; + ros::spin(); + slog::info << "--------------End of Execution--------------" << FLAGS_config << slog::endl; + +} diff --git a/sample/src/image_object_client.cpp b/vino_sample/src/image_object_client.cpp similarity index 97% rename from sample/src/image_object_client.cpp rename to vino_sample/src/image_object_client.cpp index 1cdddb9a..b377ec5c 100644 --- a/sample/src/image_object_client.cpp +++ b/vino_sample/src/image_object_client.cpp @@ -36,7 +36,7 @@ int main(int argc, char ** argv) ros::NodeHandle n; if (argc != 2) { - ROS_INFO("Usage: rosrun dynamic_vino_sample image_object_client "); + ROS_INFO("Usage: rosrun vino_sample image_object_client "); //You can find a sample image in /opt/openvino_toolkit/ros_openvino_toolkit/data/images/car.png return -1; } diff --git a/sample/src/image_object_server.cpp b/vino_sample/src/image_object_server.cpp similarity index 74% rename from sample/src/image_object_server.cpp rename to vino_sample/src/image_object_server.cpp index 25905814..976f9139 100644 --- a/sample/src/image_object_server.cpp +++ b/vino_sample/src/image_object_server.cpp @@ -40,24 +40,24 @@ #include #include -#include "dynamic_vino_lib/common.h" -#include "dynamic_vino_lib/engines/engine.h" -#include "dynamic_vino_lib/factory.h" -#include "dynamic_vino_lib/inferences/age_gender_detection.h" -#include "dynamic_vino_lib/inferences/base_inference.h" -#include "dynamic_vino_lib/inferences/emotions_detection.h" -#include "dynamic_vino_lib/inferences/face_detection.h" -#include "dynamic_vino_lib/inferences/head_pose_detection.h" -#include "dynamic_vino_lib/inputs/realsense_camera_topic.h" -#include "dynamic_vino_lib/outputs/image_window_output.h" -#include "dynamic_vino_lib/outputs/ros_topic_output.h" -#include "dynamic_vino_lib/outputs/rviz_output.h" -#include "dynamic_vino_lib/pipeline.h" -#include "dynamic_vino_lib/pipeline_manager.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/common.h" +#include "vino_core_lib/engines/engine.h" +#include "vino_core_lib/factory.h" +#include "vino_core_lib/inferences/age_gender_detection.h" +#include "vino_core_lib/inferences/base_inference.h" +#include "vino_core_lib/inferences/emotions_detection.h" +#include "vino_core_lib/inferences/face_detection.h" +#include "vino_core_lib/inferences/head_pose_detection.h" +#include "vino_core_lib/inputs/realsense_camera_topic.h" +#include "vino_core_lib/outputs/image_window_output.h" +#include "vino_core_lib/outputs/ros_topic_output.h" +#include "vino_core_lib/outputs/rviz_output.h" +#include "vino_core_lib/pipeline.h" +#include "vino_core_lib/pipeline_manager.h" +#include "vino_core_lib/slog.h" #include "inference_engine.hpp" #include "opencv2/opencv.hpp" -#include "sample/utility.hpp" +#include "vino_sample/utility.hpp" bool parseAndCheckCommandLine(int argc, char** argv) { diff --git a/sample/src/image_people_client.cpp b/vino_sample/src/image_people_client.cpp similarity index 90% rename from sample/src/image_people_client.cpp rename to vino_sample/src/image_people_client.cpp index 5dfb1b60..1b375c80 100644 --- a/sample/src/image_people_client.cpp +++ b/vino_sample/src/image_people_client.cpp @@ -23,7 +23,7 @@ #include #include -#include +#include @@ -33,15 +33,15 @@ int main(int argc, char ** argv) ros::NodeHandle n; if (argc != 2) { - ROS_INFO( "Usage: rosrun dynamic_vino_sample image_people_client" + ROS_INFO( "Usage: rosrun vino_sample image_people_client" ""); return -1; } - ros::ServiceClient client = n.serviceClient("/openvino_toolkit/service"); + ros::ServiceClient client = n.serviceClient("/openvino_toolkit/service"); - people_msgs::PeopleSrv srv; + vino_people_msgs::PeopleSrv srv; if (client.call(srv)) { diff --git a/sample/src/image_people_server.cpp b/vino_sample/src/image_people_server.cpp similarity index 71% rename from sample/src/image_people_server.cpp rename to vino_sample/src/image_people_server.cpp index 92520ec5..cde7e21d 100644 --- a/sample/src/image_people_server.cpp +++ b/vino_sample/src/image_people_server.cpp @@ -40,24 +40,24 @@ #include #include -#include "dynamic_vino_lib/common.h" -#include "dynamic_vino_lib/engines/engine.h" -#include "dynamic_vino_lib/factory.h" -#include "dynamic_vino_lib/inferences/age_gender_detection.h" -#include "dynamic_vino_lib/inferences/base_inference.h" -#include "dynamic_vino_lib/inferences/emotions_detection.h" -#include "dynamic_vino_lib/inferences/face_detection.h" -#include "dynamic_vino_lib/inferences/head_pose_detection.h" -#include "dynamic_vino_lib/inputs/realsense_camera_topic.h" -#include "dynamic_vino_lib/outputs/image_window_output.h" -#include "dynamic_vino_lib/outputs/ros_topic_output.h" -#include "dynamic_vino_lib/outputs/rviz_output.h" -#include "dynamic_vino_lib/pipeline.h" -#include "dynamic_vino_lib/pipeline_manager.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/common.h" +#include "vino_core_lib/engines/engine.h" +#include "vino_core_lib/factory.h" +#include "vino_core_lib/inferences/age_gender_detection.h" +#include "vino_core_lib/inferences/base_inference.h" +#include "vino_core_lib/inferences/emotions_detection.h" +#include "vino_core_lib/inferences/face_detection.h" +#include "vino_core_lib/inferences/head_pose_detection.h" +#include "vino_core_lib/inputs/realsense_camera_topic.h" +#include "vino_core_lib/outputs/image_window_output.h" +#include "vino_core_lib/outputs/ros_topic_output.h" +#include "vino_core_lib/outputs/rviz_output.h" +#include "vino_core_lib/pipeline.h" +#include "vino_core_lib/pipeline_manager.h" +#include "vino_core_lib/slog.h" #include "inference_engine.hpp" #include "opencv2/opencv.hpp" -#include "sample/utility.hpp" +#include "vino_sample/utility.hpp" bool parseAndCheckCommandLine(int argc, char** argv) @@ -89,7 +89,7 @@ int main(int argc, char** argv) slog::info << "service name=" << service_name << slog::endl; auto node = std::make_shared>(service_name, FLAGS_config); + >(service_name, FLAGS_config); slog::info << "Waiting for service request..." << slog::endl; diff --git a/sample/src/image_reidentification_client.cpp b/vino_sample/src/image_reidentification_client.cpp similarity index 89% rename from sample/src/image_reidentification_client.cpp rename to vino_sample/src/image_reidentification_client.cpp index 70a94991..74f04fd3 100644 --- a/sample/src/image_reidentification_client.cpp +++ b/vino_sample/src/image_reidentification_client.cpp @@ -24,7 +24,7 @@ #include #include "opencv2/opencv.hpp" -#include +#include @@ -34,10 +34,10 @@ int main(int argc, char ** argv) ros::NodeHandle n; - ros::ServiceClient client = n.serviceClient("/openvino_toolkit/service"); + ros::ServiceClient client = n.serviceClient("/openvino_toolkit/service"); - people_msgs::ReidentificationSrv srv; + vino_people_msgs::ReidentificationSrv srv; if (client.call(srv)) { @@ -68,4 +68,4 @@ int main(int argc, char ** argv) } - \ No newline at end of file + diff --git a/sample/src/image_reidentification_server.cpp b/vino_sample/src/image_reidentification_server.cpp similarity index 71% rename from sample/src/image_reidentification_server.cpp rename to vino_sample/src/image_reidentification_server.cpp index 86293ddb..3ddb5480 100644 --- a/sample/src/image_reidentification_server.cpp +++ b/vino_sample/src/image_reidentification_server.cpp @@ -40,25 +40,25 @@ #include #include -#include "dynamic_vino_lib/common.h" -#include "dynamic_vino_lib/engines/engine.h" -#include "dynamic_vino_lib/factory.h" -#include "dynamic_vino_lib/inferences/age_gender_detection.h" -#include "dynamic_vino_lib/inferences/base_inference.h" -#include "dynamic_vino_lib/inferences/emotions_detection.h" -#include "dynamic_vino_lib/inferences/face_detection.h" -#include "dynamic_vino_lib/inferences/head_pose_detection.h" -#include "dynamic_vino_lib/inputs/realsense_camera_topic.h" -#include "dynamic_vino_lib/outputs/image_window_output.h" -#include "dynamic_vino_lib/outputs/ros_topic_output.h" -#include "dynamic_vino_lib/outputs/rviz_output.h" -#include "dynamic_vino_lib/pipeline.h" -#include "dynamic_vino_lib/pipeline_manager.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/common.h" +#include "vino_core_lib/engines/engine.h" +#include "vino_core_lib/factory.h" +#include "vino_core_lib/inferences/age_gender_detection.h" +#include "vino_core_lib/inferences/base_inference.h" +#include "vino_core_lib/inferences/emotions_detection.h" +#include "vino_core_lib/inferences/face_detection.h" +#include "vino_core_lib/inferences/head_pose_detection.h" +#include "vino_core_lib/inputs/realsense_camera_topic.h" +#include "vino_core_lib/outputs/image_window_output.h" +#include "vino_core_lib/outputs/ros_topic_output.h" +#include "vino_core_lib/outputs/rviz_output.h" +#include "vino_core_lib/pipeline.h" +#include "vino_core_lib/pipeline_manager.h" +#include "vino_core_lib/slog.h" #include "inference_engine.hpp" #include "opencv2/opencv.hpp" -#include "sample/utility.hpp" -#include +#include "vino_sample/utility.hpp" +#include bool parseAndCheckCommandLine(int argc, char** argv) @@ -90,7 +90,7 @@ int main(int argc, char** argv) auto node = std::make_shared>(service_name, FLAGS_config); + >(service_name, FLAGS_config); slog::info << "Waiting for reid service request..." << slog::endl; ros::spin(); diff --git a/sample/src/image_segmentation_client.cpp b/vino_sample/src/image_segmentation_client.cpp similarity index 90% rename from sample/src/image_segmentation_client.cpp rename to vino_sample/src/image_segmentation_client.cpp index be98d04a..5ce95813 100644 --- a/sample/src/image_segmentation_client.cpp +++ b/vino_sample/src/image_segmentation_client.cpp @@ -24,7 +24,7 @@ #include #include "opencv2/opencv.hpp" -#include +#include @@ -34,10 +34,10 @@ int main(int argc, char ** argv) ros::NodeHandle n; - ros::ServiceClient client = n.serviceClient("/openvino_toolkit/service"); + ros::ServiceClient client = n.serviceClient("/openvino_toolkit/service"); - people_msgs::ObjectsInMasksSrv srv; + vino_people_msgs::ObjectsInMasksSrv srv; if (client.call(srv)) { @@ -69,4 +69,4 @@ int main(int argc, char ** argv) } - \ No newline at end of file + diff --git a/sample/src/image_segmentation_server.cpp b/vino_sample/src/image_segmentation_server.cpp similarity index 72% rename from sample/src/image_segmentation_server.cpp rename to vino_sample/src/image_segmentation_server.cpp index 34288a29..4536ba15 100644 --- a/sample/src/image_segmentation_server.cpp +++ b/vino_sample/src/image_segmentation_server.cpp @@ -40,24 +40,24 @@ #include #include -#include "dynamic_vino_lib/common.h" -#include "dynamic_vino_lib/engines/engine.h" -#include "dynamic_vino_lib/factory.h" -#include "dynamic_vino_lib/inferences/age_gender_detection.h" -#include "dynamic_vino_lib/inferences/base_inference.h" -#include "dynamic_vino_lib/inferences/emotions_detection.h" -#include "dynamic_vino_lib/inferences/face_detection.h" -#include "dynamic_vino_lib/inferences/head_pose_detection.h" -#include "dynamic_vino_lib/inputs/realsense_camera_topic.h" -#include "dynamic_vino_lib/outputs/image_window_output.h" -#include "dynamic_vino_lib/outputs/ros_topic_output.h" -#include "dynamic_vino_lib/outputs/rviz_output.h" -#include "dynamic_vino_lib/pipeline.h" -#include "dynamic_vino_lib/pipeline_manager.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/common.h" +#include "vino_core_lib/engines/engine.h" +#include "vino_core_lib/factory.h" +#include "vino_core_lib/inferences/age_gender_detection.h" +#include "vino_core_lib/inferences/base_inference.h" +#include "vino_core_lib/inferences/emotions_detection.h" +#include "vino_core_lib/inferences/face_detection.h" +#include "vino_core_lib/inferences/head_pose_detection.h" +#include "vino_core_lib/inputs/realsense_camera_topic.h" +#include "vino_core_lib/outputs/image_window_output.h" +#include "vino_core_lib/outputs/ros_topic_output.h" +#include "vino_core_lib/outputs/rviz_output.h" +#include "vino_core_lib/pipeline.h" +#include "vino_core_lib/pipeline_manager.h" +#include "vino_core_lib/slog.h" #include "inference_engine.hpp" #include "opencv2/opencv.hpp" -#include "sample/utility.hpp" +#include "vino_sample/utility.hpp" @@ -90,7 +90,7 @@ int main(int argc, char** argv) auto node = std::make_shared>(service_name, FLAGS_config); + >(service_name, FLAGS_config); slog::info << "Waiting for seg service request..." << slog::endl; ros::spin(); diff --git a/sample/src/pipeline_with_params.cpp b/vino_sample/src/pipeline_with_params.cpp similarity index 79% rename from sample/src/pipeline_with_params.cpp rename to vino_sample/src/pipeline_with_params.cpp index 475d6cb3..e1bb29e1 100644 --- a/sample/src/pipeline_with_params.cpp +++ b/vino_sample/src/pipeline_with_params.cpp @@ -40,24 +40,24 @@ #include #include -#include "dynamic_vino_lib/common.h" -#include "dynamic_vino_lib/engines/engine.h" -#include "dynamic_vino_lib/factory.h" -#include "dynamic_vino_lib/inferences/age_gender_detection.h" -#include "dynamic_vino_lib/inferences/base_inference.h" -#include "dynamic_vino_lib/inferences/emotions_detection.h" -#include "dynamic_vino_lib/inferences/face_detection.h" -#include "dynamic_vino_lib/inferences/head_pose_detection.h" -#include "dynamic_vino_lib/inputs/realsense_camera_topic.h" -#include "dynamic_vino_lib/outputs/image_window_output.h" -#include "dynamic_vino_lib/outputs/ros_topic_output.h" -#include "dynamic_vino_lib/outputs/rviz_output.h" -#include "dynamic_vino_lib/pipeline.h" -#include "dynamic_vino_lib/pipeline_manager.h" -#include "dynamic_vino_lib/slog.h" +#include "vino_core_lib/common.h" +#include "vino_core_lib/engines/engine.h" +#include "vino_core_lib/factory.h" +#include "vino_core_lib/inferences/age_gender_detection.h" +#include "vino_core_lib/inferences/base_inference.h" +#include "vino_core_lib/inferences/emotions_detection.h" +#include "vino_core_lib/inferences/face_detection.h" +#include "vino_core_lib/inferences/head_pose_detection.h" +#include "vino_core_lib/inputs/realsense_camera_topic.h" +#include "vino_core_lib/outputs/image_window_output.h" +#include "vino_core_lib/outputs/ros_topic_output.h" +#include "vino_core_lib/outputs/rviz_output.h" +#include "vino_core_lib/pipeline.h" +#include "vino_core_lib/pipeline_manager.h" +#include "vino_core_lib/slog.h" #include "inference_engine.hpp" #include "opencv2/opencv.hpp" -#include "sample/utility.hpp" +#include "vino_sample/utility.hpp" void signalHandler(int signum) {