-
Notifications
You must be signed in to change notification settings - Fork 5.9k
datasets loading module #78
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
8b4a389
initial commit
3787bf9
document updated, added simple base class for datasets and other mino…
17d7398
finished conversion project to module
bb704fc
removed generated document
e1e269d
fixed warnings for document & Win64
17616a8
fixed last warning in document
5e647ff
fixed all comments except base class enhancment and interface-impleme…
2916f89
enhanced base class: created hierarchy for objects in train\test and …
ee684c8
structure names capitalized + minor cleaning
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| set(the_description "datasets tools") | ||
| ocv_define_module(datasetstools opencv_core) | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,331 @@ | ||
| ********************************************************* | ||
| datasetstools. Tools for working with different datasets. | ||
| ********************************************************* | ||
|
|
||
| .. highlight:: cpp | ||
|
|
||
| The datasetstools module includes classes for working with different datasets. | ||
|
|
||
| First version of this module was implemented for **Fall2014 OpenCV Challenge**. | ||
|
|
||
| Action Recognition | ||
| ------------------ | ||
|
|
||
| AR_hmdb | ||
| ======= | ||
| .. ocv:class:: AR_hmdb | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"HMDB: A Large Human Motion Database"`: http://serre-lab.clps.brown.edu/resource/hmdb-a-large-human-motion-database/ | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. From link above download dataset files: hmdb51_org.rar & test_train_splits.rar. | ||
|
|
||
| 2. Unpack them. | ||
|
|
||
| 3. To load data run: ./opencv/build/bin/example_datasetstools_ar_hmdb -p=/home/user/path_to_unpacked_folders/ | ||
|
|
||
| AR_sports | ||
| ========= | ||
| .. ocv:class:: AR_sports | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"Sports-1M Dataset"`: http://cs.stanford.edu/people/karpathy/deepvideo/ | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. From link above download dataset files (git clone https://code.google.com/p/sports-1m-dataset/). | ||
|
|
||
| 2. To load data run: ./opencv/build/bin/example_datasetstools_ar_sports -p=/home/user/path_to_downloaded_folders/ | ||
|
|
||
| Face Recognition | ||
| ---------------- | ||
|
|
||
| FR_lfw | ||
| ====== | ||
| .. ocv:class:: FR_lfw | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"Labeled Faces in the Wild-a"`: http://www.openu.ac.il/home/hassner/data/lfwa/ | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. From link above download dataset file: lfwa.tar.gz. | ||
|
|
||
| 2. Unpack it. | ||
|
|
||
| 3. To load data run: ./opencv/build/bin/example_datasetstools_fr_lfw -p=/home/user/path_to_unpacked_folder/lfw2/ | ||
|
|
||
| Gesture Recognition | ||
| ------------------- | ||
|
|
||
| GR_chalearn | ||
| =========== | ||
| .. ocv:class:: GR_chalearn | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"ChaLearn Looking at People"`: http://gesture.chalearn.org/ | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. Follow instruction from site above, download files for dataset "Track 3: Gesture Recognition": Train1.zip-Train5.zip, Validation1.zip-Validation3.zip (Register on site: www.codalab.org and accept the terms and conditions of competition: https://www.codalab.org/competitions/991#learn_the_details There are three mirrors for downloading dataset files. When I downloaded data only mirror: "Universitat Oberta de Catalunya" works). | ||
|
|
||
| 2. Unpack train archives Train1.zip-Train5.zip to one folder (currently loading validation files wasn't implemented) | ||
|
|
||
| 3. To load data run: ./opencv/build/bin/example_datasetstools_gr_chalearn -p=/home/user/path_to_unpacked_folder/ | ||
|
|
||
| GR_skig | ||
| ======= | ||
| .. ocv:class:: GR_skig | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"Sheffield Kinect Gesture Dataset"`: http://lshao.staff.shef.ac.uk/data/SheffieldKinectGesture.htm | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. From link above download dataset files: subject1_dep.7z-subject6_dep.7z, subject1_rgb.7z-subject6_rgb.7z. | ||
|
|
||
| 2. Unpack them. | ||
|
|
||
| 3. To load data run: ./opencv/build/bin/example_datasetstools_gr_skig -p=/home/user/path_to_unpacked_folders/ | ||
|
|
||
| Human Pose Estimation | ||
| --------------------- | ||
|
|
||
| HPE_parse | ||
| ========= | ||
| .. ocv:class:: HPE_parse | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"PARSE Dataset"`: http://www.ics.uci.edu/~dramanan/papers/parse/ | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. From link above download dataset file: people.zip. | ||
|
|
||
| 2. Unpack it. | ||
|
|
||
| 3. To load data run: ./opencv/build/bin/example_datasetstools_hpe_parse -p=/home/user/path_to_unpacked_folder/people_all/ | ||
|
|
||
| Image Registration | ||
| ------------------ | ||
|
|
||
| IR_affine | ||
| ========= | ||
| .. ocv:class:: IR_affine | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"Affine Covariant Regions Datasets"`: http://www.robots.ox.ac.uk/~vgg/data/data-aff.html | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. From link above download dataset files: bark\\bikes\\boat\\graf\\leuven\\trees\\ubc\\wall.tar.gz. | ||
|
|
||
| 2. Unpack them. | ||
|
|
||
| 3. To load data, for example, for "bark", run: ./opencv/build/bin/example_datasetstools_ir_affine -p=/home/user/path_to_unpacked_folder/bark/ | ||
|
|
||
| IR_robot | ||
| ======== | ||
| .. ocv:class:: IR_robot | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"Robot Data Set"`: http://roboimagedata.compute.dtu.dk/?page_id=24 | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. From link above download files for dataset "Point Feature Data Set – 2010": SET001_6.tar.gz-SET055_60.tar.gz (there are two data sets: - Full resolution images (1200×1600), ~500 Gb and - Half size image (600×800), ~115 Gb.) | ||
| 2. Unpack them to one folder. | ||
|
|
||
| 3. To load data run: ./opencv/build/bin/example_datasetstools_ir_robot -p=/home/user/path_to_unpacked_folder/ | ||
|
|
||
| Image Segmentation | ||
| ------------------ | ||
|
|
||
| IS_bsds | ||
| ======= | ||
| .. ocv:class:: IS_bsds | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"The Berkeley Segmentation Dataset and Benchmark"`: https://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/ | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. From link above download dataset files: BSDS300-human.tgz & BSDS300-images.tgz. | ||
|
|
||
| 2. Unpack them. | ||
|
|
||
| 3. To load data run: ./opencv/build/bin/example_datasetstools_is_bsds -p=/home/user/path_to_unpacked_folder/BSDS300/ | ||
|
|
||
| IS_weizmann | ||
| =========== | ||
| .. ocv:class:: IS_weizmann | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"Weizmann Segmentation Evaluation Database"`: http://www.wisdom.weizmann.ac.il/~vision/Seg_Evaluation_DB/ | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. From link above download dataset files: Weizmann_Seg_DB_1obj.ZIP & Weizmann_Seg_DB_2obj.ZIP. | ||
|
|
||
| 2. Unpack them. | ||
|
|
||
| 3. To load data, for example, for 1 object dataset, run: ./opencv/build/bin/example_datasetstools_is_weizmann -p=/home/user/path_to_unpacked_folder/1obj/ | ||
|
|
||
| Multiview Stereo Matching | ||
| ------------------------- | ||
|
|
||
| MSM_epfl | ||
| ======== | ||
| .. ocv:class:: MSM_epfl | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"EPFL Multi-View Stereo"`: http://cvlabwww.epfl.ch/~strecha/multiview/denseMVS.html | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. From link above download dataset files: castle_dense\\castle_dense_large\\castle_entry\\fountain\\herzjesu_dense\\herzjesu_dense_large_bounding\\cameras\\images\\p.tar.gz. | ||
|
|
||
| 2. Unpack them in separate folder for each object. For example, for "fountain", in folder fountain/ : fountain_dense_bounding.tar.gz -> bounding/, fountain_dense_cameras.tar.gz -> camera/, fountain_dense_images.tar.gz -> png/, fountain_dense_p.tar.gz -> P/ | ||
|
|
||
| 3. To load data, for example, for "fountain", run: ./opencv/build/bin/example_datasetstools_msm_epfl -p=/home/user/path_to_unpacked_folder/fountain/ | ||
|
|
||
| MSM_middlebury | ||
| ============== | ||
| .. ocv:class:: MSM_middlebury | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"Stereo – Middlebury Computer Vision"`: http://vision.middlebury.edu/mview/ | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. From link above download dataset files: dino\\dinoRing\\dinoSparseRing\\temple\\templeRing\\templeSparseRing.zip | ||
|
|
||
| 2. Unpack them. | ||
|
|
||
| 3. To load data, for example "temple" dataset, run: ./opencv/build/bin/example_datasetstools_msm_middlebury -p=/home/user/path_to_unpacked_folder/temple/ | ||
|
|
||
| Object Recognition | ||
| ------------------ | ||
|
|
||
| OR_imagenet | ||
| =========== | ||
| .. ocv:class:: OR_imagenet | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"ImageNet"`: http://www.image-net.org/ | ||
|
|
||
| Currently implemented loading full list with urls. Planned to implement dataset from ILSVRC challenge. | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. From link above download dataset file: imagenet_fall11_urls.tgz | ||
|
|
||
| 2. Unpack it. | ||
|
|
||
| 3. To load data run: ./opencv/build/bin/example_datasetstools_or_imagenet -p=/home/user/path_to_unpacked_file/ | ||
|
|
||
| OR_sun | ||
| ====== | ||
| .. ocv:class:: OR_sun | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"SUN Database"`: http://sun.cs.princeton.edu/ | ||
|
|
||
| Currently implemented loading "Scene Recognition Benchmark. SUN397". Planned to implement also "Object Detection Benchmark. SUN2012". | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. From link above download dataset file: SUN397.tar | ||
|
|
||
| 2. Unpack it. | ||
|
|
||
| 3. To load data run: ./opencv/build/bin/example_datasetstools_or_sun -p=/home/user/path_to_unpacked_folder/SUN397/ | ||
|
|
||
| SLAM | ||
| ---- | ||
|
|
||
| SLAM_kitti | ||
| ========== | ||
| .. ocv:class:: SLAM_kitti | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"KITTI Vision Benchmark"`: http://www.cvlibs.net/datasets/kitti/eval_odometry.php | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. From link above download "Odometry" dataset files: data_odometry_gray\\data_odometry_color\\data_odometry_velodyne\\data_odometry_poses\\data_odometry_calib.zip. | ||
|
|
||
| 2. Unpack data_odometry_poses.zip, it creates folder dataset/poses/. After that unpack data_odometry_gray.zip, data_odometry_color.zip, data_odometry_velodyne.zip. Folder dataset/sequences/ will be created with folders 00/..21/. Each of these folders will contain: image_0/, image_1/, image_2/, image_3/, velodyne/ and files calib.txt & times.txt. These two last files will be replaced after unpacking data_odometry_calib.zip at the end. | ||
|
|
||
| 3. To load data run: ./opencv/build/bin/example_datasetstools_slam_kitti -p=/home/user/path_to_unpacked_folder/dataset/ | ||
|
|
||
| SLAM_tumindoor | ||
| ============== | ||
| .. ocv:class:: SLAM_tumindoor | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"TUMindoor Dataset"`: http://www.navvis.lmt.ei.tum.de/dataset/ | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. From link above download dataset files: dslr\\info\\ladybug\\pointcloud.tar.bz2 for each dataset: 11-11-28 (1st floor)\\11-12-13 (1st floor N1)\\11-12-17a (4th floor)\\11-12-17b (3rd floor)\\11-12-17c (Ground I)\\11-12-18a (Ground II)\\11-12-18b (2nd floor) | ||
|
|
||
| 2. Unpack them in separate folder for each dataset. dslr.tar.bz2 -> dslr/, info.tar.bz2 -> info/, ladybug.tar.bz2 -> ladybug/, pointcloud.tar.bz2 -> pointcloud/. | ||
|
|
||
| 3. To load each dataset run: ./opencv/build/bin/example_datasetstools_slam_tumindoor -p=/home/user/path_to_unpacked_folders/ | ||
|
|
||
| Text Recognition | ||
| ---------------- | ||
|
|
||
| TR_chars | ||
| ======== | ||
| .. ocv:class:: TR_chars | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"The Chars74K Dataset"`: http://www.ee.surrey.ac.uk/CVSSP/demos/chars74k/ | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. From link above download dataset files: EnglishFnt\\EnglishHnd\\EnglishImg\\KannadaHnd\\KannadaImg.tgz, ListsTXT.tgz. | ||
|
|
||
| 2. Unpack them. | ||
|
|
||
| 3. Move .m files from folder ListsTXT/ to appropriate folder. For example, English/list_English_Img.m for EnglishImg.tgz. | ||
|
|
||
| 4. To load data, for example "EnglishImg", run: ./opencv/build/bin/example_datasetstools_tr_chars -p=/home/user/path_to_unpacked_folder/English/ | ||
|
|
||
| TR_svt | ||
| ====== | ||
| .. ocv:class:: TR_svt | ||
|
|
||
| Implements loading dataset: | ||
|
|
||
| _`"The Street View Text Dataset"`: http://vision.ucsd.edu/~kai/svt/ | ||
|
|
||
| .. note:: Usage | ||
|
|
||
| 1. From link above download dataset file: svt.zip. | ||
|
|
||
| 2. Unpack it. | ||
|
|
||
| 3. To load data run: ./opencv/build/bin/example_datasetstools_tr_svt -p=/home/user/path_to_unpacked_folder/svt/svt1/ | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shall we make common derived classes for each type of the problem?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
currently I don't think so. But I think that there are a lot of possible enhancements for this module. Just need some time to investigate datasets and their usages and of course, time on implementation.