Skip to content

Classification task in LFW dataset format #222

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Apr 20, 2021
Merged

Classification task in LFW dataset format #222

merged 4 commits into from
Apr 20, 2021

Conversation

yasakova-anastasia
Copy link

@yasakova-anastasia yasakova-anastasia commented Apr 19, 2021 •
edited
Loading

Summary

  • Added classification task
  • Added support arbitrary file names
  • Moved attributes for re-identification task to label

How to test

Checklist

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below)
# Copyright (C) 2020 Intel Corporation
#
# SPDX-License-Identifier: MIT

@yasakova-anastasia yasakova-anastasia changed the title Support for LFW dataset format Classification task in LFW dataset format Apr 19, 2021
This was linked to issues Apr 19, 2021
Classification task in LFW dataset format #217
Closed
Support arbitrary file names in LFW format #165
Closed
@zhiltsov-max zhiltsov-max merged commit 4279bc8 into develop Apr 20, 2021
@zhiltsov-max zhiltsov-max deleted the ay/add-lfw branch April 20, 2021 15:29
zhiltsov-max pushed a commit that referenced this pull request Jun 3, 2021
@chuneuny-emily @yasakova-anastasia
@seungyoon-woo @dvkruchinin @sstrehlk
Release 0.1.9 (#277)
057740e 
* Rename 'openvino' plugin to 'openvino_plugin' (#205)

Co-authored-by: Jihyeon Yi <[email protected]>

* Make remap labels more accurate, allow explicit label deletion, add docs, update tests (#203)

* Kate/handling multiple attributes and speed up detection split (#207)

* better handling multi-attributes for classification_split

* handling multi-attributes better for detection

* bugfix in calculating required number of images for splitting 2 correct side effect of the changes for re-id split

* allow multiple subsets with arbitrary names

* rename _is_number to _is_float and improve it

* Fix voc to coco example (#209)

* Fix export filtering

* update example in readme

* Fix export filename for LabelMe format (#200)

* change export filename for LabelMe format

* Allow simple merge for datasets with no labels

* Add a more complex test on relative paths

* Support escaping in attributes

* update changelog

Co-authored-by: Maxim Zhiltsov <[email protected]>

* split unlabeled data into subsets for task-specific splitters (#211)

* split unlabeled data into subsets for classification, detection. for re-id, 'not-supported' subsets for this data

* Fix image ext on saving in cvat format (#214)

* fix image saving in cvat format

* update changelog

* Label "face" for bounding boxes in Wider Face (#215)

* add face label

* update changelog

* Adding "difficult", "truncated", "occluded" attributes when converting to Pascal VOC if they are not present (#216)

* remove check for 'difficult' attribute

* remove check for 'truncated' and 'occluded' attributes

* update changelog

* Ignore empty lines in YOLO annotations (#221)

* Ignore empty lines in yolo annotations

* Add type hints for image class, catch image opening errors in image.size

* update changelog

* Classification task in LFW dataset format (#222)

* add classification

* update changelog

* update documentation

* Add splitter for segmentation task  (#223)

* added segmentation_split

* updated changelog

* rename reidentification to reid

* Support for CIFAR-10/100 format (#225)

* add CIFAR dataset format

* add CIFAR to documentation

* update Changelog

* add validation item for instance segmentation (#227)

* add validation item for instance segmentation

* Add panoptic and stuff COCO format (#210)

* add coco stuff and panoptic formats

* update CHANGELOG

Co-authored-by: Maxim Zhiltsov <[email protected]>

* update detection splitter algorithm from # of samples to # of instances (#235)

* add documentation for validator (#233)

* add documentation for validator

* add validation item description (#237)

* Fix converter for Pascal VOC format (#239)

* User documentation for Pascal VOC format (#228)

* add user documentation for Pascal VOC format

* add integration tests

* update changelog

* Support for MNIST dataset format (#234)

* add mnist format

* add mnist csv format

* add mnist to documentation

* make formats docs folder, create COCO format documentation (#241)

* Make formats docs folder, move format docs

* Create COCO format documentation

* Fixes in CIFAR dataset format (#243)

* Add folder creation

* Update changelog

* Add user documentation file and integration tests for YOLO format (#246)

* add user documentation file for yolo

* add integraion tests

* update user manual

* update changelog

* Add Cityscapes format (#249)

* add cityscapes format

* add format docs

* update changelog

* Fix saving attribute in WiderFace extractor (#251)

* add fixes

* update changelog

* Fix spelling errors (#252)

* Configurable Threshold CLI support (#250)

* add validator cli

* add configurable validator threshold

* update changelog

* CI. Move to GitHub actions. (#263)

* Moving to GitHub Actions

* Sending a coverage report if python3.6 (#264)

* Rename workflows (#265)

* Rename workflows

* Update repo config and badge (#266)

* Update PR template

* Update build status badge

* Fix deprecation warnings (#270)

* Update RISE docs (#255)

* Update rise docs

* Update cli help

* Pytest related changes (#248)

* Tests moved to pytest. Updated CI. Updated requirements.

* Updated contribution guide

* Added annotations for tests

* Updated tests

* Added code style guide

* Fix CI (#272)

* Fix script call

* change script call to binary call

* Fix help program name, add mark_bug (#275)

* Fix prog name

* Add mark_bug test annotation

* Fix labelmap parameter in CamVid (#262)

* Fix labelmap parameter in camvid

* Release 0.1.9 (dev) (#276)

* Update version

* Update changelog

* Fix numpy conflict (#278)

Co-authored-by: Emily Chun <[email protected]>
Co-authored-by: Jihyeon Yi <[email protected]>
Co-authored-by: Kirill Sizov <[email protected]>
Co-authored-by: Anastasia Yasakova <[email protected]>
Co-authored-by: Harim Kang <[email protected]>
Co-authored-by: Zoya Maslova <[email protected]>
Co-authored-by: Roman Donchenko <[email protected]>
Co-authored-by: Seungyoon Woo <[email protected]>
Co-authored-by: Dmitry Kruchinin <[email protected]>
Co-authored-by: Slawomir Strehlke <[email protected]>
zhiltsov-max pushed a commit that referenced this pull request Jul 14, 2021
@chuneuny-emily @yasakova-anastasia
@seungyoon-woo @dvkruchinin @sstrehlk @jaesuny @jayraj-solanki
Release 1.10 (#355)
c43bb2d 
* Rename 'openvino' plugin to 'openvino_plugin' (#205)

Co-authored-by: Jihyeon Yi <[email protected]>

* Make remap labels more accurate, allow explicit label deletion, add docs, update tests (#203)

* Kate/handling multiple attributes and speed up detection split (#207)

* better handling multi-attributes for classification_split

* handling multi-attributes better for detection

* bugfix in calculating required number of images for splitting 2 correct side effect of the changes for re-id split

* allow multiple subsets with arbitrary names

* rename _is_number to _is_float and improve it

* Fix voc to coco example (#209)

* Fix export filtering

* update example in readme

* Fix export filename for LabelMe format (#200)

* change export filename for LabelMe format

* Allow simple merge for datasets with no labels

* Add a more complex test on relative paths

* Support escaping in attributes

* update changelog

Co-authored-by: Maxim Zhiltsov <[email protected]>

* split unlabeled data into subsets for task-specific splitters (#211)

* split unlabeled data into subsets for classification, detection. for re-id, 'not-supported' subsets for this data

* Fix image ext on saving in cvat format (#214)

* fix image saving in cvat format

* update changelog

* Label "face" for bounding boxes in Wider Face (#215)

* add face label

* update changelog

* Adding "difficult", "truncated", "occluded" attributes when converting to Pascal VOC if they are not present (#216)

* remove check for 'difficult' attribute

* remove check for 'truncated' and 'occluded' attributes

* update changelog

* Ignore empty lines in YOLO annotations (#221)

* Ignore empty lines in yolo annotations

* Add type hints for image class, catch image opening errors in image.size

* update changelog

* Classification task in LFW dataset format (#222)

* add classification

* update changelog

* update documentation

* Add splitter for segmentation task  (#223)

* added segmentation_split

* updated changelog

* rename reidentification to reid

* Support for CIFAR-10/100 format (#225)

* add CIFAR dataset format

* add CIFAR to documentation

* update Changelog

* add validation item for instance segmentation (#227)

* add validation item for instance segmentation

* Add panoptic and stuff COCO format (#210)

* add coco stuff and panoptic formats

* update CHANGELOG

Co-authored-by: Maxim Zhiltsov <[email protected]>

* update detection splitter algorithm from # of samples to # of instances (#235)

* add documentation for validator (#233)

* add documentation for validator

* add validation item description (#237)

* Fix converter for Pascal VOC format (#239)

* User documentation for Pascal VOC format (#228)

* add user documentation for Pascal VOC format

* add integration tests

* update changelog

* Support for MNIST dataset format (#234)

* add mnist format

* add mnist csv format

* add mnist to documentation

* make formats docs folder, create COCO format documentation (#241)

* Make formats docs folder, move format docs

* Create COCO format documentation

* Fixes in CIFAR dataset format (#243)

* Add folder creation

* Update changelog

* Add user documentation file and integration tests for YOLO format (#246)

* add user documentation file for yolo

* add integraion tests

* update user manual

* update changelog

* Add Cityscapes format (#249)

* add cityscapes format

* add format docs

* update changelog

* Fix saving attribute in WiderFace extractor (#251)

* add fixes

* update changelog

* Fix spelling errors (#252)

* Configurable Threshold CLI support (#250)

* add validator cli

* add configurable validator threshold

* update changelog

* CI. Move to GitHub actions. (#263)

* Moving to GitHub Actions

* Sending a coverage report if python3.6 (#264)

* Rename workflows (#265)

* Rename workflows

* Update repo config and badge (#266)

* Update PR template

* Update build status badge

* Fix deprecation warnings (#270)

* Update RISE docs (#255)

* Update rise docs

* Update cli help

* Pytest related changes (#248)

* Tests moved to pytest. Updated CI. Updated requirements.

* Updated contribution guide

* Added annotations for tests

* Updated tests

* Added code style guide

* Fix CI (#272)

* Fix script call

* change script call to binary call

* Fix help program name, add mark_bug (#275)

* Fix prog name

* Add mark_bug test annotation

* Fix labelmap parameter in CamVid (#262)

* Fix labelmap parameter in camvid

* Release 0.1.9 (dev) (#276)

* Update version

* Update changelog

* Fix numpy conflict (#278)

* Add changelog stub (#279)

* tests/requirements.py: remove the test_wrapper functions (#285)

* Subformat importers for VOC and COCO (#281)

* Document find_sources

* Add VOC subformat importers

* Add coco subformat importers

* Fix LFW

* Reduce voc detect dataset cases

* Reorganize coco tests, add subformat tests

* Fix default subset handling in Dataset

* Fix getting subset

* Fix coco tests

* Fix voc tests

* Update changelog

* Add image zip format (#273)

* add tests

* add image_zip format

* update changelog

Co-authored-by: Maxim Zhiltsov <[email protected]>

* Add KITTI detection and segmentation formats (#282)

* Add KITTI detection and segmentation formats

* Remove unused import

* Add KITTI user manual

Co-authored-by: Maxim Zhiltsov <[email protected]>

* Fix loading file and image processing in CIFAR (#284)

* Fix image layout and encoding problems

* Update Changelog

Co-authored-by: Maxim Zhiltsov <[email protected]>

* CLI tests for convert command for VOC dataset (#286)

* Add tests for convert command

* Convert most enum definitions from the functional style to the class style (#290)

* yolo format documentation update (#295)

* add info about coordinates in yolo format doc

* Fix merged dataset item filtering (#258)

* Add tests

* Fix xpathfilter transform

* Update changelog

* Sms/pytest marking cityscapes and zip (#298)

* Updated pytest marking for cityscapes and imagezip.

* Introduce Validator plugin type (#299)

* Introduce Validator plugin type

* Fix validator definitions (#303)

* update changelog

* Fixes in validator definitions

* Update validator cli

* Make TF availability check optional (#305)

* Make tf availability check optional

* update changelog

* Update pylint (#304)

* Add import order check in pylint

* Fix some linter problems

* Remove warning suppression comments

* Add lazy loading for builtin plugins (#306)

* Refactor env code

* Load builtin plugins lazily

* update changelog

* Update transforms handling in Dataset (#297)

* Update builtin transforms

* Optimize dataset length computation when no source

* Add filter test

* Fix transforms affecting categories

* Optimize categories transforms

* Update filters

* fix imports

* Avoid using default docstrings in plugins

* Fix patch saving in VOC, add keep_empty export parameter

* Fix flush_changes

* Fix removed images and subsets in dataset patch

* Update changelog

* Update voc doc

* Skip item transform base class in plugins

* Readable COCO and datumaro format for CJK (#307)

* Do not force ASCII in COCO and Datumaro JSONs for readable CJK

* Add tests

* Use utf-8 encoding for writing

Co-authored-by: Maxim Zhiltsov <[email protected]>

* Force utf-8 everywhere (#309)

* Fix in ImageNet_txt (#302)

* Add extensions for images to annotation file

* Remove image search in extractor

* Update changelog

Co-authored-by: Maxim Zhiltsov <[email protected]>

* Reduce duplication of dependency information (#308)

* Move requirements from setup.py to requirements-base.txt

* Add whitespace error checking to GitHub Actions (#311)

* Fix whitespace errors

As detected with `git diff --check`.

* Add a job to check for whitespace errors

I called it "lint" so that other checks could be added to it later.

* Bump copyright years in changed files

* Add initial support for the Open Images dataset (#291)

* Support reading or Labels in Open Images (v4, v5, v6)

* Add tests for the Open Images extractor/importer

* Add Open Images documentation

* Update changelog

* Fix tensorboardX dependency (#318)

* Fixing remark-lint issues. Adding remark-linter check. (#321)

* Fix remark-lint issues.

* Align continuation lines with the first line.

Apply comments

* Added remark check

* Add an upper bound on the Pillow dependency to work around a regression in 8.3 (#323)

* open_images_user_manual.md: fix image description file URLs

I accidentally swapped the URLs for test and validation sets.

* Fix COCO Panoptic (#319)

* add test

* Fix integer overflow in bgr2index

* Fix pylint issues. Added pylint checking. (#322)

* Added pylint job for CI

* Rework pip install

* Fixed remaining pylint warnings

Co-authored-by: Andrey Zhavoronkov <[email protected]>

* Open Images: add writing support (#315)

* open_images_user_manual.md: fix image description file URLs

* open_images_format: add conversion support

* open_images_format: add support for images in subdirectories

* open_images_format: add tests for writing support

* open_images_format: add documentation for the writing support

* Update the changelog entry for the Open Images support

* Add python bandit checks. (#316)

* Add bandit dependency

* Add bandit checks on CI

* Disable some warnings

Co-authored-by: Andrey Zhavoronkov <[email protected]>
Co-authored-by: Maxim Zhiltsov <[email protected]>

* Remove Pylint unused-import warning suppressions (#326)

* Remove Pylint unused-import warning suppressions

* Add a job to check import formatting using isort (#333)

* Reformat all imports using isort

* Implement a workflow for checking import formatting based on isort

* Reformat the enabled checker list in .pylintrc (#335)

Put each code on its own line and add a comment with its symbolic name.
That makes the list more understandable and easier to edit.

* Merge all linting jobs into one workflow file (#331)

Doing it this way means that on GitHub's Checks page, all jobs are displayed
under one "Linter" category, instead of multiple indistinguishable "Linter"
categories with one job each.

Move the whitespace checking job into the Linter workflow as well, since
that's where it logically belongs.

I also took the opportunity to slightly rename the jobs in order to spell
the linter names correctly.

* Fix cuboids / 3d / M6 (#320)

* CVAT-3D Milestone-6: Added Supervisely Point Cloud and KITTI Raw 3D formats

* Added Cuboid3d annotations

* Added docs for new formats

Co-authored-by: cdp <cdp123>
Co-authored-by: Jayraj <[email protected]>
Co-authored-by: Roman Donchenko <[email protected]>

* Clean up .pylintrc (#340)

* Clean up the list of messages in .pylintrc

* Remove obsolete Pylint options

* .pylintrc: move the disable setting and its documentation together

* Remove the commented-out setting.

* Revert "Add an upper bound on the Pillow dependency to work around a regression in 8.3 (#323)" (#341)

The regression was fixed in 8.3.1.

This reverts commit 9a85616.

* Enable pylint checkers that find invalid escape sequences (#344)

Fix the issues that they found.

* Factor out the images.meta loading code from YoloExtractor (#343)

* Factor out the images.meta loading code from YoloExtractor

It looks like the same thing will be needed for Open Images, so I'm
moving it to a common module.

* Rework image.meta parsing code to use shell syntax

This allows comments and improves extensibility.

* Support for CIFAR-100 (#301)

* Add support for CIFAR-100

* Update Changelog

* Update user_manual.md

* Add notes about differences in formats

* Fix importing for VGG Face 2 (#345)

* correct asset according the original vgg_face2 dataset

* fix importing of the original dataset

Co-authored-by: Maxim Zhiltsov <[email protected]>

* Dataset caching fixes (#351)

* Fix importing arbitrary file names in COCO subformats

* Optimize subset iteration in a simple scenario

* Fix subset iteration in dataset with transforms

* Cuboid 3D for Datumaro format (#349)

* Support cuboid_3d and point cloud in datumaro format

* Add cuboid_3d and point cloud tests in datumaro format

* Add image size type conversions

Co-authored-by: Maxim Zhiltsov <[email protected]>

* Add e2e tests for cuboids (#353)

* Add attr name check in kitti raw

* Add sly pcd e2e test

* Rename "object" attribute to "track_id" in sly point cloud

* Add kitti raw e2e test

* Update kitti raw example

* update changelog

* Release 0.1.10 (dev) (#354)

* Update changelog

* Add cifar security notice

* Update version

Co-authored-by: Emily Chun <[email protected]>
Co-authored-by: Jihyeon Yi <[email protected]>
Co-authored-by: Kirill Sizov <[email protected]>
Co-authored-by: Anastasia Yasakova <[email protected]>
Co-authored-by: Harim Kang <[email protected]>
Co-authored-by: Zoya Maslova <[email protected]>
Co-authored-by: Roman Donchenko <[email protected]>
Co-authored-by: Seungyoon Woo <[email protected]>
Co-authored-by: Dmitry Kruchinin <[email protected]>
Co-authored-by: Slawomir Strehlke <[email protected]>
Co-authored-by: Jaesun Park <[email protected]>
Co-authored-by: Andrey Zhavoronkov <[email protected]>
Co-authored-by: Jayraj <[email protected]>
wonjuleee added a commit that referenced this pull request Apr 5, 2023
@wonjuleee
[Doc] Reorganize content structure (#908)
1fa4e01 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary


![image](https://user-images.githubusercontent.com/89109581/229419480-c4835066-9253-4a48-8a98-6630e951c34b.png)

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
wonjuleee added a commit that referenced this pull request Apr 6, 2023
@wonjuleee
[Doc] reformat contents and add level up skills (#916)
16acf4e 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
wonjuleee added a commit to wonjuleee/datumaro that referenced this pull request Apr 6, 2023
@wonjuleee
[Doc] Reorganize content structure (open-edge-platform#908)
b85d486 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary


![image](https://user-images.githubusercontent.com/89109581/229419480-c4835066-9253-4a48-8a98-6630e951c34b.png)

<!--
Resolves open-edge-platform#111 and open-edge-platform#222.
Depends on open-edge-platform#1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem open-edge-platform#1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
wonjuleee added a commit that referenced this pull request Apr 6, 2023
@wonjuleee @chuneuny-emily @yunchu
[Doc] sphinx documentation with content reformatting (#917)
89b2111 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT
```

---------

Co-authored-by: Chun, Emily <[email protected]>
Co-authored-by: Yunchu Lee <[email protected]>
sooahleex added a commit that referenced this pull request Apr 7, 2023
@sooahleex
Fix log issue when importing celeba and align celeba dataset (#919)
ddde11d 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->
When import `celeba` or `align_celeba` dataset without format like
`dm.Dataset.import_from(path=...)`, we met error like below. The example
is the case for importing `align_celeba` dataset.
```
ERROR:root:Cannot find celeba's images directory at tests/assets/align_celeba_dataset/dataset/Img/img_celeba
```

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [X] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
yunchu added a commit that referenced this pull request Apr 10, 2023
@yunchu
Revised bandit exclusion comments (#904)
fd5aa10 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary
Revised some comments for excluding to the report of bandit.

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
yunchu added a commit that referenced this pull request Apr 10, 2023
@yunchu
Updated code_scan workflow to use tox (#924)
c425353 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
yunchu added a commit that referenced this pull request Apr 10, 2023
@yunchu
Fixed code-scan workflow for the bandit (#925)
606a0c8 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary
Fixed code-scan workflow for the bandit
<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
wonjuleee added a commit that referenced this pull request Apr 11, 2023
@wonjuleee
[Doc] add tutorials for level 3 and 4 (#920)
4672276 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

Please take a look at
http://10.225.20.174:7041/build/html/docs/level-up/basic_skills/03_dataset_import_export.html
and
http://10.225.20.174:7041/build/html/docs/level-up/basic_skills/04_detect_data_format.html#level-4-detect-data-format-from-an-unknown-dataset
for references.

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
yunchu added a commit that referenced this pull request Apr 11, 2023
@yunchu
Added toxenv for snyk scanning (#926)
66c9422 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

This one would be used to perform snyk code scanning from the internal
development environment only because the API endpoint cannot be accessed
from the external.

Once we setup the self-hosted CI instance, this one will be used by
code-scanning workflow to automate snyk scanning & reporting. before
that, this will be used to generate snyk report manually.

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
wonjuleee pushed a commit that referenced this pull request Apr 14, 2023
@cih9088
Fix arrow format patch error on python 3.7 (#939)
e6d73a0 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary
Disentangle temporary directory management

- Fixed to remove and move manually each files instead of whole
directory itself.

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT
```

Signed-off-by: Inhyuk Andy Cho <[email protected]>
wonjuleee added a commit that referenced this pull request Apr 17, 2023
@wonjuleee @vinnamkim
Fix validator and add notebooks and document for level-up validator (#…
26fba49 
…933)

<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT
```

---------

Signed-off-by: Kim, Vinnam <[email protected]>
Co-authored-by: Vinnam Kim <[email protected]>
yunchu added a commit that referenced this pull request Apr 17, 2023
@yunchu
Updates docs for release (#936)
f640c2c 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

* Added docs workflow for stable release
* Updated some docstrings
* Fixed some docs build errors

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
bonhunko added a commit that referenced this pull request Apr 19, 2023
@bonhunko
Add basic skills - Level 10: Data Generation (#953)
6529b60 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary
Ticket no. 107282
Add basic skills - Level 10: Data Generation

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [x] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
sooahleex added a commit that referenced this pull request Apr 20, 2023
@sooahleex
Add Intermediate skills - Level 9: Dataset Explorartion from a Query …
efd8a40 
…Image/Text (#959)

<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->
- Ticket no. 108906
- Add intermediate skills - Level 9: Data exploration
- Rename searcher to explorer

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [X] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
JihwanEom added a commit that referenced this pull request Apr 21, 2023
@JihwanEom
Fix documentation links (#972)
c5911ea 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary
- Fix broken documentation links

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
bonhunko added a commit that referenced this pull request Apr 24, 2023
@bonhunko
fix typo - level 10 data generation (#975)
8c30f5c 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

fix a typo in the level-up 10 data generation document

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
sooahleex added a commit to sooahleex/datumaro that referenced this pull request Sep 25, 2024
@itrushkin @williamcorsel
@sooahleex @dependabot @yunchu @wonjuleee
Fix merging of stream datasets (open-edge-platform#1609)
757b26d 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

When importing a stream dataset with multiple sources in eager mode by
specifying `error_policy` or `progress_reporting`, an error occurs:
```
'_MergedStreamDataset' object has no attribute '_data'
```

<!--
Resolves open-edge-platform#111 and open-edge-platform#222.
Depends on open-edge-platform#1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem open-edge-platform#1234
-->

<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

<!-- Put an 'x' in all the boxes that apply -->
- [x] I have added unit tests to cover my changes.​
- [x] I have added integration tests to cover my changes.​
- [x] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [x] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

- [x] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [x] I have updated the license header for each file (see an example
below).

```python
```

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Ilya Trushkin <[email protected]>
Co-authored-by: williamcorsel <[email protected]>
Co-authored-by: Sooah Lee <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yunchu Lee <[email protected]>
Co-authored-by: Wonju Lee <[email protected]>
jihyeonyi pushed a commit that referenced this pull request Sep 25, 2024
Handle path separators in the subset when exporting a datumaro dataset (
dc9bbf2 
#1615)

<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary
Ticket: 152928
Raise an error when exporting a dataset in Datumaro format if the subset
name contains path separators.

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [x] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [x] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [x] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [x] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [x] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
wonjuleee added a commit that referenced this pull request Sep 25, 2024
@sooahleex @wonjuleee
Support KITTI 3D format (#1619)
c4d7bb4 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->
- Ticket no.151424
- Support KITTI 3D format

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->
- Add unit test

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [X] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [X] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```

---------

Co-authored-by: Wonju Lee <[email protected]>
chuneuny-emily pushed a commit that referenced this pull request Sep 27, 2024
@yunchu @sooahleex
@itrushkin @williamcorsel @dependabot @wonjuleee
Mergeback 1.9.1 to develop (#1623)
fdd1ac2 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Ilya Trushkin <[email protected]>
Co-authored-by: Sooah Lee <[email protected]>
Co-authored-by: Ilya Trushkin <[email protected]>
Co-authored-by: williamcorsel <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Wonju Lee <[email protected]>
wonjuleee pushed a commit that referenced this pull request Sep 27, 2024
@sooahleex
Support subset for KITTI 3D format (#1621)
fcdb37d 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->
- Support subset structure for KITTI 3d format
- Update media type from PointCloud to Image

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->
- Add unit test

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [X] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [X] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
wonjuleee pushed a commit that referenced this pull request Sep 27, 2024
@yunchu
Update for release 1.9.1 (#1622)
4619baf 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
jihyeonyi pushed a commit that referenced this pull request Oct 11, 2024
Enhance 'id_from_image_name' transform to ensure each identifier is u…
f716078 
…nique (#1635)

<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary
Ticket: 153389
1. Enhance 'id_from_image_name' transform to ensure each identifier is
unique.
- add random suffix if the image name is not distinct:
[image_name]__[suffix]
- introduce related parameters: ensure_unique(default: false),
suffix_length (default: 3)
2. Handle VideoFrame considering its index
   - format: [video_name]_frame-[index]

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [x] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [x] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [x] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
jihyeonyi pushed a commit that referenced this pull request Oct 15, 2024
Fix datumaro keypoint loading (#1644)
acad7ec 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

This fixes the bug in loading a keypoint dataset in datumaro format. 
The visibility information will be filled correctly by this fix.

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [x] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [x] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [x] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
wonjuleee pushed a commit that referenced this pull request Oct 15, 2024
No security check for id_from_image_name (#1642)
054bd95 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary
Ignore bandit error B311 in 'id_from_image_name' transform.

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [x] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
wonjuleee pushed a commit that referenced this pull request Oct 16, 2024
@itrushkin
Convert Cuboid2D to/from KITTI 3D data (#1639)
3d533b9 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary
CVS-151427

#### New features
- New `Cuboid2D` methods:
- `Cuboid2D.from_3d(dimensions, location, rotation_y, P,
Tr_velo_to_cam)`: Creates a Cuboid2D object from KITTI 3D bbox
annotation data. Matrix `P` (`P2` in Kitti format context) is a 3x4
projection matrix in the left color camera coordinate system. Matrix
`Tr_velo_to_cam` is a 3x4 projection matrix between LiDAR and camera
coordinate systems.
- `cuboid_2d.to_3d(P_inv)`: Reconstructs approximate KITTI 3D bbox
annotation data (`dimensions`, `location` and `rotation_y`) from 2D
projection coordinates. `P_inv` matrix is a
[pseudo-inverse](https://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_inverse)
of camera-to-image projection matrix.
<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
See unit test changes

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [x] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [x] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [x] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```

---------

Signed-off-by: Ilya Trushkin <[email protected]>
wonjuleee pushed a commit that referenced this pull request Oct 16, 2024
@sooahleex
Update assets for explorer (#1647)
62bb4f4 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->
- Fixed the problem in
[action](https://github.com/openvinotoolkit/datumaro/actions/runs/11356378945/job/31587449268)
- This issue occurs occasionally

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
wonjuleee pushed a commit that referenced this pull request Oct 23, 2024
@itrushkin
Add label groups for hierarchical classification in ImageNet (#1645)
e79cca2 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary
This PR adds grouping of labels (directories for the ImageNet case) by
their groups (parent directories).
For example, for the following folder structure
```
.
├── label_1
│   └── label_1_1
│       └── 1.jpg
└── label_2
    └── label_2_1
        └── 2.jpg
```
label groups will be `label_1` and `label_2`.


**Note**: for the higher depth of nesting, names of groups will be
relative paths of second-to-last directories. For the following case:
```
.
├── label_1
│   └── label_1_1
│       └── label_1_1_1
│           └── 1.jpg
│        
└── label_2
    └── label_2_1
        └── label_2_1_1
            └── 2.jpg
```
label groups will be `label_1/label_1_1` and `label_2/label_2_1`.
<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [x] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [x] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [x] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```

---------

Signed-off-by: Ilya Trushkin <[email protected]>
jihyeonyi pushed a commit that referenced this pull request Oct 24, 2024
Make bandit to ignore B311 for id_from_image_name transform (#1657)
3265766 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary
Make bandit to ignore B311 for id_from_image_name transform
--> Re-located the comment, # nosec B311)
The result of bandit-scan is pasted:
```
Run started:2024-10-24 06:29:43.187490

Test results:
        No issues identified.

Code scanned:
        Total lines of code: 51857
        Total lines skipped (#nosec): 6
        Total potential issues skipped due to specifically being disabled (e.g., #nosec BXXX): 12

Run metrics:
        Total issues (by severity):
                Undefined: 0
                Low: 0
                Medium: 0
                High: 0
        Total issues (by confidence):
                Undefined: 0
                Low: 0
                Medium: 0
                High: 0
Files skipped (0):
```
<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
@coderabbitai coderabbitai bot mentioned this pull request Dec 24, 2024
Merged
7 tasks
djdameln added a commit that referenced this pull request Jan 28, 2025
@djdameln
Add Python 3.12 to the CI and update Ubuntu (#1701)
3d0aa04 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

This PR adds support for Python 3.12 in the project configuration and
CI. I have left the min Python version unchanged (3.9).
<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
djdameln added a commit that referenced this pull request Jan 28, 2025
@djdameln
Remove Bandit tests B320 and B410. (#1706)
41fbe5a 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

This PR disables Bandit scans which have been removed in the latest
version of Bandit.

See for background information:
* PyCQA/bandit#767
* PyCQA/bandit#1212

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
sovrasov added a commit that referenced this pull request Feb 11, 2025
@sovrasov
Del old maintainers group from code owners (#1713)
981717b 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
djdameln added a commit that referenced this pull request Mar 3, 2025
@djdameln
Disable weekly checks for python 3.12 (#1720)
57bc473 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->
### Summary

This PR disables the weekly stability checks for Python 3.12. For the
CLI fuzzing tests, the weekly checks depend on the Atheris package,
which is not compatible with Python 3.12 at this moment.

As a temporary fix, I am disabling the weekly checks for Python 3.12.
This is safe, because the fuzzing tests still run for Python <= 3.11.
For a long term solution, we need to wait until Python 3.12 support is
added to Atheris, or switch to a different package for fuzzing tests.

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [x] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
djdameln added a commit that referenced this pull request Mar 4, 2025
@djdameln
fix absl-py version in tfds extras (#1723)
5f8318b 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary
Quick PR to fix the failing weekly checks for tfds in python 3.10 (see
example
[here](https://github.com/openvinotoolkit/datumaro/actions/runs/13618405842/job/38064455696)).

The root cause was [this
issue](abseil/abseil-py#161) which causes
installation of the `absl-py` package to fail in python 3.10. The issue
was fixed in `absl-py==0.12`, so we need to ensure that we use at least
that version. `absl-py` is a dependency of `tfds`, which we only use in
the `tfds` extras, so the fix is to enforce the absl-py version for
`tfds`.



<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
djdameln added a commit that referenced this pull request Mar 5, 2025
@djdameln
Set 1.10 release date to Q1 2025 (#1724)
3a91993 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

Update the timestamp for the 1.10 release in the changelog. Q1 2025
instead of Q4 2024

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
sovrasov added a commit that referenced this pull request Mar 20, 2025
@sovrasov
Switch to trusted publishing (#1730)
ee01e56 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary
These changes allow to get rid of access tokens and related efforts to
maintain them.

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
djdameln added a commit that referenced this pull request Mar 27, 2025
@djdameln
Merge 1.10.0 release branch (#1741)
898e697 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

PR to merge the 1.10 release branch back into develop.

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2025 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
sovrasov added a commit that referenced this pull request Apr 1, 2025
@sovrasov
Update references to old organization (#1740)
1b1f0b4 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

Following the moving of the repo to the `open-edge-platform`
organization, this PR updates the references to the old organization
throughout the code base.

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [x] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2025 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
djdameln added a commit that referenced this pull request Apr 9, 2025
@djdameln
Remove deprecated dataset formats (#1745)
4298ca0 
<!-- Contributing guide:
https://github.com/openvinotoolkit/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

This PR removes two deprecated dataset formats: `Common Semantic
Segmentation` and `Common Superresolution`. Both these dataset formats
are used by OpenVINO accuracy checker, which itself is deprecated and no
longer exists in recent versions.

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [x] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [x] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2025 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
sovrasov added a commit that referenced this pull request Apr 14, 2025
@sovrasov
Add issue template (#1751)
a0d7cc7 
<!-- Contributing guide:
https://github.com/open-edge-platform/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

A requirement from GH org

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/open-edge-platform/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/open-edge-platform/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/open-edge-platform/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2025 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
AlexanderBarabanov added a commit that referenced this pull request May 20, 2025
@AlexanderBarabanov
Minor updates to workflow files (#1763)
97599ea 
<!-- Contributing guide:
https://github.com/open-edge-platform/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary
This PR introduces minor updates in workflow files:
- migration from
[unmaintained](actions-rs/toolchain#216)
`actions-rs/toolchain` to `dtolnay/rust-toolchain`
- migration from `pre-commit/action` ([this action is in
maintenance-only mode](https://github.com/pre-commit/action)) to bash
- pinning 3rd-party actions
- Zizmor upgrade, move from ignore yaml to in-line comments for
visibility purpose


<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/open-edge-platform/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/open-edge-platform/datumaro/tree/develop/docs)
accordingly

### License

- [x] I submit _my code changes_ under the same [MIT
License](https://github.com/open-edge-platform/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2025 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
sovrasov added a commit that referenced this pull request Jun 26, 2025
@sovrasov
Mark pyemd as optional since it does not support Python 3.12 (#1770)
6553a7d 
pyemd is only used for data shift analysis and does not seem to be
maintained anymore.
Most of the code does not use pyemd so it is better to mark it as
optional.

<!-- Contributing guide:
https://github.com/open-edge-platform/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/open-edge-platform/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/open-edge-platform/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/open-edge-platform/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2025 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
AlbertvanHouten added a commit that referenced this pull request Jul 4, 2025
@AlbertvanHouten
Add deprecation notice to features that will soon be removed (#1776)
2a61efd 
### Summary
This pull request introduces a new `@deprecated` decorator to mark
classes as deprecated and applies it across multiple files to notify
users of upcoming removals. Additionally, it includes a unit test to
verify the functionality of the decorator.

Resolves #1775

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [x] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [x] I have added the description of my changes into
[CHANGELOG](https://github.com/open-edge-platform/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/open-edge-platform/datumaro/tree/develop/docs)
accordingly

### License

- [x] I submit _my code changes_ under the same [MIT
License](https://github.com/open-edge-platform/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [x] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2025 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
gdlg added a commit that referenced this pull request Jul 9, 2025
@gdlg
Deprecate NDR, DSA, and some filters. Remove deprecation from tiling. (…
0f10fc3 
…#1780)

<!-- Contributing guide:
https://github.com/open-edge-platform/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary

This PR deprecates NDR, DSA (data shift analysis), model transform,
xpath filters. It also remove the deprecation of tiling as it is used by
OTX.

<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
<!-- Describe the testing procedure for reviewers, if changes are
not fully covered by unit tests or manual testing can be complicated.
-->

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/open-edge-platform/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/open-edge-platform/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/open-edge-platform/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2025 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
AlexanderBarabanov added a commit that referenced this pull request Jul 10, 2025
@AlexanderBarabanov
Security pipeline update (#1781)
301b93c 
<!-- Contributing guide:
https://github.com/open-edge-platform/datumaro/blob/develop/CONTRIBUTING.md
-->

### Summary
This PR updates security pipeline, in particular:
- security checks is implemented based on reusable actions from Anomalib
repo (later we will move it to more appropriate repo like
`reusable-workflows`) - this will simplify maintenance and support
across Geti repos.
- bandit configuration moved to `pyproject.toml`
- `.pre-commit-config.yaml` - added zizmor and bandit pre-commit checks
- removed unnecessary files

Tested on local fork.
<!--
Resolves #111 and #222.
Depends on #1000 (for series of dependent commits).

This PR introduces this capability to make the project better in this
and that.

- Added this feature
- Removed that feature
- Fixed the problem #1234
-->

### How to test
Tested in fork https://github.com/AlexanderBarabanov/datumaro/actions

### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [ ] I have added the description of my changes into
[CHANGELOG](https://github.com/open-edge-platform/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/open-edge-platform/datumaro/tree/develop/docs)
accordingly

### License

- [ ] I submit _my code changes_ under the same [MIT
License](https://github.com/open-edge-platform/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [ ] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2025 Intel Corporation
#
# SPDX-License-Identifier: MIT
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhiltsov-max zhiltsov-max zhiltsov-max approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Classification task in LFW dataset format Support arbitrary file names in LFW format
2 participants
@yasakova-anastasia @zhiltsov-max