Skip to content

Commit 3faf3ed

Browse files
zhiltsov-maxFernando Martínez González
zhiltsov-max
authored and
Fernando Martínez González
committed
Merge annotations and dataset_manager apps (cvat-ai#1352)
1 parent 9e2be4a commit 3faf3ed

File tree

91 files changed

+3542
-3994
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

91 files changed

+3542
-3994
lines changed

‎.travis.yml

+3-2
Original file line numberDiff line numberDiff line change
@@ -21,11 +21,12 @@ before_script:
2121
- mkdir -m a=rwx -p ${HOST_COVERAGE_DATA_DIR}
2222

2323
script:
24-
- docker-compose -f docker-compose.yml -f docker-compose.ci.yml run cvat_ci /bin/bash -c 'coverage run -a manage.py test cvat/apps utils/cli && coverage run -a --source cvat/apps/ manage.py test --pattern="_tests.py" cvat/apps/dataset_manager && mv .coverage ${CONTAINER_COVERAGE_DATA_DIR}'
24+
# FIXME: Git package and application name conflict in PATH
25+
- docker-compose -f docker-compose.yml -f docker-compose.ci.yml run cvat_ci /bin/bash -c 'coverage run -a manage.py test cvat/apps && coverage run -a manage.py test --pattern="_test*.py" cvat/apps/dataset_manager/tests cvat/apps/engine/tests utils/cli && coverage run -a manage.py test datumaro/ && mv .coverage ${CONTAINER_COVERAGE_DATA_DIR}'
2526
- docker-compose -f docker-compose.yml -f docker-compose.ci.yml run cvat_ci /bin/bash -c 'cd cvat-data && npm install && cd ../cvat-core && npm install && npm run test && mv ./reports/coverage/lcov.info ${CONTAINER_COVERAGE_DATA_DIR}'
2627

2728
after_success:
2829
# https://coveralls-python.readthedocs.io/en/latest/usage/multilang.html
2930
- mv ${HOST_COVERAGE_DATA_DIR}/* .
3031
- coveralls-lcov -v -n lcov.info > coverage.json
31-
- coveralls --merge=coverage.json
32+
- coveralls --merge=coverage.json

‎.vscode/settings.json

+2-1
Original file line numberDiff line numberDiff line change
@@ -33,5 +33,6 @@
3333
"./datumaro",
3434
],
3535
"licenser.license": "Custom",
36-
"licenser.customHeader": "Copyright (C) @YEAR@ Intel Corporation\n\nSPDX-License-Identifier: MIT"
36+
"licenser.customHeader": "Copyright (C) @YEAR@ Intel Corporation\n\nSPDX-License-Identifier: MIT",
37+
"files.trimTrailingWhitespace": true
3738
}

‎CHANGELOG.md

+17-3
Original file line numberDiff line numberDiff line change
@@ -6,18 +6,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
66

77
## [1.0.0] - Unreleased
88
### Added
9-
-
9+
- Added `datumaro_project` export format (https://github.com/opencv/cvat/pull/1352)
1010

1111
### Changed
12-
- cvat-core: session.annotations.put() now returns identificators of added objects (<https://github.com/opencv/cvat/pull/1493>)
12+
- Downloaded file name in annotations export became more informative (https://github.com/opencv/cvat/pull/1352)
13+
- Added auto trimming for trailing whitespaces style enforsement (https://github.com/opencv/cvat/pull/1352)
14+
- REST API: updated `GET /task/<id>/annotations`: parameters are `format`, `filename` (now optional), `action` (optional) (https://github.com/opencv/cvat/pull/1352)
15+
- REST API: removed `dataset/formats`, changed format of `annotation/formats` (https://github.com/opencv/cvat/pull/1352)
16+
- Exported annotations are stored for N hours instead of indefinitely (https://github.com/opencv/cvat/pull/1352)
17+
- Formats: CVAT format now accepts ZIP and XML (https://github.com/opencv/cvat/pull/1352)
18+
- Formats: COCO format now accepts ZIP and JSON (https://github.com/opencv/cvat/pull/1352)
19+
- Formats: most of formats renamed, no extension in title (https://github.com/opencv/cvat/pull/1352)
20+
- Formats: definitions are changed, are not stored in DB anymore (https://github.com/opencv/cvat/pull/1352)
21+
- cvat-core: session.annotations.put() now returns identificators of added objects (https://github.com/opencv/cvat/pull/1493)
1322

1423
### Deprecated
1524
-
1625

1726
### Removed
18-
-
27+
- `annotation` application is replaced with `dataset_manager` (https://github.com/opencv/cvat/pull/1352)
1928

2029
### Fixed
30+
- Categories for empty projects with no sources are taken from own dataset (https://github.com/opencv/cvat/pull/1352)
31+
- Added directory removal on error during `extract` command (https://github.com/opencv/cvat/pull/1352)
32+
- Added debug error message on incorrect XPath (https://github.com/opencv/cvat/pull/1352)
33+
- Exporting frame stepped task (https://github.com/opencv/cvat/issues/1294, https://github.com/opencv/cvat/issues/1334)
34+
- Fixed broken command line interface for `cvat` export format in Datumaro (https://github.com/opencv/cvat/issues/1494)
2135
- Updated Rest API document, Swagger document serving instruction issue (https://github.com/opencv/cvat/issues/1495)
2236

2337
### Security

‎CONTRIBUTING.md

+1
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,7 @@ for development
7575
- [ESLint](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint)
7676
- [vscode-remark-lint](https://marketplace.visualstudio.com/items?itemName=drewbourne.vscode-remark-lint)
7777
- [licenser](https://marketplace.visualstudio.com/items?itemName=ymotongpoo.licenser)
78+
- [Trailing Spaces](https://marketplace.visualstudio.com/items?itemName=shardulm94.trailing-spaces)
7879

7980
- Reload Visual Studio Code from virtual environment
8081

‎README.md

+23-13
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,10 @@
77
[![codebeat badge](https://codebeat.co/badges/53cd0d16-fddc-46f8-903c-f43ed9abb6dd)](https://codebeat.co/projects/g.yxqyang.asia-opencv-cvat-develop)
88
[![DOI](https://zenodo.org/badge/139156354.svg)](https://zenodo.org/badge/latestdoi/139156354)
99

10-
CVAT is free, online, interactive video and image annotation tool for computer vision. It is being used by our team to annotate million of objects with different properties. Many UI and UX decisions are based on feedbacks from professional data annotation team.
10+
CVAT is free, online, interactive video and image annotation
11+
tool for computer vision. It is being used by our team to
12+
annotate million of objects with different properties. Many UI
13+
and UX decisions are based on feedbacks from professional data annotation team.
1114

1215
![CVAT screenshot](cvat/apps/documentation/static/documentation/images/cvat.jpg)
1316

@@ -34,21 +37,23 @@ CVAT is free, online, interactive video and image annotation tool for computer v
3437

3538
## Supported annotation formats
3639

37-
Format selection is possible after clicking on the Upload annotation / Dump annotation button.
38-
[Datumaro](datumaro/README.md) dataset framework allows additional dataset transformations
39-
via its command line tool.
40+
Format selection is possible after clicking on the Upload annotation
41+
and Dump annotation buttons. [Datumaro](datumaro/README.md) dataset
42+
framework allows additional dataset transformations
43+
via its command line tool and Python library.
4044

41-
| Annotation format | Dumper | Loader |
45+
| Annotation format | Import | Export |
4246
| ------------------------------------------------------------------------------------------ | ------ | ------ |
43-
| [CVAT XML v1.1 for images](cvat/apps/documentation/xml_format.md#annotation) | X | X |
44-
| [CVAT XML v1.1 for a video](cvat/apps/documentation/xml_format.md#interpolation) | X | X |
45-
| [Pascal VOC](http://host.robots.ox.ac.uk/pascal/VOC/) | X | X |
47+
| [CVAT for images](cvat/apps/documentation/xml_format.md#annotation) | X | X |
48+
| [CVAT for a video](cvat/apps/documentation/xml_format.md#interpolation) | X | X |
49+
| [Datumaro](datumaro/README.md) | | X |
50+
| [PASCAL VOC](http://host.robots.ox.ac.uk/pascal/VOC/) | X | X |
51+
| Segmentation masks from [PASCAL VOC](http://host.robots.ox.ac.uk/pascal/VOC/) | X | X |
4652
| [YOLO](https://pjreddie.com/darknet/yolo/) | X | X |
4753
| [MS COCO Object Detection](http://cocodataset.org/#format-data) | X | X |
48-
| PNG class mask + instance mask as in [Pascal VOC](http://host.robots.ox.ac.uk/pascal/VOC/) | X | X |
4954
| [TFrecord](https://www.tensorflow.org/tutorials/load_data/tf_records) | X | X |
5055
| [MOT](https://motchallenge.net/) | X | X |
51-
| [LabelMe](http://labelme.csail.mit.edu/Release3.0) | X | X |
56+
| [LabelMe 3.0](http://labelme.csail.mit.edu/Release3.0) | X | X |
5257

5358
## Links
5459
- [Intel AI blog: New Computer Vision Tool Accelerates Annotation of Digital Images and Video](https://www.intel.ai/introducing-cvat)
@@ -57,14 +62,19 @@ via its command line tool.
5762

5863
## Online Demo
5964

60-
[Onepanel](https://www.onepanel.io/) has added CVAT as an environment into their platform and a running demo of CVAT can be accessed at [CVAT Public Demo](https://c.onepanel.io/onepanel-demo/projects/cvat-public-demo/workspaces?utm_source=cvat).
65+
[Onepanel](https://www.onepanel.io/) has added CVAT as an environment
66+
into their platform and a running demo of CVAT can be accessed at
67+
[CVAT Public Demo](https://c.onepanel.io/onepanel-demo/projects/cvat-public-demo/workspaces?utm_source=cvat).
6168

62-
If you have any questions, please contact Onepanel directly at [email protected]. If you are in the Onepanel application, you can also use the chat icon in the bottom right corner.
69+
If you have any questions, please contact Onepanel directly at
70+
[email protected]. If you are in the Onepanel application, you can also
71+
use the chat icon in the bottom right corner.
6372

6473
## REST API
6574

6675
Automatically generated Swagger documentation for Django REST API is
67-
available on ``<cvat_origin>/api/swagger`` (default: ``localhost:8080/api/swagger``).
76+
available on ``<cvat_origin>/api/swagger``
77+
(default: ``localhost:8080/api/swagger``).
6878

6979
Swagger documentation is visiable on allowed hostes, Update environement variable in docker-compose.yml file with cvat hosted machine IP or domain name. Example - ``ALLOWED_HOSTS: 'localhost, 127.0.0.1'``)
7080

‎cvat-core/package.json

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "cvat-core",
3-
"version": "1.0.0",
3+
"version": "2.0.0",
44
"description": "Part of Computer Vision Tool which presents an interface for client-side integration",
55
"main": "babel.config.js",
66
"scripts": {

‎cvat-core/src/annotation-format.js renamed to ‎cvat-core/src/annotation-formats.js

+14-103
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,8 @@
1212
class Loader {
1313
constructor(initialData) {
1414
const data = {
15-
display_name: initialData.display_name,
16-
format: initialData.format,
17-
handler: initialData.handler,
15+
name: initialData.name,
16+
format: initialData.ext,
1817
version: initialData.version,
1918
};
2019

@@ -27,7 +26,7 @@
2726
* @readonly
2827
* @instance
2928
*/
30-
get: () => data.display_name,
29+
get: () => data.name,
3130
},
3231
format: {
3332
/**
@@ -39,16 +38,6 @@
3938
*/
4039
get: () => data.format,
4140
},
42-
handler: {
43-
/**
44-
* @name handler
45-
* @type {string}
46-
* @memberof module:API.cvat.classes.Loader
47-
* @readonly
48-
* @instance
49-
*/
50-
get: () => data.handler,
51-
},
5241
version: {
5342
/**
5443
* @name version
@@ -71,9 +60,8 @@
7160
class Dumper {
7261
constructor(initialData) {
7362
const data = {
74-
display_name: initialData.display_name,
75-
format: initialData.format,
76-
handler: initialData.handler,
63+
name: initialData.name,
64+
format: initialData.ext,
7765
version: initialData.version,
7866
};
7967

@@ -86,7 +74,7 @@
8674
* @readonly
8775
* @instance
8876
*/
89-
get: () => data.display_name,
77+
get: () => data.name,
9078
},
9179
format: {
9280
/**
@@ -98,16 +86,6 @@
9886
*/
9987
get: () => data.format,
10088
},
101-
handler: {
102-
/**
103-
* @name handler
104-
* @type {string}
105-
* @memberof module:API.cvat.classes.Dumper
106-
* @readonly
107-
* @instance
108-
*/
109-
get: () => data.handler,
110-
},
11189
version: {
11290
/**
11391
* @name version
@@ -127,108 +105,41 @@
127105
* @memberof module:API.cvat.classes
128106
* @hideconstructor
129107
*/
130-
class AnnotationFormat {
108+
class AnnotationFormats {
131109
constructor(initialData) {
132110
const data = {
133-
created_date: initialData.created_date,
134-
updated_date: initialData.updated_date,
135-
id: initialData.id,
136-
owner: initialData.owner,
137-
name: initialData.name,
138-
handler_file: initialData.handler_file,
111+
exporters: initialData.exporters.map((el) => new Dumper(el)),
112+
importers: initialData.importers.map((el) => new Loader(el)),
139113
};
140114

141-
data.dumpers = initialData.dumpers.map((el) => new Dumper(el));
142-
data.loaders = initialData.loaders.map((el) => new Loader(el));
143-
144115
// Now all fields are readonly
145116
Object.defineProperties(this, {
146-
id: {
147-
/**
148-
* @name id
149-
* @type {integer}
150-
* @memberof module:API.cvat.classes.AnnotationFormat
151-
* @readonly
152-
* @instance
153-
*/
154-
get: () => data.id,
155-
},
156-
owner: {
157-
/**
158-
* @name owner
159-
* @type {integer}
160-
* @memberof module:API.cvat.classes.AnnotationFormat
161-
* @readonly
162-
* @instance
163-
*/
164-
get: () => data.owner,
165-
},
166-
name: {
167-
/**
168-
* @name name
169-
* @type {string}
170-
* @memberof module:API.cvat.classes.AnnotationFormat
171-
* @readonly
172-
* @instance
173-
*/
174-
get: () => data.name,
175-
},
176-
createdDate: {
177-
/**
178-
* @name createdDate
179-
* @type {string}
180-
* @memberof module:API.cvat.classes.AnnotationFormat
181-
* @readonly
182-
* @instance
183-
*/
184-
get: () => data.created_date,
185-
},
186-
updatedDate: {
187-
/**
188-
* @name updatedDate
189-
* @type {string}
190-
* @memberof module:API.cvat.classes.AnnotationFormat
191-
* @readonly
192-
* @instance
193-
*/
194-
get: () => data.updated_date,
195-
},
196-
handlerFile: {
197-
/**
198-
* @name handlerFile
199-
* @type {string}
200-
* @memberof module:API.cvat.classes.AnnotationFormat
201-
* @readonly
202-
* @instance
203-
*/
204-
get: () => data.handler_file,
205-
},
206117
loaders: {
207118
/**
208119
* @name loaders
209120
* @type {module:API.cvat.classes.Loader[]}
210-
* @memberof module:API.cvat.classes.AnnotationFormat
121+
* @memberof module:API.cvat.classes.AnnotationFormats
211122
* @readonly
212123
* @instance
213124
*/
214-
get: () => [...data.loaders],
125+
get: () => [...data.importers],
215126
},
216127
dumpers: {
217128
/**
218129
* @name dumpers
219130
* @type {module:API.cvat.classes.Dumper[]}
220-
* @memberof module:API.cvat.classes.AnnotationFormat
131+
* @memberof module:API.cvat.classes.AnnotationFormats
221132
* @readonly
222133
* @instance
223134
*/
224-
get: () => [...data.dumpers],
135+
get: () => [...data.exporters],
225136
},
226137
});
227138
}
228139
}
229140

230141
module.exports = {
231-
AnnotationFormat,
142+
AnnotationFormats,
232143
Loader,
233144
Dumper,
234145
};

‎cvat-core/src/annotations.js

+1-1
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
const {
1818
Loader,
1919
Dumper,
20-
} = require('./annotation-format.js');
20+
} = require('./annotation-formats.js');
2121
const {
2222
ScriptingError,
2323
DataError,

‎cvat-core/src/api-implementation.js

+2-7
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@
2626
} = require('./enums');
2727

2828
const User = require('./user');
29-
const { AnnotationFormat } = require('./annotation-format.js');
29+
const { AnnotationFormats } = require('./annotation-formats.js');
3030
const { ArgumentError } = require('./exceptions');
3131
const { Task } = require('./session');
3232

@@ -66,12 +66,7 @@
6666

6767
cvat.server.formats.implementation = async () => {
6868
const result = await serverProxy.server.formats();
69-
return result.map((el) => new AnnotationFormat(el));
70-
};
71-
72-
cvat.server.datasetFormats.implementation = async () => {
73-
const result = await serverProxy.server.datasetFormats();
74-
return result;
69+
return new AnnotationFormats(result);
7570
};
7671

7772
cvat.server.register.implementation = async (username, firstName, lastName,

‎cvat-core/src/api.js

+1-15
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,7 @@ function build() {
109109
* @method formats
110110
* @async
111111
* @memberof module:API.cvat.server
112-
* @returns {module:API.cvat.classes.AnnotationFormat[]}
112+
* @returns {module:API.cvat.classes.AnnotationFormats}
113113
* @throws {module:API.cvat.exceptions.PluginError}
114114
* @throws {module:API.cvat.exceptions.ServerError}
115115
*/
@@ -118,20 +118,6 @@ function build() {
118118
.apiWrapper(cvat.server.formats);
119119
return result;
120120
},
121-
/**
122-
* Method returns available dataset export formats
123-
* @method exportFormats
124-
* @async
125-
* @memberof module:API.cvat.server
126-
* @returns {module:String[]}
127-
* @throws {module:API.cvat.exceptions.PluginError}
128-
* @throws {module:API.cvat.exceptions.ServerError}
129-
*/
130-
async datasetFormats() {
131-
const result = await PluginRegistry
132-
.apiWrapper(cvat.server.datasetFormats);
133-
return result;
134-
},
135121
/**
136122
* Method allows to register on a server
137123
* @method register

0 commit comments

Comments
 (0)