Skip to content

Commit 3a299ac

Browse files
kellrottkellrott
authored
Merge pull request #65 from buchanae/readme
Update readme
2 parents 8fbe0fe + e39f43e commit 3a299ac

File tree

1 file changed

+126
-125
lines changed

1 file changed

+126
-125
lines changed

‎README.md

+126-125
Original file line numberDiff line numberDiff line change
@@ -1,149 +1,150 @@
1-
# task-execution-schemas
2-
31
GA4GH Task Execution Schema (TES)
42
=================================
53

6-
The Task Execution Schema proposal can be found at https://github.com/ga4gh/task-execution-schemas
7-
The Protocol Buffer Based Schema can be found at https://github.com/ga4gh/task-execution-schemas/blob/master/proto/task_execution.proto
8-
The swagger translation can be viewed at http://editor.swagger.io/#/?import=https://github.com/ga4gh/task-execution-schemas/raw/master/swagger/proto/task_execution.swagger.json
4+
The Task Execution Schema (TES) project is an effort to define a standardized schema
5+
and API for describing batch execution tasks. A task defines a set of input files,
6+
a set of (Docker) containers and commands to run, a set of output files,
7+
and some other logging and metadata.
98

10-
Example Task Message
11-
```
12-
{
13-
"name" : "TestMD5",
14-
"projectId" : "MyProject",
15-
"description" : "My Desc",
16-
"inputs" : [
17-
{
18-
"name" : "infile",
19-
"description" : "File to be MD5ed",
20-
"location" : "s3://my-bucket/input_file",
21-
"path" : "/tmp/test_file"
22-
}
23-
],
24-
"outputs" : [
25-
{
26-
"location" : "s3://my-bucket/output_file",
27-
"path" : "/tmp/test_out"
28-
}
29-
],
30-
"resources" : {
31-
"volumes" : [{
32-
"name" : "test_disk",
33-
"sizeGb" : 5,
34-
"mountPoint" : "/tmp"
35-
}]
36-
},
37-
"docker" : [
38-
{
39-
"imageName" : "ubuntu",
40-
"cmd" : ["md5sum", "/tmp/test_file"],
41-
"stdout" : "/tmp/test_out"
42-
}
43-
]
44-
}
45-
```
9+
The schema and APIs is defined [here](./task_execution.proto) in [protocol buffers](https://developers.google.com/protocol-buffers/). Clients may use JSON and REST to communicate
10+
with a service implementing the TES API.
4611

47-
Example Task Message:
48-
```
12+
13+
Create a task
14+
---------------------------------
15+
16+
Here's an example of a complete task message, defining a task which calculates
17+
an MD5 checksum on an input file and uploads the output:
18+
```JSON
4919
{
50-
"jobId" : "6E57CA6B-0BC7-44FB-BA2C-0CBFEC629C63",
51-
"metadata" : { Custom service metadata },
52-
"task" : {Task Message Above},
53-
"state" : "Running",
54-
"logs" : [
55-
{ Job Log }
56-
]
20+
"name": "MD5 example",
21+
"description": "Task which runs md5sum on the input file.",
22+
"project": "tes-example-project-id",
23+
"tags": {
24+
"custom-tag": "tag-value",
25+
},
26+
"inputs": [
27+
{
28+
"name": "infile",
29+
"description": "md5sum input file",
30+
"url": "/path/to/input_file",
31+
"path": "/container/input",
32+
"type": "FILE",
33+
}
34+
],
35+
"outputs" : [
36+
{
37+
"url" : "/path/to/output_file",
38+
"path" : "/container/output",
39+
}
40+
],
41+
"resources" : {
42+
"cpu_cores": 1,
43+
"ram_gb": 1.0,
44+
"size_gb": 100.0,
45+
"preemptible": false,
46+
},
47+
"executors" : [
48+
{
49+
"image_name" : "ubuntu",
50+
"cmd" : ["md5sum", "/container/input"],
51+
"stdout" : "/container/output",
52+
"stderr" : "/container/stderr",
53+
"workdir": "/tmp",
54+
}
55+
]
5756
}
5857
```
5958

60-
Example Job Log Message:
61-
```
59+
A minimal version of the same task, including only the required fields looks like:
60+
```JSON
6261
{
63-
"cmd" : ["md5sum", "/tmp/test_file"],
64-
"startTime" : "2016-09-18T23:08:27Z",
65-
"endTime" : "2016-09-18T23:38:00Z",
66-
"stdout": "f6913671da6018ff8afcb1517983ab24 test_file",
67-
"stderr": "",
68-
"exitCode" = 0
62+
"inputs": [
63+
{
64+
"url": "/path/to/input_file",
65+
"path": "/container/input",
66+
}
67+
],
68+
"outputs" : [
69+
{
70+
"url" : "/path/to/output_file",
71+
"path" : "/container/output",
72+
}
73+
],
74+
"executors" : [
75+
{
76+
"image_name" : "ubuntu",
77+
"cmd" : ["md5sum", "/container/input"],
78+
"stdout" : "/container/output",
79+
}
80+
]
6981
}
7082
```
7183

72-
Example Task Conversation:
84+
To create the task, send an HTTP POST request:
85+
```HTTP
86+
POST /v1/tasks
7387
74-
Get meta-data about service
88+
{ "id": "task-1234" }
7589
```
76-
GET /v1/jobs-service
90+
91+
The return value is a task ID.
92+
93+
94+
Get a task
95+
--------------------------------
96+
97+
To get a task by ID:
98+
99+
```HTTP
100+
GET /v1/tasks/task-1234
101+
102+
{ "id": "task-1234", "state": "RUNNING" }
77103
```
78-
Returns (from reference server)
79-
{"storageConfig":{"baseDir":"/var/run/task-execution-server/storage","storageType":"sharedFile"}}
80104

105+
The return value will be a minimal description of the task state.
106+
107+
To get more information, you can change the task view using the `view` URL query parameter.
81108

82-
Post Job
109+
The `basic` view will include all task fields except a few which might be
110+
large strings (stdout/err logging, input parameter contents).
111+
112+
```HTTP
113+
GET /v1/tasks/task-1234?view=BASIC
114+
115+
{ "id": "task-1234", "state": "RUNNING", "name": "MD5 example", etc... }
83116
```
84-
POST /v1/jobs {JSON body message above}
85-
Return:
86-
{ "value" : "{job uuid}"}
117+
118+
The `full` view includes stdout/err logs and full input parameters:
119+
120+
```HTTP
121+
GET /v1/tasks/task-1234?view=FULL
122+
123+
{ "id": "task-1234", "state": "RUNNING", "name": "MD5 example",
124+
"logs": [{ "stdout": "stdout content..." }], etc... }
87125
```
88126

89-
Get Job Info
127+
List tasks
128+
------------------------------------
129+
130+
To list tasks:
131+
132+
```HTTP
133+
GET /v1/tasks
134+
135+
{ "tasks": [{ "id": "task-1234", "state": "RUNNING"}, etc...] }
90136
```
91-
GET /v1/jobs/{job uuid}
137+
138+
Similar to getting a task by ID, you may change the task view:
139+
```HTTP
140+
GET /v1/tasks?view=BASIC
92141
```
93-
Returns Job Body Example:
142+
143+
144+
Cancel a task
145+
-------------------------------------
146+
147+
To cancel a task, send an HTTP POST to the cancel endpoint:
148+
```HTTP
149+
POST /v1/tasks/task-1234:cancel
94150
```
95-
{
96-
"jobId" : "06b170b4-6ae8-4f11-7fc6-4417f1778b74",
97-
"logs" : [
98-
{
99-
"exitCode" : -1
100-
}
101-
],
102-
"task" : {
103-
"projectId" : "test",
104-
"inputs" : [
105-
{
106-
"location" : "fs://README.md",
107-
"description" : "input",
108-
"path" : "/mnt/README.md",
109-
"name" : "input"
110-
}
111-
],
112-
"name" : "funnel workflow",
113-
"taskId" : "06b170b4-6ae8-4f11-7fc6-4417f1778b74",
114-
"resources" : {
115-
"minimumRamGb" : 1,
116-
"minimumCpuCores" : 1,
117-
"volumes" : [
118-
{
119-
"sizeGb" : 10,
120-
"name" : "data",
121-
"mountPoint" : "/mnt"
122-
}
123-
]
124-
},
125-
"outputs" : [
126-
{
127-
"location" : "fs://output/sha",
128-
"path" : "/mnt/sha",
129-
"name" : "stdout",
130-
"description" : "tool stdout"
131-
}
132-
],
133-
"docker" : [
134-
{
135-
"imageName" : "bmeg/openssl",
136-
"workdir" : "/mnt/sha",
137-
"cmd" : [
138-
"openssl",
139-
"dgst",
140-
"-sha",
141-
"/mnt/README.md"
142-
]
143-
}
144-
],
145-
"description" : "CWL TES task"
146-
},
147-
"state" : "Error"
148-
}
149-
```

0 commit comments

Comments
 (0)