Skip to content

Commit 646dd7e

Browse files
tpoisonooomegvii-mge
tpoisonooo
and
megvii-mge
authored
Add tutorial 01 (#12)
* feat(project): fix memleak; add tutorial-01 Co-authored-by: MegEngine <[email protected]>
1 parent 7cdd925 commit 646dd7e

35 files changed

+1783
-199
lines changed

‎Cargo.lock

+8-8
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎README.md

+7-6
Original file line numberDiff line numberDiff line change
@@ -21,12 +21,13 @@ MegFlow 是一个面向视觉应用的流式计算框架, 目标是简单、高
2121
- 基础测试工具，支持插件沙盒，用于单测插件
2222

2323
## HowTo
24-
* [how to build with docker](docs/how-to-build-with-docker.zh.md)
25-
* [how to build from source](docs/how-to-build-from-source.zh.md)
26-
* [how to pack Python .whl](docs/how-to-pack-python-whl.zh.md)
27-
* [how to add my service](docs/how-to-add-graph.zh.md)
28-
* [how to add plugins](docs/how-to-add-plugins.zh.md)
29-
* [how to optimize and debug](docs/how-to-debug.zh.md)
24+
* how to build and run
25+
* [build with docker](docs/how-to-build-and-run/build-with-docker.zh.md)
26+
* [build from source](docs/how-to-build-and-run/build-from-source.zh.md)
27+
* [generate rtsp](docs/how-to-build-and-run/generate-rtsp.zh.md)
28+
* how to use
29+
* [add my first service](docs/how-to-add-my-service/01-single-classification-model.zh.md)
30+
* [how to optimize and debug](docs/how-to-debug.zh.md)
3031
* [how to contribute](docs/how-to-contribute.zh.md)
3132
* [FAQ](docs/FAQ.zh.md)
3233

‎doc_link_checker.py

+40
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
import os
2+
import re
3+
4+
pattern = re.compile(r'\[.*?\]\(.*?\)')
5+
def analyze_doc(home, path):
6+
problem_list = []
7+
with open(path) as f:
8+
lines = f.readlines()
9+
for line in lines:
10+
if '[' in line and ']' in line and '(' in line and ')' in line:
11+
all = pattern.findall(line)
12+
for item in all:
13+
start = item.find('(')
14+
end = item.find(')')
15+
ref = item[start+1: end]
16+
if ref.startswith('http'):
17+
continue
18+
fullpath = os.path.join(home, ref)
19+
if not os.path.exists(fullpath):
20+
problem_list.append(ref)
21+
# print(f' {fullpath} in {path} not exist!')
22+
else:
23+
continue
24+
if len(problem_list) > 0:
25+
print(f'{path}:')
26+
for item in problem_list:
27+
print(f'\t {item}')
28+
print('\n')
29+
30+
def traverse(_dir):
31+
for home, dirs, files in os.walk(_dir):
32+
if "./target" in home or "./.github" in home:
33+
continue
34+
for filename in files:
35+
if filename.endswith('.md'):
36+
path = os.path.join(home, filename)
37+
analyze_doc(home, path)
38+
39+
if __name__ == "__main__":
40+
traverse(".")

‎docs/how-to-add-my-service/01-single-classification-model.zh.md

+149
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
# 把分类模型变成服务
2+
3+
尽管 MegFlow 解决的是多个（20+）模型组织成 pipeline 的问题，但凡事总要一步步来。本文介绍如何 step by step 集成 1 个分类模型，最终成为图片/视频 http 服务。
4+
5+
## 准备分类模型
6+
7+
[MegEngine models]() 有现成的 imagenet 预训模型。这里把模型 dump 成 .mge。
8+
9+
新增 [dump.py](../../flow-python/examples/simple_classification/dump.py)，按 [1, 3, 224, 224] 尺寸 trace 模型，打开推理优化选项，保存为 `model.mge`。
10+
11+
```bash
12+
$ git clone https://github.com/MegEngine/models
13+
$ cd models
14+
$ export PYHTONPATH=${PWD}:${PYTHONPATH}
15+
$ cd official/vision/classification/resnet
16+
$ python3 dump.py
17+
$ ls -lah model.mge
18+
...
19+
```
20+
`dump.py` 正在 PR 到 MegEngine/models
21+
```bash
22+
$ cat dump.py
23+
...
24+
data = mge.Tensor(np.random.random((1, 3, 224, 224))) # 准备一个样例输入
25+
26+
@jit.trace(capture_as_const=True)
27+
def pred_func(data):
28+
outputs = model(data) # trace 每个 opr 的 shape
29+
return outputs
30+
31+
pred_func(data)
32+
pred_func.dump( # 保存模型
33+
graph_name,
34+
arg_names=["data"],
35+
optimize_for_inference=True, # 打开推理优化选项
36+
enable_fuse_conv_bias_nonlinearity=True, # 打开 fuse conv+bias+ReLU pass 推理更快
37+
)
38+
...
39+
```
40+
41+
## 模型单测
42+
开发模型的推理封装，对外提供[功能内聚](https://baike.baidu.com/item/%E9%AB%98%E5%86%85%E8%81%9A%E4%BD%8E%E8%80%A6%E5%90%88/5227009)的接口。调用方传入一张或多张图片、直接获取结果，尽量避免关心内部实现（如用何种 backbone、预处理是什么、后处理是什么）。
43+
44+
```bash
45+
$ cat flow-python/examples/simple_classification/lite.py
46+
...
47+
def inference(self, mat):
48+
# 设置输入
49+
inp_data =self.net.get_io_tensor("data")
50+
inp_data.set_data_by_share(img)
51+
52+
# 推理
53+
self.net.forward()
54+
self.net.wait()
55+
56+
# 取输出
57+
output_keys = self.net.get_all_output_name()
58+
output = self.net.get_io_tensor(output_keys[0]).to_numpy()
59+
return np.argmax(output[0])
60+
...
61+
$ python3 lite.py --model model.mge --path test.jpg # 测试
62+
2021-09-14 11:45:02.406 | INFO | __main__:<module>:81 - 285
63+
```
64+
65+
`285` 是分类模型最后一层的 argmax，对应含义需要查[ imagenet 数据集分类表](../../flow-python/examples/simple_classification/synset_words.txt) ，这里是 “Egyptian cat”（下标从 0 开始）。
66+
67+
## 配置计算图
68+
`flow-python/examples`增加`simple_classification/image_cpu.toml`
69+
70+
```bash
71+
$ cat flow-python/examples/simple_classification/image_cpu.toml
72+
main = "tutorial_01_image"
73+
74+
[[graphs]]
75+
name = "subgraph"
76+
inputs = [{ name = "inp", cap = 16, ports = ["classify:inp"] }] # 一、输入输出结点
77+
outputs = [{ name = "out", cap = 16, ports = ["classify:out"] }]
78+
connections = [
79+
]
80+
81+
[[graphs.nodes]] # 二、结点参数
82+
name = "classify"
83+
ty = "Classify"
84+
path = "models/simple_classification_models/resnet18.mge"
85+
device = "cpu"
86+
device_id = 0
87+
...
88+
[[graphs.nodes]] # 三、服务类型配置
89+
name = "source"
90+
ty = "ImageServer"
91+
port = 8084 # 端口号 8084
92+
response = "json"
93+
...
94+
```
95+
开发时直接从别处复制一个过来即可，图片单模型服务只需要关心 3 处
96+
* 计算图输入、输出结点的名字。这里是`classify`
97+
* `classify` 结点的参数。最重要的是 `ty="Classify"`指明了类名，MegFlow 将在当前目录搜索`Classify`类。path/device/device_id 分别是模型路径/用 CPU 推理/用哪个核，属于用户自定义配置
98+
* 服务类型。这里想运行图片服务 `ty = "ImageServer"`，如果想运行视频解析服务改 `ty = "VideoServer"`；图片服务默认返回图片，想返回 string 需要配置 `response = "json"`
99+
100+
[完整的计算图 config 定义](appendix-A-graph-definition.md)
101+
102+
## 实现配置中的 node
103+
104+
创建文件`classify.py`，把之前实现的模型推理调起来即可
105+
```bash
106+
$ cat flow-python/examples/simple_classification/classify.py
107+
...
108+
@register(inputs=['inp'], outputs=['out'])
109+
class Classify:
110+
def __init__(self, name, args):
111+
logger.info("loading Resnet18 Classification...")
112+
self.name = name
113+
114+
# load model and warmup
115+
self._model = PredictorLite(path=args['path'], device=args['device'], device_id=args['device_id'])
116+
warmup_data = np.zeros((224, 224, 3), dtype=np.uint8)
117+
self._model.inference(warmup_data)
118+
logger.info("Resnet18 loaded.")
119+
120+
def exec(self):
121+
envelope = self.inp.recv()
122+
if envelope is None:
123+
return
124+
125+
data = envelope.msg['data']
126+
result = self._model.inference(data)
127+
self.out.send(envelope.repack(json.dumps(str(result))))
128+
```
129+
实现只有 2 点：
130+
* `__init__` 里加载模型，做个 warmup 防止首次推理太慢
131+
* 解码成 BGR 的 data 在 `envelope.msg['data']`，推理，send 返回 json string
132+
133+
[更多 node 说明](appendix-B-python-plugin.zh.md)
134+
135+
## 运行测试
136+
137+
运行服务
138+
```bash
139+
$ cd flow-python/examples
140+
$ cargo run --example run_with_plugins -- -c simple_classification/image_cpu.toml -p simple_classification # 源码/docker 编译方式用这条命令
141+
```
142+
143+
浏览器打开 8084 端口服务（例如 http://10.122.101.175:8084/docs ），选择一张图“try it out”即可。
144+
145+
## 其他
146+
147+
一、http 客户端开发
148+
149+
rweb/Swagger 提供了 http RESTful API 描述文件，例如在 http://10.122.101.175:8084/openapi.json 。`swagger_codegen` 可用描述文件生成各种语言的调用代码。更多教程见 [swagger codegen tutorial ](https://swagger.io/tools/swagger-codegen/)。

‎docs/how-to-add-graph.zh.md renamed to ‎docs/how-to-add-my-service/appendix-A-graph-definition.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ connections = [
6262

6363
对应可视化的计算图：
6464

65-
![](../flow-python/examples/cat_finder/images/image.png)
65+
![](../../flow-python/examples/cat_finder/images/image.png)
6666

6767
## 视频范例
6868

@@ -137,7 +137,7 @@ connections = [
137137
```
138138

139139
视频解析服务可视化结果和图片接近：
140-
![](../flow-python/examples/cat_finder/images/video.png)
140+
![](../../flow-python/examples/cat_finder/images/video.png)
141141

142142

143143
## 完整定义

‎docs/how-to-add-plugins.zh.md renamed to ‎docs/how-to-add-my-service/appendix-B-python-plugin.zh.md

-4
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,3 @@ MegFlow也提供了一系列异步工具
3939
3. `join(tasks)`, `tasks`参数是一个函数列表，`join`堵塞直到`tasks`中的函数都执行完毕
4040
4. `create_future(callback)`, `callback`参数是一个函数, 默认值为None，`create_future`返回一个`(Future, Waker)`对象
4141
- `Future::wait`, 堵塞直到`Waker::wake`被调用，返回`Waker::wake(result)`传入的`result`参数
42-
43-
# Rust Plugins
44-
45-
Coming soon

‎docs/how-to-build-from-source.zh.md renamed to ‎docs/how-to-build-and-run/build-from-source.zh.md

+5-10
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
### 安装 Rust
66
```bash
7-
$ sudo apt install curl
7+
$ sudo apt install yasm git build-essential ffmpeg curl
88
$ curl --proto '=https' --tlsv1.2 https://sh.rustup.rs -sSf | sh
99
```
1010

@@ -14,11 +14,6 @@ $ cargo --version
1414
cargo 1.53.0 (4369396ce 2021-04-27)
1515
```
1616

17-
如果不成功，提示`Command 'cargo' not found`，可以按照提示加载一下环境变量(重新连接或打开终端也可以)：
18-
```
19-
source $HOME/.cargo/env
20-
```
21-
2217
> `cargo` 是 Rust 的包管理器兼编译辅助工具。类似 Java maven/go pkg/C++ CMake 的角色，更易使用。
2318
2419
### 安装 python3.x （推荐 conda）
@@ -49,7 +44,7 @@ $ conda activate py38
4944
MegFlow 需要编译 ffmpeg。考虑到 ffmpeg 依赖较多、本身又是常用工具，最简单的办法就是直接装 ffmpeg 把编译依赖装上
5045

5146
```bash
52-
$ sudo apt install yasm git build-essential ffmpeg
47+
$ sudo apt install ffmpeg
5348
$ ffmpeg
5449
ffmpeg version 3.4.8...
5550
$ sudo apt install clang
@@ -68,7 +63,7 @@ $ cd flow-python
6863
$ python3 setup.py install --user
6964
```
7065

71-
**FAQ**：`error while loading shared libraries: libpython3.8.xxx`。如果使用 conda 只需要
66+
**常见问题**：`error while loading shared libraries: libpython3.8.xxx`，意为 libpython.so 找不到。如果使用 conda 就在 miniconda 安装目录下面，只需要设置环境变量
7267

7368
```bash
7469
$ export LD_LIBRARY_PATH=`conda info --base`/pkgs/python-3.8.11-xxx/lib:${LD_LIBRARY_PATH}
@@ -88,11 +83,11 @@ $ cargo run --example run_with_plugins -- -p logical_test
8883

8984
接下来开始运行好玩的 Python 应用
9085

91-
* [猫猫围栏运行手册](../flow-python/examples/cat_finder/README.md)
86+
* [猫猫围栏运行手册](../../flow-python/examples/cat_finder/README.md)
9287
* 图片注册猫猫
9388
* 部署视频围栏，注册的猫离开围栏时会发通知
9489
* 未注册的不会提示
95-
* [电梯电瓶车告警](../flow-python/examples/electric_bicycle/README.md)
90+
* [电梯电瓶车告警](../../flow-python/examples/electric_bicycle/README.md)
9691
* 电梯里看到电瓶车立即报警
9792
* Comming Soon
9893
* OCR： 通用字符识别

‎docs/how-to-build-with-docker.zh.md renamed to ‎docs/how-to-build-and-run/build-with-docker.zh.md

+4-3
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,14 @@
22

33
## Build Docker Image
44

5-
MegFlow 提供了 [Dockerfile](../Dockerfile)，能够“可复现地”生成运行环境、减少依赖缺失的痛苦
5+
MegFlow 提供了 [Dockerfile](../../Dockerfile)，能够“可复现地”生成运行环境、减少依赖缺失的痛苦
66

77
```bash
88
$ cd MegFlow
99
$ docker build -t megflow .
1010
```
1111
稍等一段时间（取决于网络和 CPU）镜像构建完成并做了基础自测
12+
> 注意：**不要移动 Dockerfile 文件的位置**。受 [EAR](https://www.federalregister.gov/documents/2019/10/09/2019-22210/addition-of-certain-entities-to-the-entity-list) 约束，MegFlow 无法提供现成的 docker 镜像，需要自己 build 出来，这个过程用了相对路径。
1213
```bash
1314
$ docker images
1415
REPOSITORY TAG IMAGE ID CREATED SIZE
@@ -23,11 +24,11 @@ $ docker run -p 18081:8081 -p 18082:8082 -i -t c65e37e1df6c /bin/bash
2324

2425
接下来开始运行好玩的 Python 应用
2526

26-
* [猫猫围栏运行手册](../flow-python/examples/cat_finder/README.md)
27+
* [猫猫围栏运行手册](../../flow-python/examples/cat_finder/README.md)
2728
* 图片注册猫猫
2829
* 部署视频围栏，注册的猫离开围栏时会发通知
2930
* 未注册的不会提示
30-
* [电梯电瓶车告警](../flow-python/examples/electric_bicycle/README.md)
31+
* [电梯电瓶车告警](../../flow-python/examples/electric_bicycle/README.md)
3132
* 电梯里看到电瓶车立即报警
3233
* Comming Soon
3334
* OCR： 通用字符识别

‎docs/how-to-generate-rtsp.zh.md renamed to ‎docs/how-to-build-and-run/generate-rtsp.zh.md

File renamed without changes.

0 commit comments

Comments
 (0)