Skip to content

Commit ef7edb1

Browse files
forgivedengkaiforgivedengkai
authored
Merge pull request #132 from FederatedAI/develop
Develop
2 parents 1f1cf4d + 28ca862 commit ef7edb1

File tree

19 files changed

+75
-51
lines changed

19 files changed

+75
-51
lines changed

‎RELEASE.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,8 @@
1+
# Release 2.1.3
2+
## Major Features and Improvements
3+
* Fix the HttpAdapter thread leakage problem.
4+
* Upgrade log4j dependent version.
5+
16
# Release 2.1.2
27
## Major Features and Improvements
38
* Fix serious security vulnerabilities in log4j.

‎document/docs/config/proxy.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -116,7 +116,7 @@
116116
</table>
117117

118118
### route_table配置
119-
下面的json代码大致说明了router_table.json的填写规则，根据具体案例来配置。
119+
下面的json代码大致说明了route_table.json的填写规则，根据具体案例来配置。
120120

121121
>启动注册中心时，只配置默认对外转发地址（default）即可 ;
122122
>在2.1.0版本开始支持HTTP接口配置，2.1.0之前的版本只支持GRPC配置。
@@ -173,7 +173,7 @@
173173
"default": {
174174
"default": [
175175
{
176-
"url":"http://127.0.0.1:8879/unary"
176+
"url":"http://127.0.0.1:8869/unary"
177177
}
178178
]
179179
},

‎document/docs/example/deploy.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ zk.url=192.168.0.1:2181,192.168.0.2:2181,192.168.0.3:2181
1111
proxy.grpc.intra.port=8879
1212
proxy.grpc.inter.port=9370
1313
```
14-
• guest 的serving-proxy router_table.json配置：
14+
• guest 的serving-proxy route_table.json配置：
1515

1616
由于guest的请求只会向外发送，所以只需要配置出口ip端口就好， 如下代码所示只需要配置default转发规则，则会将所有请求转发至出口ip，而出口ip需要与host端proxy.grpc.inter.port对齐。
1717
```yml
@@ -59,7 +59,7 @@ zk.url=192.134.0.1:2181,192.134.0.2:2181,192.134.0.3:2181
5959
proxy.grpc.intra.port=8879
6060
proxy.grpc.inter.port=9370
6161
```
62-
• host 的serving-proxy router_table.json配置：
62+
• host 的serving-proxy route_table.json配置：
6363
```yml
6464
xxxxxxxxxx
6565
{

‎document/docs/quickstart/quickstart.md

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,9 @@ $ mvn clean package
1515
随后拷贝，比如 serving-server/target/fate-serving-server-{version}-release.zip 到想要部署的路径下，并解压。（version为当前版本号）
1616

1717
用户也可选择下载已编译好的版本，链接如下:
18-
• [fate-serving-server-2.1.2-release.zip](https://webank-ai-1251170195.cos.ap-guangzhou.myqcloud.com/fate-serving-server-2.1.2-release.zip)
19-
• [fate-serving-proxy-2.1.2-release.zip](https://webank-ai-1251170195.cos.ap-guangzhou.myqcloud.com/fate-serving-proxy-2.1.2-release.zip)
20-
• [fate-serving-admin-2.1.2-release.zip](https://webank-ai-1251170195.cos.ap-guangzhou.myqcloud.com/fate-serving-admin-2.1.2-release.zip)
18+
• [fate-serving-server-2.1.3-release.zip](https://webank-ai-1251170195.cos.ap-guangzhou.myqcloud.com/fate-serving-server-2.1.3-release.zip)
19+
• [fate-serving-proxy-2.1.3-release.zip](https://webank-ai-1251170195.cos.ap-guangzhou.myqcloud.com/fate-serving-proxy-2.1.3-release.zip)
20+
• [fate-serving-admin-2.1.3-release.zip](https://webank-ai-1251170195.cos.ap-guangzhou.myqcloud.com/fate-serving-admin-2.1.3-release.zip)
2121

2222

2323
### zookeeper部署
@@ -54,15 +54,17 @@ export PATH=$PATH:$JAVA_HOME/bin
5454

5555
### serving-proxy部署
5656
1.根据需求修改部署目录下 conf/application.properties文件，具体配置项解释见配置文件详解[proxy.md](../config/proxy.md)中application.properties配置。
57-
2.配置router_table.json ，具体配置项解释见文件详解[proxy.md](../config/proxy.md)中route_table配置。
58-
>对router_table.json的修改是定时刷新生效，可以不需要重启serving-proxy；配置本身为json格式，修改时需要注意是否满足json格式。
57+
2.配置route_table.json ，具体配置项解释见文件详解[proxy.md](../config/proxy.md)中route_table配置。
58+
>对route_table.json的修改是定时刷新生效，可以不需要重启serving-proxy；配置本身为json格式，修改时需要注意是否满足json格式。
5959
6060
3.sh service.sh restart (或 ./service.sh restart) 启动应用（windows 脚本暂时不支持，如有需要可自行编写）
6161
有可能出现的问题：
6262
>jdk 没有安装成功，可以尝试执行 java -version 查看java命令是否能正常执行，可在bin/service.sh中指定jdk
63+
```yaml
64+
xxxxxxxxxx
6365
export JAVA_HOME=/data/projects/fate/common/jdk/jdk-8u192
6466
export PATH=$PATH:$JAVA_HOME/bin
65-
>
67+
```
6668
4.检查日志与端口看启动是否正常
6769

6870
### serving-admin部署
@@ -75,7 +77,9 @@ serving-admin提供了集群的可视化操作界面，可以展示集群中各
7577
2.sh service.sh restart (或 ./service.sh restart) 启动应用（windows 脚本暂时不支持，如有需要可自行编写）
7678
有可能出现的问题：
7779
>jdk 没有安装成功，可以尝试执行 java -version 查看java命令是否能正常执行，可在bin/service.sh中指定jdk
80+
```yaml
81+
xxxxxxxxxx
7882
export JAVA_HOME=/data/projects/fate/common/jdk/jdk-8u192
7983
export PATH=$PATH:$JAVA_HOME/bin
80-
>
84+
```
8185
3.通过浏览器访问admin页面，默认访问地址: [http://127.0.0.1:8350](http://127.0.0.1:8350)

‎document/docs/release/change_log.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,9 @@
1+
## FATE-SERVING 2.1.3
2+
```text
3+
1.修复内置HttpAdapter线程泄露问题。
4+
2.升级log4j依赖版本。
5+
```
6+
17
## FATE-SERVING 2.1.2
28
```text
39
1.修复log4j的严重安全漏洞。

‎document/docs/service/adapter.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ feature.batch.adaptor=com.webank.ai.fate.serving.adaptor.dataaccess.CustomBatchA
5151
>serving-server部署目录下extension``已加载到类路径
5252

5353
## 预设Adapter
54-
fate-serving-extension中预设了6种Adapter的简单实现
54+
fate-serving-extension中预设了6中Adapter的简单实现
5555

5656
#### MockAdapter
5757
固定返回mock特征数据
@@ -63,15 +63,17 @@ fate-serving-extension中预设了6种Adapter的简单实现
6363
用于批量预测，原理同上
6464

6565
#### TestFileAdapter
66-
从host_data.csv种读取特征数据，每次调用返回值为csv种所有内容, host_data.csv需上传至Host方serving-server实例部署根目录下，
66+
从host_data.csv中读取特征数据，每次调用返回值为csv中所有内容, host_data.csv需上传至Host方serving-server实例部署根目录下，
6767
>x0:-0.320167,x1:0.58883,x2:-0.18408,x3:-0.384207,x4:2.201839,x5:1.68401,x6:1.219096,x7:1.150692,x8:1.9656,x9:1.572462,x10:-0.35685
6868
x0:1,x1:5,x2:13,x3:58,x4:95,x5:352,x6:418,x7:833,x8:888,x9:937,x10:32776
6969

7070
#### HttpAdapter
7171
在serving-server.properties文件中配置属性feature.single.adaptor和http.adapter.url，feature.single.adaptor为继承AbstractSingleFeatureDataAdaptor
72-
接口，url为调用获取数据接口地址。
73-
feature.single.adaptor=com..webank.ai.fate.serving.adaptor.dataaccess.HttpAdapter
72+
接口，url为调用获取数据接口地址。
73+
```yaml
74+
feature.single.adaptor=com.webank.ai.fate.serving.adaptor.dataaccess.HttpAdapter
7475
http.adapter.url=http://127.0.0.1:9380/v1/http/adapter/getFeature
76+
```
7577

76-
### HttpBatchAdapter
78+
#### HttpBatchAdapter
7779
用于批量预测，需将feature.single.adaptor配置改为feature.batch.adaptor

‎document/docs/service/client.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
11
FATE-Serving提供了fate-serving-client工具
22
### 下载对应版本的fate-serving-client
3-
>linux版本：[fate-serving-client-2.1.2-linux.tar.gz](https://webank-ai-1251170195.cos.ap-guangzhou.myqcloud.com/fate-serving-client-2.1.2-linux.tar.gz )
4-
>mac版本：[fate-serving-client-2.1.2-darwin.tar.gz](https://webank-ai-1251170195.cos.ap-guangzhou.myqcloud.com/fate-serving-client-2.1.2-darwin.tar.gz)
3+
>linux版本：[fate-serving-client-2.1.3-linux.tar.gz](https://webank-ai-1251170195.cos.ap-guangzhou.myqcloud.com/fate-serving-client-2.1.3-linux.tar.gz )
4+
>mac版本：[fate-serving-client-2.1.3-darwin.tar.gz](https://webank-ai-1251170195.cos.ap-guangzhou.myqcloud.com/fate-serving-client-2.1.3-darwin.tar.gz)
55
6-
需要先确认下载解压出来的文件有没有可执行权限，没有权限就执行 `chmod +777 fate-serving-client-2.1.2-linux`授权。
6+
需要先确认下载解压出来的文件有没有可执行权限，没有权限就执行 `chmod +777 fate-serving-client-2.1.3-linux`授权。
77

88
### 启动client
9-
在终端中使用`./fate-serving-client-2.1.2-linux`启动client
9+
在终端中使用`./fate-serving-client-2.1.3-linux`启动client
1010
![connect](../img/client-connect.jpg)
1111

1212
### 提供了以下几种指令

‎fate-serving-admin-ui/package.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "FATE-Serving",
3-
"version": "2.1.2",
3+
"version": "2.1.3",
44
"private": true,
55
"author": "FATE Serving",
66
"scripts": {
@@ -47,7 +47,7 @@
4747
"sass-loader": "^7.1.0",
4848
"svg-sprite-loader": "^3.9.2",
4949
"vue-template-compiler": "^2.6.10",
50-
"webpack-bundle-analyzer": "^2.13.1",
50+
"webpack-bundle-analyzer": ">=3.3.2",
5151
"webpack-cli": "^3.2.3"
5252
}
5353
}

‎fate-serving-admin/bin/service.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ basepath=$(cd `dirname $0`;pwd)
2424
configpath=$(cd $basepath/conf;pwd)
2525
module=serving-admin
2626
main_class=com.webank.ai.fate.serving.admin.Bootstrap
27-
module_version=2.1.2
27+
module_version=2.1.3
2828

2929
case "$1" in
3030
start)

‎fate-serving-common/src/main/java/com/webank/ai/fate/serving/common/health/HealthCheckItemEnum.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
package com.webank.ai.fate.serving.common.health;
22

33
public enum HealthCheckItemEnum {
4-
CHECK_ROUTER_FILE("check router_table.json exist",HealthCheckComponent.SERVINGPROXY),
4+
CHECK_ROUTER_FILE("check route_table.json exist",HealthCheckComponent.SERVINGPROXY),
55
CHECK_ZOOKEEPER_CONFIG("check zk config",HealthCheckComponent.ALL),
66
CHECK_ROUTER_NET("check router",HealthCheckComponent.SERVINGPROXY),
77
CHECK_MEMORY_USAGE("check memory usage",HealthCheckComponent.ALL),

‎fate-serving-common/src/main/java/com/webank/ai/fate/serving/common/utils/HttpClientPool.java

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -87,13 +87,16 @@ public static void initPool() {
8787
requestConfig = RequestConfig.custom().setConnectionRequestTimeout(
8888
connectionRequestTimeout).setSocketTimeout(socketTimeout).setConnectTimeout(
8989
connectTimeout).build();
90-
httpClient = getConnection();
90+
httpClient = createConnection();
9191
} catch (NoSuchAlgorithmException | KeyStoreException | KeyManagementException ex) {
9292
logger.error("init http client pool failed:", ex);
9393
}
9494
}
95-
9695
public static CloseableHttpClient getConnection() {
96+
return httpClient;
97+
}
98+
99+
public static CloseableHttpClient createConnection() {
97100
CloseableHttpClient httpClient = HttpClients.custom()
98101
.setConnectionManager(poolConnManager)
99102
.setDefaultRequestConfig(requestConfig)

‎fate-serving-core/src/main/java/com/webank/ai/fate/serving/core/bean/MetaInfo.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@
2424
import java.util.Map;
2525

2626
public class MetaInfo {
27-
public static final long CURRENT_VERSION = 212;
27+
public static final long CURRENT_VERSION = 213;
2828
public static List<AdaptorDescriptor.ParamDescriptor> inferenceParamDescriptorList;
2929
public static List<AdaptorDescriptor.ParamDescriptor> batchInferenceParamDescriptorList;
3030
public static Boolean PROPERTY_REMOTE_MODEL_INFERENCE_RESULT_CACHE_SWITCH;

‎fate-serving-proxy/bin/service.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ basepath=$(cd `dirname $0`;pwd)
2424
configpath=$(cd $basepath/conf;pwd)
2525
module=serving-proxy
2626
main_class=com.webank.ai.fate.serving.proxy.bootstrap.Bootstrap
27-
module_version=2.1.2
27+
module_version=2.1.3
2828

2929

3030
case "$1" in

‎fate-serving-proxy/src/main/java/com/webank/ai/fate/serving/proxy/common/RouterTableUtils.java

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,7 @@ public static JsonObject initRouter() {
9898
// long now = new Date().getTime();
9999
// JsonObject routerJson = loadRoutTable();
100100
// if (routerJson == null) {
101-
// throw new RouterInfoOperateException("router_table.json not exists");
101+
// throw new RouterInfoOperateException("route_table.json not exists");
102102
// }
103103
// JsonObject route_table = routerJson.getAsJsonObject("route_table");
104104
// if (route_table == null) {
@@ -126,8 +126,8 @@ public static JsonObject initRouter() {
126126
// }
127127
// routerJson.add("route_table", route_table);
128128
// if (writeRouterFile(JsonUtil.formatJson(routerJson.toString()))) {
129-
// logger.error("write router_table.json error");
130-
// throw new RouterInfoOperateException("write router_table.json error");
129+
// logger.error("write route_table.json error");
130+
// throw new RouterInfoOperateException("write route_table.json error");
131131
// }
132132
// } catch (RouterInfoOperateException routerEx) {
133133
// throw new RouterInfoOperateException(routerEx.getMessage());
@@ -141,7 +141,7 @@ public static JsonObject initRouter() {
141141
// try {
142142
// JsonObject routerJson = loadRoutTable();
143143
// if (routerJson == null) {
144-
// throw new RouterInfoOperateException("router_table.json not exists");
144+
// throw new RouterInfoOperateException("route_table.json not exists");
145145
// }
146146
// JsonObject route_table = routerJson.getAsJsonObject("route_table");
147147
// if (route_table == null) {
@@ -167,8 +167,8 @@ public static JsonObject initRouter() {
167167
// partyIdRouter.add(routerInfo.getServerType(), serverTypeArray);
168168
// }
169169
// if (writeRouterFile(JsonUtil.formatJson(routerJson.toString()))) {
170-
// logger.error("write router_table.json error");
171-
// throw new RouterInfoOperateException("write router_table.json error");
170+
// logger.error("write route_table.json error");
171+
// throw new RouterInfoOperateException("write route_table.json error");
172172
// }
173173
// } catch (RouterInfoOperateException routerEx) {
174174
// throw new RouterInfoOperateException(routerEx.getMessage());
@@ -182,7 +182,7 @@ public static JsonObject initRouter() {
182182
// try {
183183
// JsonObject routerJson = loadRoutTable();
184184
// if (routerJson == null) {
185-
// throw new RouterInfoOperateException("router_table.json not exists");
185+
// throw new RouterInfoOperateException("route_table.json not exists");
186186
// }
187187
// JsonObject route_table = routerJson.getAsJsonObject("route_table");
188188
// if (route_table == null) {
@@ -193,8 +193,8 @@ public static JsonObject initRouter() {
193193
// if (StringUtils.isBlank(routerInfo.getServerType())) {
194194
// route_table.remove(routerInfo.getPartyId());
195195
// if (writeRouterFile(JsonUtil.formatJson(routerJson.toString()))) {
196-
// logger.error("write router_table.json error");
197-
// throw new RouterInfoOperateException("write router_table.json error");
196+
// logger.error("write route_table.json error");
197+
// throw new RouterInfoOperateException("write route_table.json error");
198198
// }
199199
// return;
200200
// }
@@ -216,8 +216,8 @@ public static JsonObject initRouter() {
216216
// }
217217
// routerJson.add("route_table", route_table);
218218
// if (writeRouterFile(JsonUtil.formatJson(routerJson.toString()))) {
219-
// logger.error("write router_table.json error");
220-
// throw new RouterInfoOperateException("write router_table.json error");
219+
// logger.error("write route_table.json error");
220+
// throw new RouterInfoOperateException("write route_table.json error");
221221
// }
222222
// } catch (RouterInfoOperateException routerEx) {
223223
// throw new RouterInfoOperateException(routerEx.getMessage());
@@ -230,17 +230,17 @@ public static JsonObject initRouter() {
230230
public static void saveRouter(String routerInfo){
231231
try {
232232
if (!RouteTableJsonValidator.isJSON(routerInfo)) {
233-
logger.error("validate router_table.json format error");
233+
logger.error("validate route_table.json format error");
234234
}
235235
} catch (Exception e) {
236-
throw new RouterInfoOperateException("validate router_table.json format error:" + e.getMessage());
236+
throw new RouterInfoOperateException("validate route_table.json format error:" + e.getMessage());
237237
}
238238
try {
239239
if (writeRouterFile(JsonUtil.formatJson(routerInfo))) {
240-
logger.error("write router_table.json fail");
240+
logger.error("write route_table.json fail");
241241
}
242242
} catch (Exception e) {
243-
throw new RouterInfoOperateException("router_table.json : " + e.getMessage());
243+
throw new RouterInfoOperateException("route_table.json : " + e.getMessage());
244244
}
245245
}
246246

‎fate-serving-proxy/src/main/java/com/webank/ai/fate/serving/proxy/rpc/services/HealthCheckEndPointService.java

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -121,11 +121,11 @@ private void checkRouterInfo(HealthCheckResult healthCheckResult){
121121

122122
healthCheckResult.getRecords().add(new HealthCheckRecord(HealthCheckItemEnum.CHECK_ROUTER_FILE.getItemName(),"check router file : no router info found",HealthCheckStatus.error));
123123

124-
// healthCheckResult.getErrorList().add("check router_table.json "+": no router info found");
124+
// healthCheckResult.getErrorList().add("check route_table.json "+": no router info found");
125125
}else{
126126
healthCheckResult.getRecords().add(new HealthCheckRecord(HealthCheckItemEnum.CHECK_ROUTER_FILE.getItemName(),"check router file : router info is found",HealthCheckStatus.ok));
127127

128-
// healthCheckResult.getOkList().add("check router_table.json "+": router_table.json is found");
128+
// healthCheckResult.getOkList().add("check route_table.json "+": route_table.json is found");
129129
}
130130

131131
routerInfoCheck(healthCheckResult);
@@ -221,7 +221,7 @@ public HealthCheckResult check(Context context) {
221221
return healthCheckComponent == HealthCheckComponent.ALL || healthCheckComponent == HealthCheckComponent.SERVINGPROXY;
222222
}).forEach((item) -> {
223223
switch (item) {
224-
// CHECK_ROUTER_FILE("check router_table.json exist",HealthCheckComponent.SERVINGPROXY),
224+
// CHECK_ROUTER_FILE("check route_table.json exist",HealthCheckComponent.SERVINGPROXY),
225225
// CHECK_ZOOKEEPER_CONFIG("check zk config",HealthCheckComponent.ALL),
226226
// CHECK_ROUTER_NET("check router",HealthCheckComponent.SERVINGPROXY),
227227
// CHECK_MEMORY_USAGE("check memory usage",HealthCheckComponent.ALL),

‎fate-serving-proxy/src/main/resources/route_table.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
"default": [
55
{
66
"ip": "127.0.0.1",
7-
"port": 8879,
7+
"port": 8869,
88
"useSSL": false
99
}
1010
]

‎fate-serving-server/bin/service.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ basepath=$(cd `dirname $0`;pwd)
2424
configpath=$(cd $basepath/conf;pwd)
2525
module=serving-server
2626
main_class=com.webank.ai.fate.serving.Bootstrap
27-
module_version=2.1.2
27+
module_version=2.1.3
2828

2929

3030
case "$1" in

‎fate-serving-server/src/main/resources/serving-server.properties

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,4 +50,8 @@ zk.url=localhost:2181,localhost:2182,localhost:2183
5050
# zk acl
5151
#acl.enable=false
5252
#acl.username=
53-
#acl.password=
53+
#acl.password=
54+
55+
# LR algorithm config
56+
#lr.split.size=500
57+
#lr.use.parallel=false

‎pom.xml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@
3737
</modules>
3838

3939
<properties>
40-
<fate.version>2.1.2</fate.version>
40+
<fate.version>2.1.3</fate.version>
4141
<java.version>1.8</java.version>
4242
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
4343
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
@@ -50,7 +50,7 @@
5050
<spring.boot.version>2.2.0.RELEASE</spring.boot.version>
5151
<jackson.version>2.10.0</jackson.version>
5252
<jedis.version>2.9.0</jedis.version>
53-
<log4j2.version>2.16.0</log4j2.version>
53+
<log4j2.version>2.17.0</log4j2.version>
5454
<skipTests>true</skipTests>
5555
</properties>
5656

0 commit comments

Comments
 (0)