@@ -167,18 +167,18 @@ Here CPU utilization dropped to 0, and so HPA autoscaled the number of replicas
167
167
## Autoscaling on multiple metrics and custom metrics
168
168
169
169
You can introduce additional metrics to use when autoscaling the ` php-apache ` Deployment
170
- by making use of the ` autoscaling/v2beta1 ` API version.
170
+ by making use of the ` autoscaling/v2beta2 ` API version.
171
171
172
- First, get the YAML of your HorizontalPodAutoscaler in the ` autoscaling/v2beta1 ` form:
172
+ First, get the YAML of your HorizontalPodAutoscaler in the ` autoscaling/v2beta2 ` form:
173
173
174
174
``` shell
175
- $ kubectl get hpa.v2beta1 .autoscaling -o yaml > /tmp/hpa-v2.yaml
175
+ $ kubectl get hpa.v2beta2 .autoscaling -o yaml > /tmp/hpa-v2.yaml
176
176
```
177
177
178
178
Open the ` /tmp/hpa-v2.yaml ` file in an editor, and you should see YAML which looks like this:
179
179
180
180
``` yaml
181
- apiVersion : autoscaling/v2beta1
181
+ apiVersion : autoscaling/v2beta2
182
182
kind : HorizontalPodAutoscaler
183
183
metadata :
184
184
name : php-apache
@@ -194,7 +194,9 @@ spec:
194
194
- type : Resource
195
195
resource :
196
196
name : cpu
197
- targetAverageUtilization : 50
197
+ target :
198
+ type : Utilization
199
+ averageUtilization : 50
198
200
status :
199
201
observedGeneration : 1
200
202
lastScaleTime : <some-time>
@@ -204,8 +206,9 @@ status:
204
206
- type : Resource
205
207
resource :
206
208
name : cpu
207
- currentAverageUtilization : 0
208
- currentAverageValue : 0
209
+ current :
210
+ averageUtilization : 0
211
+ averageValue : 0
209
212
` ` `
210
213
211
214
Notice that the ` targetCPUUtilizationPercentage` field has been replaced with an array called `metrics`.
@@ -215,40 +218,49 @@ the only other supported resource metric is memory. These resources do not chan
215
218
to cluster, and should always be available, as long as the `metrics.k8s.io` API is available.
216
219
217
220
You can also specify resource metrics in terms of direct values, instead of as percentages of the
218
- requested value. To do so, use the `targetAverageValue` field instead of the `targetAverageUtilization`
219
- field.
221
+ requested value, by using a `target` type of `AverageValue` instead of `AverageUtilization`, and
222
+ setting the corresponding `target.averageValue` field instead of the `target.averageUtilization` .
220
223
221
224
There are two other types of metrics, both of which are considered *custom metrics* : pod metrics and
222
225
object metrics. These metrics may have names which are cluster specific, and require a more
223
226
advanced cluster monitoring setup.
224
227
225
228
The first of these alternative metric types is *pod metrics*. These metrics describe pods, and
226
229
are averaged together across pods and compared with a target value to determine the replica count.
227
- They work much like resource metrics, except that they *only* have the `targetAverageValue` field .
230
+ They work much like resource metrics, except that they *only* support a `target` type of `AverageValue` .
228
231
229
232
Pod metrics are specified using a metric block like this :
230
233
231
234
` ` ` yaml
232
235
type: Pods
233
236
pods:
234
- metricName: packets-per-second
235
- targetAverageValue: 1k
237
+ metric:
238
+ name: packets-per-second
239
+ target:
240
+ type: AverageValue
241
+ averageValue: 1k
236
242
` ` `
237
243
238
- The second alternative metric type is *object metrics*. These metrics describe a different
239
- object in the same namespace, instead of describing pods. Note that the metrics are not
240
- fetched from the object -- they simply describe it. Object metrics do not involve averaging,
241
- and look like this :
244
+ The second alternative metric type is *object metrics*. These metrics describe a different
245
+ object in the same namespace, instead of describing pods. The metrics are not necessarily
246
+ fetched from the object; they only describe it. Object metrics support `target` types of
247
+ both `Value` and `AverageValue`. With `Value`, the target is compared directly to the returned
248
+ metric from the API. With `AverageValue`, the value returned from the custom metrics API is divided
249
+ by the number of pods before being compared to the target. The following example is the YAML
250
+ representation of the `requests-per-second` metric.
242
251
243
252
` ` ` yaml
244
253
type: Object
245
254
object:
246
- metricName: requests-per-second
247
- target:
255
+ metric:
256
+ name: requests-per-second
257
+ describedObject:
248
258
apiVersion: extensions/v1beta1
249
259
kind: Ingress
250
260
name: main-route
251
- targetValue: 2k
261
+ target:
262
+ type: Value
263
+ value: 2k
252
264
` ` `
253
265
254
266
If you provide multiple such metric blocks, the HorizontalPodAutoscaler will consider each metric in turn.
@@ -275,19 +287,25 @@ spec:
275
287
- type: Resource
276
288
resource:
277
289
name: cpu
278
- targetAverageUtilization: 50
290
+ target:
291
+ kind: AverageUtilization
292
+ averageUtilization: 50
279
293
- type: Pods
280
294
pods:
281
- metricName: packets-per-second
295
+ metric:
296
+ name: packets-per-second
282
297
targetAverageValue: 1k
283
298
- type: Object
284
299
object:
285
- metricName: requests-per-second
286
- target:
300
+ metric:
301
+ name: requests-per-second
302
+ describedObject:
287
303
apiVersion: extensions/v1beta1
288
304
kind: Ingress
289
305
name: main-route
290
- targetValue: 10k
306
+ target:
307
+ kind: Value
308
+ value: 10k
291
309
status:
292
310
observedGeneration: 1
293
311
lastScaleTime: <some-time>
@@ -297,48 +315,84 @@ status:
297
315
- type: Resource
298
316
resource:
299
317
name: cpu
300
- currentAverageUtilization: 0
301
- currentAverageValue: 0
318
+ current:
319
+ averageUtilization: 0
320
+ averageValue: 0
321
+ - type: Object
322
+ object:
323
+ metric:
324
+ name: requests-per-second
325
+ describedObject:
326
+ apiVersion: extensions/v1beta1
327
+ kind: Ingress
328
+ name: main-route
329
+ current:
330
+ value: 10k
302
331
` ` `
303
332
304
333
Then, your HorizontalPodAutoscaler would attempt to ensure that each pod was consuming roughly
305
334
50% of its requested CPU, serving 1000 packets per second, and that all pods behind the main-route
306
335
Ingress were serving a total of 10000 requests per second.
307
336
337
+ # ## Autoscaling on more specific metrics
338
+
339
+ Many metrics pipelines allow you to describe metrics either by name or by a set of additional
340
+ descriptors called _labels_. For all non-resource metric types (pod, object, and external,
341
+ described below), you can specify an additional label selector which is passed to your metric
342
+ pipeline. For instance, if you collect a metric `http_requests` with the `verb`
343
+ label, you can specify the following metric block to scale only on GET requests :
344
+
345
+ ` ` ` yaml
346
+ type: Object
347
+ object:
348
+ metric:
349
+ name: ` http_requests`
350
+ selector : ` verb=GET`
351
+ ` ` `
352
+
353
+ This selector uses the same syntax as the full Kubernetes label selectors. The monitoring pipeline
354
+ determines how to collapse multiple series into a single value, if the name and selector
355
+ match multiple series. The selector is additive, and cannot select metrics
356
+ that describe objects that are **not** the target object (the target pods in the case of the ` Pods`
357
+ type, and the described object in the case of the `Object` type).
358
+
308
359
# ## Autoscaling on metrics not related to Kubernetes objects
309
360
310
361
Applications running on Kubernetes may need to autoscale based on metrics that don't have an obvious
311
362
relationship to any object in the Kubernetes cluster, such as metrics describing a hosted service with
312
363
no direct correlation to Kubernetes namespaces. In Kubernetes 1.10 and later, you can address this use case
313
364
with *external metrics*.
314
365
315
- Using external metrics requires a certain level of knowledge of your monitoring system, and it requires a cluster
316
- monitoring setup similar to one required for using custom metrics. With external metrics, you can autoscale
317
- based on any metric available in your monitoring system by providing a `metricName` field in your
318
- HorizontalPodAutoscaler manifest. Additionally you can use a `metricSelector` field to limit which
319
- metrics' time series you want to use for autoscaling. If multiple time series are matched by `metricSelector`,
366
+ Using external metrics requires knowledge of your monitoring system; the setup is
367
+ similar to that required when using custom metrics. External metrics allow you to autoscale your cluster
368
+ based on any metric available in your monitoring system. Just provide a `metric` block with a
369
+ ` name ` and `selector`, as above, and use the `External` metric type instead of `Object`.
370
+ If multiple time series are matched by the `metricSelector`,
320
371
the sum of their values is used by the HorizontalPodAutoscaler.
372
+ External metrics support both the `Value` and `AverageValue` target types, which function exactly the same
373
+ as when you use the `Object` type.
321
374
322
375
For example if your application processes tasks from a hosted queue service, you could add the following
323
376
section to your HorizontalPodAutoscaler manifest to specify that you need one worker per 30 outstanding tasks.
324
377
325
378
` ` ` yaml
326
379
- type: External
327
380
external:
328
- metricName: queue_messages_ready
329
- metricSelector:
330
- matchLabels:
331
- queue: worker_tasks
332
- targetAverageValue: 30
381
+ metric:
382
+ name: queue_messages_ready
383
+ selector: "queue=worker_tasks"
384
+ target:
385
+ type: AverageValue
386
+ averageValue: 30
333
387
` ` `
334
388
335
- If your metric describes work or resources that can be divided between autoscaled pods the `targetAverageValue`
336
- field describes how much of that work each pod can handle. Instead of using the `targetAverageValue` field, you could use the
337
- ` targetValue ` to define a desired value of your external metric .
389
+ When possible, it's preferrable to use the custom metric target types instead of external metrics, since it's
390
+ easier for cluster administrators to secure the custom metrics API. The external metrics API potentially allows
391
+ access to any metric, so cluster administrators should take care when exposing it .
338
392
339
393
# # Appendix: Horizontal Pod Autoscaler Status Conditions
340
394
341
- When using the `autoscaling/v2beta1 ` form of the HorizontalPodAutoscaler, you will be able to see
395
+ When using the `autoscaling/v2beta2 ` form of the HorizontalPodAutoscaler, you will be able to see
342
396
*status conditions* set by Kubernetes on the HorizontalPodAutoscaler. These status conditions indicate
343
397
whether or not the HorizontalPodAutoscaler is able to scale, and whether or not it is currently restricted
344
398
in any way.
0 commit comments