Skip to content

Commit 5459711

Browse files
wukkowukko
committed
docs: create a dedicated doc for api instance env variables
& also move "api key file format" section to the actually relevant doc, aka `protect-an-instance`
1 parent f70f88b commit 5459711

File tree

3 files changed

+283
-91
lines changed

3 files changed

+283
-91
lines changed

‎docs/api-env-variables.md

+228
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,228 @@
1+
# cobalt api instance environment variables
2+
you can customize your processing instance's behavior using these environment variables. all of them but `API_URL` are optional.
3+
this document is not final and will expand over time. feel free to improve it!
4+
5+
### general vars
6+
| name | default | value example |
7+
|:--------------------|:----------|:--------------------------------------|
8+
| API_URL | | `https://api.url.example/` |
9+
| API_PORT | `9000` | `1337` |
10+
| COOKIE_PATH | | `/cookies.json` |
11+
| PROCESSING_PRIORITY | | `10` |
12+
| API_INSTANCE_COUNT | | `6` |
13+
| API_REDIS_URL | | `redis://localhost:6379` |
14+
| DISABLED_SERVICES | | `bilibili,youtube` |
15+
16+
[*view details*](#general)
17+
18+
### networking vars
19+
| name | default | value example |
20+
|:--------------------|:----------|:--------------------------------------|
21+
| API_LISTEN_ADDRESS | `0.0.0.0` | `127.0.0.1` |
22+
| API_EXTERNAL_PROXY | | `http://user:[email protected]:8080` |
23+
| FREEBIND_CIDR | | `2001:db8::/32` |
24+
25+
[*view details*](#networking)
26+
27+
### limit vars
28+
| name | default | value example |
29+
|:-------------------------|:--------|:--------------|
30+
| DURATION_LIMIT | `10800` | `18000` |
31+
| TUNNEL_LIFESPAN | `90` | `120` |
32+
| RATELIMIT_WINDOW | `60` | `120` |
33+
| RATELIMIT_MAX | `20` | `30` |
34+
| SESSION_RATELIMIT_WINDOW | `60` | `60` |
35+
| SESSION_RATELIMIT | `10` | `10` |
36+
37+
[*view details*](#limits)
38+
39+
### security vars
40+
| name | default | value example |
41+
|:------------------|:--------|:--------------------------------------|
42+
| CORS_WILDCARD | `1` | `0` |
43+
| CORS_URL | | `https://web.url.example` |
44+
| TURNSTILE_SITEKEY | | `1x00000000000000000000BB` |
45+
| TURNSTILE_SECRET | | `1x0000000000000000000000000000000AA` |
46+
| JWT_SECRET | | see [details](#security) |
47+
| JWT_EXPIRY | `120` | `240` |
48+
| API_KEY_URL | | `file://keys.json` |
49+
| API_AUTH_REQUIRED | | `1` |
50+
51+
[*view details*](#security)
52+
53+
### service-specific vars
54+
| name | value example |
55+
|:---------------------------------|:-------------------------|
56+
| CUSTOM_INNERTUBE_CLIENT | `IOS` |
57+
| YOUTUBE_SESSION_SERVER | `http://localhost:8080/` |
58+
| YOUTUBE_SESSION_INNERTUBE_CLIENT | `WEB_EMBEDDED` |
59+
60+
[*view details*](#service-specific)
61+
62+
## general
63+
[*jump to the table*](#general-vars)
64+
65+
### API_URL
66+
> [!NOTE]
67+
> API_URL is required to run the API instance.
68+
69+
the URL from which your instance will be accessible. can be external or internal, but it must be a valid URL or else tunnels will not work.
70+
71+
the value is a URL.
72+
73+
### API_PORT
74+
port from which the API server will be accessible.
75+
76+
the value is a number from 1024 to 65535.
77+
78+
### COOKIE_PATH
79+
path to the `cookies.json` file relative to the current working directory of your cobalt instance (usually the main (src/api) folder).
80+
81+
### PROCESSING_PRIORITY
82+
`nice` value for ffmpeg subprocesses. available only on unix systems.
83+
84+
note: the higher the nice value, the lower the priority. you can [read more about nice here](https://en.wikipedia.org/wiki/Nice_(Unix)).
85+
86+
the value is a number.
87+
88+
### API_INSTANCE_COUNT
89+
supported only on linux and node.js `>=23.1.0`. when configured, cobalt will spawn multiple sub-instances amongst which requests will be balanced. `API_REDIS_URL` is required to use this option.
90+
91+
the value is a number.
92+
93+
### API_REDIS_URL
94+
when configured, cobalt will use this redis instance for tunnel cache. required when `API_INSTANCE_COUNT` is more than 1, because else sub-instance wouldn't be able to share cache.
95+
96+
the value is a URL.
97+
98+
### DISABLED_SERVICES
99+
comma-separated list which disables certain services from being used.
100+
101+
the value is a string of cobalt-supported services.
102+
103+
## networking
104+
[*jump to the table*](#networking-vars)
105+
106+
### API_LISTEN_ADDRESS
107+
defines the local address for the api instance. if you are using a docker container, you usually don't need to configure this.
108+
109+
the value is a local IP address.
110+
111+
### API_EXTERNAL_PROXY
112+
URL of the proxy that will be passed to [`ProxyAgent`](https://undici.nodejs.org/#/docs/api/ProxyAgent) and used for all external requests. HTTP(S) only.
113+
114+
if some feature breaks when using a proxy, please make a new issue about it!
115+
116+
the value is a URL.
117+
118+
### FREEBIND_CIDR
119+
IPv6 prefix used for randomly assigning addresses to cobalt requests. available only on linux systems.
120+
121+
setting a `FREEBIND_CIDR` allows cobalt to pick a random IP for every download and use it for all requests it makes for that particular download.
122+
123+
to use freebind in cobalt, you need to follow its [setup instructions](https://github.com/imputnet/freebind.js?tab=readme-ov-file#setup) first.
124+
125+
if you want to use this option and run cobalt in a docker container, you also need to set the `API_LISTEN_ADDRESS` env variable to `127.0.0.1` and set `network_mode` for the container to `host`.
126+
127+
the value is an IPv6 range.
128+
129+
## limits
130+
[*jump to the table*](#limit-vars)
131+
132+
### DURATION_LIMIT
133+
media duration limit, in **seconds**
134+
135+
the value is a number.
136+
137+
### TUNNEL_LIFESPAN
138+
the duration for which tunnel info is stored in ram, **in seconds**.
139+
140+
it's recommended to keep this value either default or as low as possible to preserve efficiency and user privacy.
141+
142+
the value is a number.
143+
144+
### RATELIMIT_WINDOW
145+
rate limit time window for api requests, but not session requests, in **seconds**.
146+
147+
the value is a number.
148+
149+
### RATELIMIT_MAX
150+
amount of api requests to be allowed within the time window of `RATELIMIT_WINDOW`.
151+
152+
the value is a number.
153+
154+
### SESSION_RATELIMIT_WINDOW
155+
rate limit time window for session creation requests, in **seconds**.
156+
157+
the value is a number.
158+
159+
### SESSION_RATELIMIT
160+
amount of session requests to be allowed within the time window of `SESSION_RATELIMIT_WINDOW`.
161+
162+
the value is a number.
163+
164+
## security
165+
[*jump to the table*](#security-vars)
166+
167+
> [!NOTE]
168+
> in order to enable turnstile bot protection, `TURNSTILE_SITEKEY`, `TURNSTILE_SECRET`, and `JWT_SECRET` must be set. all three at once.
169+
170+
### CORS_WILDCARD
171+
defines whether cross-origin resource sharing is enabled. when enabled, your instance will be accessible from foreign web pages.
172+
173+
the value is a number. 0: disabled. 1: enabled.
174+
175+
### CORS_URL
176+
configures the [cross-origin resource sharing origin](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Access-Control-Allow-Origin). your instance will be available only from this URL if `CORS_WILDCARD` is set to `0`.
177+
178+
the value is a URL.
179+
180+
### TURNSTILE_SITEKEY
181+
[cloudflare turnstile](https://www.cloudflare.com/products/turnstile/) sitekey used by the web client to request & solve a challenge to prove that the user is not a bot.
182+
183+
the value is a specific key.
184+
185+
### TURNSTILE_SECRET
186+
[cloudflare turnstile](https://www.cloudflare.com/products/turnstile/) secret used by the processing instance to verify that the client solved the challenge successfully.
187+
188+
the value is a specific key.
189+
190+
### JWT_SECRET
191+
the secret used for issuing JWT tokens for request authentication. the value must be a random, secure, and long string (over 16 characters).
192+
193+
the value is a specific key.
194+
195+
### JWT_EXPIRY
196+
the duration of how long a cobalt-issued JWT token will remain valid, in seconds.
197+
198+
the value is a number.
199+
200+
### API_KEY_URL
201+
the URL to the the external or local key database. for local files you have to specify a local path using the `file://` protocol.
202+
203+
see [the api key section](/docs/protect-an-instance.md#api-key-file-format) in the "how to protect your cobalt instance" document for more details.
204+
205+
the value is a URL.
206+
207+
### API_AUTH_REQUIRED
208+
when set to `1`, the user always needs to be authenticated in some way before they can access the API (either via an api key or via turnstile, if enabled).
209+
210+
the value is a number.
211+
212+
## service-specific
213+
[*jump to the table*](#service-specific-vars)
214+
215+
### CUSTOM_INNERTUBE_CLIENT
216+
innertube client that will be used instead of the default one.
217+
218+
the value is a string.
219+
220+
### YOUTUBE_SESSION_SERVER
221+
URL to an instance of [yt-session-generator](https://github.com/imputnet/yt-session-generator). used for automatically pulling `poToken` & `visitor_data` for youtube. can be local or remote.
222+
223+
the value is a URL.
224+
225+
### YOUTUBE_SESSION_INNERTUBE_CLIENT
226+
innertube client that's compatible with botguard's (web) `poToken` and `visitor_data`.
227+
228+
the value is a string.

‎docs/protect-an-instance.md

+53-1
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,7 @@ if you want to use your instance outside of web interface, you'll need an api ke
114114
>
115115
> if api keys leak, you'll have to update/remove all UUIDs to revoke them.
116116

117-
1. create a `keys.json` file following [the schema and example here](/docs//run-an-instance.md#api-key-file-format).
117+
1. create a `keys.json` file following [the schema and example down below](#api-key-file-format).
118118

119119
2. expose the `keys.json` to the docker container:
120120
```yml
@@ -148,3 +148,55 @@ environment:
148148
### why not make keys exclusive by default?
149149
keys may be useful for going around rate limiting,
150150
while keeping the rest of api rate limited, with no turnstile in place.
151+
152+
## api key file format
153+
the file is a JSON-serialized object with the following structure:
154+
```typescript
155+
156+
type KeyFileContents = Record<
157+
UUIDv4String,
158+
{
159+
name?: string,
160+
limit?: number | "unlimited",
161+
ips?: (CIDRString | IPString)[],
162+
userAgents?: string[]
163+
}
164+
>;
165+
```
166+
167+
where *`UUIDv4String`* is a stringified version of a UUIDv4 identifier.
168+
- **name** is a field for your own reference, it is not used by cobalt anywhere.
169+
170+
- **`limit`** specifies how many requests the API key can make during the window specified in the `RATELIMIT_WINDOW` env.
171+
- when omitted, the limit specified in `RATELIMIT_MAX` will be used.
172+
- it can be also set to `"unlimited"`, in which case the API key bypasses all rate limits.
173+
174+
- **`ips`** contains an array of allowlisted IP ranges, which can be specified both as individual ips or CIDR ranges (e.g. *`["192.168.42.69", "2001:db8::48", "10.0.0.0/8", "fe80::/10"]`*).
175+
- when specified, only requests from these ip ranges can use the specified api key.
176+
- when omitted, any IP can be used to make requests with that API key.
177+
178+
- **`userAgents`** contains an array of allowed user agents, with support for wildcards (e.g. *`["cobaltbot/1.0", "Mozilla/5.0 * Chrome/*"]`*).
179+
- when specified, requests with a `user-agent` that does not appear in this array will be rejected.
180+
- when omitted, any user agent can be specified to make requests with that API key.
181+
182+
- if both `ips` and `userAgents` are set, the tokens will be limited by both parameters.
183+
- if cobalt detects any problem with your key file, it will be ignored and a warning will be printed to the console.
184+
185+
an example key file could look like this:
186+
```json
187+
{
188+
"b5c7160a-b655-4c7a-b500-de839f094550": {
189+
"limit": 10,
190+
"ips": ["10.0.0.0/8", "192.168.42.42"],
191+
"userAgents": ["*Chrome*"]
192+
},
193+
"b00b1234-a3e5-99b1-c6d1-dba4512ae190": {
194+
"limit": "unlimited",
195+
"ips": ["192.168.1.2"],
196+
"userAgents": ["cobaltbot/1.0"]
197+
}
198+
}
199+
```
200+
201+
if you are configuring a key file, **do not use the UUID from the example** but instead generate your own. you can do this by running the following command if you have node.js installed:
202+
`node -e "console.log(crypto.randomUUID())"`

‎docs/run-an-instance.md

+2-90
Original file line numberDiff line numberDiff line change
@@ -54,93 +54,5 @@ sudo apt install nscd
5454
sudo service nscd start
5555
```
5656

57-
## list of environment variables for api
58-
| variable name | default | example | description |
59-
|:----------------------|:----------|:------------------------|:------------|
60-
| `API_PORT` | `9000` | `9000` | changes port from which api server is accessible. |
61-
| `API_LISTEN_ADDRESS` | `0.0.0.0` | `127.0.0.1` | changes address from which api server is accessible. **if you are using docker, you usually don't need to configure this.** |
62-
| `API_URL` | ➖ | `https://api.cobalt.tools/` | changes url from which api server is accessible. <br> ***REQUIRED TO RUN THE API***. |
63-
| `API_NAME` | `unknown` | `ams-1` | api server name that is shown in `/api/serverInfo`. |
64-
| `API_EXTERNAL_PROXY` | ➖ | `http://user:[email protected]:8080`| url of the proxy that will be passed to [`ProxyAgent`](https://undici.nodejs.org/#/docs/api/ProxyAgent) and used for all external requests. HTTP(S) only. |
65-
| `CORS_WILDCARD` | `1` | `0` | toggles cross-origin resource sharing. <br> `0`: disabled. `1`: enabled. |
66-
| `CORS_URL` | not used | `https://cobalt.tools` | cross-origin resource sharing url. api will be available only from this url if `CORS_WILDCARD` is set to `0`. |
67-
| `COOKIE_PATH` | not used | `/cookies.json` | path for cookie file relative to main folder. |
68-
| `PROCESSING_PRIORITY` | not used | `10` | changes `nice` value* for ffmpeg subprocess. available only on unix systems. |
69-
| `FREEBIND_CIDR` | ➖ | `2001:db8::/32` | IPv6 prefix used for randomly assigning addresses to cobalt requests. only supported on linux systems. see below for more info. |
70-
| `RATELIMIT_WINDOW` | `60` | `120` | rate limit time window in **seconds**. |
71-
| `RATELIMIT_MAX` | `20` | `30` | max requests per time window. requests above this amount will be blocked for the rate limit window duration. |
72-
| `DURATION_LIMIT` | `10800` | `18000` | max allowed video duration in **seconds**. |
73-
| `TUNNEL_LIFESPAN` | `90` | `120` | the duration for which tunnel info is stored in ram, **in seconds**. |
74-
| `TURNSTILE_SITEKEY` | ➖ | `1x00000000000000000000BB` | [cloudflare turnstile](https://www.cloudflare.com/products/turnstile/) sitekey used by browser clients to request a challenge.\*\* |
75-
| `TURNSTILE_SECRET` | ➖ | `1x0000000000000000000000000000000AA` | [cloudflare turnstile](https://www.cloudflare.com/products/turnstile/) secret used by cobalt to verify the client successfully solved the challenge.\*\* |
76-
| `JWT_SECRET` | ➖ | ➖ | the secret used for issuing JWT tokens for request authentication. to choose a value, generate a random, secure, long string (ideally >=16 characters).\*\* |
77-
| `JWT_EXPIRY` | `120` | `240` | the duration of how long a cobalt-issued JWT token will remain valid, in seconds. |
78-
| `API_KEY_URL` | ➖ | `file://keys.json` | the location of the api key database. for loading API keys, cobalt supports HTTP(S) urls, or local files by specifying a local path using the `file://` protocol. see the "api key file format" below for more details. |
79-
| `API_AUTH_REQUIRED` | ➖ | `1` | when set to `1`, the user always needs to be authenticated in some way before they can access the API (either via an api key or via turnstile, if enabled). |
80-
| `API_REDIS_URL` | ➖ | `redis://localhost:6379` | when set, cobalt uses redis instead of internal memory for the tunnel cache. |
81-
| `API_INSTANCE_COUNT` | ➖ | `2` | supported only on Linux and node.js `>=23.1.0`. when configured, cobalt will spawn multiple sub-instances amongst which requests will be balanced. |
82-
| `DISABLED_SERVICES` | ➖ | `bilibili,youtube` | comma-separated list which disables certain services from being used. |
83-
| `CUSTOM_INNERTUBE_CLIENT` | ➖ | `IOS` | innertube client that will be used instead of the default one. |
84-
| `YOUTUBE_SESSION_SERVER` | ➖ | `http://localhost:8080/` | url to an instance of [invidious' youtube-trusted-session-generator](https://github.com/iv-org/youtube-trusted-session-generator) or its fork/counterpart. used for automatically pulling poToken & visitor_data for youtube. can be local or remote. |
85-
86-
\* the higher the nice value, the lower the priority. [read more here](https://en.wikipedia.org/wiki/Nice_(Unix)).
87-
88-
\*\* in order to enable turnstile bot protection, all three **`TURNSTILE_SITEKEY`, `TURNSTILE_SECRET` and `JWT_SECRET`** need to be set.
89-
90-
#### FREEBIND_CIDR
91-
setting a `FREEBIND_CIDR` allows cobalt to pick a random IP for every download and use it for all
92-
requests it makes for that particular download. to use freebind in cobalt, you need to follow its [setup instructions](https://github.com/imputnet/freebind.js?tab=readme-ov-file#setup) first. if you configure this option while running cobalt
93-
in a docker container, you also need to set the `API_LISTEN_ADDRESS` env to `127.0.0.1`, and set
94-
`network_mode` for the container to `host`.
95-
96-
## api key file format
97-
the file is a JSON-serialized object with the following structure:
98-
```typescript
99-
100-
type KeyFileContents = Record<
101-
UUIDv4String,
102-
{
103-
name?: string,
104-
limit?: number | "unlimited",
105-
ips?: (CIDRString | IPString)[],
106-
userAgents?: string[]
107-
}
108-
>;
109-
```
110-
111-
where *`UUIDv4String`* is a stringified version of a UUIDv4 identifier.
112-
- **name** is a field for your own reference, it is not used by cobalt anywhere.
113-
114-
- **`limit`** specifies how many requests the API key can make during the window specified in the `RATELIMIT_WINDOW` env.
115-
- when omitted, the limit specified in `RATELIMIT_MAX` will be used.
116-
- it can be also set to `"unlimited"`, in which case the API key bypasses all rate limits.
117-
118-
- **`ips`** contains an array of allowlisted IP ranges, which can be specified both as individual ips or CIDR ranges (e.g. *`["192.168.42.69", "2001:db8::48", "10.0.0.0/8", "fe80::/10"]`*).
119-
- when specified, only requests from these ip ranges can use the specified api key.
120-
- when omitted, any IP can be used to make requests with that API key.
121-
122-
- **`userAgents`** contains an array of allowed user agents, with support for wildcards (e.g. *`["cobaltbot/1.0", "Mozilla/5.0 * Chrome/*"]`*).
123-
- when specified, requests with a `user-agent` that does not appear in this array will be rejected.
124-
- when omitted, any user agent can be specified to make requests with that API key.
125-
126-
- if both `ips` and `userAgents` are set, the tokens will be limited by both parameters.
127-
- if cobalt detects any problem with your key file, it will be ignored and a warning will be printed to the console.
128-
129-
an example key file could look like this:
130-
```json
131-
{
132-
"b5c7160a-b655-4c7a-b500-de839f094550": {
133-
"limit": 10,
134-
"ips": ["10.0.0.0/8", "192.168.42.42"],
135-
"userAgents": ["*Chrome*"]
136-
},
137-
"b00b1234-a3e5-99b1-c6d1-dba4512ae190": {
138-
"limit": "unlimited",
139-
"ips": ["192.168.1.2"],
140-
"userAgents": ["cobaltbot/1.0"]
141-
}
142-
}
143-
```
144-
145-
if you are configuring a key file, **do not use the UUID from the example** but instead generate your own. you can do this by running the following command if you have node.js installed:
146-
`node -e "console.log(crypto.randomUUID())"`
57+
## list of environment variables
58+
[this section has moved](/docs/api-env-variables.md) to a dedicated document that is way easier to understand and maintain. go check it out!

0 commit comments

Comments
 (0)