Skip to main content

A provider from an HTTP API

Call url() in a Source's data line. The daemon makes the request itself, with no curl process, and caches the fields you pick out of the response.

A public endpoint​

# ~/.config/beachcomber/conf.d/github-status.toml
[daemon.providers.status.sources.main]
data = "{{ url('https://www.githubstatus.com/api/v2/status.json') | fromjson }}"
result.indicator = "{{ data.status.indicator }}"
result.description = "{{ data.status.description }}"
return = "{{ result | poll('60s') | failback('5s', 3) }}"
comb get status
description=All Systems Operational
indicator=none

url() returns the response body as text; | fromjson parses it. result.<field> lines pick fields out of the parsed body, and result = "{{ data.rate }}" would make one whole object the provider's fields instead:

[daemon.providers.gh_rate.sources.main]
data = '''{{ url('https://api.github.com/rate_limit', headers={'Accept': 'application/json'})
| fromjson }}'''
result = "{{ data.rate }}"
return = "{{ result | poll('5m') }}"
comb get gh_rate
limit=60
remaining=58
reset=1793898342
used=2

For a body that is not JSON, leave out fromjson and use the text: result.body = "{{ data | trim }}".

url() arguments​

url(address, method='GET', headers={}, body=none, timeout='10s')

argumentmeaning
addressan absolute http:// or https:// URL
methodGET, POST, PUT or PATCH
headersa map of header names to string values
bodya string; needs a method other than GET. Build JSON with | tojson
timeouthow long the request may take, a duration

A POST with a token and a JSON body:

[daemon.providers.echo.sources.main]
data = '''{{ url('https://httpbin.org/anything',
method='POST',
headers={
'Authorization': 'Bearer ' ~ env.API_TOKEN,
'Content-Type': 'application/json',
},
body={'q': env.REGION} | tojson,
timeout='5s') | fromjson }}'''
result.method = "{{ data.method }}"
result.sent = "{{ data.json }}"
return = "{{ result | poll('1h') }}"
comb get echo -f json
{
"method": "POST",
"sent": {
"q": "eu"
}
}

url() runs only in the daemon. In a client template it is an error, url() runs only in a Source's lines.

Keep secrets in the env file​

env in a Source's lines is the daemon's environment, not your shell's. A daemon started on demand inherits whatever environment started it, so put the variables a Source needs in the env file, which the daemon loads at startup:

# ~/.config/beachcomber/env
# KEY=VALUE, one per line; blank lines and #comments are ignored; values may be quoted.
API_TOKEN="replace-me"
REGION=eu
chmod 600 ~/.config/beachcomber/env
comb kill # the next read starts a daemon that has loaded it

The token stays out of the configuration files, and comb check config reads the same file, so it checks with the variables the daemon would have. An unset variable renders empty: 'Bearer ' ~ env.API_TOKEN then sends Bearer and the API refuses it. To keep the file elsewhere, set [daemon] env_file = "~/.secrets/comb.env".

A secret manager can write the file for you, for example op read "op://Vault/comb/env" > ~/.config/beachcomber/env.

When a request fails​

A non-2xx status, a connection error, a body that is not UTF-8 and a timeout all fail the run. The last good value stays in the cache and readers keep getting it. The log says what happened, with the query string left out:

WARN beachcomber::run: Source 'broken.main' failed: broken.main: data: url(): https://api.github.com/this-does-not-exist: status 404

failback(interval, count) on return sets how the Source backs off: after count failures in a row, retries are spaced from interval, doubling. The run as a whole is cut off at [daemon] timeout (default 10s), or at | timeout('30s') on the Source's return.

Poll an API no faster than it changes and no faster than its rate limit allows: poll('86400s') for a daily figure.