The UBIRCH client provides signature and chaining services to seal original data generated on-premise. It takes care of packaging the hashed data into a chained UBIRCH PROTOCOL PACKET ("UPP"), signing the package and sending it to the UBIRCH backend for storage, anchoring in the blockchain, and as a source for verification requests.
The original data must be stored in a customer database to be able to execute verification requests at a later stage. UBIRCH does not store any original sensitive data!
UPP data is sent to the UBIRCH client via HTTP requests. The client can receive either the original data as a JSON data package which will be formatted and hashed (SHA256) by the client, or the binary representation of a SHA256 hash that is directly added as payload to the UPP.
The UBIRCH client is able to handle multiple cryptographic identities (i.e. for multiple LAN based devices). For each identity, the signing key and the signature of the last UPP are stored persistently to ensure an intact chain after a restart.
The UBIRCH client is provided as a multi-architecture docker image that can be configured and run on any system that can run docker (Intel/AMD64 or ARM64 architecture).
Docker Hub Address: ubirch/ubirch-client
See how to run client in Docker container
- System based on Intel/AMD64 or ARM
- Docker installation
- Space to store last used signatures and key material, either:
- disk space, mounted into the docker pod, or
- as SQL database
- Possibility to send a HTTP request to UBIRCH.com domains
- A unique identifier (UUID) registered with the UBIRCH console
- A corresponding authentication token (acquired via UBIRCH console)
- A password for the client’s key store
- (OPTIONAL) a private ECDSA (ecdsa-prime256v1) key (otherwise a new key is generated by the client itself)
There are multiple make targets, to simplify the building process.
make # default target: build
make all # create all available artifacts
make build # create all supported binaries
make pack # compresses the binaries to be much smaller (requires UPX installation)
make image # create docker image
make publish # builds images for amd64 armv7 and arm64, publishes under versioned tag
# takes IMAGE_TAG= to specify a different tag name
make clean # delete all artifactsCompiled artifacts will be saved to the build/ directory.
make publish IMAGE_TAG=stable # will tag a multi-arch image with the selected tag and upload it
# to the dockerhub.The configuration can be set via a configuration file (config.json) or environment variables.
There are two mandatory configurations:
- a 32 byte base64 encoded secret, which will be used to encrypt the signing keys in the database
- the desired database driver and DSN (see Context Management)
You can generate a random 32 byte base64 encoded secret in a Linux/macOS terminal with
head -c 32 /dev/urandom | base64
At start-up, the client will first check if the UBIRCH_SECRET32 environment variable exists and, if it does exist,
load the configuration from the environment variables. If the UBIRCH_SECRET32 environment variable is not set or
empty, the client will try to load the config.json-file from the working directory. If neither exist, the client will
abort and exit with status 1.
config.json:
{
"secret32": "<32 byte secret (base64 encoded)>"
}See example_config.json as an example for file-based configuration.
UBIRCH_SECRET32=<32 byte secret (base64 encoded)>See example.env as an example for environment-based configuration.
All further configuration parameters have default values, that can be changed as described under Optional Configurations.
The identity context is stored persistently in a database. The user can choose between connecting to a postgres database or using the local file system, i.e. SQLite.
config.json:"dbDriver": "<postgres | sqlite>",
- or environment variable:
UBIRCH_DB_DRIVER=<postgres | sqlite>
In order to connect the client to a postgres database, the DSN must be set in the configuration.
- add the following key-value pair to your
config.json:"dbDSN": "postgres://<username>:<password>@<hostname>:5432/<database>",
- or set the following environment variable:
UBIRCH_DB_DSN=postgres://<username>:<password>@<hostname>:5432/<database>
If the driver is set to sqlite, the client will by default create a SQLite database file sqlite.db in the mounted
volume upon first startup.
Alternatively, a custom path (relative to the mounted volume) and filename can be set in the configuration.
It is also possible to overwrite the default SQLite database configuration by appending a ? followed by a query string
to the filename. For more information about this, see https://pkg.go.dev/modernc.org/sqlite#Driver.Open
config.json:"dbDSN": "path/to/sqlite.db",
- or environment variable:
UBIRCH_DB_DSN=path/to/sqlite.db
The use of a SQLite database is appropriate in case the application is running on a system with limited space, like embedded devices, and only one or very few identities need to be managed.
When compared to postgreSQL, a drawback of SQLite is the performance while handling a high load of chaining requests for multiple identities at the same time.
| JSON | env | description | default value |
|---|---|---|---|
dbMaxOpenConns |
UBIRCH_DB_MAX_OPEN_CONNS |
maximum number of open connections to the database | 0 (unlimited) |
dbMaxIdleConns |
UBIRCH_DB_MAX_IDLE_CONNS |
maximum number of connections in the idle connection pool To disable idle connections, set to -1 |
10 |
dbConnMaxLifetimeSec |
DB_CONN_MAX_LIFETIME_SEC |
maximum amount of seconds a connection may be reused To disable connections being closed due to a connection's age, set to -1 |
600 |
dbConnMaxIdleTimeSec |
DB_CONN_MAX_IDLE_TIME_SEC |
maximum amount of seconds a connection may be idle To disable connections being closed due to a connection's idle time, set to -1 |
60 |
The UBIRCH client is able to handle multiple cryptographic identities. An identity has a universally unique identifier (UUID), private and public key, and an auth token. Before signing requests can be processed, a UUID with its auth token has to be registered at the UBIRCH client.
Registering a new UUID triggers an identity initialization, which consists of three parts:
- key pair generation (ECDSA)
- public key registration at the UBIRCH backend
- storing of the identity context in the database
Identities can be registered at the UBIRCH client in two ways:
- via configuration
- via HTTP request
If identity registration via HTTP requests is desired, the respective endpoint must be enabled and a static authentication token must be set in the configuration. This token is then used to authenticate requests against the identity registration endpoint.
- json:
"staticAuth": "<static auth token>",
"enableRegistrationEndpoint": true,- env:
UBIRCH_STATIC_AUTH=<static auth token>
UBIRCH_ENABLE_REGISTRATION_ENDPOINT=trueBefore registering a new identity at the UBIRCH client, the first step is to register the identity's UUID with the UBIRCH backend and acquire an authentication token for that identity.
A UUID can easily be generated in a Linux/macOS terminal with the
uuidgencommand.
- Create an account at the UBIRCH web UI and log in.
- Go to Things (in the menu on the left) and click the green
+ ADD NEW DEVICE-button. - Enter your UUID to the ID field and, optionally, a description. Then click on
register. - After successful registration, you can click on your UUID to open the settings and copy the "password" (which
looks like a UUID) from the
apiConfig. This is the UBIRCH backend authentication token for your device.
It is possible to declare identities (devices) in the configuration with a devices-map, which maps device UUIDs to
their authentication token.
- add the
devices-map to yourconfig.json-file:"devices": { "<UUID>": "<ubirch backend auth token>", "<another UUID>": "<another ubirch backend auth token>" }
- or as environment variable:
UBIRCH_DEVICES=<UUID>:<ubirch backend auth token>,<another UUID>:<another ubirch backend auth token>
Alternatively, the device UUIDs and their corresponding authentication tokens can also be set through a file
identities.json. See example: example_identities.json
Once the identities have been initialized successfully, their UUIDs and auth tokens are persistently stored in the connected database and can be removed from the configuration.
To get the latest multi-architecture image, check the releases and pull the latest release from Docker Hub using the release tag, e.g.:
docker pull ubirch/ubirch-client:v3.x.xTo start the multi-arch Docker image on any system, run:
docker run -v $(pwd):/data --network host ubirch/ubirch-client:v3.x.xThe configuration directory inside the docker container is /data. The docker image mounts the current
directory ($(pwd)) into the /data path to load the configuration file (if configuration is not set via environment
variables), and the TLS certificate and key files (if TLS is enabled).
It is also possible to pass an absolute path instead of $(pwd).
If the /data path is not used for either configuration file, SQLite DB, nor TLS cert files,
the -v $(pwd):/data parameter can be omitted.
Sending a registration request invokes the generation of a ECDSA key pair for signing UPPs. On success, the response contains an X.509 Certificate Signing Request in PEM format.
curl ${host}/register -X PUT \
-H "X-Auth-Token: ${staticAuth}" \
-H "Content-Type: application/json" \
-d '{"uuid":${device_uuid}, "password":${password}}' \
-i
The "password" is the UBIRCH backend authentication token.
If CSR creation via HTTP requests is desired, the respective endpoint must be enabled and a static authentication token must be set in the configuration. This token is then used to authenticate requests against the CSR creation endpoint.
- json:
"staticAuth": "<static auth token>",
"enableCSRCreationEndpoint": true,- env:
UBIRCH_STATIC_AUTH=<static auth token>
UBIRCH_ENABLE_CSR_CREATION_ENDPOINT=trueA CSR for an already registered identity can be retrieved from the CSR endpoint.
curl ${host}/${uuid}/csr -X GET \
-H "X-Auth-Token: ${staticAuth}" \
-i
The UBIRCH client provides HTTP endpoints for both original data and direct hash injection, i.e. the SHA256 digest of the original data. If the client receives original data, it will create a SHA256 hash before any further processing.
This means, UBIRCH will never see your original data. It also means that the original data will have to be stored independently in order to be able to verify it later.
When receiving a JSON data package, the UBIRCH client will sort the keys alphabetically and remove insignificant space characters before hashing.
Signing service endpoints require an authentication token, which corresponds to the UUID used in the request. The
token must be sent with the request header.
| Request Header | Description |
|---|---|
X-Auth-Token |
UBIRCH backend token related to <UUID> |
- chained
| Method | Path | Content-Type | Description |
|---|---|---|---|
| POST | /<UUID> |
application/octet-stream |
original data (binary) will be hashed, chained, signed, and anchored |
| POST | /<UUID> |
application/json |
original data (JSON data package) will be hashed, chained, signed, and anchored |
| POST | /<UUID>/hash |
application/octet-stream |
SHA256 hash (binary) will be chained, signed, and anchored |
| POST | /<UUID>/hash |
text/plain |
SHA256 hash (base64 string repr.) will be chained, signed, and anchored |
- no chain
| Method | Path | Content-Type | Description |
|---|---|---|---|
| POST | /<UUID>/anchor |
application/octet-stream |
original data (binary) will be hashed, signed, and anchored |
| POST | /<UUID>/anchor |
application/json |
original data (JSON data package) will be hashed, signed, and anchored |
| POST | /<UUID>/anchor/hash |
application/octet-stream |
SHA256 hash (binary) will be signed, and anchored |
| POST | /<UUID>/anchor/hash |
text/plain |
SHA256 hash (base64 string repr.) will be signed, and anchored |
Beside anchoring, the client can request hash update operations from the UBIRCH backend, i.e. disable, enable
and delete.
| Update Operation | Path (original data) | Path (hash) |
|---|---|---|
| disable | /<UUID>/disable |
/<UUID>/disable/hash |
| enable | /<UUID>/enable |
/<UUID>/enable/hash |
| delete | /<UUID>/delete |
/<UUID>/delete/hash |
Hash update requests to the UBIRCH backend must come from the same UUID that anchored said hash and be signed by the same private key that signed the anchoring request.
The client supports offline hash sealing, where the created UPP is not sent to the UBIRCH backend, but only returned as part of the HTTP response content.
- chained
| Method | Path | Content-Type | Description |
|---|---|---|---|
| POST | /<UUID>/offline |
application/octet-stream |
original data (binary) will be hashed, chained, and signed |
| POST | /<UUID>/offline |
application/json |
original data (JSON data package) will be hashed, chained, and signed |
| POST | /<UUID>/offline/hash |
application/octet-stream |
SHA256 hash (binary) will be chained and signed |
| POST | /<UUID>/offline/hash |
text/plain |
SHA256 hash (base64 string repr.) will be chained and signed |
- no chain
| Method | Path | Content-Type | Description |
|---|---|---|---|
| POST | /<UUID>/anchor/offline |
application/octet-stream |
original data (binary) will be hashed and signed |
| POST | /<UUID>/anchor/offline |
application/json |
original data (JSON data package) will be hashed and signed |
| POST | /<UUID>/anchor/offline/hash |
application/octet-stream |
SHA256 hash (binary) will be signed |
| POST | /<UUID>/anchor/offline/hash |
text/plain |
SHA256 hash (base64 string repr.) will be signed |
Response codes indicate the successful delivery of the UPP to the UBIRCH backend. Any code other than 200 should be
considered a failure. The client does not retry itself. A good approach to handle errors is to add a flag to the
original data storage that indicates whether the UBIRCH blockchain anchoring was successful and retry at a later point
if necessary.
The response body consists of either an error message, or a JSON map with
- the data hash
- the executed operation, i.e. chain, anchor, disable, enable or delete
- the UPP, which contains that data hash and was sent to the UBIRCH backend by the client
- the public key, that corresponds to the private key with which the UPP was signed
- the response from the UBIRCH backend
- the unique request ID
- a flag indicating if the backend response signature has been verified ( see Enable Backend Response Verification)
- a flag indicating if the backend response chain has been verified, i.e. if the niomon response UPP contains the
signature of the sent UPP in the
previous signaturefield ( see Enable Backend Response Verification)
{
"hash": "<base64 encoded data hash>",
"operation": "<chain | anchor | disable | enable | delete>",
"upp": "<base64 encoded UPP containing the data hash>",
"publicKey": "<base64 encoded raw public key>",
"response": {
"statusCode": <backend response status code (int)>,
"header": {<backend response header (map[string][]string)>},
"content": "<base64 encoded backend response content>"
},
"requestID": "<request ID (standard hex string representation)>",
"responseSignatureVerified": <true | false>,
"responseChainVerified": <true | false>
}
UPPs (such as the backend response content) are MessagePack formatted and can be decoded using an online tool like this MessagePack to JSON Converter.
| HTTP response status code | orig. data | hash | description |
|---|---|---|---|
| 200 - OK | x | x | success |
| 400 - Bad Request | x | x | unable to read request body |
| x | invalid content-type for original data (≠application/octet-stream or application/json) |
||
| x | unable to parse JSON request body (only for content-type application/json) |
||
| x | invalid content-type for hash (≠application/octet-stream or text/plain) |
||
| x | decoding hash failed (only for content-type text/plain) |
||
| x | invalid SHA256 hash size (≠32 bytes) | ||
| 401 - Unauthorized | x | x | unknown UUID |
| x | x | invalid auth token | |
| 403 - Forbidden | x | x | UPP signature verification failed (only for verification) |
| 404 - Not Found | x | x | invalid UUID |
| x | x | invalid operation (≠anchor / disable / enable / delete) |
|
| 500 - Internal Server Error | x | x | signing failed |
| 502 - Bad Gateway | x | x | sending request to UBIRCH backend failed |
| 503 - Service Temporarily Unavailable | x | x | service busy |
| 504 - Gateway Timeout | x | x | service was unable to produce a timely response |
Internally, the client sends a request to the UBIRCH Trust Service (Niomon) and forwards its response back to
the sender (i.e. the "response"-filed in the JSON response body of the client). If no errors occurred before sending
the request to Niomon, the client will simply forward the HTTP response status code that Niomon returned.
See the swagger documentation for Niomon error codes.
-
original data (JSON):
- anchor hash (chained)
curl localhost:8080/ba70ad8b-a564-4e58-9a3b-224ac0f0153f \ -H "X-Auth-Token: 32e325d5-b6a9-4800-b750-49c53b9350fc" \ -H "Content-Type: application/json" \ -d '{"id": "ba70ad8b-a564-4e58-9a3b-224ac0f0153f", "ts": 1585838578, "data": "1234567890"}' \ -i
- anchor hash (no chain)
curl localhost:8080/ba70ad8b-a564-4e58-9a3b-224ac0f0153f/anchor \ -H "X-Auth-Token: 32e325d5-b6a9-4800-b750-49c53b9350fc" \ -H "Content-Type: application/json" \ -d '{"id": "ba70ad8b-a564-4e58-9a3b-224ac0f0153f", "ts": 1585838578, "data": "1234567890"}' \ -i
- disable hash
curl localhost:8080/ba70ad8b-a564-4e58-9a3b-224ac0f0153f/disable \ -H "X-Auth-Token: 32e325d5-b6a9-4800-b750-49c53b9350fc" \ -H "Content-Type: application/json" \ -d '{"id": "ba70ad8b-a564-4e58-9a3b-224ac0f0153f", "ts": 1585838578, "data": "1234567890"}' \ -i
- enable hash
curl localhost:8080/ba70ad8b-a564-4e58-9a3b-224ac0f0153f/enable \ -H "X-Auth-Token: 32e325d5-b6a9-4800-b750-49c53b9350fc" \ -H "Content-Type: application/json" \ -d '{"id": "ba70ad8b-a564-4e58-9a3b-224ac0f0153f", "ts": 1585838578, "data": "1234567890"}' \ -i
- delete hash
curl localhost:8080/ba70ad8b-a564-4e58-9a3b-224ac0f0153f/delete \ -H "X-Auth-Token: 32e325d5-b6a9-4800-b750-49c53b9350fc" \ -H "Content-Type: application/json" \ -d '{"id": "ba70ad8b-a564-4e58-9a3b-224ac0f0153f", "ts": 1585838578, "data": "1234567890"}' \ -i
- anchor hash (chained)
-
direct data hash injection
- anchor hash (chained)
curl localhost:8080/ba70ad8b-a564-4e58-9a3b-224ac0f0153f/hash \ -H "X-Auth-Token: 32e325d5-b6a9-4800-b750-49c53b9350fc" \ -H "Content-Type: text/plain" \ -d "wp1WK/3z5yHiGBYUZReiMN4UVM2lUJzAtGg9kFtdy3A=" \ -i
- anchor hash (no chain)
curl localhost:8080/ba70ad8b-a564-4e58-9a3b-224ac0f0153f/anchor/hash \ -H "X-Auth-Token: 32e325d5-b6a9-4800-b750-49c53b9350fc" \ -H "Content-Type: text/plain" \ -d "wp1WK/3z5yHiGBYUZReiMN4UVM2lUJzAtGg9kFtdy3A=" \ -i
- disable hash
curl localhost:8080/ba70ad8b-a564-4e58-9a3b-224ac0f0153f/disable/hash \ -H "X-Auth-Token: 32e325d5-b6a9-4800-b750-49c53b9350fc" \ -H "Content-Type: text/plain" \ -d "wp1WK/3z5yHiGBYUZReiMN4UVM2lUJzAtGg9kFtdy3A=" \ -i
- enable hash
curl localhost:8080/ba70ad8b-a564-4e58-9a3b-224ac0f0153f/enable/hash \ -H "X-Auth-Token: 32e325d5-b6a9-4800-b750-49c53b9350fc" \ -H "Content-Type: text/plain" \ -d "wp1WK/3z5yHiGBYUZReiMN4UVM2lUJzAtGg9kFtdy3A=" \ -i
- delete hash
curl localhost:8080/ba70ad8b-a564-4e58-9a3b-224ac0f0153f/delete/hash \ -H "X-Auth-Token: 32e325d5-b6a9-4800-b750-49c53b9350fc" \ -H "Content-Type: text/plain" \ -d "wp1WK/3z5yHiGBYUZReiMN4UVM2lUJzAtGg9kFtdy3A=" \ -i
- anchor hash (chained)
Verification endpoints do not require an authentication token.
| Method | Path | Content-Type | Description |
|---|---|---|---|
| POST | /verify |
application/octet-stream |
verify hash of original data (binary) |
| POST | /verify |
application/json |
verify hash of original data (JSON data package) |
| POST | /verify/hash |
application/octet-stream |
verify hash (binary) |
| POST | /verify/hash |
text/plain |
verify hash (base64 string repr.) |
It is possible to verify that a UPP contains a given data hash and has a valid signature of a known identity without an internet connection.
Just like the standard verification, the offline verification endpoint expects the data or data hash in the request
body, but additionally expects the base64 representation of the UPP in the header X-Ubirch-UPP.
curl ${host}/verify/offline -X POST \
-H "X-Ubirch-UPP: liPEEO6QegJtYkRNpXhDJV/hplXEQGLoJh2SbSjQ7datOhGsWokSqO7Sckts1LGOxiBQ8SZoeql8ypLHpDLiYXQZ4MJ9vx1y/5rXwKl7VV+PG8eNEFAAxCCSV78s1WG2QGMS5vBVZOF51/JDHjjBqk/8x3VgpfL+dMRA7xvnoOSTNAYMJxItAkbzAMcD+YP2AX1bkfkV8EJUCNk7oI8DSPKmnNZ8gsb7fEv6DXGMTdFkGTtOvgpmkCNU7g==" \
-H "Content-Type: application/json" \
-d '{"id": "ee907a02-6d62-444d-a578-43255fe1a655", "ts": 1651746633, "data": "1234567890"}'curl ${host}/verify/offline/hash -X POST \
-H "X-Ubirch-UPP: liPEEO6QegJtYkRNpXhDJV/hplXEQGLoJh2SbSjQ7datOhGsWokSqO7Sckts1LGOxiBQ8SZoeql8ypLHpDLiYXQZ4MJ9vx1y/5rXwKl7VV+PG8eNEFAAxCCSV78s1WG2QGMS5vBVZOF51/JDHjjBqk/8x3VgpfL+dMRA7xvnoOSTNAYMJxItAkbzAMcD+YP2AX1bkfkV8EJUCNk7oI8DSPKmnNZ8gsb7fEv6DXGMTdFkGTtOvgpmkCNU7g==" \
-H "Content-Type: text/plain" \
-d "kle/LNVhtkBjEubwVWThedfyQx44wapP/Md1YKXy/nQ="A 200 response code indicates the successful verification of the data in the UBIRCH backend as well as a local
verification of the validity of the retrieved UPP.
The response body consists of either an error message, or a JSON map with
- the requested data hash,
- the UPP, which contains the requested data hash and was retrieved from the UBIRCH backend by the client,
- the UUID of the device from which the data originated,
- the public key of that device, which was used to verify the signature of the retrieved UPP
- possibly: a description of an occurred error (the
error-key is only present in case an error occurred)
{
"hash": "<base64 encoded requested data hash>",
"upp": "<base64 encoded UPP containing the requested data hash",
"uuid": "<standard hex string representation of the device UUID>",
"pubKey": "<base64 encoded public key used for signature verification>",
"error": "error message"
}If key de- and re-activation via HTTP requests is desired, the respective endpoint must be enabled and a static authentication token must be set in the configuration. This token is then used to authenticate requests against the deactivation endpoint.
- json:
"staticAuth": "<static auth token>",
"enableDeactivationEndpoint": true,- env:
UBIRCH_STATIC_AUTH=<static auth token>
UBIRCH_ENABLE_DEACTIVATION_ENDPOINT=trueA key can be deactivated with the following request. Signing requests for identities with deactivated key will fail with
status code 400.
curl ${host}/device/updateActive -X PUT \
-H "X-Auth-Token: ${staticAuth}" \
-H "Content-Type: application/json" \
-d '{"id":${device_uuid},"active":false}' \
-i
A deactivated key can be reactivated with the following request.
curl ${host}/device/updateActive -X PUT \
-H "X-Auth-Token: ${staticAuth}" \
-H "Content-Type: application/json" \
-d '{"id":${device_uuid},"active":true}' \
-i
The client exposes port :8080 for HTTP requests. When running the client locally, the default base address is:
http://localhost:8080
(or https://localhost:8080, if TLS is enabled)
Here is an example of a request to the client using CURL.
-
original data (JSON):
curl localhost:8080/<UUID> \ -H "X-Auth-Token: <AUTH_TOKEN>" \ -H "Content-Type: application/json" \ -d '{"id": "605b91b4-49be-4f17-93e7-f1b14384968f", "ts": 1585838578, "data": "1234567890"}' \ -i
-
direct data hash injection:
curl localhost:8080/<UUID>/hash \ -H "X-Auth-Token: <AUTH_TOKEN>" \ -H "Content-Type: text/plain" \ -d "bTawDQO7nnB+3h55/6VyQ+Tmd1RTV9R0cFcf7CRWzQQ=" \ -i
Every anchored data hash, and therefore the data, must be unique. The UBIRCH backend will reject the request with
response code 409 if the same hash has been sent previously.
Uniqueness can be achieved by adding a UUID and timestamp to the data before hashing. For example:
{
"id": "605b91b4-49be-4f17-93e7-f1b14384968f",
"ts": 1585838578,
"data": "1234567890"
}It is essential for the hashes to be reproducible in order to use them for verification of the data at a later time. Since the JSON format is non-deterministic, we need to define rules for converting it to a binary representation before calculating the hash.
If the client receives a JSON data package, it will generate a sorted compact rendering before calculating the hash, i.e. it will first create a string representation of the JSON formatted data where the keys are in alphabetical order and insignificant space characters were elided.
- JSON data package:
{ "id": "605b91b4-49be-4f17-93e7-f1b14384968f", "ts": 1585838578, "data": { "T": "26.250", "H": "65" } } - sorted compact rendering (string):
{"data":{"H":"65","T":"26.250"},"id":"605b91b4-49be-4f17-93e7-f1b14384968f","ts":1585838578} - SHA256 digest (base64):
uVXpb1vR8UlQnow/FoIcNbvcJ5bY1r2B+DZwe8AYSkE=
Floating-point numbers and integers greater than 253 are not allowed as values for the JSON data package!
If you need to sign floating-point numbers or numbers greater than 9,007,199,254,740,992, you can pass the string representation, e.g.
{ "float": "5.321", "bigNum": "9007199254740993" }
If no postgres DSN is set, the client defaults to the usage of a SQLite database with the following DSN
sqlite.db?_txlock=EXCLUSIVE&_pragma=journal_mode(WAL)&_pragma=synchronous(FULL)&_pragma=wal_autocheckpoint(4)&_pragma=wal_checkpoint(PASSIVE)&_pragma=journal_size_limit(32000)&_pragma=busy_timeout(100)
The default values can be overwritten by adding a custom SQLite DSN to the configuration.
- json:
"sqliteDSN": "<database file name>[?<query string>]",
- env:
UBIRCH_SQLITE_DSN=<database file name>[?<query string>]
A query string can optionally be appended to the database file name. If no query string is appended, the defaults from above will be used. More information about the query string can be found in the documentation of the sqlite library: https://pkg.go.dev/modernc.org/sqlite#Driver.Open
The env configuration refers to the UBIRCH backend environment. The default value is prod, which is the production
environment. For development, the environment may be set to demo, which is a test system that works like the
production environment, but stores data only in a blockchain test net. However, we suggest using prod in general as
demo may not always be available.
Note that the UUIDs must be registered at the according UBIRCH backend environment, i.e. https://console.demo.ubirch.com/.
To switch to the demo backend environment
- add the following key-value pair to your
config.json:"env": "demo"
- or set the following environment variable:
UBIRCH_ENV=demo
The UBIRCH Trust Service (niomon) when receiving a request responds with a UPP that is signed with the service's key
and contains the signature of the received UPP in the previous signature field. The verification of the response UPP
on client side, i.e. signature verification and chain check, can be enabled by setting the following flag.
| JSON | env | description | default value |
|---|---|---|---|
verifyNiomonResponse |
UBIRCH_VERIFY_NIOMON_RESPONSE |
true to enable response verification |
false |
If this flag is set, a request to the UBIRCH client will fail (502 - Bad Gateway) if the backend response can not be
verified. The JSON response body contains two fields responseSignatureVerified and responseChainVerified indicating
if the backend response signature verification and chain verification have been successful.
If backend response verification is disabled, these flags will always be false.
Note that the signature will be verified first. If the signature verification fails, chain verification will not be executed.
In some cases it is possible that the response UPP has a valid signature, but is not a chained UPP and therefore does
not contain the signature of the request UPP in the previous signature field. In those cases the response would
contain:
"responseSignatureVerified": true,
"responseChainVerified": false
The default identities, i.e. UUID and public key, for the dev, demo and prod environment are stored in files in
the server-identities directory and automatically loaded at application startup. The files are also
part of the docker image, so the user does not have to do anything if the default identities are valid.
However, the default identities can be overwritten by setting the niomon identity in the configuration.
config.json:"niomonIdentity": { "uuid": "<niomon UUID>", "publicKey": "<niomon public key bytes [base64]>" }
- environment variable:
UBIRCH_NIOMON_IDENTITY=uuid:<niomon UUID>,publicKey:<niomon public key bytes [base64]>
You can specify the TCP address for the server to listen on, in the form host:port. If empty, port 8080 is used.
- add the following key-value pair to your
config.json:"TCP_addr": ":8080",
- or set the following environment variable:
UBIRCH_TCP_ADDR=:8080
This describes how to configure the TCP port the client exposes, not the docker container. If you are running the client in a docker container, you can configure the exposed TCP port (
<host_port>) with the according argument when starting the client withdocker run:
docker run -p <host_port>:8080 ubirch/ubirch-client:vx.x.x
-
Create a self-signed TLS certificate
In order to serve HTTPS endpoints, you can run the following command to create a self-signed certificate with openssl. With this command it will be valid for ten years.
openssl req -x509 -newkey rsa:4096 -keyout key.pem -nodes -out cert.pem -days 3650
-
Enable TLS in configuration
- add the following key-value pair to your
config.json:"TLS": true
- or set the following environment variable:
UBIRCH_TLS=true
- add the following key-value pair to your
-
Set path and filename (optional)
By default, client will look for the
key.pemandcert.pemfiles in the working directory (same location as the config file), but it is possible to define a different location (relative to the working directory) and/or filename by adding them to your configuration file.- add the following key-value pairs to your
config.json:"TLSCertFile": "<path/to/TLS-cert-filename>", "TLSKeyFile": "<path/to/TLS-key-filename>"
- or set the following environment variables:
UBIRCH_TLS_CERTFILE=certs/cert.pem UBIRCH_TLS_KEYFILE=certs/key.pem
- add the following key-value pairs to your
Cross Origin Resource Sharing (CORS) can only be enabled if the UBIRCH backend environment is set to demo
and will be ignored on production stage.
To enable CORS and configure a list of allowed origins, i.e. origins a cross-domain request can be executed from,
- add the following key-value pairs to your
config.json:"CORS": true, "CORS_origins": ["<allowed origin>"]
- or set the following environment variables:
UBIRCH_CORS=true UBIRCH_CORS_ORIGINS=<allowed origin>
An origin may contain a wildcard (*) to replace 0 or more characters (e.g.: http://*.domain.com). Only one wildcard
can be used per origin.
Setting allowed origins is optional. If CORS is enabled, but no allowed origins are specified, the default value
is ["*"]
which means, all origins will be allowed.
The client creates X.509 Certificate Signing Requests (CSRs) for the public keys of the devices it is managing. The * Common Name* of the CSR subject is the UUID associated with the public key. The values for the Organization and * Country* of the CSR subject can be set through the configuration.
- add the following key-value pairs to your
config.json:"CSR_country": "<CSR Subject Country Name (2 letter code)>", "CSR_organization": "<CSR Subject Organization Name (e.g. company)>"
- or set the following environment variables:
UBIRCH_CSR_COUNTRY=<CSR Subject Country Name (2 letter code)> UBIRCH_CSR_ORGANIZATION=<CSR Subject Organization Name (e.g. company)>
To set the logging level to debug and so enable extended debug output,
- add the following key-value pair to your
config.json:"debug": true
- or set the following environment variable:
UBIRCH_DEBUG=true
By default, the log of the client is in JSON format. To change it to a (more human-eye-friendly) text format,
- add the following key-value pair to your
config.json:"logTextFormat": true
- or set the following environment variable:
UBIRCH_LOGTEXTFORMAT=true
To log the UUIDs of all known (registered) identities at startup,
- add the following key-value pair to your
config.json:"logKnownIdentities": true
- or set the following environment variable:
UBIRCH_LOG_KNOWN_IDENTITIES=true
The following request-related timeouts can be configured.
| json | env | description | default value |
|---|---|---|---|
identityServiceTimeoutMs |
IDENTITY_SERVICE_TIMEOUT_MS |
time limit for requests to the UBIRCH identity service in milliseconds | 10000 |
authServiceTimeoutMs |
AUTH_SERVICE_TIMEOUT_MS |
time limit for requests to the UBIRCH authentication service (niomon) in milliseconds | 2000 |
verifyServiceTimeoutMs |
VERIFY_SERVICE_TIMEOUT_MS |
time limit for requests to the UBIRCH verification service in milliseconds | 600 |
verificationTimeoutMs |
VERIFICATION_TIMEOUT_MS |
time limit for repeated attempts to verify a hash at the UBIRCH verification service in milliseconds | 2000 |
If a hash can not be verified by the UBIRCH verification service, a possible reason is that the verification was
attempted too early after anchoring and that a subsequent request will be successful. Because of this, the client
retries the verification if it fails with an HTTP response code 404.
verifyServiceTimeoutMsis the HTTP client timeout for each individual request to the verification serviceverificationTimeoutMsis the max. duration that verification will be attempted repeatedly
When the client receives a verification request for a UPP that was signed by an identity that is unknown to the client, i.e. an external identity, the default behaviour is to request the public key of that identity from the UBIRCH identity service in order to verify the signature locally.
To disable that behaviour and only verify UPPs that were signed by a known identity, i.e. an identity for which the public key exists in the database,
- add the following key-value pair to your
config.json:"verifyFromKnownIdentitiesOnly": true
- or set the following environment variable:
UBIRCH_VERIFY_FROM_KNOWN_IDENTITIES_ONLY=true
A simple workflow for setting up a system which locks out new data sources/identities, even if they are registered with the UBIRCH backend, could be the following:
During setup and test operation, set
verifyFromKnownIdentitiesOnlyto false and make sure to test verification from all intended sending devices. The client will pull all necessary public keys from the backend and save them locally.As soon as the setup phase is over, set
verifyFromKnownIdentitiesOnlyto true to lock out any new devices.
Version 1 of the client used a file-based context management. In order to update the client from v1 to a newer version (>v2) while keeping the existing context, the context can be migrated from the legacy context files into a database.
First, add the new mandatory configurations to your existing configuration.
To start the migration process, run the client with the command-line flag --migrate.
docker run -v $(pwd):/data --network host ubirch/ubirch-client:v3.0.0 /data --migrateAfter successful migration, the process will exit with status 0. In case of failed migration, the exit status is 1.
Lastly, the legacy context files can be deleted.
rm -rf keys.json keys.json.bck signaturesThe support for migration from file-based context into a database was dropped after version
v3.0.0.
-
Configuration
First, you will need a device UUID, that is registered with the UBIRCH backend, and a corresponding authentication token for that device. You will also need a secret to encrypt the locally stored private keys:
- Generate a UUID for your device. On Linux or macOS, simply enter
uuidgenin your terminal. Alternatively, you can use an online tool. - Get your device auth token:
- Create an account at the UBIRCH web UI and log in.
- Go to Things (in the menu on the left) and click the green
+ ADD NEW DEVICE-button. - Enter your UUID to the ID field and, optionally, a description. Then click on
register. - After successful registration, you can click on your UUID to open the settings and copy the "password"
from the
apiConfigas your device auth token.
- Generate a 32 byte secret in base64 format. You can enter
head -c 32 /dev/urandom | base64in a Linux/macOS terminal or encode 32 ASCII characters in an online base64 encoder.
Create a file
config.jsonin your working directory with the following content:{ "devices": { "<YOUR_DEVICE_UUID>": "<YOUR_DEVICE_AUTH_TOKEN>" }, "secret32": "<YOUR_32_BYTE_SECRET(base64 encoded)>", "dbDriver": "sqlite", "verifyNiomonResponse": true, "logTextFormat": true, "logKnownIdentities": true }- Replace
<YOUR_DEVICE_UUID>with your device UUID from step 1.1. - Replace
<YOUR_DEVICE_AUTH_TOKEN>with your device auth token from step 1.2. - Replace
<YOUR_32_BYTE_SECRET(base64 encoded)>with your secret from step 1.3.
Your
config.jsonshould now look like this:{ "devices": { "e5085a89-a881-4397-902e-a630f021afd8": "f83a888f-cbf8-4d78-82a2-3e3f253f181d" }, "secret32": "kwNWDv1K8z/T4Muk8La4uzoUl2Q1G923rmm7kA5NrIE=", "dbDriver": "sqlite", "verifyNiomonResponse": true, "logTextFormat": true, "logKnownIdentities": true } - Generate a UUID for your device. On Linux or macOS, simply enter
-
Run the client
To run the dockerized UBIRCH client, you will need to have Docker installed on your computer. Then enter the following two lines in the terminal in your working directory:
docker pull ubirch/ubirch-client:v3.0.0 docker run -v $(pwd):/data -p 8080:8080 ubirch/ubirch-client:v3.0.0When the client is first started, it will create an ECDSA key pair for your device and register the public key at the UBIRCH backend.
You should see a console output like this:
{"level":"info","message":"UBIRCH client (version=devbuild, revision=0000000)","time":"2022-10-31T07:36:28Z"} {"level":"info","message":"arg #1: /data","time":"2022-10-31T07:36:28Z"} {"level":"info","message":"loading configuration from file: /data/config.json","time":"2022-10-31T07:36:28Z"} time="2022-10-31 07:36:28.152 +0000" level=warning msg="identity registration endpoint disabled. To enable, set json:\"enableRegistrationEndpoint\" env:\"UBIRCH_ENABLE_REGISTRATION_ENDPOINT\" =true" time="2022-10-31 07:36:28.153 +0000" level=warning msg="CSR creation endpoint disabled. To enable, set json:\"enableCSRCreationEndpoint\" env:\"UBIRCH_ENABLE_CSR_CREATION_ENDPOINT\" =true" time="2022-10-31 07:36:28.154 +0000" level=warning msg="key deactivation endpoint disabled. To enable, set json:\"enableDeactivationEndpoint\" env:\"ENABLE_DEACTIVATION_ENDPOINT\" =true" time="2022-10-31 07:36:28.155 +0000" level=info msg="UBIRCH backend environment: prod" time="2022-10-31 07:36:28.155 +0000" level=info msg="initializing sqlite database connection" time="2022-10-31 07:36:28.285 +0000" level=info msg="0 known internal identities (signing and verification)" time="2022-10-31 07:36:28.287 +0000" level=info msg="0 known external identities (verification only)" time="2022-10-31 07:36:28.296 +0000" level=info msg="e5085a89-a881-4397-902e-a630f021afd8: initializing identity" time="2022-10-31 07:36:28.750 +0000" level=info msg="e5085a89-a881-4397-902e-a630f021afd8: key certificate: {\"pubKeyInfo\":{\"algorithm\":\"ecdsa-p256v1\",\"created\":\"2022-10-31T07:36:28.739Z\",\"hwDeviceId\":\"e5085a89-a881-4397-902e-a630f021afd8\",\"pubKey\":\"//3eUKJOrGaYCoPBOMMUquX3cn+EXHMqCKu7IJWu/Xs1x7oJ4HU6LLWksf8toG0ir1VreFo8A5tJEGvxmQbe0w==\",\"pubKeyId\":\"//3eUKJOrGaYCoPBOMMUquX3cn+EXHMqCKu7IJWu/Xs1x7oJ4HU6LLWksf8toG0ir1VreFo8A5tJEGvxmQbe0w==\",\"validNotAfter\":\"2032-10-28T07:36:28.739Z\",\"validNotBefore\":\"2022-10-31T07:36:28.739Z\"},\"signature\":\"GpGZzgTtvZ0InzvqNlNh3CEMkNxLY+G/og1qBe8J/ouhHs4OS5us1JEenzyym+cKJaHAaNYMscZA3jdrFxnZ+w==\"}" time="2022-10-31 07:36:30.231 +0000" level=info msg="e5085a89-a881-4397-902e-a630f021afd8: creating CSR" time="2022-10-31 07:36:30.257 +0000" level=info msg="e5085a89-a881-4397-902e-a630f021afd8: CSR [PEM]: -----BEGIN CERTIFICATE REQUEST-----\nMIIBDjCBtAIBADBSMQswCQYDVQQGEwJERTEUMBIGA1UEChMLdWJpcmNoIEdtYkgx\nLTArBgNVBAMTJGU1MDg1YTg5LWE4ODEtNDM5Ny05MDJlLWE2MzBmMDIxYWZkODBZ\nMBMGByqGSM49AgEGCCqGSM49AwEHA0IABP/93lCiTqxmmAqDwTjDFKrl93J/hFxz\nKgiruyCVrv17Nce6CeB1Oiy1pLH/LaBtIq9Va3haPAObSRBr8ZkG3tOgADAKBggq\nhkjOPQQDAgNJADBGAiEA8UANAK6JLUk+TQMZ4FtWsJQJT/dWyhonF/ZbUuV03n0C\nIQCQj7U/la0wf9FuBYvn813sQ3FE/P1E43fwLni0pxTH2g==\n-----END CERTIFICATE REQUEST-----\n" time="2022-10-31 07:36:30.270 +0000" level=info msg="starting HTTP server"
That means the client is running and ready!
If you want, you can now go back to the UBIRCH web UI and see the freshly registered public key under
PublicKeys.
WARNING
The client stores the encrypted signing keys in a local file
sqlite.db, which will be created in the working directory upon first start-up. Do not delete this file, as our backend will not accept the registration of a new key once a device already has a registered key.
-
Seal your data
The client is now listening for HTTP requests on port
8080. You can send either...- JSON data packages to the
/<UUID>-endpoint withContent-Type: application/json-header, or - SHA256 hashes of your data to the
/<UUID>/hash-endpoint withContent-Type: application/octet-stream-header.
Since the data hash for every UPP must be unique, ensure that the body of each request has a unique content. You can do that, for example, by adding an ID and a timestamp to the JSON data package. For more information see Uniqueness of hashes.
Floating-point numbers and integers greater than 253 are not allowed as values for the JSON data package!
You also need to set the
X-Auth-Token-header with your UBIRCH backend auth token from step 1.Here is an example of how a request to the client would look like using
CURL:curl localhost:8080/<YOUR_DEVICE_UUID> \ -H "X-Auth-Token: <YOUR_DEVICE_AUTH_TOKEN>" \ -H "Content-Type: application/json" \ -d '{"id": "e5085a89-a881-4397-902e-a630f021afd8", "ts": 1667202152, "data": "1234567890"}' \ -i -s
Insert
<YOUR_DEVICE_UUID>and<YOUR_DEVICE_AUTH_TOKEN>and a request body with your own unique content to ensure a unique hash! In case of hash collision, the request will fail with status code409.When the client receives a request, it hashes the data from the request body and creates a chained Ubirch Protocol Package (UPP) with the data hash as payload. The UPP will be signed with the private key of the device and sent to the UBIRCH backend. There, the signature will be verified with the previously registered public key.
The console output of the client should look like this:
time="2022-10-31 07:44:48.178 +0000" level=info msg="create UPP: uuid: e5085a89-a881-4397-902e-a630f021afd8, hash: 5snVjoqWbqLbhABMD1L5OguJyvcsxbJOECQSurDqs5k=, operation: chain, offline: false" time="2022-10-31 07:44:48.909 +0000" level=info msg="e5085a89-a881-4397-902e-a630f021afd8: request ID: 3c6e0e19-63b6-4d42-b316-3f46481f14cc"
Take note of the hash for verification.
If your request was successful, you'll get the HTTP response code
200.The HTTP response body is a JSON map. It contains the data hash, the UPP, which was created by the client and sent to the UBIRCH backend, and the UBIRCH backend response. The content of the UBIRCH backend response is a UPP as well. UPPs are in MessagePack format (base64 encoded) and can be decoded using, for example, this MessagePack to JSON Converter. You can read more about UPPs here.
{ "hash": "5snVjoqWbqLbhABMD1L5OguJyvcsxbJOECQSurDqs5k=", "operation": "chain", "upp": "liPEEOUIWomogUOXkC6mMPAhr9jEQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAxCDmydWOipZuotuEAEwPUvk6C4nK9yzFsk4QJBK6sOqzmcRA0fYMXgkdjoAOPE9jXV/gfBxb9kl9WierPozz+usi+WLUNTD98al0QX6TWB3i1pg43XDL0/lHf8E+4AhWfFFlCQ==", "publicKey": "//3eUKJOrGaYCoPBOMMUquX3cn+EXHMqCKu7IJWu/Xs1x7oJ4HU6LLWksf8toG0ir1VreFo8A5tJEGvxmQbe0w==", "response": { "statusCode": 200, "header": { "Content-Length": [ "187" ], "Content-Type": [ "application/octet-stream" ], "Date": [ "Mon, 31 Oct 2022 07:44:48 GMT" ], "Server": [ "ubirch-trust-service/1.0" ], "Strict-Transport-Security": [ "max-age=15552000; includeSubDomains; preload" ] }, "content": "liPEEBCy4aRWs0//mtrMjCD5MBbEQNH2DF4JHY6ADjxPY11f4HwcW/ZJfVonqz6M8/rrIvli1DUw/fGpdEF+k1gd4taYON1wy9P5R3/BPuAIVnxRZQkAxCA8bg4ZY7ZNQrMWP0ZIHxTMAAAAAAAAAAAAAAAAAAAAAMRAtCx79HokXAELQRbiEoE9YVPLfx4Zdh9fC93QO4X4e60HXseQUdVFtbuQBQiz2yqHBuoQyMQVsu2fBPreNihifA==" }, "requestID": "3c6e0e19-63b6-4d42-b316-3f46481f14cc", "responseSignatureVerified": true, "responseChainVerified": true }If you get a response code other than
200, it means that something went wrong. In this case the client will respond with an error message. You can also find error messages in the console output of the client. - JSON data packages to the
-
To stop the client, press
ctrl+c.
You should now be able to see that your UPP was received and verified by the UBIRCH backend under Your Things in the UBIRCH web UI.
To look at the anchoring of your data hash in public blockchains, go to the UBIRCH web UI verification page and enter your data hash in the search field.
It is also possible to verify the hash using the API by sending a POST request with the hash you wish to verify to the UBIRCH verification service:
https://verify.prod.ubirch.com/api/upp/verify/anchor
e.g.
curl -d '5snVjoqWbqLbhABMD1L5OguJyvcsxbJOECQSurDqs5k=' https://verify.prod.ubirch.com/api/upp/verify/anchor
This endpoint checks if the UPP, which contains the data hash has arrived correctly and was verifiable, gives information about the chain (previous UPP) as well as blockchain info on the time frame (the upper and lower bounds) when the data was received, i.e. the closest blockchain transactions before and after the data was received by the UBIRCH backend (anchors).
If the verification was successful, the service will send a 200 response with a JSON formatted body like this:
{
"upp": "liPEEOUIWomogUOXkC6mMPAhr9jEQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAxCDmydWOipZuotuEAEwPUvk6C4nK9yzFsk4QJBK6sOqzmcRA0fYMXgkdjoAOPE9jXV/gfBxb9kl9WierPozz+usi+WLUNTD98al0QX6TWB3i1pg43XDL0/lHf8E+4AhWfFFlCQ==",
"prev": null,
"anchors": [
{
"label": "PUBLIC_CHAIN",
"properties": {
"timestamp": "2022-10-31T07:45:21.625Z",
"hash": "5d9f841fed2d4b7693b91586d8b7312d681dcfb8e070ba7153a8032835bb109d",
"public_chain": "IOTA_MAINNET_IOTA_MAINNET_NETWORK",
"prev_hash": "65cf37759c94accebf1d7344a9d5a5bcafef3b5198772fff8b63ed7458ee155d5735e2e7d9eb6f09c7517115038dc72b15d5ed997a55bf7b59ccef708636c394"
}
},
{
"label": "PUBLIC_CHAIN",
"properties": {
"timestamp": "2022-10-31T07:47:03.100Z",
"hash": "0xe87fdf636a781638ff6e1be573172699986f82ea2f8d06595f39a39e433c420b",
"public_chain": "ETHEREUM-CLASSIC_MAINNET_ETHERERUM_CLASSIC_MAINNET_NETWORK",
"prev_hash": "3f2129956066cdb62f690bb080d7c2c029d64a657db88ec3b043f0b9472400f946f9d28c4af49c6005424f33dc1349140df9a2020fa470c58c955d80dac297e5"
}
}
]
}Note that the first UPP to be anchored will not have a 'previous' package to be chained to. The
"prev"value will therefore benull.
It can take up to 10 minutes before the anchoring in public blockchains can be verified, but there is also an endpoint for a quick check, that verifies that the hash was received by the UBIRCH backend:
https://verify.prod.ubirch.com/api/upp
... and another endpoint, which additionally checks the chain, but not the blockchain anchor:
https://verify.prod.ubirch.com/api/upp/verify
A 404 response with an empty body means the hash could not be verified (yet).
You can find more information on the services and functionalities of the UBIRCH backend in the developer documentation.
Copyright (c) 2019-2020 ubirch GmbH
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.