Skip to content

Commit

Permalink
s3-satis
Browse files Browse the repository at this point in the history
  • Loading branch information
kduma committed Dec 18, 2023
1 parent ec7455f commit 9b16e7c
Show file tree
Hide file tree
Showing 2 changed files with 308 additions and 0 deletions.
307 changes: 307 additions & 0 deletions content/1.systems/2.serverless-satis/1.s3-satis.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,307 @@
---
title: `s3-satis` tool
list_title: `s3-satis`
type: app
platform: PHP
active: true
github: https://github.com/kduma-OSS/CLI-s3-satis
---

# `s3-satis` tool

Tool to generate a [Composer](https://getcomposer.org/){:target="_blank"} PHP packages repository
(based on [Satis - static Composer repository generator](https://github.com/composer/satis){:target="_blank"})
and synchronize generated repository with a [Amazon S3](https://aws.amazon.com/s3/){:target="_blank"}
(or compatible) bucket.


:button-link[GitHub Repository]{icon="bxl:github" href="https://github.com/kduma-OSS/CLI-s3-satis" blank}
 
:button-link[Download s3-satis.phar]{icon="material-symbols:cloud-download" href="https://github.com/kduma-OSS/CLI-s3-satis/releases/latest/download/s3-satis.phar" blank}


## Setup

You can install `s3-satis` tool in four ways:

- As a [Docker](https://www.docker.com/){:target="_blank"} container -using image [ghcr.io/kduma-oss/s3-satis](https://github.com/kduma-OSS/CLI-s3-satis/pkgs/container/s3-satis){:target="_blank"}
- Global composer installation - tool will be available globally as `s3-satis` command
- You can download phar executable file from [GitHub Releases](https://github.com/kduma-OSS/CLI-s3-satis/releases/latest){:target="_blank"} page
- Download source code form [GitHub](https://github.com/kduma-OSS/CLI-s3-satis){:target="_blank"} to run

::code-group
```bash [Docker]
docker run \
--volume "${PWD}/satis.json:/usr/src/satis/satis.json" \
--volume "${COMPOSER_HOME:-$HOME/.composer}:/root/.composer/" \
--env S3_ACCESS_KEY_ID=<S3_ACCESS_KEY_ID> \
--env S3_SECRET_ACCESS_KEY=<S3_SECRET_ACCESS_KEY> \
--env S3_REGION=<S3_REGION> \
--env S3_BUCKET=<S3_BUCKET> \
--env S3_ENDPOINT=<S3_ENDPOINT> \
--env S3_USE_PATH_STYLE_ENDPOINT=<S3_USE_PATH_STYLE_ENDPOINT> \
ghcr.io/kduma-oss/s3-satis:latest build /usr/src/satis/satis.json
```
```bash [Composer Global]
composer global require kduma/s3-satis
```
```bash [phar executable]
wget https://github.com/kduma-OSS/CLI-pdf-scan-splitter/releases/latest/download/s3-satis.phar
chmod +x s3-satis.phar
mv s3-satis.phar /usr/local/bin/s3-satis
```
```bash [Sources]
git clone https://github.com/kduma-OSS/CLI-pdf-scan-splitter.git s3-satis
cd s3-satis
composer install
```
::

## Requirements

You need to have prepared a S3 (or compatible) bucket with a provider of your choice,
and generated an access key and secret key with write access for created bucket.

Usually you need also to have authorized your `composer` installation to access your private repositories.
If you want to create a repository consisting of publicly available packages on GitHub,
you will probably also need to generate a GitHub access token, because of GitHub API rate limits.

## Usage

First prepare a `satis.json` file with your repository configuration.
This tool is based on [Satis - static Composer repository generator](https://github.com/composer/satis){:target="_blank"},
so please check [Satis documentation](https://getcomposer.org/doc/articles/handling-private-packages-with-satis.md){:target="_blank"}
for configuration options.

::code-group
```json [Simple satis.json example]
{
"name": "my/repo",

"homepage": "https://satis.example.com",

"repositories": [
{ "type": "vcs", "url": "https://github.com/laravel/framework" }
],

"require-all": true
}
```
```json [Simple satis.json example with archiving]
{
"name": "my/repo",

"homepage": "https://satis.example.com",

"repositories": [
{ "type": "vcs", "url": "https://github.com/laravel/framework" }
],

"require-all": true,

"archive": {
"directory": "dist"
}
}
```
::

Second, configure your environment variables (or `.env` file) with your S3 bucket credentials:
```env
S3_ACCESS_KEY_ID=
S3_SECRET_ACCESS_KEY=
S3_REGION=us-east-1
S3_BUCKET=
S3_ENDPOINT=
S3_USE_PATH_STYLE_ENDPOINT=false
```

Then run `s3-satis` tool to generate repository and upload it to S3 bucket:
```bash
s3-satis build satis.json
```


## Extensions

s3-satis tool provides a few extensions to Satis functionality.
You can enable them by adding them to your `satis.json` file, in `s3-satis.plugins` section,
or by providing them as a command line argument like: `s3-satis build --plugin=my_plugin_name:param1=value1:param2=value2`.

In `satis.json` you can provide them like this:
```json
{
"s3-satis": {
"plugins": {
"my_plugin_name": {
"enabled": true,
"param1": "value1",
"param2": "value2"
}
}
}
}
```
or, if you don't need to provide any parameters, you can omit configuration and only provide boolean value:
```json
{
"s3-satis": {
"plugins": {
"my_plugin_name": true
}
}
}
```

### Cache Extension: `cache`

This extension adds an ability to cache downloaded metadata files in local directory and not download them from S3 bucket on every run.
In `path` parameter you can provide a path to directory where cache files will be stored and retrieved, and if you provide `temp` as a value - files will be stored in system temp directory.
If you provide `copy` parameter as `true`, files will be copied from cache directory, otherwise they will be moved - so if for any reason script crashes during execution, the cache will be left empty.

```json
{
"s3-satis": {
"plugins": {
"cache": {
"enabled": true,
"path": "temp",
"copy": false
}
}
}
}
```

### File Restrictions Map Generator Extension: `file-restrictions-map-generator`

This extension will generate a files in repository root directory, which will contain a version and package identifiers of archives in repository.

It there is a file `dist/vendor/package/package-xxxxxxx.tar` generated, and it has a version of `1.0.0.0`
in corresponding `packages.json` file, there will be generated `.tags/dist/vendor/package/package-xxxxxxx.tar.json` file with following content:

```json
[
"vendor/package:1.0.0.0",
"vendor/package:1.0.0.x",
"vendor/package:1.0.x",
"vendor/package:1.x"
]
```

Or if it is not released version, but a branch, the file will be generated with `dev-{branch}` tag:
```json
[
"vendor/package:dev-master"
]
```

You can use this extension with CloudFlare Worker to provide a selective access to packages in your private repository.

```json
{
"s3-satis": {
"plugins": {
"file-restrictions-map-generator": true
}
}
}
```

### RemoveFieldsFromJsonExtension: `remove-fields-from-json`

When enabled, this extension will remove specified fields from exported json files.
Only fields listed bellow are supported, other fields will be ignored.

```json
{
"s3-satis": {
"plugins": {
"remove-fields-from-json": {
"enabled": true,
"remove": [
"source",
"authors",
"homepage",
"support"
]
}
}
}
}
```

### SkipRemoteVersionsExtension: `skip-remote-versions`

When enabled, this extension will filter out all entries in exported json files which doesn't provide local download link (e.g. are linked directly to GitHub repository)

```json
{
"s3-satis": {
"plugins": {
"skip-remote-versions": true
}
}
}
```

### Debugging Extensions

There are few extensions which can be used for debugging purposes:

#### SkipStepAfterHooksExtension: `skip-step-after-hook`

Allows to skip execution of specified builtin steps.

```json
{
"s3-satis": {
"plugins": {
"skip-step-after-hook": {
"enabled": true,
"skip": [
"BEFORE_INITIAL_CLEAR_TEMP_DIRECTORY",
"BEFORE_CREATE_TEMP_DIRECTORY",
"BEFORE_DOWNLOAD_FROM_S3",
"BEFORE_BUILD_SATIS_REPOSITORY",
"BEFORE_UPLOAD_TO_S3",
"BEFORE_REMOVE_MISSING_FILES_FROM_S3",
"BEFORE_FINAL_CLEAR_TEMP_DIRECTORY"
]
}
}
}
}
```

#### PauseAtHooksExtension: `pause-at-hook`

Allows to pause execution of at specified hook points. Press enter to continue execution.

```json
{
"s3-satis": {
"plugins": {
"pause-at-hook": {
"enabled": true,
"pause": [
"BEFORE_INITIAL_CLEAR_TEMP_DIRECTORY",
"AFTER_INITIAL_CLEAR_TEMP_DIRECTORY",
"BEFORE_CREATE_TEMP_DIRECTORY",
"AFTER_CREATE_TEMP_DIRECTORY",
"BEFORE_DOWNLOAD_FROM_S3",
"AFTER_DOWNLOAD_FROM_S3",
"BEFORE_BUILD_SATIS_REPOSITORY",
"AFTER_BUILD_SATIS_REPOSITORY",
"BEFORE_UPLOAD_TO_S3",
"AFTER_UPLOAD_TO_S3",
"BEFORE_REMOVE_MISSING_FILES_FROM_S3",
"AFTER_REMOVE_MISSING_FILES_FROM_S3",
"BEFORE_FINAL_CLEAR_TEMP_DIRECTORY",
"AFTER_FINAL_CLEAR_TEMP_DIRECTORY"
]
}
}
}
}
```
1 change: 1 addition & 0 deletions content/1.systems/2.serverless-satis/_dir.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
title: Serverless Satis

0 comments on commit 9b16e7c

Please sign in to comment.