Skip to content

Commit

Permalink
docs: new documentation website using docusaurus (#99)
Browse files Browse the repository at this point in the history
* docs: new documentation website

* docs: add quick links

* chore: add missing yarn.lock

* chore: apply codacy rules to readme files

* chore: exclude website from codacy
  • Loading branch information
Gui ⚡️ Guilherme Ferreira authored Feb 6, 2023
1 parent 985fcc5 commit d6fe944
Show file tree
Hide file tree
Showing 26 changed files with 29,979 additions and 129 deletions.
3 changes: 3 additions & 0 deletions .codacy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
---
exclude_paths:
- "website/**"
38 changes: 38 additions & 0 deletions .github/workflows/deploy-website.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Deploy to GitHub Pages

on:
push:
branches:
- main
- release-*

workflow_dispatch:

jobs:
deploy:
name: Deploy to GitHub Pages
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3

- uses: actions/setup-node@v3
with:
node-version: 18
cache: yarn
cache-dependency-path: website/yarn.lock


- name: Install dependencies
working-directory: ./website
run: yarn install --frozen-lockfile

- name: Build website
working-directory: ./website
run: yarn build

- name: Deploy to GitHub Pages
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./website/build
25 changes: 25 additions & 0 deletions .github/workflows/test-deploy-website.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: Test deployment

on:
pull_request:

jobs:
test-deploy:
name: Test deployment
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- uses: actions/setup-node@v3
with:
node-version: 18
cache: yarn
cache-dependency-path: website/yarn.lock

- name: Install dependencies
working-directory: ./website
run: yarn install --frozen-lockfile

- name: Test build website
working-directory: ./website
run: yarn build
156 changes: 27 additions & 129 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,137 +1,30 @@
# KafkaFlow Retry Extensions
[![Codacy Badge](https://app.codacy.com/project/badge/Grade/2a86b45f0ec2487fb63dfd581071465a)](https://www.codacy.com/gh/Farfetch/kafkaflow-retry-extensions/dashboard?utm_source=github.com&utm_medium=referral&utm_content=Farfetch/kafkaflow-retry-extensions&utm_campaign=Badge_Grade)

KafkaFlow Retry is a .NET framework to implement easy resilience on consumers.

KafkaFlow Retry is an extension of [Kafka Flow](https://github.com/Farfetch/kafkaflow).

## Resilience policies

|Policy| Description | Aka| Required Packages|
| ------------- | ------------- |:-------------: |------------- |
|**Simple Retry** <br/>(policy family)<br/><sub>([quickstart](#simple)&nbsp;;&nbsp;deep)</sub>|Many faults are transient and may self-correct after a short delay.| "Maybe it's just a blip" | KafkaFlow.Retry |
|**Forever Retry**<br/>(policy family)<br/><sub>([quickstart](#forever)&nbsp;;&nbsp;deep)</sub>|Many faults are semi-transient and may self-correct after multiple retries. | "Never give up" | KafkaFlow.Retry |
|**Durable Retry**<br/><sub>([quickstart](#durable)&nbsp;;&nbsp;deep)</sub>|Beyond a certain amount of retries and wait, you want to keep processing next-in-line messages but you can't loss the current offset message. As persistance databases, MongoDb or SqlServer are available. And you can manage in-retry messages through HTTP API.| "I can't stop processing messages but I can't loss messages" | KafkaFlow.Retry <br/>KafkaFlow.Retry.API<br/><br/>KafkaFlow.Retry.SqlServer<br/>or<br/>KafkaFlow.Retry.MongoDb |

## Installing via NuGet
Install packages related to your context. The Core package is required for all other packages.

## Requirements
**.NET Core 2.1 and later using Hosted Service**

## Packages

|Name |nuget.org|
|---------------------------------|----|
|KafkaFlow.Retry|[![Nuget Package](https://img.shields.io/nuget/v/KafkaFlow.Retry.svg?logo=nuget)](https://www.nuget.org/packages/KafkaFlow.Retry/) ![Nuget downloads](https://img.shields.io/nuget/dt/KafkaFlow.Retry.svg)
|KafkaFlow.Retry.API|[![Nuget Package](https://img.shields.io/nuget/v/KafkaFlow.Retry.API.svg?logo=nuget)](https://www.nuget.org/packages/KafkaFlow.Retry.API/) ![Nuget downloads](https://img.shields.io/nuget/dt/KafkaFlow.Retry.API.svg)
|KafkaFlow.Retry.MongoDb|[![Nuget Package](https://img.shields.io/nuget/v/KafkaFlow.Retry.MongoDb.svg?logo=nuget)](https://www.nuget.org/packages/KafkaFlow.Retry.MongoDb/) ![Nuget downloads](https://img.shields.io/nuget/dt/KafkaFlow.Retry.MongoDb.svg)
|KafkaFlow.Retry.SqlServer|[![Nuget Package](https://img.shields.io/nuget/v/KafkaFlow.Retry.SqlServer.svg?logo=nuget)](https://www.nuget.org/packages/KafkaFlow.Retry.SqlServer/) ![Nuget downloads](https://img.shields.io/nuget/dt/KafkaFlow.Retry.SqlServer.svg)

## Core package
Install-Package KafkaFlow.Retry

## HTTP API package
Install-Package KafkaFlow.Retry.API

## MongoDb package
Install-Package KafkaFlow.Retry.MongoDb

## SqlServer package
Install-Package KafkaFlow.Retry.SqlServer

## Usage &ndash; Simple and Forever retries policies
### Simple

```csharp
.AddMiddlewares(
middlewares => middlewares // KafkaFlow middlewares
.RetrySimple(
(config) => config
.Handle<ExternalGatewayException>() // Exceptions to be handled
.TryTimes(3)
.WithTimeBetweenTriesPlan((retryCount) =>
TimeSpan.FromMilliseconds(Math.Pow(2, retryCount)*1000) // exponential backoff
)
)
```

### Forever

```csharp
.AddMiddlewares(
middlewares => middlewares // KafkaFlow middlewares
.RetryForever(
(config) => config
.Handle<DatabaseTimeoutException>() // Exceptions to be handled
.WithTimeBetweenTriesPlan(
TimeSpan.FromMilliseconds(500),
TimeSpan.FromMilliseconds(1000)
)
)

```

## Usage &ndash; Durable retry policy

### Durable

```csharp
.AddMiddlewares(
middlewares => middlewares // KafkaFlow middlewares
.RetryDurable(
config => config
.Handle<NonBlockingException>() // Exceptions to be handled
.WithMessageType(typeof(TestMessage)) // Message type to be consumed
.WithEmbeddedRetryCluster( // Retry consumer config
cluster,
config => config
.WithRetryTopicName("test-topic-retry")
.WithRetryConsumerBufferSize(4)
.WithRetryConsumerWorkersCount(2)
.WithRetryConusmerStrategy(RetryConsumerStrategy.GuaranteeOrderedConsumption)
.WithRetryTypedHandlers(
handlers => handlers
.WithHandlerLifetime(InstanceLifetime.Transient)
.AddHandler<Handler>()
).Enabled(true)
)
.WithQueuePollingJobConfiguration( // Polling configuration
config => config
.WithId("custom_search_key")
.WithCronExpression("0 0/1 * 1/1 * ? *")
.WithExpirationIntervalFactor(1)
.WithFetchSize(10)
.Enabled(true)
)
.WithMongoDbDataProvider( // Persistence configuration
mongoDbconnectionString,
mongoDbdatabaseName,
mongoDbretryQueueCollectionName,
mongoDbretryQueueItemCollectionName
)
.WithRetryPlanBeforeRetryDurable( // Chained simple retry before triggering durable
config => config
.TryTimes(3)
.WithTimeBetweenTriesPlan(
TimeSpan.FromMilliseconds(250),
TimeSpan.FromMilliseconds(500),
TimeSpan.FromMilliseconds(1000))
.ShouldPauseConsumer(false)
)
)
)
```

See the [setup page](https://github.com/Farfetch/kafkaflow-retry-extensions/wiki/Setup) and [samples](https://github.com/Farfetch/kafkaflow-retry-extensions/tree/main/samples) for more details
# KafkaFlow Retry Extensions · [![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/farfetch/kafkaflow-retry-extensions/blob/main/LICENSE) [![nuget version](https://img.shields.io/nuget/v/kafkaflow.retry.svg?style=flat)](https://www.nuget.org/packages/KafkaFlow.Retry/) ![Build Main](https://github.com/Farfetch/kafkaflow-retry-extensions/workflows/Build/badge.svg?branch=main) [![Codacy Badge](https://app.codacy.com/project/badge/Grade/2a86b45f0ec2487fb63dfd581071465a)](https://www.codacy.com/gh/Farfetch/kafkaflow-retry-extensions/dashboard?utm_source=github.com&utm_medium=referral&utm_content=Farfetch/kafkaflow-retry-extensions&utm_campaign=Badge_Grade)

## Introduction

🔁 KafkaFlow Retry is an extension to [KafkaFlow](https://github.com/Farfetch/kafkaflow) that implements resilience on Apache Kafka consumers.

Want to give it a try? Check out our [Quickstart](https://farfetch.github.io/kafkaflow-retry-extensions/docs/getting-started/quickstart)!

### Resilience policies

| Policy | Description | Aka | Required Packages |
| ------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------: | ----------------- |
| **Simple Retry** <br/>(policy family)<br/><sub>([quickstart](#simple) ; deep)</sub> | Many faults are transient and may self-correct after a short delay. | "Maybe it's just a blip" | KafkaFlow.Retry |
| **Forever Retry**<br/>(policy family)<br/><sub>([quickstart](#forever) ; deep)</sub> | Many faults are semi-transient and may self-correct after multiple retries. | "Never give up" | KafkaFlow.Retry |
| **Durable Retry**<br/><sub>([quickstart](#durable) ; deep)</sub> | Beyond a certain amount of retries and waiting, you want to keep processing next-in-line messages but you can't lose the current offset message. As persistence databases, MongoDb or SqlServer is available. And you can manage in-retry messages through HTTP API."I can't stop processing messages but I can't lose messages" | KafkaFlow.Retry <br/>KafkaFlow.Retry.API<br/><br/>KafkaFlow.Retry.SqlServer<br/>or<br/>KafkaFlow.Retry.MongoDb | |

## Installation

[Read the docs](https://farfetch.github.io/kafkaflow-retry-extensions/docs/getting-started/installation) for any further information.

## Documentation

[Wiki Page](https://github.com/Farfetch/kafkaflow-retry-extensions/wiki)
Learn more about using KafkaFlow Retry Extensions [here](https://farfetch.github.io/kafkaflow-retry-extensions/docs/)!

## Contributing

Read the [Contributing guidelines](CONTRIBUTING.md)
Read our [contributing guidelines](CONTRIBUTING.md) to learn about our development process, how to propose bugfixes and improvements, and how to build and test your changes.

## Maintainers

Expand All @@ -144,7 +37,12 @@ Read the [Contributing guidelines](CONTRIBUTING.md)
- [Rodrigo Belo](https://github.com/rodrigobelo)
- [Sérgio Ribeiro](https://github.com/sergioamribeiro)

## Get in touch

You can find us at:

- [GitHub Issues](https://github.com/Farfetch/kafkaflow-retry-extensions/issues)

## License

[MIT](LICENSE.md)
KafkaFlow Retry is a free and open source project, released under the permissible [MIT license](LICENSE).
20 changes: 20 additions & 0 deletions website/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Dependencies
/node_modules

# Production
/build

# Generated files
.docusaurus
.cache-loader

# Misc
.DS_Store
.env.local
.env.development.local
.env.test.local
.env.production.local

npm-debug.log*
yarn-debug.log*
yarn-error.log*
31 changes: 31 additions & 0 deletions website/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Website

This website is built using [Docusaurus 2](https://docusaurus.io/), a modern static website generator.

## Installation

$ yarn

## Local Development

$ yarn start

This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.

## Build

$ yarn build

This command generates static content into the `build` directory and can be served using any static contents hosting service.

## Deployment

Using SSH:

$ USE_SSH=true yarn deploy

Not using SSH:

$ GIT_USER=<Your GitHub username> yarn deploy

If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch.
3 changes: 3 additions & 0 deletions website/babel.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
module.exports = {
presets: [require.resolve('@docusaurus/core/lib/babel/preset')],
};
7 changes: 7 additions & 0 deletions website/docs/getting-started/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"label": "Getting Started",
"position": 2,
"link": {
"type": "generated-index"
}
}
62 changes: 62 additions & 0 deletions website/docs/getting-started/installation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
sidebar_position: 1
---


# Installation

KafkaFlow Retry Extensions is a set of NuGet packages that can extend [KafkaFlow](https://github.com/Farfetch/kafkaflow).


## Prerequisites

- One of the following .NET versions
- .NET Core 2.1 or above.
- .NET Framework 4.6.1 or above.
- KafkaFlow NuGet package installed.


## Installing

On an application with KafkaFlow configured, install KafkaFlow Retry Extensions using NuGet package management.

Required Package:

* [KafkaFlow.Retry](https://www.nuget.org/packages/KafkaFlow.Retry/)


You can quickly install it using .NET CLI 👇
```shell
dotnet add package KafkaFlow.Retry
```

You can find a complete list of the available packages [here](packages).

## Setup

Types are in the `KafkaFlow.Retry` namespace.

```csharp
using KafkaFlow;
using KafkaFlow.Retry;
```

The Retry Extensions library exposes a middleware. To use them, edit your KafkaFlow configuration add register a new middleware, as you can see below.

```csharp

.AddMiddlewares(
middlewares => middlewares // KafkaFlow middlewares
.RetrySimple(
(config) => config
.Handle<ExternalGatewayException>() // Exception to be handled
.TryTimes(3)
.WithTimeBetweenTriesPlan((retryCount) =>
TimeSpan.FromMilliseconds(Math.Pow(2, retryCount)*1000) // exponential backoff
)
)
```

You can use other types of retry policies such as [Forever](../guides/forever-retries) or [Durable](../guides/durable-retries).


12 changes: 12 additions & 0 deletions website/docs/getting-started/packages.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
sidebar_position: 4
---

# Packages

|Name |nuget.org|
|---------------------------------|----|
|KafkaFlow.Retry|[![Nuget Package](https://img.shields.io/nuget/v/KafkaFlow.Retry.svg?logo=nuget)](https://www.nuget.org/packages/KafkaFlow.Retry/) ![Nuget downloads](https://img.shields.io/nuget/dt/KafkaFlow.Retry.svg)
|KafkaFlow.Retry.API|[![Nuget Package](https://img.shields.io/nuget/v/KafkaFlow.Retry.API.svg?logo=nuget)](https://www.nuget.org/packages/KafkaFlow.Retry.API/) ![Nuget downloads](https://img.shields.io/nuget/dt/KafkaFlow.Retry.API.svg)
|KafkaFlow.Retry.MongoDb|[![Nuget Package](https://img.shields.io/nuget/v/KafkaFlow.Retry.MongoDb.svg?logo=nuget)](https://www.nuget.org/packages/KafkaFlow.Retry.MongoDb/) ![Nuget downloads](https://img.shields.io/nuget/dt/KafkaFlow.Retry.MongoDb.svg)
|KafkaFlow.Retry.SqlServer|[![Nuget Package](https://img.shields.io/nuget/v/KafkaFlow.Retry.SqlServer.svg?logo=nuget)](https://www.nuget.org/packages/KafkaFlow.Retry.SqlServer/) ![Nuget downloads](https://img.shields.io/nuget/dt/KafkaFlow.Retry.SqlServer.svg)
Loading

0 comments on commit d6fe944

Please sign in to comment.