[Proposal] Plugin system

[mantas-sidlauskas]

Is your feature request related to a problem? Please describe.
Cadence provides multiple options/clients for components like:

Storage:

• Cassandra
• MySQL
• PostgreSQL
• Others

Document search:

• Elasticsearch
• Opensearch

Archival:

• S3
• GCloud

Having so many components built-in makes dependency version control complicated. Adding another component/client most likely will bring more dependencies to the code.

This also makes problematic to keep up with dependency versioning and introducing/experimenting with additional features will require approval from Cadence team.

Proposed Solution
I propose to implement a "plugin" system where you could declare external repository to be used.
Plugins interface can be declared to represent a concrete component in Cadence, for example:

• Storage plugin
• RPC Middleware plugin
• Archival plugin
• Etc.

One of possible implementation method: generate an entry main.go file where "import" section would be generated based on plugins you defined.

This gives at least two benefits:

• Having a possibility to declare a plugin as go module, community could use publicly available plugin which is not placed in Cadence github repository.
• Cadence code will have less internal dependencies, or even some existing components could be placed on another repository.

Additional context
Relates to #5584

[tfcace]

this would be perfect for advanced visibility without Kafka