Account management implementation (#239)

* account management keystore

* added manager

* some lint fix

* feed test

* linters and test fix

* lint fix

* test fix

* test fix

* test fix

* to abs path

* custom json unmarshall

* commands added

* unnecessary files deletion

* interface unnecessary functions

* manager integration

* skip TestUpdatedKeyFileContents

* introduction of external signer

* lint fix

* lint

* lint again

* personal endpoint

* To one key file

* keys to one file and account_cache_test fix

* keystore-test fix

* comment delete

* lint fix

* fix personal

* linter and personal

* lint

* test fix and remove panic

* test fix

* unnecessary files deletion

* jsonrpc functions not exposed

* command fix

* mixed case deleted

* fix some comments

* url deleted

* file cache delted

* comment fix

* forks in time fix

* in-house pub sub

* lint and better fork managemt, also interface for manager

* lint

* some minor changes

* Rebase

* fix for tests

* small code changes

* code optimizations

* lint fix

* commands update

* comment fix

* Update accounts/event/event_handler_test.go

TestMultipleSubscribers optimization

Co-authored-by: oliverbundalo <[email protected]>

* comment fix and rename keyStore interface functions

* refresh wallets optimization

* comments and change names

* small changes

* lint fix

* lint fix 2

* lint fix 2

* tests fix

* lint again

* switch to if in personal

* fuzz test fix

* manager test

* manager unlock change

* manager test optimziation

* readme for account management

* to mermaid

* readme fix

---------

Co-authored-by: Goran Rojovic <[email protected]>
Co-authored-by: oliverbundalo <[email protected]>

accounts/README.MD

@@ -0,0 +1,78 @@
+Account management
+===
+
+## Problem
+Account management is a feature that implements the logic for storing private keys for non-validator users. These private keys are used for signing various types of data. Currently, we have implemented file storage on the local disk. The keys are stored in a text file in JSON format. To ensure security, the private keys are encrypted, so opening the file does not compromise sensitive data.
+
+## Account storage
+Account storage is component of account management which is used to manage keys file. When system starts it can create a new file if neccessary or load account from existing file.Account cache adds, delete or update accounts in file. Its primary purpose is to keep the key file updated with the latest account data.
+
+## Key storage
+Keys are stored in a file in JSON format. All keys are stored in the same file. The data structure used to marshal the keys is map[Address]EncryptedKey, making it very easy to retrieve a keys from the file while keeping the sensitive data secure.
+
+## Keystore
+The keystore is a layer between the AccountManager and the AccountStore, where cryptographic operations take place. The keystore maintains its own list of wallets (wrappers around accounts) and notifies the account manager about any changes (such as accounts being added or deleted). In this layer, transactions and data are signed, and private keys are kept unlocked for actions that require an unlocked private key (such as eth_sign).
+
+## Account manager
+The account manager is the top layer, and the system interacts with it for every account management task. It handles all types of key storage (currently, we only support local keystore storage) and merges all data from different storage types into one list.
+
+## Interacting with account management
+Accounts are required for some features. So our software support two diffent ways of interaction with account management. You can interact with commands or with different json rpc calls.
+
+### Suported json-rpc calls
+* **personal_listAccount** - return addresses of all accounts
+* **personal_newAccount** - create new account and return address of account
+* **personal_importRawKey** - insert hex raw key and storing them in storage
+* **personal_unlockAccount** - unlockes account so that can sign transaction with account private key, it used for eth_sign and other calls that doesn't send password for decrypt private key
+* **personal_lockAccount** - lock unlocked account
+* **personal_updatePassphrase** - change passphrase of existing account
+
+### Supported commands
+* **create** - create new account and return address of account
+* **insert** - insert command insert existing private key and store in keystore
+* **update** - change passphrase of existing account
+
+## ImportRawKey JSON-RPC Flow
+
+``` mermaid
+    sequenceDiagram
+    Network->>Personal endpoint:personal_improtRawKey
+    Personal endpoint->>Manager:WalletsManager()\nKeyStoreType
+    Manager->>Personal endpoint:KeyStore\n(WalletManager)
+    Personal endpoint->>KeyStore:ImportECDSA()
+    KeyStore->>KeyStore:KeyEncrypt()
+    KeyStore->>AccountStore:add()
+    KeyStore->>Manager:WalletEvent
+    Manager->>Manager:Updater
+    AccountStore->>AccountStore:add
+    Note right of AccountStore:KeyFile
+```
+
+## Insert Command Flow
+``` mermaid
+    sequenceDiagram
+    Command->>Personal endpoint:personal_improtRawKey
+    Personal endpoint->>Manager:WalletsManager()\nKeyStoreType
+    Manager->>Personal endpoint:KeyStore\n(WalletManager)
+    Personal endpoint->>KeyStore:ImportECDSA()
+    KeyStore->>KeyStore:KeyEncrypt()
+    KeyStore->>AccountStore:add()
+    KeyStore->>Manager:WalletEvent
+    Manager->>Manager:Updater
+    AccountStore->>AccountStore:add
+    Note right of AccountStore:KeyFile
+```
+
+
+
+## Private keys encryption 
+Encryption of private keys is crucial for protecting blockchain users from unauthorized actions. We use the AES-128-CTR cryptographic algorithm to encrypt private keys, ensuring a high level of security. AES-128-CTR is a symmetric encryption algorithm, meaning the same key is used for both encryption and decryption.
+
+To enhance security, we introduce an authentication string used to encrypt and decrypt private keys. From this authentication string, we derive an appropriate encryption key, which serves as the private key for AES-128-CTR. To access private keys, users must know their authentication password, which allows for the decryption of the private keys.
+
+Once a private key is decrypted, it is temporarily stored in memory and deleted after each use to maintain security. Every user, when creating a new account or importing an existing key, determines their authentication string and must remember it.
+
+
+
+
+

accounts/accounts.go

@@ -0,0 +1,131 @@
+package accounts
+
+import (
+	"fmt"
+	"reflect"
+
+	"github.com/0xPolygon/polygon-edge/accounts/event"
+	"github.com/0xPolygon/polygon-edge/crypto"
+	"github.com/0xPolygon/polygon-edge/types"
+	"golang.org/x/crypto/sha3"
+)
+
+type Account struct {
+	Address types.Address `json:"address"`
+}
+
+// Wallet represents a software or hardware wallet that might contain one or more
+// accounts (derived from the same seed).
+type Wallet interface {
+	// Status returns a textual status to aid the user in the current state of the
+	// wallet
+	Status() (string, error)
+
+	// Open initializes access to a wallet instance.
+	Open(passphrase string) error
+
+	// Close releases any resources held by an open wallet instance.
+	Close() error
+
+	// Accounts retrieves the list of signing accounts the wallet is currently aware
+	// of
+	Accounts() []Account
+
+	// Contains returns whether an account is part of this particular wallet or not.
+	Contains(account Account) bool
+
+	// SignData requests the wallet to sign the hash of the given data
+	SignData(account Account, mimeType string, data []byte) ([]byte, error)
+
+	// SignDataWithPassphrase is identical to SignData, but also takes a password
+	SignDataWithPassphrase(account Account, passphrase, mimeType string, data []byte) ([]byte, error)
+
+	// SignText requests the wallet to sign the hash of a given piece of data, prefixed
+	// by the Ethereum prefix scheme
+	// This method should return the signature in 'canonical' format, with v 0 or 1.
+	SignText(account Account, text []byte) ([]byte, error)
+
+	// SignTextWithPassphrase is identical to Signtext, but also takes a password
+	SignTextWithPassphrase(account Account, passphrase string, hash []byte) ([]byte, error)
+
+	// SignTx requests the wallet to sign the given transaction.
+	SignTx(account Account, tx *types.Transaction) (*types.Transaction, error)
+
+	// SignTxWithPassphrase is identical to SignTx, but also takes a password
+	SignTxWithPassphrase(account Account, passphrase string,
+		tx *types.Transaction) (*types.Transaction, error)
+}
+
+func TextHash(data []byte) []byte {
+	hash, _ := textAndHash(data)
+
+	return hash
+}
+
+func textAndHash(data []byte) ([]byte, string) {
+	msg := fmt.Sprintf("\x19Ethereum Signed Message:\n%d%s", len(data), data)
+	hasher := sha3.NewLegacyKeccak256()
+	hasher.Write([]byte(msg))
+
+	return hasher.Sum(nil), msg
+}
+
+type WalletEventType int
+
+const (
+	// WalletArrived is fired when a new wallet is detected either via USB or via
+	// a filesystem event in the keystore.
+	WalletArrived WalletEventType = iota
+
+	// WalletOpened is fired when a wallet is successfully opened with the purpose
+	// of starting any background processes such as automatic key derivation.
+	WalletOpened
+
+	// WalletDropped
+	WalletDropped
+)
+
+// WalletEvent is an event fired by an account backend when a wallet arrival or
+// departure is detected.
+type WalletEvent struct {
+	Wallet Wallet          // Wallet instance arrived or departed
+	Kind   WalletEventType // Event type that happened in the system
+}
+
+func (WalletEvent) Type() event.EventType {
+	return event.WalletEventType
+}
+
+type WalletManager interface {
+	// Wallets retrieves the list of wallets the backend is currently aware of
+	Wallets() []Wallet
+
+	// SetEventHandler set eventHandler on backend to push events
+	SetEventHandler(eventHandler *event.EventHandler)
+
+	// SetManager sets backend manager
+	SetManager(manager AccountManager)
+}
+
+type AccountManager interface {
+	// Checks for active forks at current block number and return signer
+	GetSigner() crypto.TxSigner
+
+	// Close stop updater in manager
+	Close() error
+
+	// Adds wallet manager to list of wallet managers
+	AddWalletManager(walletManager WalletManager)
+
+	// Return specific type of wallet manager
+	WalletManagers(kind reflect.Type) []WalletManager
+
+	// Return list of all wallets
+	Wallets() []Wallet
+
+	// Return all accounts
+	Accounts() []types.Address
+
+	// Search wallet with specific account
+	Find(account Account) (Wallet, error)
+}

accounts/accounts_test.go

@@ -0,0 +1,19 @@
+package accounts
+
+import (
+	"bytes"
+	"testing"
+
+	"github.com/0xPolygon/polygon-edge/types"
+)
+
+func TestTextHash(t *testing.T) {
+	t.Parallel()
+
+	hash := TextHash([]byte("Hello Joe"))
+	want := types.StringToBytes("0xa080337ae51c4e064c189e113edd0ba391df9206e2f49db658bb32cf2911730b")
+
+	if !bytes.Equal(hash, want) {
+		t.Fatalf("wrong hash: %x", hash)
+	}
+}

accounts/event/event_handler.go

@@ -0,0 +1,61 @@
+package event
+
+import "sync"
+
+type EventHandler struct {
+	subscribers map[string][]chan Event
+	mu          sync.RWMutex
+}
+
+func NewEventHandler() *EventHandler {
+	return &EventHandler{
+		subscribers: make(map[string][]chan Event),
+	}
+}
+
+func (ps *EventHandler) Subscribe(topic string, event chan Event) {
+	ps.mu.Lock()
+	defer ps.mu.Unlock()
+
+	ps.subscribers[topic] = append(ps.subscribers[topic], event)
+}
+
+func (ps *EventHandler) Publish(topic string, msg Event) {
+	ps.mu.RLock()
+	defer ps.mu.RUnlock()
+
+	if chans, ok := ps.subscribers[topic]; ok {
+		for _, ch := range chans {
+			ch <- msg
+		}
+	}
+}
+
+func (ps *EventHandler) Unsubscribe(topic string, sub <-chan Event) {
+	ps.mu.Lock()
+	defer ps.mu.Unlock()
+
+	if chans, ok := ps.subscribers[topic]; ok {
+		for i, ch := range chans {
+			if ch == sub {
+				ps.subscribers[topic] = append(chans[:i], chans[i+1:]...)
+
+				close(ch)
+
+				break
+			}
+		}
+	}
+}
+
+type EventType byte
+
+const (
+	WalletEventType EventType = 0x01
+
+	NewWalletManagerType EventType = 0x02
+)
+
+type Event interface {
+	Type() EventType
+}