Unified Offer and Trade Generic Processor

[karthikiyer56]

The ask described here supersedes the Generic Offer CDP Transfomer/Processor as captured in #5414

Note: This issue has gone through a few updates vis-a-vis data model design. To skip to the most return data model, skip to the end

It intuitively makes sense to think of offers updates, liquiditypool updates and trades together for the following reason - a trade is simply:

• updates to an offer ledger entry's state caused by an operation (manageBuy/Sell, pathPayment) in the Stellar blockchain
• updates to a liquidity pool ledger entry's state caused by an operation (in this case only a pathPayment operation can cause updates to a liquidity pool entry) in the stellar blockchain

This ticket serves to unify the 2 sentiments under a single external facing CDP transformer.

** Top Level Goal **

Given a ledger, derive trade events by working at the cadence of operation-level changes within a given transaction

type TradeEvent interface{
..... // details below in data model section
}

func ProcessAllTradesFromLedger(ledger xdr.LedgerCloseMeta) ([]TradeEvent, error)

Event Model (Tentative):

The lifecycle of an offer through the stellar blockchain is as follows:

• An Offer is created by an operation - MangeBuy/ManageSell/CreatePasssiveOffer

• An Offer's state is updated
  - By an operation to update the price/amount of an existing offer
  OR
  - When an offer is filled, either partially or fully. i.e by a taker - either due to another counterpart ManageBuy/MangeSell operation OR by a PathPayment operation

• An offer is finally removed from the blockchain
  - Because an offer is fully filled
  OR
  - Because of a close operation initiated by a user
  OR
  - Because of a system (stellar blockchain) triggered update. For e,g as a part of a protocol upgrade, for some reason, an offer was removed (very rare)

type OfferBase struct {
	SellingAsset  Asset
	BuyingAsset   Asset
	Amount   Int64
	Price    Price
	Flags    Uint32
}

• OfferCreatedEvent --> This indicates the first time an offerEntry was created on the n/w

{
	SellerId AccountId
	OfferId  Int64
        offerState OfferBaseEntries
        EventType --> OfferCreatedEnum
        createdLedgerSeq int32
        Fills FillInfo[]
}

• OfferUpdatedEvent --> This indicates that a previous offerEntry was updated

{
    PrevUpdatedLedgerSeq int64
    PreviousOfferState OfferBase
    UpdatedOfferState OfferBase
    UpdatedLedgerSeq int32
    EventType --> OfferUpdatedEnum
    Fills       FillInfo[]
}

• OfferClosed --> Synthetic event that will be generated by code when an offer is evicted for the reasons mentioned above

{
    LastOfferState OfferBase
    ClosedLedgerSeq int32
    EventType --> OfferClosedEnum
}

The FillInfo structure gives us information about the CounterParty that caused updates to the Offer and is modelled as follows:
Note: There can be more than 1 fills in the FillInfo

type FillInfo struct {
	FillType    FillType    // Enum indicating the type of fill: OrderBook or LiquidityPool
	Counterparty Counterparty // Details of the counterparty, encapsulating OrderBook or LiquidityPool specifics
	AssetSold    Asset       // Asset sold in this fill
	AmountSold   Int64       // Amount of the asset sold in this fill
	AssetBought  Asset       // Asset bought in this fill
	AmountBought Int64       // Amount of the asset bought in this fill
	LedgerSeq    int32       // Ledger sequence in which the fill occurred
}

type FillType int
const (
	FillTypeUnknown FillType = iota // Default value for uninitialized FillType
	FillTypeOrderBook               // Fill against an order in the order book
	FillTypeLiquidityPool           // Fill against a liquidity pool
)

type Counterparty struct {
	OrderBook     *OrderBookCounterparty     // Filled from order book, contains offerId and sellerId
	LiquidityPool *LiquidityPoolCounterparty // Filled from a liquidity pool, contains poolId
}

type OrderBookCounterparty struct {
	SellerId string // Account ID of the seller in the order book
	OfferId  Int64  // Offer ID in the order book
}

type LiquidityPoolCounterparty struct {
	PoolId string // Liquidity pool ID
}

The 3 event types OfferCreated, OfferUpdated, OfferClosed can be thought of as concrete implementations of the interface -

type TradeEvent interface {
	GetTradeEventType() TradeEventType // Method to retrieve the type of the trade event
}

UPDATE 1

The distinct piece missing in the model described above is updates to LiquidityPools
An incoming path payment operation can match against a liquidity pool or against other offers in the orderbook or both.

For example, consider the path: XLM -> USDC.
Presume that the best path for XLM to USDC goes through BTC.
The following combinations are possible:
XLM -> BTC (via orderbook) , BTC -> USDC (via orderbook)
XLM -> BTC (via liquidity pool) , BTC -> USDC (via liquidity pool)
XLM -> BTC (via liquidity pool) , BTC -> USDC (via orderbook)
XLM -> BTC (via orderbook) , BTC -> USDC (via liquidity pool)

Each hop within this path payment, for e.g XLM-BTC via orderbook or BTC - USDC via liquidity pool, is called a trade.
It is also entirely possible that a path payment from A -- B goes entirely via liquidity pools, and doesnt cause updates to any existing offer.
These also need to be captured in the unified model

So, the updated model can be thought to be as follows

type TradeEvent interface {
	GetTradeEventType() TradeEventType // Method to retrieve the type of the trade event
}

const (
	TradeEventTypeUnknown              TradeEventType = iota // Default value
	TradeEventTypeOfferCreated                               // Offer created event
	TradeEventTypeOfferUpdated                               // Offer updated event
	TradeEventTypeOfferClosed                                // Offer closed event
	TradeEventTypeLiquidityPoolUpdated                       // Liquidity pool update event
)

type LiquidityPoolUpdateEvent struct {
	Fills []FillInfo // List of fills for this liquidity pool update
}

Tentative Implementation functions that can be exposed by this unified trade package:

func processAllTradeEventsInLedger(ledger xdr.LedgerCloseMeta) ([]TradeEvent, error)
func processAllTradeEventsInTransaction(tx LedgerTransaction) ([]TradeEvent, error)

Update 2

There are some nuances that cannot be accurately captured by the previously described Counterparty Counterparty field in FillInfo

1. An incoming ManageBuy/ManageSell operation that fills completely will not have an offerId field, since the taker offerEntry was never created. To account for that, the offerId field should be optional. The counter party here would be the sourceAccount of the
   operation (or the account on the transaction)

2. If an offer is filled, the counterparty can never be a liquidity pool. In other words, an incoming ManageBuy/ManageSell operation will never fill against a liquidity pool. Keeping that in mind, it makes sense to include the Source of the fill - i.e what operation caused the fill to happen

3. The only way to sweep liquidity off the Liquidity pool is via a path payment operation. It makes sense to include the source account and destination account of the path payment in the counterparty info.

4. It is possible that a path payment operation can fill against a resting offer. In this case, also, it makes sense to include the source account and destination account of the path payment in the counterparty info

It is evident that the notion of CounterParty doesnt exactly fit answer the question - what caused this change
A better option is to include a FillSource in the FillInfo

type FillSource struct {	      
       // Type of the operation that caused this fill (ManageBuyOffer,  ManageSellOffer, PathPaymentStrictSend, PathPaymentStrictReceive)
       SourceOperation      FillSourceOperationType

        // The taker's information. Who caused this fill???
	ManageOfferInfo *ManageOfferInfo  // Details of a ManageBuy/ManageSell operation (optional)
	PathPaymentInfo *PathPaymentInfo  // Details of a PathPayment operation (optional)
}

// ManageBuy/ManageSell operation details
type ManageOfferInfo struct {
        // Account that initiated the operation. Source of operation or source of transaction
	SourceAccount AccountId   

         // Did the taking operation create an offerId/offerEntry that rested after being partially filled OR Was it fully filled
         OfferFullyFilled bool 

	OfferId       *Int64      // Offer ID, if an offer entry was created (nil if fully filled)
}

type PathPaymentInfo struct {
        SourceAccount      AccountId // Source account of the PathPayment
	DestinationAccount AccountId // Destination account of the PathPayment
}

And the FillInfo then can be represented as

type FillInfo struct {
	AssetSold    Asset       // Asset sold in this fill
	AmountSold   Int64       // Amount of the asset sold in this fill
	AssetBought  Asset       // Asset bought in this fill
	AmountBought Int64       // Amount of the asset bought in this fill
	LedgerSeq    int32       // Ledger sequence in which the fill occurred

         FillSource FillSource // Details about what operation (and details) caused this fill 
}