API or documentation? #2
Comments
|
We were planning to add documentation a bit later. But, yes, there is an overview, it was posted on reddit:
Each library uses the more low-level one under the hood, e.g. There is a demo, but it's a bit outdated: http://killerstorm.xen.prgmr.com/alex/cw/demo-eng.html (and a faucet is very slow because it uses outdated libraries.) You can easily build this demo out of chromawallet-proto. Please let me know if you need more info. |
|
A pattern that I've found useful with Bitcoin metatransactions protocols (like OpenAssets, BlockSign, etc) is to have a client that can build complete unsigned transactions when provided with these basic building blocks: someProtocol.build({
options: { ... },
address: address,
unspentOutputs: unspentOutputs
}, function(error, builtTx) {
console.log("an unsigned built transaction", builtTx);
});or even going further when provided with a function that will sign and propagate the transaction as well: someProtocol.buildSignAndPropagate({
options: { ... },
address: address,
unspentOutputs: unspentOutputs,
propagateTransaction: propagateTransaction,
signTransaction: signTransaction
}, function(error, receipt) {
console.log("the receipt after building, signing and propagating the tx", receipt);
});Another interesting approach is taken by the CoinPrism API. It is a REST interface that builds and returns unsigned transactions to wallet software. You can see examples here: http://docs.coinprism.apiary.io/#reference/transaction-builder/post Does or will The main point is that the best interface I see for these kinds of Bitcoin metatransaction protocols is one where the client libraries do not control or manage private keys. That is, that they do not function as wallets, merely as transaction builders with perhaps support for helper functions that will call out for signing and propagating. Also, can you point me to some ChromaWay colored coin transactions as they exist on tesnet3 or the Bitcoin mainnet? I'd love to see some live examples. Even better would be a description of that transaction that breaks down how it works! BTW, I'm building a blockchain metadata browser. Here's a screenshot:
I'd love to be showing ChromaWay transactions as well! BTW, the tool also authors these transactions. So far just OpenAssets and BlockSign but I'm adding them as quick as I can. I'd love to be able to build ChromaWay transactions with my own wallet Right now I'm still discovering the best interfaces for each of these metatransaction protocol types. I'd like for there to be a standard protocol. I live in San Francisco and have started reaching out to everyone in hopes of getting this discussion started. Does anyone on your team live in the Bay Area? |
|
Also, do you have a ChromaWay blockchain browser like CoinPrism? https://www.coinprism.info/tx/4309b7f2bb85f18dac820d4e6d38b73110b4b28c61364a14b979d31890b03a99 |
|
And where exactly do you store your metadata? Not in OP_RETURN? |
This requires you to trust the server, as it can give you a bad transaction to sign.
Yes. but it's quite a bit verbose.
True, that's why we implemented coloredcoinlib which is fully separate from the wallet.
OK, I'll dig one up.
No.
We had at one point, but I think it no longer works... are planning to implement another one at some point.
There are only 12 bits of metadata per transaction, it is stored in nSequence of the first input. Otherwise, colorvalues are encoded using satoshi values. |
The transaction can still be parsed and inspected by the client before signing so I don't really see that as an issue. Client software that creates transactions need to trust the client libraries that they use. Of course this could be done with just a single audit as opposed to having to check every transaction as with a REST interface. However, don't make assumptions on how much trust developers require from your protocol or implementations. There will always be varying needs. The important thing about a REST interface for a building transactions is that it makes it easier for developers to work with. Protocols that are not developer friendly won't be used as much as protocols that are. Right now it is easier to view, understand, scan and create OpenAsset transactions than any other colored coin protocol. There are big limitations with OpenAssets and I would definitely like to use colored coin protocols that are capable of non-reissuable coins and tokens but because OpenAssets has the best blockchain explorer, developer tools and documentation I was more inclined to implement that protocol first. Again, I'm interested in supporting any and all forms of Bitcoin metatransactions.
Make an easy-to-use interface for this and you'll incentivize developers to use your tools! |
|
FWIW coloredcoinjs-lib is supposed to be compatible with every imaginable colored coin protocol. E.g. if you implement OpenAssetsColorDefinition it will able to understand OpenAssets colors.
|
How many colored coin protocols are currently supported by
The solution that you described involving OperationalTx, EPOBCColorDefinition.makeComposedTx and ComposedTX are internal systems that I know nothing about. I would have to blindly study your source code to figure this out. This will definitely take more than a couple of hours! Good and useful public APIs don't need to expose this internal functionality. Perhaps you could make some documentation or examples that show how this would work? Perhaps when I have some time I could fork one of these projects and implement the kind of interface that makes sense for my needs. I think you'll find all of these kinds of APIs would benefit from the assumption that colored transactions will be signed by a system that manages it's own private keys. This makes the API easier to consume by existing wallet software. |
Just one. We implemented ~5 different color kernels in the Python version of coloredcoinlib until settled on EPOBC.
These aren't internal systems, it is an API. E.g. OperationalTx is an interface which describes which functions coloredcoinjs expects wallet to provide. I.e. instead of providing 10 parameters and options, you just provide an instance of OperationalTx which encapsulates all of this stuff. It's just called in a weird way, I guess, for historic reasons. Should be something like WalletInterface or Context, perhaps. It was called OperationalTxSpec in Python coloredcoinlib, the idea was that you provide colordcoinlib a specification of the transaction you want a compose, and it composes it. coloredcoinlib was also supposed to be fully abstract of scripting and signing. It just composes inputs and outputs according to requirements of color kernel, and then the wallet constructs actual scriptSig/scriptPubKey as needed. I agree that it's a bit too complex in case one doesn't need all this flexibility, so if there is a demand for it, it would make sense to make a simplified API on top of raw functions.
Yes, that's the plan.
This is why we're providing coloredcoinlib. It doesn't even have a notion of signing a transaction. And it's fully abstract of scripting too. Whether you wallet uses plain P2PKH, P2SH, multi-sig or steal addresses, we can work with it. Even if you need complex scripts to implement trading across chain contract, you can still use stock version of coloredcoinlib to compose transactions involving colored coin cross-chain trade. It doesn't seem to be possible with OpenAssets API, as it seems to be able to have a fixed repertoire of transactions it can do. Their strategy was to start with something simple (send colored coin to a single address) and add more and more complex transaction types. But we implemented an API which can compose any kind of a transaction and factored all irrelevant parts out of it. |
What format are those inputs and outputs in? Is it portable?
Perhaps there is a demand for the services provided by your APIs but the price of entry is too high! A simpler API for developers is a less costly API for developers. The cost is in the time it takes to familiarize oneself with the new API. A great side effect is that the documentation for modules with simple interfaces practically writes itself! The associated tests will tend to be very descriptive examples of how to use the module. Have you see modules that ship with tests with integration tests that interface with testnet3? Helloblock.io has a test faucet and testnet API that https://www.npmjs.com/package/helloblock-js 'getRawTransaction' and the other Raw Transactions JSON-RPC API methods seem to be a very portable solution and natural interface. |
|
And here's an example of another OpenAssets implementation that exposes a different set of problems: https://coins.assembly.com/docs The server does all of the signing. Private keys are a problematic interface for client libraries. The raw bytes of a transaction, either signed or unsigned, are the best interface point for Bitcoin client libraries. Any sort of multi-signature will also benefit from raw transactions bytecode as we can almost always assume that different signatures will come from different types of wallet implementations. If the state of the system requires the full Bitcoin blockchain to be back-scanned we need to be building libraries that interface with bitcoind and can build and verify the state from the ground up much like https://www.npmjs.com/package/openassets. Again a natural interface seems to be the Raw Transaction JSON-RPC API. It seems like we should all build these... and if we were extra nice about it, have common interfaces to various data stores like postgres, redis, mongo, cassandra... What about a middlewear system between bitcoind and a data store? Either this or we can choose to trust the state of public API endpoints like with https://api.coinprism.com - and there are definitely use cases that justify both approaches! What I find interesting is that there are two product implementations built on top of OpenAssets that have incompatible metadata systems. They can see each other's transactions and trace the spends but they have no idea what the names of any of these things are. Perhaps this is fine and it is up to them to release their own client libraries. I've been treating them as two different types and hitting both of their public APIs to retrieve this metadata and get names or images of the assets. How does ChromaWay plan to deal with metadata and will you have a public API and client libraries for this metadata? |
|
What about a middlewear system between bitcoind and a data store? Each transaction could have an associated metatype identifier and a JSON metadata document that is built by different processors... OpenAssets, BlockSign CoinPrism, Assembly, ChromaWay, ColoredCoins, etc. So you back-scan and build the state and all metadata for each protocol storing it in a bunch of supported data stores like postgres, redis, etc. The processors themselves should be be separate from this bitcoind interface and should still work if only for read-only remote APIs. This would allow for fully functional web browser clients that could have a websocket listening for and parsing incoming live transactions complete with associated metadata. |
We treat inputs as opaque objects. Whatever format wallet uses, it gets them back. Outputs are just (value, script) pairs, where script is hex-encoded.
Well, I would be surprised it's more than one day. At least when one sees usage examples.
We have integration tests of this kind for cc-wallet-core.
Well, I believe that doing it in a layered fashion is the best way to do it. Low-level coloring level is fully agnostic of metadata: it doesn't care what kind of an asset is being transferred, it works with color identifiers. Meta-data appears on higher level. Let's say if somebody is building a stock market, he needs metadata specific to stocks. (And within our libraries we have an AssetDefinition class, which concrete applications can extend in any way.) As for discoverable meta-data, I believe it is utterly stupid and outright malicious concept. Any metadata which is published in the blockchain or other medium can be trivially duplicated. So suppose I buy oracle-stock.com, get an SSL cert for it, register ORCL OpenAssets asset. Then I can try to sell it to somebody, and it will show up in CoinPrism as Oracle stock. Unsuspecting user might believe it's true. We use a different model in ChromaWallet: user explicitly needs to choose to install asset definitions he trusts. In future we're going to introduce a notion of asset directories. E.g. a user who wishes to trade stocks can go to a broker and connect to him (thus fetching a list of asset definitions), getting an ability to trade ORCL, MSFT, etc. Obviously, he needs to trust this broker. Or it might be a bank or an exchange users connects to. Important thing here is that user should explicitly trust the source of asset definitions. And the whole list of all issued assets is definitely useless, as quite likely most of them will be junk, fake, scam or test assets. I don't think anybody is going to enjoy looking through thousands of such assets. (We actually had a global list of assets in WebcoinX... Ended up with 100+ assets people made for testing purposes.) |
|
Here's an example of EPOBC issuance transaction: http://tbtc.blockr.io/tx/info/b77b5d214b2f9fd23b377cbbf443a9da445fd7c6c24ba1b92d3a3bfdf26aabf2 And the next one is an EPOBC spend: http://tbtc.blockr.io/tx/info/550a61f2fc5473053a4b91134fd1ff230e8fcda475260505ed40809148a82d24 EPOBC belongs to a class of order-based color kernels, so uncolored outputs and inputs go after colored ones. It also uses padding (argh, anti-dust rules), so it's not always easy to deduce colored output values from satoshi amounts. |
Can you explain further what you mean? What are example of these discoverable meta-data that you find objectionable and why? |
I do! Wether or not I believe those assets and transactions to be real or fake is up for me to decide. I'm perfectly capable of creating filters on my own end. Do you not expect people to make their asset definitions public knowledge and publish that metadata on the web somewhere? Is that what mean when you say asset directories? How can I make a blockchain explorer that consumes the asset definitions that I've created from within ChromaWallet? Can I export these somehow? How can I share these asset definitions with other people? |
|
Example is in very next paragraph:
|
It's up to them to decide whether to make it public or not.
Please check "Publishing the Asset you Created" section here: http://chromawallet.com/guide.html
Asset definition has a "color descriptor" which looks like this: Then if you like to use JS version you need this stuff: So then you can query colorvalues for specific transactions: getTxFn is a function which gives a Transaction for txId (it is async, so via callback). |
|
William, I write some examples (issuance, transferring and calculate color value) and also add SimpleOperationalTx. I hope this will be helpful for understanding how |
Wow, these are great and super descriptive! I immediately have a much better understanding of how this works. 😄
All of the various plugins/filters and protocols to parse and author these transactions will be open source, so anyone with a copy of bitcoind or a list of rawTransactions will be able to process transactions and get meaningful metadata. I'm not going to release anything related to transport (like, a web server interface), storage (how/where this metadata is stored) or presentation (HTML/CSS) because I feel like people will have their own ideas. In additional I plan on releasing an open access public HTTP API that will return the metatype and associated metadata when supplied with a rawTransaction or txHash/txID. The architecture for these plugins/filters hasn't stabilized just yet but the processing engine is tentatively called I imagine in the ecosystem there will be I'm hoping to give the owners of these protocols the opportunity to take ownership of these various plugins/filters so they can have the Github and npm registrations and then I'll just fork from them. Since some of these protocols are based on back-scanning the blockchain, they will need to be an interface for these to use the JSON-RPC API or a 3rd party blockchain explorer HTTP interface in order to look up previous transaction information. There could also be an interface for plugins to get metadata from a trusted 3rd party API such as the case with api.coinprism.com. (Two options, back-scan or trust a 3rd party) This means that people could use these plugins/filters to build their own databases and network interfaces. Most of this functionality is already working and has been developed along with the blockchain metadata browser I've been building. I still need to spend some time to clean up the interfaces and write documentation to get it ready for publishing. |
|
Hey @fanatid, a few of questions: When calling cclib.ColorDefinitionManager.getGenesis(), how does this relate to a color descriptor like Could this transaction be parsed to get the color descriptor? Can you explain how a color descriptor is related to the raw transactions? How is the transaction from the describe('coloredcoinjs-lib (balance) test related to the color descriptor |
|
EPOBC color descriptor is of form So we get a color descriptor after the genesis transaction is built: we just compute the hash of the transaction and concatenate it with other parts. So if you want a complete list of EPOBC colors, you can scan the blockchain for transactions which has nSequence of the first input equal to 37 and construct color descriptors from hashes of those transactions. Now if you received a transaction which looks like EPOBC transaction (check nSequence of the first input), you can, in principle, find the genesis of its color by going backwards through blockchain until you see that genesis. This is what
This is just a placeholder color definition which we use to denote colorvalue of the genesis transaction. Can't use the actual color definition as it doesn't exist yet at that point. |
|
Do you know of anyone who maintains an API endpoint of the complete list of EPOBC colors? and perhaps a complete list of all the transactions? only the EPOBC color descriptors would be needed, not any metadata. Wallet clients should at least be able to get that information without having to have a local copy of entire blockchain, don't you think? How do you see clients scanning for and being made aware of asset issuances and transactions? Doesn't it make sense to have a public server that caches all of the back-scanning for clients? It's all public data parsed by open-source protocols. Clients still have the opportunity to validate the data from the public API in another manner. An HTTP server with CORS (cross-origin resources) that served information about EPOBC color descriptor issuance, transactions and balances would be great for web clients. Do you do metrics on how many assets have been issued and how many transactions there have been? |
Fairly certain nobody does that.
I don't see how this information can be useful aside for statistical purposes, or block-explorer kind of an application.
When client receives an unspent transaction output, it scans transactions backwards until it finds a transaction it is already aware of, or an issuance transaction. This is how it works now.
Client still needs to verify all the data itself, as trustlessness is the whole point of colored coins (vs. a centralized solution). Thus pre-computation on server will only reduce latency, but not the amount of information being sent. We already implemented batch fetches in Python client/server, and are going to add it to JS libraries once performance becomes a problem.
I see no point in implementing an API which will assist development of non-secure wallets.
No. |
What about block explorers? What about other kinds of clients that are read-only? Do you expect every client to have a full 25 GB copy of the blockchain to query locally?
That could be a lot of back and forth and quite slow over a networked mobile client.
Wouldn't you like to know who is using your software and why? Statistics and published compiled data sets would only help all aspects of a project like this! A blog post with analysis could be great for marketing. |
Yea, certainly it would be nice to provide this kind of service so block explorers would be able to integrate EPOBC view with a minimal effort. (Although it would be better if they do the scanning themselves so people will be able to cross-check information. Having a single point of failure sucks.) But, having limited amount of resources, we have to prioritize things. Currently our focus in on delivering rock-solid wallet software. Then again, we already provide code which scans the blockchain. So if anyone in the community is interested in this kind of a service, it will probably take like 50 lines of code to do it. Hardly a big feat.
Perhaps it isn't clear, but we already have a web wallet which does all the processing in browser, requiring only a generic service like blockr.io or chain.com to work.
Yes, but the whole point of using colored coins is that it is decentralized and trustless. Making wallets which need to trust servers is not the right solution here. (You might as well switch to Open Transactions if you're find with it.) In theory, the amount of data client needs to scan is proportional to the size of transaction history for specific color(s). We plan to use snapshots to address the scalability issue: issuer will provide time-stamped and signed snapshots which clients will be able to use to scan only a limited part of history. I believe this provides a reasonable trade-off between security and scalability. Applications which work with colors with a small tx history size (e.g. smart property) can remain 100% trustless, doing complete scan and verification client-side. Otherwise, when scanning a complete history is not feasible, clients can rely on issuer-provided snapshots, but we can still provide a certain degree of security: wallets will remain secure even if issuer's server are compromised, as long as the issuer will restore control within 1 week, for example.
We certainly would like to know that, but we also have things which are higher on the priority list right now. |
Is there an overview of the architecture for ChromaWay colored coins?
What's the relationship between all of these different projects: cc-wallet-engine, coloredcoinjs-lib, chromawallet-proto, cc-wallet-core
Are there demos of these projects running on a website somewhere?
The text was updated successfully, but these errors were encountered: