Setting up with BCH #578

phewitt · 2018-04-18T22:55:13Z

I literally have looked everywhere on how to setup insight-api with bitcoin cash. I see that you guys have a cash branch on a lot of your repos, however I have no idea how these tie into getting them to work with insight-api. I also have seem many other people asking for this. Is there ANYWHERE I can find some guidance on how to at least begin setting up BCH to work with insight.

maektwain · 2018-04-30T18:09:44Z

Its quite easy

Start your node
Configure bitcore with Bitcioin cash by adding this inside you config so, my config looks like this

{
  "version": "5.0.0-beta.44",
  "network": "livenet",
  "port": 3001,
  "services": [
    "address",
    "block",
    "db",
    "fee",
    "header",
    "mempool",
    "p2p",
    "timestamp",
    "transaction",
    "web",
    "insight-api",
    "insight-ui"
  ],
  "datadir": "/home/username/.bitcore/data",
  "servicesConfig": {
    "insight-api": {
      "cwdRequirePath": "insight-api",
      "routePrefix": "api"
    },
    "insight-ui": {
      "cwdRequirePath": "insight-ui",
      "routePrefix": "",
      "apiPrefix": "api"
    },
    "p2p": {
      "peers": [
        {
          "ip": {
            "v4": "nodeIP"
          },
          "port": "8333"
        }
      ]
    }
  }
}

But there is a problem with bitcore-lib which come pre-installed(the network properties belongs to bitcoin) , either you can change that before installing bitcore by switching to cash branch.(to be clear bitcore-lib is this https://github.com/bitpay/bitcore-lib/blob/master/lib/networks.js)

Now once everything is done.

Fire up.

It will work

dviramontes · 2018-05-08T15:53:21Z

@maektwain what version of insight-api are you using with that config ^^^ ?

j4ys0n · 2018-05-10T18:49:11Z

Been trying to get this working for a couple days now - just tried the strategy above, no luck.

it runs and connects to the bitcoin-abc node but i get "Invalid address" when i try to request data for a bitcoin cash address. Is there something else that needs to happen?

j4ys0n · 2018-05-10T18:57:38Z

@phewitt @dviramontes did either of you get this working?

maektwain · 2018-05-14T10:59:03Z

@dviramontes I am using the latest clone from the git.

Invalid address is because the conversion strategy is old, you need to manually update the conversion strategy from legacy to bitcoin cash.

This is how I did it.

nano node_modules/insight-api/lib/addresstranslator.js

Remove your existing AddressTranslator.translate function to this

AddressTranslator.translate = function(addresses, coin, origCoin) {
  var wasArray = true;
	console.log(coin + "Coin at coin bch");
	console.log(origCoin + "Coin at origCoin btc");
  if (!_.isArray(addresses)) {
    wasArray = false;
    addresses = [addresses];
  }
origCoin = origCoin || AddressTranslator.getAddressCoin(addresses[0]);
console.log("getAddresCoin Method" + origCoin);
  var ret =  _.map(addresses, function(x) {
	console.log(x);
    var orig = new Bitcore_[origCoin].Address(x).toObject();
if (origCoin === 'bch') {
      return Bitcore_[coin].Address.fromObject(orig).toString();
    }
var origC = Bitcore_[coin].Address.fromObject(orig).toString();
var bchAddress = Bitcore_['bch'].Address.fromObject(orig).toString();
console.log("BCH Address" + bchAddress);
console.log("Inside the map " + origC);
    return Bitcore_[coin].Address.fromObject(orig).toCashAddress();
  });

  if (wasArray) 
    return ret;
  else 
    return ret[0];

};

buraktt · 2018-05-18T11:26:13Z

You can add usecashaddr=0 to bitcoin.conf to use legacy addresses.

zhousi666 · 2018-06-11T10:13:51Z

@maektwain can you tell me how to Setting up with bitcoin cash node , i can't find node_modules/insight-api/lib/addresstranslator.js
The following is my deployment process：
(1) install bitcore
cd /root
npm i npm@latest -g
npm install -g --unsafe-perm [email protected]
bitcore create bitcore-cash-node
(2) install insight-api insight-ui
/root/bitcore-cash-node
bitcore install insight-api insight-ui
(3) link bitcoin-abc.(bitcoind)
/usr/local/lib/node_modules/bitcore/node_modules/bitcore-node/bin
https://download.bitcoinabc.org/0.17.2/linux/bitcoin-abc-0.17.2-x86_64-linux-gnu.tar.gz ./
tar -xzf bitcoin-abc-0.17.2-x86_64-linux-gnu.tar.gz
ln -sf bitcoin-abc-0.17.2/bin/bitcoind bitcoind

zjb0807 · 2018-06-12T13:30:42Z

@maektwain "But there is a problem with bitcore-lib which come pre-installed(the network properties belongs to bitcoin) , either you can change that before installing bitcore by switching to cash branch"
how to do this? thanks!

phewitt · 2018-06-12T20:36:00Z

I was able to get it to work with [email protected] and adding usecashaddr=0 to my bitcoin.conf. I used this bitcoin cash node https://api.github.com/repos/bitprim/bitcoin-abc/tarball/tag0.17.1-bitcore. Hope that helps!

Basically.
download and compile the bitprim bitcoin-abc node (you have to have modified versions of the underlying nodes that have added the required RPC methods for insight)
npm install -g [email protected]
bitcore-node create mynode
bitcore-node install insight-api
then either run the bitcoincash node to generate the bitcoin.conf file or copy the file where you set the data directory. add usecashaddr=0 to the bitcoin.conf (for legacy format). Make sure you have insight config pointed at the bitcoin cash node and not the bitcoin node that it installs by default. Hope this helps someone!

joadr · 2018-06-13T01:53:49Z

Does this branch work?
https://github.com/bitpay/insight-api/tree/cash

or I have to do all the above?

psiofxt · 2018-06-28T22:50:36Z

@phewitt Thanks a bunch for your steps -- I pretty much did the exact same thing and everything works great except I'm having trouble with specific insight-api methods like /addr. Could you elaborate what you mean by:

Make sure you have insight config pointed at bitcoin cash node...

I can't seem to find anywhere the ability to change this config.

joadr · 2018-06-28T22:56:22Z

@psiofxt Could you make like a very simple step by step guide to achieve this?

Also.. How big is the blockchain size in GB once synced?

Thanks!

psiofxt · 2018-06-28T23:03:11Z

@joadr Absolutely. I'll type something up either tonight or tomorrow. I'm also planning on writing an even more detailed blog post/guide next week on setting up bitcore with litecoin/bitcoin/bitcoin cash all on the same server -- I'll post it here.

Currently I'm only testing with BCH testnet so it's a mere 13GB.

edit: To clarify, this is with the @latest tag install for bitcore, so 3.1.3

phewitt · 2018-06-29T15:39:28Z

@psiofxt if you just place it in your specified "datadir" it will not overwrite if one already exists. So, what I did is make my own place it where I specify my "datadir" directory and then fire up the node. Same goes for the bitcore-node.config just make your own place it where in the bitcore-node project directory and it should use it. Hope this helps

phewitt · 2018-06-29T15:41:29Z

Place your "bitcore-node.json" file in your Mynode directory or whatever you named the local node you ran bitcore-node create on. here is my config for reference.

{
  "network": "livenet",
  "port": 3001,
  "services": [
    "bitcoind",
    "insight-api",
    "web"
  ],  
  "servicesConfig": {
    "insight-api": {
      "routePrefix": "api",
      "disableRateLimiter": true
    },  
    "bitcoind": {
      "spawn": {
        "datadir": "/home/cqdev/.bitcoin",
        "exec": "/usr/local/bin/bitcoind"
      }   
    }   
  }
}

and just point datadir where you want to store chaindata and have your bitcoin.conf and point exec at the bitcoin-abc binary. I think we may have made a softlink called bitcoind instead of point directly at bitcoin-abc either way that's how I did it!

Also, I setup litecoin aswell and it is a little more involved. You can't use bitpays default repos you have to modify them. I used this fork https://github.com/litecoin-project/litecore-node. Hope it helps.

psiofxt · 2018-06-29T17:31:19Z

@phewitt Hmm, I believe I'm doing that except I renamed all of the Bitcoin-ABC files to bitcoinCashd for instance.

It definitely runs just fine and syncs up with the BCH testnet chain, insight is just producing strange results for anything that isn't /api/block-index/<height>. I'm using the data directory that is inside the myNode folder. My config looks identical to yours except I'm using testnet and my bitcoind spawn config is

"bitcoind": {
      "spawn": {
        "datadir": "./data",
        "exec": "/usr/local/bin/bitcoinCashd"
      }

edit: Did you change any of your bitcore = require('bitcore-lib') to require('bitcore-lib-cash') in the .js files?

phewitt · 2018-06-29T19:25:09Z

@psiofxt No I didn't have to change any of that because I am using the old cash address (aka bitcoin addresses) with the usecashaddr=0 in the .config. What are the results? I might be able to help you if I know your error messages or anything else.

phewitt · 2018-06-29T19:28:26Z

Here is my bitcoin.conf file. for the bitcoin-abc node.

whitelist=127.0.0.1
txindex=1
addressindex=1
timestampindex=1
spentindex=1
zmqpubrawtx=tcp://127.0.0.1:28332
zmqpubhashblock=tcp://127.0.0.1:28332
rpcallowip=127.0.0.1
rpcuser=bitcoin
rpcpassword=local321
uacomment=bitcore
usecashaddr=0
rpcworkqueue=32

phewitt · 2018-06-29T19:30:20Z

I honestly haven't used testnet either. Not sure if or why that would cause any issues. But it is worth my pointing out.

psiofxt · 2018-06-29T19:38:01Z

@phewitt Your help is immensely appreciated, thank you. I'm going to re-index my node and try again.

I'll get back to you with results, but it is highly likely testnet may be causing the issues. If I do /insight-api/addr/mqXbNGz9EbVTmQY3otshAbMR65RaQrsPUD (some random address on BCH testnet found on blocktrail.com/tBCC) I get a Method not found error. Examining some tx id with /insight-api/tx/<tx_hash> I see block height=-1, value=NaN and a few other odd results.

phewitt · 2018-06-29T22:28:27Z

@phewitt Okay so method not found issues means your underlying node doesn't have the correct RPC calls. You have to download that modified version I linked above because the normal nodes do not have all the RPC methods that Insight uses. Bitpay actually forked bitcoin and added certain RPC methods that they use. I saw this issue many times and generally means that either your insight is not pointed at the correct bitcoind / bitcoin-abc that you want OR the bitcoind /bitcoin-abc you installed doesn't actually have the additional RPC methods.

Here is a commit list of some of the additional RPC methods I have seen added to nodes. This is BTG however the same RPC methods are needed on whatever node you plan to use insight with. BTCGPU/BTCGPU@1c1d93d

psiofxt · 2018-06-29T22:36:33Z

@phewitt Aha! You are a god, I didn't even notice it was a bitcore specific fork of bitcoin-abc. My bitcoin abc .tar came from https://download.bitcoinabc.org/. I'll start from scratch with this and let you know.

phewitt · 2018-06-29T22:39:43Z

Hope this helps others too. I wish bitpay would have made more of an effort documenting this stuff. I understand it's a lot of extra work on their end with no real gain for them though. You should look into the bitcore V8.0.0 branch too it's really neat looking and seems like it will be fixing a lot of problems with standing up other nodes other than bitcoin. https://github.com/bitpay/bitcore/tree/v8.0.0. I've been dabbling with it and it doesn't have all the same endpoints that the older api has yet, but it seems like it is more active and a MUCH better architecture IMO. Here is a high level article on some of it's advantages. https://blog.bitpay.com/bitcore-node-v8-beta/

osagga · 2018-07-03T22:12:47Z

@phewitt Do you know where I could find the binary for the bitcoin-abc you linked (https://api.github.com/repos/bitprim/bitcoin-abc/tarball/tag0.17.1-bitcore)? It seems like the only option is to build from source, but it would be nice if I can just use a pre-built binary (like what I can find on https://download.bitcoinabc.org/0.17.2/linux/)

psiofxt · 2018-07-03T22:19:19Z

@phewitt After compiling the bitcore fork of bitcoin-abc I got BCH working beautifully. Thanks again. To all those reading this from google or elsewhere I'll be writing up one-by-one steps this week to get everything working. I've glanced at v8.0.0 and it does look promising, especially being able to include multiple networks in one config file.

@osagga I looked for that option as well when I was deploying, had to build the binaries myself.

usecashaddr=0 must be included for all of this to work with bitcore version <5, but I wonder if you switch all of the .js files to bitcore-lib-cash if it would work with cash addrs.

osagga · 2018-07-05T20:43:19Z

@phewitt Thanks for sharing the link to the custom bitcoin-abc, the setup works!

@psiofxt I have just made a guide on how to setup bitcore with BCH according the steps in this thread, you can check it out here: https://github.com/osagga/bch-bitcore-setup/blob/master/README.md, feel free to add to it or fix a mistake.

Also I manged to modify the API to show BCH addresses (explained in the guide), so the setup should be complete now.

hxzqlh · 2018-07-06T12:18:28Z

@osagga The bitcore-abc((https://github.com/bitprim/bitcoin-abc/tree/tag0.17.1-bitcore) add some rpc metheod to default bitcore-abc (eg: https://download.bitcoinabc.org/0.17.1/linux/bitcoin-abc-0.17.1-x86_64-linux-gnu.tar.gz), it can work together withe insight-api(v0.4.3) that supporting /addr api. Your bitcore-node-cash use bitcore-abc(https://download.bitcoinabc.org/0.17.2/) and insight-api ("osagga/insight-api#cash_v1") that also work. Does your version insight-api add some rpc implementation(eg:/addr method)? I think the cash address supporting comes from bitcore-abc itself or insight-api.

osagga · 2018-07-06T12:26:23Z

@hxzqlh So for bitcore-node-cash, I didn't add any additional RPC commands, but instead, I modified the bitcore-node (v5) to handle new BCH addresses (mainly using bitcore-lib-cash instead of bitcore-lib, you can check the commits for more info). Recently I've found that the setup using bitcore-node (v3) and only modifiying the insight-api is more stable and the sync speed is way faster since the bitcore-node doesn't need to validate the blocks from the node (it's considered trusted), not like the design of v5 where the peer is not trusted, and therefore the sync speed is way slower given the verification process.

Both options work so far, it's just a design decision to chose which version. Hopefully v8 could support both options.

hxzqlh · 2018-07-06T12:50:35Z

@osagga I understand, as for now, there are two ways to support cash addr for bitcore-node:

your way: bitcore-node (v5)+insight-api(osagga/insight-api#cash_v1)+bitcore-abc-0.17.2(https://download.bitcoinabc.org/0.17.2/), sync speed is slower than follow.
@phewitt way: bitcore-node (v3.1.3)+insight-api(v0.4.3)+bitcore-abc-0.17.1(https://github.com/bitprim/bitcoin-abc/tree/tag0.17.1-bitcore)

as you said, you are managing to work on another insight-api(osagga/insight-api#cash_v4) that will enable input addr format as BTC lagecy address or new BCH address, thank you.

osagga · 2018-07-06T13:11:42Z

@hxzqlh The second way doesn't support BCH addresses by default (using insight-api v0.4.3), it can only support BTC legacy addresses. use osagga/insight-api#cash_v4 as the api if you want to enable the cash addresses

osagga/insight-api#cash_v4 is ready to use, it allows you to use BCH addresses with the second setup (bitcore-node v3.1.3) I tested it on all three networks, let me know if you gave it a try.

haydencarlson · 2018-07-19T20:22:25Z

@osagga Thanks I'm using your insight-api#cash_v4. Got my node syncing now, I noticed that when I try to make a request to
[insight_host_url]/api/addr/bitcoincash:qzq2myx5qdvplgalgcyx4ydjm82pyhdkcyga74fzrw for example I'm getting an Invalid address. Code:-5

My bitcoin.conf

server=1
whitelist=127.0.0.1
txindex=1
addressindex=1
timestampindex=1
spentindex=1
zmqpubrawtx=tcp://127.0.0.1:28332
zmqpubhashblock=tcp://127.0.0.1:28332
rpcallowip=127.0.0.1
rpcuser=bitcoin
rpcpassword=local321
uacomment=bitcore
usecashaddr=0

Then when trying to use the same url to look up the legacy address equivalent I get
Invalid address: Address has mismatched network type.. Code:1

Any suggestions?

osagga · 2018-07-19T20:48:59Z

@haydencarlson that's weird, can you post your bitcore-node.json ? Also it would be better if you can open an issue on my fork of insight-api about this problem.

haydencarlson · 2018-07-19T20:50:20Z

@osagga Ok sounds good I'll open this there.

Edit: Looks like I'm unable to open an issue there, it's not enabled.

{
  "network": "livenet",
  "port": 3024,
  "services": [
    "bitcoind",
    "insight-ui",
    "insight-api",
    "web"
  ],
  "servicesConfig": {
    "insight-ui": {
      "routePrefix": "",
      "apiPrefix": "api"
    },
    "insight-api": {
      "routePrefix": "api",
      "disableRateLimiter": true
    },
    "bitcoind": {
      "spawn": {
        "datadir": "/home/[user]/.bitcoin",
        "exec": "/usr/local/bin/bitcoind"
      }
    }
  }
}

If you re-enable it I'll repost to your fork.

osagga · 2018-07-19T20:56:26Z

@haydencarlson issues should be open now

qazxcvio · 2018-11-09T07:51:54Z

@phewitt Aha! You are a god, I didn't even notice it was a bitcore specific fork of bitcoin-abc. My bitcoin abc .tar came from https://download.bitcoinabc.org/. I'll start from scratch with this and let you know.

I also encountered the problem that bch insight /addr could not access. Return me Method not found. Code:-32601, can you ask how this is solved?

1scrooge · 2018-11-12T09:01:45Z

Hi All. I have the Bitcoin node with insight-api. And I have a problem with 556034 block, Insight-api can not download 556034 block, because it is large (32 Mb). Could anybody help me?
Insight-api log:

[2018-11-12T06:59:11.627Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:16.632Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:21.638Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:26.643Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:31.648Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:36.650Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:41.655Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:46.660Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:51.665Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:56.670Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T07:00:01.676Z] info: Block Service: download progress: 556033/556297  (99.9525%)

bitcore-node.json:

{
  "version": "5.0.0-beta.44",
  "network": "livenet",
  "port": 3001,
  "services": [
    "address",
    "block",
    "db",
    "fee",
    "header",
    "mempool",
    "p2p",
    "timestamp",
    "transaction",
    "web",
    "api"
  ],
  "datadir": "/ext/bitcore",
  "servicesConfig": {
"p2p": {
      "peers": [
        {
          "ip": {
            "v4": "127.0.0.1"
          },
          "port": 8333
        }
      ]
    }

matiu · 2018-11-12T15:38:44Z

Hi,

Bitcore v8 was able to handle 32mb blocks without any problem.

The older version had a hard time handling them. You need to up the mem to 7.5GB and wait a long time (around 1hr) to process the 32GB on a regular server hardware.

If possible, migrate to v8 to work for BCH.

some-dev · 2018-11-12T18:55:12Z

What do you mean by "migrate to v8 to work for BCH." ?
Could you explain a bit, please.

some-dev · 2018-11-12T21:27:33Z

We up the memory to 14 gb, but the issue still remain..

msaikat · 2019-01-16T19:36:23Z

@phewitt Thanks for sharing the link to the custom bitcoin-abc, the setup works!

@psiofxt I have just made a guide on how to setup bitcore with BCH according the steps in this thread, you can check it out here: https://github.com/osagga/bch-bitcore-setup/blob/master/README.md, feel free to add to it or fix a mistake.

Also I manged to modify the API to show BCH addresses (explained in the guide), so the setup should be complete now.

Hi @osagga,

Is there anything that can be followed for bitcoin sv. bch-bitcore-setup guide was helpful, I was looking for something similar for bsv. I could already run the test node. However, I a stuck with insight-api integration so that I can explore programatically.

Any kind of help is highly appreciated.

Thanks

matiu closed this as completed Nov 12, 2018

matiu reopened this Nov 12, 2018

Setting up with BCH #578

Setting up with BCH #578

Comments

phewitt commented Apr 18, 2018

maektwain commented Apr 30, 2018

dviramontes commented May 8, 2018 • edited Loading

j4ys0n commented May 10, 2018

j4ys0n commented May 10, 2018

maektwain commented May 14, 2018

buraktt commented May 18, 2018

zhousi666 commented Jun 11, 2018

zjb0807 commented Jun 12, 2018 • edited Loading

phewitt commented Jun 12, 2018 • edited Loading

joadr commented Jun 13, 2018

psiofxt commented Jun 28, 2018 • edited Loading

joadr commented Jun 28, 2018

psiofxt commented Jun 28, 2018 • edited Loading

phewitt commented Jun 29, 2018

phewitt commented Jun 29, 2018 • edited Loading

psiofxt commented Jun 29, 2018 • edited Loading

phewitt commented Jun 29, 2018 • edited Loading

phewitt commented Jun 29, 2018

phewitt commented Jun 29, 2018

psiofxt commented Jun 29, 2018

phewitt commented Jun 29, 2018 • edited Loading

psiofxt commented Jun 29, 2018

phewitt commented Jun 29, 2018 • edited Loading

osagga commented Jul 3, 2018

psiofxt commented Jul 3, 2018

osagga commented Jul 5, 2018 • edited Loading

hxzqlh commented Jul 6, 2018

osagga commented Jul 6, 2018

hxzqlh commented Jul 6, 2018

osagga commented Jul 6, 2018

haydencarlson commented Jul 19, 2018 • edited Loading

osagga commented Jul 19, 2018

haydencarlson commented Jul 19, 2018 • edited Loading

osagga commented Jul 19, 2018

qazxcvio commented Nov 9, 2018

1scrooge commented Nov 12, 2018

matiu commented Nov 12, 2018

some-dev commented Nov 12, 2018

some-dev commented Nov 12, 2018

msaikat commented Jan 16, 2019

dviramontes commented May 8, 2018 •

edited

Loading

zjb0807 commented Jun 12, 2018 •

edited

Loading

phewitt commented Jun 12, 2018 •

edited

Loading

psiofxt commented Jun 28, 2018 •

edited

Loading

psiofxt commented Jun 28, 2018 •

edited

Loading

phewitt commented Jun 29, 2018 •

edited

Loading

psiofxt commented Jun 29, 2018 •

edited

Loading

phewitt commented Jun 29, 2018 •

edited

Loading

phewitt commented Jun 29, 2018 •

edited

Loading

phewitt commented Jun 29, 2018 •

edited

Loading

osagga commented Jul 5, 2018 •

edited

Loading

haydencarlson commented Jul 19, 2018 •

edited

Loading

haydencarlson commented Jul 19, 2018 •

edited

Loading