Bitcore-explorers getUnspentUtxos( ) returns empty result


(Chano Palis) #1

i have 2 utxos on my address with $20 and $25 amounts, then i spent $30 using that address and it uses the 2 utxos, transaction goes well(but still unconfirmed) but when i get the utxos using the bitcore-explorers getUnspentUtxos( ) it returns no result… i also use the bitcore playground to get the uxtos of my address it returns none also… but when the transaction was CONFIRMED… the amount that should be remaining on my address was there now, and the bitcore-explorers getUnspentUtxos( ) is returning result…

but then i tried it on single utxo with $50 amount and spend $10 and get the utxo after the transaction using bitcore-explorers getUnspentUtxos( ) and it returns result…

so the issue is, if an address has multiple utxos and if all utxos of the address was used in transaction, the supposed to be change output( change utxo ) is not being returned by getUnspentUtxos( ) until the transaction is CONFIRMED…

but this issue is not occuring when the address has single(1) utxo and was use in transaction.

also this issue is not occuring if there’s a utxo that was not used in transaction…
like address have 3 utxos and only 2 was used…

any idea what cause this?

Thank you.


(Braydon Fuller) #2

Changes in the mempool are reflected in utxos responses, I don’t see any issues here, unless I am missing something.

Please see relevant code:

Insight API Route: https://github.com/bitpay/insight-api/blob/master/lib/index.js#L192
Insight API Controller: https://github.com/bitpay/insight-api/blob/master/lib/addresses.js#L138-149
Bitcore Node: https://github.com/bitpay/bitcore-node/blob/master/lib/services/bitcoind.js#L1062-1153
Bitcoin: https://github.com/bitpay/bitcoin/blob/0.12-bitcore/src/txmempool.cpp#L420-L476


(Braydon Fuller) #3

Expanded a few tests for the mempool indexing for multiple outputs, and tests are passing:


(Braydon Fuller) #4

Can you describe how to reproduce in more detail?


(Chano Palis) #5

thanks @braydon for the reply… here’s my code for spending manually, please check if I’m doing something wrong… thank you.

router.get('/manual/spent', function(req,res, next) {
    var publicKeys = ['0342d6169806e84153a7d9756c5f43115cf1b5c508248d69c9dfe8e58c22333af1', '0212a699415e0aa3425aadfbf259d322e0b0666727d0490dcd08705c2d194d6c11', '028c7153e34e09244f3162f11be27dc456c07efeadec7495527fc501d76d3b4fbf'];
    var privateKeys = ['241aea6c2213c951fa481141254267973574249c7d5c9e867a7d765f47018e07', '267061f4b5d60297bb88a094593a9d079a2b08f3547dfd726e508f664f36d137'];

    var redeemScript = bitcore.Script.buildMultisigOut(publicKeys, 2);
    var s = redeemScript.toScriptHashOut();

    var utxos = [{
        "address": "2MyNSAsUDbh6WzSNNC5XDrmx2u8ncyyduJC",
        "txid": "63472459757a154cb65f1816deedd8caa8a6cbf771e71f2cdde889d52e73aa1f",
        "vout": 0,
        "script": s,
        "amount": 0.00872692
    }, {
        "address": "2MyNSAsUDbh6WzSNNC5XDrmx2u8ncyyduJC",
        "txid": "6592277fcc62aa68cfc5634ce3f8a83af68248b26b7142587f25f688d12f5c15",
        "vout": 1,
        "script": s,
        "amount": 0.00872692
    }, {
        "address": "2MyNSAsUDbh6WzSNNC5XDrmx2u8ncyyduJC",
        "txid": "e7b0a53e1b5411872e5d486fabaf375e1297a53c21ec919080da447f8d1c077e",
        "vout": 0,
        "script": s,
        "amount": 0.01221768
    }];

    var transaction = new bitcore.Transaction();

    transaction.from(utxos, publicKeys, 2);
    transaction.change("2MyNSAsUDbh6WzSNNC5XDrmx2u8ncyyduJC");
    transaction.to('n2te8MVwJJBcV3Y3RkM7AKCfbsWssfNfJh', Unit.fromBTC(0.01393728).toSatoshis() );
    transaction.fee(5026).sign(privateKeys);

    var insight = new Insight("https://test-insight.bitpay.com", "testnet");
    insight.broadcast(transaction, function(err, returnedTxId) {

        if (err) {
            res.send(err)
        } else {
            res.json({ transaction_id: returnedTxId });
        }

    });
});

(Chano Palis) #6

hi @braydon, actually the above is ONLY USING 2 utxos… so i created new one. this uses 3 of the 3 utxos… please use this.

router.get('/manual/spent', function(req, res, next) {
    var publicKeys = ['03a6cbab1fd975d563f065837d80250559bf7d4951ff3db67b9e9d6d63f8c4714c', '0292c558bfc231a28ec78a63861a7c138321cc8458c8da92378261efd40c79592f', '023523e50660cb34418a870b000d7d09c5e4d71b5cb90ca10deb7e18383112eaeb'];
    var privateKeys = ['d3e312967f9a6febc3d3c2feba3846c970d2e21e1c3ce698ab719b9711e9d15f', '5e3e1d40fe09f2e46fc6625c165f2b559add1091937deaddc821ca6aeda94133'];

    var redeemScript = bitcore.Script.buildMultisigOut(publicKeys, 2);
    var s = redeemScript.toScriptHashOut();

    var utxos = [{
        "address": "2N7nWWBC5fq9LUunmbKwStLs67VZYGt2J8v",
        "txid": "57bbc09cef3812b4811414421690d3bd2642594a582ea3cea8cdefcd1d1e5a37",
        "vout": 1,
        "scriptPubKey": s,
        "amount": 0.0087108
    }, {
        "address": "2N7nWWBC5fq9LUunmbKwStLs67VZYGt2J8v",
        "txid": "32c6685cbd424b688c7fb7166be114d0f400b7a2a0ddaab6a16f15374da3d209",
        "vout": 1,
        "scriptPubKey": s,
        "amount": 0.0087108
    }, {
        "address": "2N7nWWBC5fq9LUunmbKwStLs67VZYGt2J8v",
        "txid": "4e1da9453c3841f3190d9d64ba7d909ed5772038a932796e84a6031cb26c112a",
        "vout": 0,
        "scriptPubKey": s,
        "amount": 0.01219512
    }];

    var transaction = new bitcore.Transaction();

    transaction.from(utxos, publicKeys, 2);
    transaction.change("2N7nWWBC5fq9LUunmbKwStLs67VZYGt2J8v");
    transaction.to('n2te8MVwJJBcV3Y3RkM7AKCfbsWssfNfJh', Unit.fromBTC(0.0244311).toSatoshis());
    transaction.fee(5026).sign(privateKeys);

    var insight = new Insight("https://test-insight.bitpay.com", "testnet");
    insight.broadcast(transaction, function(err, returnedTxId) {

        if (err) {
            res.send(err)
        } else {
            res.json({ transaction_id: returnedTxId });
        }

    });
});

screenshot of playground before sending…

screenshot of playground after sending…

thank you.


(Chano Palis) #7

here’s the screen video capture of my entire process

thank you.


(Braydon Fuller) #8

Thanks for the info.

I’ve added a couple of more tests to bitcore-node in relation to this, with tests passing: https://github.com/bitpay/bitcore-node/pull/456/files


(Braydon Fuller) #9

Okay, so I went through and did a full test, building a transaction and broadcasting and have been able to reproduce it now. Will look into it.

Edit: Identified that the problem is not with bitcore-node, but lower level. Running with a regtest bitcoind node, I’m getting three spending deltas, but not a receiving delta from getaddressmempool rpc call from bitcoind.

Edit: This is also not p2sh vs. p2pkh related

Edit: Created a test case to reproduce in bitcoind, and it looks related to reusing the change address

Here is the reproducable test case added to qa/rpc-tests/addressindex.py:

privkey1 = "cQY2s58LhzUCmEXN8jtAp1Etnijx78YRZ466w4ikX1V4UpTpbsf8"
address1 = "myAUWSHnwsQrhuMWv4Br6QsCnpB41vFwHn"
address1hash = "c192bff751af8efec15135d42bfeedf91a6f3e34".decode("hex")
address1script = CScript([OP_DUP, OP_HASH160, address1hash, OP_EQUALVERIFY, OP_CHECKSIG])

self.nodes[0].sendtoaddress(address1, 10)
self.nodes[0].generate(1)
self.sync_all()

utxos = self.nodes[1].getaddressutxos({"addresses": [address1]})
assert_equal(len(utxos), 1)

tx = CTransaction()
tx.vin = [
  CTxIn(COutPoint(int(utxos[0]["txid"], 16), utxos[0]["outputIndex"]))
]
amount = utxos[0]["satoshis"] - 1000
tx.vout = [CTxOut(amount, address1script)]
tx.rehash()
self.nodes[0].importprivkey(privkey1)
signed_tx = self.nodes[0].signrawtransaction(binascii.hexlify(tx.serialize()).decode("utf-8"))
mem_txid = self.nodes[0].sendrawtransaction(signed_tx["hex"], True)

self.sync_all()
mempool_deltas = self.nodes[2].getaddressmempool({"addresses": [address1]})
assert_equal(len(mempool_deltas), 2)

(Braydon Fuller) #10

Okay I have found the problem and have a patch for the fix:

In summary the bug was caused by an input and output in a transaction that would share the same address and index. In this case the output would not appear as it was conflicting with the already existing key or the input.


(Chano Palis) #11

thank you @braydon


(Chano Palis) #12

@braydon when is the estimate date that your fix be merged?

thank you.


#13

@braydon

It would be awesome if we could get this fix applied on bitpay’s public instance of insight. Any ETA on that?


#14

I see that the fix was merged, but the issue is still unfixed on bitpay’s public instance of insight.

How soon do you expect to update the server?


(Braydon Fuller) #15

We have a v0.12.1-bitcore-rc1 release out https://github.com/bitpay/bitcoin/releases/tag/v0.12.1-bitcore-rc1 with this fix, as well as upgrading/rebasing changes on the 0.12.1 release of bitcoind that includes BIP68, BIP112, and BIP113 and are in the process of testing.


(Braydon Fuller) #16

FYI, we have the new version running on test-insight.bitpay.com, and I’ve run the test case to confirm it’s patched.


#17

@braydon

Thanks. I’ve also confirmed that its working correctly now on test-insight.bitpay.com. We had switched to blockcypher in the meantime, but we just reverted to using insight. Do you know when the livenet explorer will be updated?