tfirst commit - electrum-personal-server - Maximally lightweight electrum server for a single user
HTML git clone https://git.parazyd.org/electrum-personal-server
DIR Log
DIR Files
DIR Refs
DIR README
---
DIR commit 99e2ff189e133a58ed70741aa85e145e8bfd8a5c
HTML Author: chris-belcher <chris-belcher@users.noreply.github.com>
Date: Thu, 8 Feb 2018 16:45:35 +0000
first commit
Diffstat:
A .gitignore | 4 ++++
A LICENCE | 20 ++++++++++++++++++++
A README.md | 88 +++++++++++++++++++++++++++++++
A bitcoin/__init__.py | 13 +++++++++++++
A bitcoin/deterministic.py | 128 +++++++++++++++++++++++++++++++
A bitcoin/main.py | 504 +++++++++++++++++++++++++++++++
A bitcoin/py2specials.py | 85 +++++++++++++++++++++++++++++++
A bitcoin/py3specials.py | 115 +++++++++++++++++++++++++++++++
A bitcoin/secp256k1_deterministic.py | 92 +++++++++++++++++++++++++++++++
A bitcoin/secp256k1_main.py | 375 +++++++++++++++++++++++++++++++
A bitcoin/secp256k1_transaction.py | 452 +++++++++++++++++++++++++++++++
A bitcoin/transaction.py | 490 +++++++++++++++++++++++++++++++
A config.cfg_sample | 37 +++++++++++++++++++++++++++++++
A jsonrpc.py | 59 +++++++++++++++++++++++++++++++
A run-server.bat | 4 ++++
A server.py | 634 +++++++++++++++++++++++++++++++
A util.py | 365 +++++++++++++++++++++++++++++++
17 files changed, 3465 insertions(+), 0 deletions(-)
---
DIR diff --git a/.gitignore b/.gitignore
t@@ -0,0 +1,4 @@
+*.pyc
+*.swp
+config.cfg
+
DIR diff --git a/LICENCE b/LICENCE
t@@ -0,0 +1,20 @@
+The MIT License (MIT)
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
DIR diff --git a/README.md b/README.md
t@@ -0,0 +1,88 @@
+# Electrum Personal Server
+
+Electrum Personal Server is an implementation of the Electrum server protocol
+which fulfills the specific need of using the Electrum UI with full node
+verification and privacy, but without the heavyweight server backend, for a
+single user. It allows the user to benefit from all of Bitcoin Core's
+resource-saving features like pruning, blocksonly and disabled txindex. All
+of Electrum's feature-richness like hardware wallet integration,
+multisignature wallets, offline signing, mnemonic recovery phrases and so on
+can still be used, but backed by the user's own full node.
+
+Using Electrum with Electrum Personal Server is probably the most
+resource-efficent way right now to use a hardware wallet connected to your
+own full node.
+
+For a longer explaination of this project and why it's important, see the
+[bitcointalk thread](https://bitcointalk.org/index.php?topic=2664747.msg27179198).
+
+See also the Electrum bitcoin wallet [website](https://electrum.org/).
+
+## How To Use
+
+This application requires python3 and a Bitcoin full node built with wallet
+capability.
+
+Download the latest release or clone the git repository. Enter the directory
+and rename the file `config.cfg_sample` to `config.cfg`, edit this file to
+configure your bitcoin node json-rpc authentication details. Next add your
+wallet addresses to the `[wallets]` section.
+
+Finally run `./server.py` on Linux or double-click `run-server.bat` on Windows.
+The first time the server is run it will import all configured addresses as
+watch-only into the Bitcoin node, and then exit giving you a chance to
+`-rescan` if your wallet contains historical transactions.
+
+Electrum wallet must be configured to connect to the server. SSL must be
+disabled which can be done either by `Tools` -> `Connection` -> Uncheck box
+`Use SSL`, or starting with the command line flag `--nossl`, depending on the
+version of Electrum. Tell Electrum to connect to the server in
+`Tools` -> `Server`, usually `localhost` if running on the same machine.
+
+Note that you can also try this with on [testnet bitcoin](https://en.bitcoin.it/wiki/Testnet).
+Electrum can be started in testnet mode with the command line flag `--testnet`.
+
+#### Exposure to the Internet
+
+You really don't want other people connecting to your server. They won't be
+able to synchronize their wallet, and they could potentially learn all your
+wallet addresses.
+
+By default the server will bind to and accept connections only from `localhost`
+so you should either run Electrum wallet from the same computer or use a SSH
+tunnel.
+
+## Project Readiness
+
+This project is in alpha stages as there are several essential missing
+features such as:
+
+* Merkle proofs are not handled, so every confirmed transaction is labelled
+ `Not Verified`.
+
+* The server does not support SSL so Electrum must be configured to disable it.
+
+* Deterministic wallets and master public keys are not supported. Addresses
+ must be imported individually.
+
+* Bech32 bitcoin addresses are not supported.
+
+* The Electrum server protocol has a caveat about multiple transactions included
+ in the same block. So there may be weird behaviour if that happens.
+
+* There's no way to turn off debug messages, so the console will be spammed by
+ them when used.
+
+When trying this, make sure you report any crashes, odd behaviour or times when
+Electrum disconnects (which indicates the server behaved unexpectedly).
+
+Someone should try running this on a Raspberry PI.
+
+## Contributing
+
+I welcome contributions. Please keep lines under 80 characters in length and
+ideally don't add any external dependencies to keep this as easy to install as
+possible.
+
+I can be contacted on freenode IRC on the `#bitcoin` and `#electrum` channels.
+
DIR diff --git a/bitcoin/__init__.py b/bitcoin/__init__.py
t@@ -0,0 +1,13 @@
+from bitcoin.py2specials import *
+from bitcoin.py3specials import *
+secp_present = False
+try:
+ import secp256k1
+ secp_present = True
+ from bitcoin.secp256k1_main import *
+ from bitcoin.secp256k1_transaction import *
+ from bitcoin.secp256k1_deterministic import *
+except ImportError as e:
+ from bitcoin.main import *
+ from bitcoin.deterministic import *
+ from bitcoin.transaction import *
DIR diff --git a/bitcoin/deterministic.py b/bitcoin/deterministic.py
t@@ -0,0 +1,128 @@
+from bitcoin.main import *
+import hmac
+import hashlib
+from binascii import hexlify
+
+# Below code ASSUMES binary inputs and compressed pubkeys
+MAINNET_PRIVATE = b'\x04\x88\xAD\xE4'
+MAINNET_PUBLIC = b'\x04\x88\xB2\x1E'
+TESTNET_PRIVATE = b'\x04\x35\x83\x94'
+TESTNET_PUBLIC = b'\x04\x35\x87\xCF'
+PRIVATE = [MAINNET_PRIVATE, TESTNET_PRIVATE]
+PUBLIC = [MAINNET_PUBLIC, TESTNET_PUBLIC]
+
+# BIP32 child key derivation
+
+def raw_bip32_ckd(rawtuple, i):
+ vbytes, depth, fingerprint, oldi, chaincode, key = rawtuple
+ i = int(i)
+
+ if vbytes in PRIVATE:
+ priv = key
+ pub = privtopub(key)
+ else:
+ pub = key
+
+ if i >= 2**31:
+ if vbytes in PUBLIC:
+ raise Exception("Can't do private derivation on public key!")
+ I = hmac.new(chaincode, b'\x00' + priv[:32] + encode(i, 256, 4),
+ hashlib.sha512).digest()
+ else:
+ I = hmac.new(chaincode, pub + encode(i, 256, 4),
+ hashlib.sha512).digest()
+
+ if vbytes in PRIVATE:
+ newkey = add_privkeys(I[:32] + B'\x01', priv)
+ fingerprint = bin_hash160(privtopub(key))[:4]
+ if vbytes in PUBLIC:
+ newkey = add_pubkeys(compress(privtopub(I[:32])), key)
+ fingerprint = bin_hash160(key)[:4]
+
+ return (vbytes, depth + 1, fingerprint, i, I[32:], newkey)
+
+
+def bip32_serialize(rawtuple):
+ vbytes, depth, fingerprint, i, chaincode, key = rawtuple
+ i = encode(i, 256, 4)
+ chaincode = encode(hash_to_int(chaincode), 256, 32)
+ keydata = b'\x00' + key[:-1] if vbytes in PRIVATE else key
+ bindata = vbytes + from_int_to_byte(
+ depth % 256) + fingerprint + i + chaincode + keydata
+ return changebase(bindata + bin_dbl_sha256(bindata)[:4], 256, 58)
+
+
+def bip32_deserialize(data):
+ dbin = changebase(data, 58, 256)
+ if bin_dbl_sha256(dbin[:-4])[:4] != dbin[-4:]:
+ raise Exception("Invalid checksum")
+ vbytes = dbin[0:4]
+ depth = from_byte_to_int(dbin[4])
+ fingerprint = dbin[5:9]
+ i = decode(dbin[9:13], 256)
+ chaincode = dbin[13:45]
+ key = dbin[46:78] + b'\x01' if vbytes in PRIVATE else dbin[45:78]
+ return (vbytes, depth, fingerprint, i, chaincode, key)
+
+
+def raw_bip32_privtopub(rawtuple):
+ vbytes, depth, fingerprint, i, chaincode, key = rawtuple
+ newvbytes = MAINNET_PUBLIC if vbytes == MAINNET_PRIVATE else TESTNET_PUBLIC
+ return (newvbytes, depth, fingerprint, i, chaincode, privtopub(key))
+
+
+def bip32_privtopub(data):
+ return bip32_serialize(raw_bip32_privtopub(bip32_deserialize(data)))
+
+
+def bip32_ckd(data, i):
+ return bip32_serialize(raw_bip32_ckd(bip32_deserialize(data), i))
+
+
+def bip32_master_key(seed, vbytes=MAINNET_PRIVATE):
+ I = hmac.new(
+ from_string_to_bytes("Bitcoin seed"), seed, hashlib.sha512).digest()
+ return bip32_serialize((vbytes, 0, b'\x00' * 4, 0, I[32:], I[:32] + b'\x01'
+ ))
+
+
+def bip32_bin_extract_key(data):
+ return bip32_deserialize(data)[-1]
+
+
+def bip32_extract_key(data):
+ return safe_hexlify(bip32_deserialize(data)[-1])
+
+# Exploits the same vulnerability as above in Electrum wallets
+# Takes a BIP32 pubkey and one of the child privkeys of its corresponding
+# privkey and returns the BIP32 privkey associated with that pubkey
+
+def raw_crack_bip32_privkey(parent_pub, priv):
+ vbytes, depth, fingerprint, i, chaincode, key = priv
+ pvbytes, pdepth, pfingerprint, pi, pchaincode, pkey = parent_pub
+ i = int(i)
+
+ if i >= 2**31:
+ raise Exception("Can't crack private derivation!")
+
+ I = hmac.new(pchaincode, pkey + encode(i, 256, 4), hashlib.sha512).digest()
+
+ pprivkey = subtract_privkeys(key, I[:32] + b'\x01')
+
+ newvbytes = MAINNET_PRIVATE if vbytes == MAINNET_PUBLIC else TESTNET_PRIVATE
+ return (newvbytes, pdepth, pfingerprint, pi, pchaincode, pprivkey)
+
+
+def crack_bip32_privkey(parent_pub, priv):
+ dsppub = bip32_deserialize(parent_pub)
+ dspriv = bip32_deserialize(priv)
+ return bip32_serialize(raw_crack_bip32_privkey(dsppub, dspriv))
+
+def bip32_descend(*args):
+ if len(args) == 2:
+ key, path = args
+ else:
+ key, path = args[0], map(int, args[1:])
+ for p in path:
+ key = bip32_ckd(key, p)
+ return bip32_extract_key(key)
DIR diff --git a/bitcoin/main.py b/bitcoin/main.py
t@@ -0,0 +1,504 @@
+#!/usr/bin/python
+from .py2specials import *
+from .py3specials import *
+import binascii
+import hashlib
+import re
+import sys
+import os
+import base64
+import time
+import random
+import hmac
+
+is_python2 = sys.version_info.major == 2
+
+# Elliptic curve parameters (secp256k1)
+
+P = 2**256 - 2**32 - 977
+N = 115792089237316195423570985008687907852837564279074904382605163141518161494337
+A = 0
+B = 7
+Gx = 55066263022277343669578718895168534326250603453777594175500187360389116729240
+Gy = 32670510020758816978083085130507043184471273380659243275938904335757337482424
+G = (Gx, Gy)
+
+# Extended Euclidean Algorithm
+def inv(a, n):
+ lm, hm = 1, 0
+ low, high = a % n, n
+ while low > 1:
+ r = high // low
+ nm, new = hm - lm * r, high - low * r
+ lm, low, hm, high = nm, new, lm, low
+ return lm % n
+
+# Elliptic curve Jordan form functions
+# P = (m, n, p, q) where m/n = x, p/q = y
+
+def isinf(p):
+ return p[0] == 0 and p[1] == 0
+
+
+def jordan_isinf(p):
+ return p[0][0] == 0 and p[1][0] == 0
+
+
+def mulcoords(c1, c2):
+ return (c1[0] * c2[0] % P, c1[1] * c2[1] % P)
+
+
+def mul_by_const(c, v):
+ return (c[0] * v % P, c[1])
+
+
+def addcoords(c1, c2):
+ return ((c1[0] * c2[1] + c2[0] * c1[1]) % P, c1[1] * c2[1] % P)
+
+
+def subcoords(c1, c2):
+ return ((c1[0] * c2[1] - c2[0] * c1[1]) % P, c1[1] * c2[1] % P)
+
+
+def invcoords(c):
+ return (c[1], c[0])
+
+
+def jordan_add(a, b):
+ if jordan_isinf(a):
+ return b
+ if jordan_isinf(b):
+ return a
+
+ if (a[0][0] * b[0][1] - b[0][0] * a[0][1]) % P == 0:
+ if (a[1][0] * b[1][1] - b[1][0] * a[1][1]) % P == 0:
+ return jordan_double(a)
+ else:
+ return ((0, 1), (0, 1))
+ xdiff = subcoords(b[0], a[0])
+ ydiff = subcoords(b[1], a[1])
+ m = mulcoords(ydiff, invcoords(xdiff))
+ x = subcoords(subcoords(mulcoords(m, m), a[0]), b[0])
+ y = subcoords(mulcoords(m, subcoords(a[0], x)), a[1])
+ return (x, y)
+
+
+def jordan_double(a):
+ if jordan_isinf(a):
+ return ((0, 1), (0, 1))
+ num = addcoords(mul_by_const(mulcoords(a[0], a[0]), 3), (A, 1))
+ den = mul_by_const(a[1], 2)
+ m = mulcoords(num, invcoords(den))
+ x = subcoords(mulcoords(m, m), mul_by_const(a[0], 2))
+ y = subcoords(mulcoords(m, subcoords(a[0], x)), a[1])
+ return (x, y)
+
+
+def jordan_multiply(a, n):
+ if jordan_isinf(a) or n == 0:
+ return ((0, 0), (0, 0))
+ if n == 1:
+ return a
+ if n < 0 or n >= N:
+ return jordan_multiply(a, n % N)
+ if (n % 2) == 0:
+ return jordan_double(jordan_multiply(a, n // 2))
+ if (n % 2) == 1:
+ return jordan_add(jordan_double(jordan_multiply(a, n // 2)), a)
+
+
+def to_jordan(p):
+ return ((p[0], 1), (p[1], 1))
+
+
+def from_jordan(p):
+ return (p[0][0] * inv(p[0][1], P) % P, p[1][0] * inv(p[1][1], P) % P)
+
+def fast_multiply(a, n):
+ return from_jordan(jordan_multiply(to_jordan(a), n))
+
+
+def fast_add(a, b):
+ return from_jordan(jordan_add(to_jordan(a), to_jordan(b)))
+
+# Functions for handling pubkey and privkey formats
+
+
+def get_pubkey_format(pub):
+ if is_python2:
+ two = '\x02'
+ three = '\x03'
+ four = '\x04'
+ else:
+ two = 2
+ three = 3
+ four = 4
+
+ if isinstance(pub, (tuple, list)): return 'decimal'
+ elif len(pub) == 65 and pub[0] == four: return 'bin'
+ elif len(pub) == 130 and pub[0:2] == '04': return 'hex'
+ elif len(pub) == 33 and pub[0] in [two, three]: return 'bin_compressed'
+ elif len(pub) == 66 and pub[0:2] in ['02', '03']: return 'hex_compressed'
+ elif len(pub) == 64: return 'bin_electrum'
+ elif len(pub) == 128: return 'hex_electrum'
+ else: raise Exception("Pubkey not in recognized format")
+
+
+def encode_pubkey(pub, formt):
+ if not isinstance(pub, (tuple, list)):
+ pub = decode_pubkey(pub)
+ if formt == 'decimal': return pub
+ elif formt == 'bin':
+ return b'\x04' + encode(pub[0], 256, 32) + encode(pub[1], 256, 32)
+ elif formt == 'bin_compressed':
+ return from_int_to_byte(2 + (pub[1] % 2)) + encode(pub[0], 256, 32)
+ elif formt == 'hex':
+ return '04' + encode(pub[0], 16, 64) + encode(pub[1], 16, 64)
+ elif formt == 'hex_compressed':
+ return '0' + str(2 + (pub[1] % 2)) + encode(pub[0], 16, 64)
+ elif formt == 'bin_electrum':
+ return encode(pub[0], 256, 32) + encode(pub[1], 256, 32)
+ elif formt == 'hex_electrum':
+ return encode(pub[0], 16, 64) + encode(pub[1], 16, 64)
+ else:
+ raise Exception("Invalid format!")
+
+
+def decode_pubkey(pub, formt=None):
+ if not formt: formt = get_pubkey_format(pub)
+ if formt == 'decimal': return pub
+ elif formt == 'bin':
+ return (decode(pub[1:33], 256), decode(pub[33:65], 256))
+ elif formt == 'bin_compressed':
+ x = decode(pub[1:33], 256)
+ beta = pow(int(x * x * x + A * x + B), int((P + 1) // 4), int(P))
+ y = (P - beta) if ((beta + from_byte_to_int(pub[0])) % 2) else beta
+ return (x, y)
+ elif formt == 'hex':
+ return (decode(pub[2:66], 16), decode(pub[66:130], 16))
+ elif formt == 'hex_compressed':
+ return decode_pubkey(safe_from_hex(pub), 'bin_compressed')
+ elif formt == 'bin_electrum':
+ return (decode(pub[:32], 256), decode(pub[32:64], 256))
+ elif formt == 'hex_electrum':
+ return (decode(pub[:64], 16), decode(pub[64:128], 16))
+ else:
+ raise Exception("Invalid format!")
+
+
+def get_privkey_format(priv):
+ if isinstance(priv, int_types): return 'decimal'
+ elif len(priv) == 32: return 'bin'
+ elif len(priv) == 33: return 'bin_compressed'
+ elif len(priv) == 64: return 'hex'
+ elif len(priv) == 66: return 'hex_compressed'
+ else:
+ bin_p = b58check_to_bin(priv)
+ if len(bin_p) == 32: return 'wif'
+ elif len(bin_p) == 33: return 'wif_compressed'
+ else: raise Exception("WIF does not represent privkey")
+
+
+def encode_privkey(priv, formt, vbyte=0):
+ if not isinstance(priv, int_types):
+ return encode_privkey(decode_privkey(priv), formt, vbyte)
+ if formt == 'decimal': return priv
+ elif formt == 'bin': return encode(priv, 256, 32)
+ elif formt == 'bin_compressed': return encode(priv, 256, 32) + b'\x01'
+ elif formt == 'hex': return encode(priv, 16, 64)
+ elif formt == 'hex_compressed': return encode(priv, 16, 64) + '01'
+ elif formt == 'wif':
+ return bin_to_b58check(encode(priv, 256, 32), 128 + int(vbyte))
+ elif formt == 'wif_compressed':
+ return bin_to_b58check(
+ encode(priv, 256, 32) + b'\x01', 128 + int(vbyte))
+ else:
+ raise Exception("Invalid format!")
+
+
+def decode_privkey(priv, formt=None):
+ if not formt: formt = get_privkey_format(priv)
+ if formt == 'decimal': return priv
+ elif formt == 'bin': return decode(priv, 256)
+ elif formt == 'bin_compressed': return decode(priv[:32], 256)
+ elif formt == 'hex': return decode(priv, 16)
+ elif formt == 'hex_compressed': return decode(priv[:64], 16)
+ elif formt == 'wif': return decode(b58check_to_bin(priv), 256)
+ elif formt == 'wif_compressed':
+ return decode(b58check_to_bin(priv)[:32], 256)
+ else:
+ raise Exception("WIF does not represent privkey")
+
+
+def add_pubkeys(p1, p2):
+ f1, f2 = get_pubkey_format(p1), get_pubkey_format(p2)
+ return encode_pubkey(
+ fast_add(
+ decode_pubkey(p1, f1), decode_pubkey(p2, f2)), f1)
+
+
+def add_privkeys(p1, p2):
+ f1, f2 = get_privkey_format(p1), get_privkey_format(p2)
+ return encode_privkey(
+ (decode_privkey(p1, f1) + decode_privkey(p2, f2)) % N, f1)
+
+
+def multiply(pubkey, privkey):
+ f1, f2 = get_pubkey_format(pubkey), get_privkey_format(privkey)
+ pubkey, privkey = decode_pubkey(pubkey, f1), decode_privkey(privkey, f2)
+ # http://safecurves.cr.yp.to/twist.html
+ if not isinf(pubkey) and (
+ pubkey[0]**3 + B - pubkey[1] * pubkey[1]) % P != 0:
+ raise Exception("Point not on curve")
+ return encode_pubkey(fast_multiply(pubkey, privkey), f1)
+
+
+def divide(pubkey, privkey):
+ factor = inv(decode_privkey(privkey), N)
+ return multiply(pubkey, factor)
+
+
+def compress(pubkey):
+ f = get_pubkey_format(pubkey)
+ if 'compressed' in f: return pubkey
+ elif f == 'bin':
+ return encode_pubkey(decode_pubkey(pubkey, f), 'bin_compressed')
+ elif f == 'hex' or f == 'decimal':
+ return encode_pubkey(decode_pubkey(pubkey, f), 'hex_compressed')
+
+
+def decompress(pubkey):
+ f = get_pubkey_format(pubkey)
+ if 'compressed' not in f: return pubkey
+ elif f == 'bin_compressed':
+ return encode_pubkey(decode_pubkey(pubkey, f), 'bin')
+ elif f == 'hex_compressed' or f == 'decimal':
+ return encode_pubkey(decode_pubkey(pubkey, f), 'hex')
+
+
+def privkey_to_pubkey(privkey):
+ f = get_privkey_format(privkey)
+ privkey = decode_privkey(privkey, f)
+ if privkey >= N:
+ raise Exception("Invalid privkey")
+ if f in ['bin', 'bin_compressed', 'hex', 'hex_compressed', 'decimal']:
+ return encode_pubkey(fast_multiply(G, privkey), f)
+ else:
+ return encode_pubkey(fast_multiply(G, privkey), f.replace('wif', 'hex'))
+
+
+privtopub = privkey_to_pubkey
+
+
+def privkey_to_address(priv, magicbyte=0):
+ return pubkey_to_address(privkey_to_pubkey(priv), magicbyte)
+
+
+privtoaddr = privkey_to_address
+
+
+def neg_pubkey(pubkey):
+ f = get_pubkey_format(pubkey)
+ pubkey = decode_pubkey(pubkey, f)
+ return encode_pubkey((pubkey[0], (P - pubkey[1]) % P), f)
+
+
+def neg_privkey(privkey):
+ f = get_privkey_format(privkey)
+ privkey = decode_privkey(privkey, f)
+ return encode_privkey((N - privkey) % N, f)
+
+
+def subtract_pubkeys(p1, p2):
+ f1, f2 = get_pubkey_format(p1), get_pubkey_format(p2)
+ k2 = decode_pubkey(p2, f2)
+ return encode_pubkey(
+ fast_add(
+ decode_pubkey(p1, f1), (k2[0], (P - k2[1]) % P)), f1)
+
+
+def subtract_privkeys(p1, p2):
+ f1, f2 = get_privkey_format(p1), get_privkey_format(p2)
+ k2 = decode_privkey(p2, f2)
+ return encode_privkey((decode_privkey(p1, f1) - k2) % N, f1)
+
+# Hashes
+
+
+def bin_hash160(string):
+ intermed = hashlib.sha256(string).digest()
+ digest = ''
+ digest = hashlib.new('ripemd160', intermed).digest()
+ return digest
+
+
+def hash160(string):
+ return safe_hexlify(bin_hash160(string))
+
+
+def bin_sha256(string):
+ binary_data = string if isinstance(string, bytes) else bytes(string,
+ 'utf-8')
+ return hashlib.sha256(binary_data).digest()
+
+
+def sha256(string):
+ return bytes_to_hex_string(bin_sha256(string))
+
+
+def bin_ripemd160(string):
+ digest = hashlib.new('ripemd160', string).digest()
+ return digest
+
+
+def ripemd160(string):
+ return safe_hexlify(bin_ripemd160(string))
+
+
+def bin_dbl_sha256(s):
+ bytes_to_hash = from_string_to_bytes(s)
+ return hashlib.sha256(hashlib.sha256(bytes_to_hash).digest()).digest()
+
+
+def dbl_sha256(string):
+ return safe_hexlify(bin_dbl_sha256(string))
+
+
+def bin_slowsha(string):
+ string = from_string_to_bytes(string)
+ orig_input = string
+ for i in range(100000):
+ string = hashlib.sha256(string + orig_input).digest()
+ return string
+
+
+def slowsha(string):
+ return safe_hexlify(bin_slowsha(string))
+
+
+def hash_to_int(x):
+ if len(x) in [40, 64]:
+ return decode(x, 16)
+ return decode(x, 256)
+
+
+def num_to_var_int(x):
+ x = int(x)
+ if x < 253: return from_int_to_byte(x)
+ elif x < 65536: return from_int_to_byte(253) + encode(x, 256, 2)[::-1]
+ elif x < 4294967296: return from_int_to_byte(254) + encode(x, 256, 4)[::-1]
+ else: return from_int_to_byte(255) + encode(x, 256, 8)[::-1]
+
+
+# WTF, Electrum?
+def electrum_sig_hash(message):
+ padded = b"\x18Bitcoin Signed Message:\n" + num_to_var_int(len(
+ message)) + from_string_to_bytes(message)
+ return bin_dbl_sha256(padded)
+
+# Encodings
+
+def b58check_to_bin(inp):
+ leadingzbytes = len(re.match('^1*', inp).group(0))
+ data = b'\x00' * leadingzbytes + changebase(inp, 58, 256)
+ assert bin_dbl_sha256(data[:-4])[:4] == data[-4:]
+ return data[1:-4]
+
+
+def get_version_byte(inp):
+ leadingzbytes = len(re.match('^1*', inp).group(0))
+ data = b'\x00' * leadingzbytes + changebase(inp, 58, 256)
+ assert bin_dbl_sha256(data[:-4])[:4] == data[-4:]
+ return ord(data[0])
+
+
+def hex_to_b58check(inp, magicbyte=0):
+ return bin_to_b58check(binascii.unhexlify(inp), magicbyte)
+
+
+def b58check_to_hex(inp):
+ return safe_hexlify(b58check_to_bin(inp))
+
+
+def pubkey_to_address(pubkey, magicbyte=0):
+ if isinstance(pubkey, (list, tuple)):
+ pubkey = encode_pubkey(pubkey, 'bin')
+ if len(pubkey) in [66, 130]:
+ return bin_to_b58check(
+ bin_hash160(binascii.unhexlify(pubkey)), magicbyte)
+ return bin_to_b58check(bin_hash160(pubkey), magicbyte)
+
+
+pubtoaddr = pubkey_to_address
+
+# EDCSA
+
+
+def encode_sig(v, r, s):
+ vb, rb, sb = from_int_to_byte(v), encode(r, 256), encode(s, 256)
+
+ result = base64.b64encode(vb + b'\x00' * (32 - len(rb)) + rb + b'\x00' * (
+ 32 - len(sb)) + sb)
+ return result if is_python2 else str(result, 'utf-8')
+
+
+def decode_sig(sig):
+ bytez = base64.b64decode(sig)
+ return from_byte_to_int(bytez[0]), decode(bytez[1:33], 256), decode(
+ bytez[33:], 256)
+
+# https://tools.ietf.org/html/rfc6979#section-3.2
+
+
+def deterministic_generate_k(msghash, priv):
+ v = b'\x01' * 32
+ k = b'\x00' * 32
+ priv = encode_privkey(priv, 'bin')
+ msghash = encode(hash_to_int(msghash), 256, 32)
+ k = hmac.new(k, v + b'\x00' + priv + msghash, hashlib.sha256).digest()
+ v = hmac.new(k, v, hashlib.sha256).digest()
+ k = hmac.new(k, v + b'\x01' + priv + msghash, hashlib.sha256).digest()
+ v = hmac.new(k, v, hashlib.sha256).digest()
+ return decode(hmac.new(k, v, hashlib.sha256).digest(), 256)
+
+
+def ecdsa_raw_sign(msghash, priv):
+
+ z = hash_to_int(msghash)
+ k = deterministic_generate_k(msghash, priv)
+
+ r, y = fast_multiply(G, k)
+ s = inv(k, N) * (z + r * decode_privkey(priv)) % N
+
+ return 27 + (y % 2), r, s
+
+
+def ecdsa_sign(msg, priv):
+ return encode_sig(*ecdsa_raw_sign(electrum_sig_hash(msg), priv))
+
+def ecdsa_raw_verify(msghash, vrs, pub):
+ v, r, s = vrs
+
+ w = inv(s, N)
+ z = hash_to_int(msghash)
+
+ u1, u2 = z * w % N, r * w % N
+ x, y = fast_add(fast_multiply(G, u1), fast_multiply(decode_pubkey(pub), u2))
+
+ return r == x
+
+def ecdsa_verify(msg, sig, pub):
+ return ecdsa_raw_verify(electrum_sig_hash(msg), decode_sig(sig), pub)
+
+def estimate_tx_size(ins, outs, txtype='p2pkh'):
+ '''Estimate transaction size.
+ Assuming p2pkh:
+ out: 8+1+3+2+20=34, in: 1+32+4+1+1+~73+1+1+33=147,
+ ver:4,seq:4, +2 (len in,out)
+ total ~= 34*len_out + 147*len_in + 10 (sig sizes vary slightly)
+ '''
+ if txtype=='p2pkh':
+ return 10 + ins*147 +34*outs
+ else:
+ raise NotImplementedError("Non p2pkh transaction size estimation not"+
+ "yet implemented")
DIR diff --git a/bitcoin/py2specials.py b/bitcoin/py2specials.py
t@@ -0,0 +1,85 @@
+import sys, re
+import binascii
+import os
+import hashlib
+
+if sys.version_info.major == 2:
+ string_types = (str, unicode)
+ string_or_bytes_types = string_types
+ int_types = (int, float, long)
+
+ # Base switching
+ code_strings = {
+ 2: '01',
+ 10: '0123456789',
+ 16: '0123456789abcdef',
+ 32: 'abcdefghijklmnopqrstuvwxyz234567',
+ 58: '123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz',
+ 256: ''.join([chr(x) for x in range(256)])
+ }
+
+ def bin_dbl_sha256(s):
+ bytes_to_hash = from_string_to_bytes(s)
+ return hashlib.sha256(hashlib.sha256(bytes_to_hash).digest()).digest()
+
+ def lpad(msg, symbol, length):
+ if len(msg) >= length:
+ return msg
+ return symbol * (length - len(msg)) + msg
+
+ def get_code_string(base):
+ if base in code_strings:
+ return code_strings[base]
+ else:
+ raise ValueError("Invalid base!")
+
+ def changebase(string, frm, to, minlen=0):
+ if frm == to:
+ return lpad(string, get_code_string(frm)[0], minlen)
+ return encode(decode(string, frm), to, minlen)
+
+ def bin_to_b58check(inp, magicbyte=0):
+ inp_fmtd = chr(int(magicbyte)) + inp
+ leadingzbytes = len(re.match('^\x00*', inp_fmtd).group(0))
+ checksum = bin_dbl_sha256(inp_fmtd)[:4]
+ return '1' * leadingzbytes + changebase(inp_fmtd + checksum, 256, 58)
+
+ def bytes_to_hex_string(b):
+ return b.encode('hex')
+
+ def safe_from_hex(s):
+ return s.decode('hex')
+
+ def from_int_to_byte(a):
+ return chr(a)
+
+ def from_byte_to_int(a):
+ return ord(a)
+
+ def from_string_to_bytes(a):
+ return a
+
+ def safe_hexlify(a):
+ return binascii.hexlify(a)
+
+ def encode(val, base, minlen=0):
+ base, minlen = int(base), int(minlen)
+ code_string = get_code_string(base)
+ result = ""
+ while val > 0:
+ result = code_string[val % base] + result
+ val //= base
+ return code_string[0] * max(minlen - len(result), 0) + result
+
+ def decode(string, base):
+ base = int(base)
+ code_string = get_code_string(base)
+ result = 0
+ if base == 16:
+ string = string.lower()
+ while len(string) > 0:
+ result *= base
+ result += code_string.find(string[0])
+ string = string[1:]
+ return result
+
DIR diff --git a/bitcoin/py3specials.py b/bitcoin/py3specials.py
t@@ -0,0 +1,115 @@
+import sys, os
+import binascii
+import hashlib
+
+if sys.version_info.major == 3:
+ string_types = (str)
+ string_or_bytes_types = (str, bytes)
+ int_types = (int, float)
+ # Base switching
+ code_strings = {
+ 2: '01',
+ 10: '0123456789',
+ 16: '0123456789abcdef',
+ 32: 'abcdefghijklmnopqrstuvwxyz234567',
+ 58: '123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz',
+ 256: ''.join([chr(x) for x in range(256)])
+ }
+
+ def bin_dbl_sha256(s):
+ bytes_to_hash = from_string_to_bytes(s)
+ return hashlib.sha256(hashlib.sha256(bytes_to_hash).digest()).digest()
+
+ def lpad(msg, symbol, length):
+ if len(msg) >= length:
+ return msg
+ return symbol * (length - len(msg)) + msg
+
+ def get_code_string(base):
+ if base in code_strings:
+ return code_strings[base]
+ else:
+ raise ValueError("Invalid base!")
+
+ def changebase(string, frm, to, minlen=0):
+ if frm == to:
+ return lpad(string, get_code_string(frm)[0], minlen)
+ return encode(decode(string, frm), to, minlen)
+
+ def bin_to_b58check(inp, magicbyte=0):
+ inp_fmtd = from_int_to_byte(int(magicbyte)) + inp
+
+ leadingzbytes = 0
+ for x in inp_fmtd:
+ if x != 0:
+ break
+ leadingzbytes += 1
+
+ checksum = bin_dbl_sha256(inp_fmtd)[:4]
+ return '1' * leadingzbytes + changebase(inp_fmtd + checksum, 256, 58)
+
+ def bytes_to_hex_string(b):
+ if isinstance(b, str):
+ return b
+
+ return ''.join('{:02x}'.format(y) for y in b)
+
+ def safe_from_hex(s):
+ return bytes.fromhex(s)
+
+ def from_int_to_byte(a):
+ return bytes([a])
+
+ def from_byte_to_int(a):
+ return a
+
+ def from_string_to_bytes(a):
+ return a if isinstance(a, bytes) else bytes(a, 'utf-8')
+
+ def safe_hexlify(a):
+ return str(binascii.hexlify(a), 'utf-8')
+
+ def encode(val, base, minlen=0):
+ base, minlen = int(base), int(minlen)
+ code_string = get_code_string(base)
+ result_bytes = bytes()
+ while val > 0:
+ curcode = code_string[val % base]
+ result_bytes = bytes([ord(curcode)]) + result_bytes
+ val //= base
+
+ pad_size = minlen - len(result_bytes)
+
+ padding_element = b'\x00' if base == 256 else b'1' \
+ if base == 58 else b'0'
+ if (pad_size > 0):
+ result_bytes = padding_element * pad_size + result_bytes
+
+ result_string = ''.join([chr(y) for y in result_bytes])
+ result = result_bytes if base == 256 else result_string
+
+ return result
+
+ def decode(string, base):
+ if base == 256 and isinstance(string, str):
+ string = bytes(bytearray.fromhex(string))
+ base = int(base)
+ code_string = get_code_string(base)
+ result = 0
+ if base == 256:
+
+ def extract(d, cs):
+ return d
+ else:
+
+ def extract(d, cs):
+ return cs.find(d if isinstance(d, str) else chr(d))
+
+ if base == 16:
+ string = string.lower()
+ while len(string) > 0:
+ result *= base
+ result += extract(string[0], code_string)
+ string = string[1:]
+ return result
+
DIR diff --git a/bitcoin/secp256k1_deterministic.py b/bitcoin/secp256k1_deterministic.py
t@@ -0,0 +1,92 @@
+from bitcoin.secp256k1_main import *
+import hmac
+import hashlib
+from binascii import hexlify
+
+# Below code ASSUMES binary inputs and compressed pubkeys
+MAINNET_PRIVATE = b'\x04\x88\xAD\xE4'
+MAINNET_PUBLIC = b'\x04\x88\xB2\x1E'
+TESTNET_PRIVATE = b'\x04\x35\x83\x94'
+TESTNET_PUBLIC = b'\x04\x35\x87\xCF'
+PRIVATE = [MAINNET_PRIVATE, TESTNET_PRIVATE]
+PUBLIC = [MAINNET_PUBLIC, TESTNET_PUBLIC]
+
+# BIP32 child key derivation
+
+def raw_bip32_ckd(rawtuple, i):
+ vbytes, depth, fingerprint, oldi, chaincode, key = rawtuple
+ i = int(i)
+
+ if vbytes in PRIVATE:
+ priv = key
+ pub = privtopub(key, False)
+ else:
+ pub = key
+
+ if i >= 2**31:
+ if vbytes in PUBLIC:
+ raise Exception("Can't do private derivation on public key!")
+ I = hmac.new(chaincode, b'\x00' + priv[:32] + encode(i, 256, 4),
+ hashlib.sha512).digest()
+ else:
+ I = hmac.new(chaincode, pub + encode(i, 256, 4),
+ hashlib.sha512).digest()
+
+ if vbytes in PRIVATE:
+ newkey = add_privkeys(I[:32] + B'\x01', priv, False)
+ fingerprint = bin_hash160(privtopub(key, False))[:4]
+ if vbytes in PUBLIC:
+ newkey = add_pubkeys([privtopub(I[:32] + '\x01', False), key], False)
+ fingerprint = bin_hash160(key)[:4]
+
+ return (vbytes, depth + 1, fingerprint, i, I[32:], newkey)
+
+def bip32_serialize(rawtuple):
+ vbytes, depth, fingerprint, i, chaincode, key = rawtuple
+ i = encode(i, 256, 4)
+ chaincode = encode(hash_to_int(chaincode), 256, 32)
+ keydata = b'\x00' + key[:-1] if vbytes in PRIVATE else key
+ bindata = vbytes + from_int_to_byte(
+ depth % 256) + fingerprint + i + chaincode + keydata
+ return changebase(bindata + bin_dbl_sha256(bindata)[:4], 256, 58)
+
+def bip32_deserialize(data):
+ dbin = changebase(data, 58, 256)
+ if bin_dbl_sha256(dbin[:-4])[:4] != dbin[-4:]:
+ raise Exception("Invalid checksum")
+ vbytes = dbin[0:4]
+ depth = from_byte_to_int(dbin[4])
+ fingerprint = dbin[5:9]
+ i = decode(dbin[9:13], 256)
+ chaincode = dbin[13:45]
+ key = dbin[46:78] + b'\x01' if vbytes in PRIVATE else dbin[45:78]
+ return (vbytes, depth, fingerprint, i, chaincode, key)
+
+def raw_bip32_privtopub(rawtuple):
+ vbytes, depth, fingerprint, i, chaincode, key = rawtuple
+ newvbytes = MAINNET_PUBLIC if vbytes == MAINNET_PRIVATE else TESTNET_PUBLIC
+ return (newvbytes, depth, fingerprint, i, chaincode, privtopub(key, False))
+
+def bip32_privtopub(data):
+ return bip32_serialize(raw_bip32_privtopub(bip32_deserialize(data)))
+
+def bip32_ckd(data, i):
+ return bip32_serialize(raw_bip32_ckd(bip32_deserialize(data), i))
+
+def bip32_master_key(seed, vbytes=MAINNET_PRIVATE):
+ I = hmac.new(
+ from_string_to_bytes("Bitcoin seed"), seed, hashlib.sha512).digest()
+ return bip32_serialize((vbytes, 0, b'\x00' * 4, 0, I[32:], I[:32] + b'\x01'
+ ))
+
+def bip32_extract_key(data):
+ return safe_hexlify(bip32_deserialize(data)[-1])
+
+def bip32_descend(*args):
+ if len(args) == 2:
+ key, path = args
+ else:
+ key, path = args[0], map(int, args[1:])
+ for p in path:
+ key = bip32_ckd(key, p)
+ return bip32_extract_key(key)
DIR diff --git a/bitcoin/secp256k1_main.py b/bitcoin/secp256k1_main.py
t@@ -0,0 +1,375 @@
+#!/usr/bin/python
+from .py2specials import *
+from .py3specials import *
+import binascii
+import hashlib
+import re
+import sys
+import os
+import base64
+import time
+import random
+import hmac
+import secp256k1
+
+ctx = secp256k1.lib.secp256k1_context_create(secp256k1.ALL_FLAGS)
+
+def privkey_to_address(priv, from_hex=True, magicbyte=0):
+ return pubkey_to_address(privkey_to_pubkey(priv, from_hex), magicbyte)
+
+privtoaddr = privkey_to_address
+
+# Hashes
+def bin_hash160(string):
+ intermed = hashlib.sha256(string).digest()
+ return hashlib.new('ripemd160', intermed).digest()
+
+def hash160(string):
+ return safe_hexlify(bin_hash160(string))
+
+def bin_sha256(string):
+ binary_data = string if isinstance(string, bytes) else bytes(string,
+ 'utf-8')
+ return hashlib.sha256(binary_data).digest()
+
+def sha256(string):
+ return bytes_to_hex_string(bin_sha256(string))
+
+def bin_dbl_sha256(s):
+ bytes_to_hash = from_string_to_bytes(s)
+ return hashlib.sha256(hashlib.sha256(bytes_to_hash).digest()).digest()
+
+def dbl_sha256(string):
+ return safe_hexlify(bin_dbl_sha256(string))
+
+def hash_to_int(x):
+ if len(x) in [40, 64]:
+ return decode(x, 16)
+ return decode(x, 256)
+
+def num_to_var_int(x):
+ x = int(x)
+ if x < 253: return from_int_to_byte(x)
+ elif x < 65536: return from_int_to_byte(253) + encode(x, 256, 2)[::-1]
+ elif x < 4294967296: return from_int_to_byte(254) + encode(x, 256, 4)[::-1]
+ else: return from_int_to_byte(255) + encode(x, 256, 8)[::-1]
+
+# WTF, Electrum?
+def electrum_sig_hash(message):
+ padded = b"\x18Bitcoin Signed Message:\n" + num_to_var_int(len(
+ message)) + from_string_to_bytes(message)
+ return bin_dbl_sha256(padded)
+
+# Encodings
+def b58check_to_bin(inp):
+ leadingzbytes = len(re.match('^1*', inp).group(0))
+ data = b'\x00' * leadingzbytes + changebase(inp, 58, 256)
+ assert bin_dbl_sha256(data[:-4])[:4] == data[-4:]
+ return data[1:-4]
+
+def get_version_byte(inp):
+ leadingzbytes = len(re.match('^1*', inp).group(0))
+ data = b'\x00' * leadingzbytes + changebase(inp, 58, 256)
+ assert bin_dbl_sha256(data[:-4])[:4] == data[-4:]
+ return ord(data[0])
+
+def hex_to_b58check(inp, magicbyte=0):
+ return bin_to_b58check(binascii.unhexlify(inp), magicbyte)
+
+def b58check_to_hex(inp):
+ return safe_hexlify(b58check_to_bin(inp))
+
+def pubkey_to_address(pubkey, magicbyte=0):
+ if len(pubkey) in [66, 130]:
+ return bin_to_b58check(
+ bin_hash160(binascii.unhexlify(pubkey)), magicbyte)
+ return bin_to_b58check(bin_hash160(pubkey), magicbyte)
+
+pubtoaddr = pubkey_to_address
+
+def wif_compressed_privkey(priv, vbyte=0):
+ """Convert privkey in hex compressed to WIF compressed
+ """
+ if len(priv) != 66:
+ raise Exception("Wrong length of compressed private key")
+ if priv[-2:] != '01':
+ raise Exception("Private key has wrong compression byte")
+ return bin_to_b58check(binascii.unhexlify(priv), 128 + int(vbyte))
+
+
+def from_wif_privkey(wif_priv, compressed=True, vbyte=0):
+ """Convert WIF compressed privkey to hex compressed.
+ Caller specifies the network version byte (0 for mainnet, 0x6f
+ for testnet) that the key should correspond to; if there is
+ a mismatch an error is thrown. WIF encoding uses 128+ this number.
+ """
+ bin_key = b58check_to_bin(wif_priv)
+ claimed_version_byte = get_version_byte(wif_priv)
+ if not 128+vbyte == claimed_version_byte:
+ raise Exception(
+ "WIF key version byte is wrong network (mainnet/testnet?)")
+ if compressed and not len(bin_key) == 33:
+ raise Exception("Compressed private key is not 33 bytes")
+ if compressed and not bin_key[-1] == '\x01':
+ raise Exception("Private key has incorrect compression byte")
+ return safe_hexlify(bin_key)
+
+def ecdsa_sign(msg, priv, usehex=True):
+ #Compatibility issue: old bots will be confused
+ #by different msg hashing algo; need to keep electrum_sig_hash, temporarily.
+ hashed_msg = electrum_sig_hash(msg)
+ if usehex:
+ #arguments to raw sign must be consistently hex or bin
+ hashed_msg = binascii.hexlify(hashed_msg)
+ dersig = ecdsa_raw_sign(hashed_msg, priv, usehex, rawmsg=True)
+ #see comments to legacy* functions
+ #also, note those functions only handles binary, not hex
+ if usehex:
+ dersig = binascii.unhexlify(dersig)
+ sig = legacy_ecdsa_sign_convert(dersig)
+ return base64.b64encode(sig)
+
+def ecdsa_verify(msg, sig, pub, usehex=True):
+ #See note to ecdsa_sign
+ hashed_msg = electrum_sig_hash(msg)
+ sig = base64.b64decode(sig)
+ #see comments to legacy* functions
+ sig = legacy_ecdsa_verify_convert(sig)
+ if usehex:
+ #arguments to raw_verify must be consistently hex or bin
+ hashed_msg = binascii.hexlify(hashed_msg)
+ sig = binascii.hexlify(sig)
+ return ecdsa_raw_verify(hashed_msg, pub, sig, usehex, rawmsg=True)
+
+#A sadly necessary hack until all joinmarket bots are running secp256k1 code.
+#pybitcointools *message* signatures (not transaction signatures) used an old signature
+#format, basically: [27+y%2] || 32 byte r || 32 byte s,
+#instead of DER. These two functions translate the new version into the old so that
+#counterparty bots can verify successfully.
+def legacy_ecdsa_sign_convert(dersig):
+ #note there is no sanity checking of DER format (e.g. leading length byte)
+ dersig = dersig[2:] #e.g. 3045
+ rlen = ord(dersig[1]) #ignore leading 02
+ #length of r and s: ALWAYS <=33, USUALLY >=32 but can be shorter
+ if rlen > 33:
+ raise Exception("Incorrectly formatted DER sig:" + binascii.hexlify(
+ dersig))
+ if dersig[2] == '\x00':
+ r = dersig[3:2 + rlen]
+ ssig = dersig[2 + rlen:]
+ else:
+ r = dersig[2:2 + rlen]
+ ssig = dersig[2 + rlen:]
+
+ slen = ord(ssig[1]) #ignore leading 02
+ if slen > 33:
+ raise Exception("Incorrectly formatted DER sig:" + binascii.hexlify(
+ dersig))
+ if len(ssig) != 2 + slen:
+ raise Exception("Incorrectly formatted DER sig:" + binascii.hexlify(
+ dersig))
+ if ssig[2] == '\x00':
+ s = ssig[3:2 + slen]
+ else:
+ s = ssig[2:2 + slen]
+
+ #the legacy version requires padding of r and s to 32 bytes with leading zeros
+ r = '\x00' * (32 - len(r)) + r
+ s = '\x00' * (32 - len(s)) + s
+
+ #note: in the original pybitcointools implementation,
+ #verification ignored the leading byte (it's only needed for pubkey recovery)
+ #so we just ignore parity here.
+ return chr(27) + r + s
+
+def legacy_ecdsa_verify_convert(sig):
+ sig = sig[1:] #ignore parity byte
+ r, s = sig[:32], sig[32:]
+ if not len(s) == 32:
+ #signature is invalid.
+ return False
+ #legacy code can produce high S. Need to reintroduce N ::cry::
+ N = 115792089237316195423570985008687907852837564279074904382605163141518161494337
+ s_int = decode(s, 256)
+ # note // is integer division operator in both 2.7 and 3
+ s_int = N - s_int if s_int > N // 2 else s_int #enforce low S.
+
+ #on re-encoding, don't use the minlen parameter, because
+ #DER does not used fixed (32 byte) length values, so we
+ #don't prepend zero bytes to shorter numbers.
+ s = encode(s_int, 256)
+
+ #as above, remove any front zero padding from r.
+ r = encode(decode(r, 256), 256)
+
+ #canonicalize r and s
+ r, s = ['\x00' + x if ord(x[0]) > 127 else x for x in [r, s]]
+ rlen = chr(len(r))
+ slen = chr(len(s))
+ total_len = 2 + len(r) + 2 + len(s)
+ return '\x30' + chr(total_len) + '\x02' + rlen + r + '\x02' + slen + s
+
+#Use secp256k1 to handle all EC and ECDSA operations.
+#Data types: only hex and binary.
+#Compressed and uncompressed private and public keys.
+def hexbin(func):
+ '''To enable each function to 'speak' either hex or binary,
+ requires that the decorated function's final positional argument
+ is a boolean flag, True for hex and False for binary.
+ '''
+
+ def func_wrapper(*args, **kwargs):
+ if args[-1]:
+ newargs = []
+ for arg in args[:-1]:
+ if isinstance(arg, (list, tuple)):
+ newargs += [[x.decode('hex') for x in arg]]
+ else:
+ newargs += [arg.decode('hex')]
+ newargs += [False]
+ returnval = func(*newargs, **kwargs)
+ if isinstance(returnval, bool):
+ return returnval
+ else:
+ return binascii.hexlify(returnval)
+ else:
+ return func(*args, **kwargs)
+
+ return func_wrapper
+
+def read_privkey(priv):
+ if len(priv) == 33:
+ if priv[-1] == '\x01':
+ compressed = True
+ else:
+ raise Exception("Invalid private key")
+ elif len(priv) == 32:
+ compressed = False
+ else:
+ raise Exception("Invalid private key")
+ return (compressed, priv[:32])
+
+@hexbin
+def privkey_to_pubkey_inner(priv, usehex):
+ '''Take 32/33 byte raw private key as input.
+ If 32 bytes, return compressed (33 byte) raw public key.
+ If 33 bytes, read the final byte as compression flag,
+ and return compressed/uncompressed public key as appropriate.'''
+ compressed, priv = read_privkey(priv)
+ #secp256k1 checks for validity of key value.
+ newpriv = secp256k1.PrivateKey(privkey=priv, ctx=ctx)
+ return newpriv.pubkey.serialize(compressed=compressed)
+
+def privkey_to_pubkey(priv, usehex=True):
+ '''To avoid changing the interface from the legacy system,
+ allow an *optional* hex argument here (called differently from
+ maker/taker code to how it's called in bip32 code), then
+ pass to the standard hexbin decorator under the hood.
+ '''
+ return privkey_to_pubkey_inner(priv, usehex)
+
+privtopub = privkey_to_pubkey
+
+@hexbin
+def multiply(s, pub, usehex, rawpub=True):
+ '''Input binary compressed pubkey P(33 bytes)
+ and scalar s(32 bytes), return s*P.
+ The return value is a binary compressed public key.
+ Note that the called function does the type checking
+ of the scalar s.
+ ('raw' options passed in)
+ '''
+ newpub = secp256k1.PublicKey(pub, raw=rawpub, ctx=ctx)
+ res = newpub.tweak_mul(s)
+ return res.serialize()
+
+@hexbin
+def add_pubkeys(pubkeys, usehex):
+ '''Input a list of binary compressed pubkeys
+ and return their sum as a binary compressed pubkey.'''
+ r = secp256k1.PublicKey(ctx=ctx) #dummy holding object
+ pubkey_list = [secp256k1.PublicKey(x,
+ raw=True,
+ ctx=ctx).public_key for x in pubkeys]
+ r.combine(pubkey_list)
+ return r.serialize()
+
+@hexbin
+def add_privkeys(priv1, priv2, usehex):
+ '''Add privkey 1 to privkey 2.
+ Input keys must be in binary either compressed or not.
+ Returned key will have the same compression state.
+ Error if compression state of both input keys is not the same.'''
+ y, z = [read_privkey(x) for x in [priv1, priv2]]
+ if y[0] != z[0]:
+ raise Exception("cannot add privkeys, mixed compression formats")
+ else:
+ compressed = y[0]
+ newpriv1, newpriv2 = (y[1], z[1])
+ p1 = secp256k1.PrivateKey(newpriv1, raw=True, ctx=ctx)
+ res = p1.tweak_add(newpriv2)
+ if compressed:
+ res += '\x01'
+ return res
+
+@hexbin
+def ecdsa_raw_sign(msg,
+ priv,
+ usehex,
+ rawpriv=True,
+ rawmsg=False,
+ usenonce=None):
+ '''Take the binary message msg and sign it with the private key
+ priv.
+ By default priv is just a 32 byte string, if rawpriv is false
+ it is assumed to be DER encoded.
+ If rawmsg is True, no sha256 hash is applied to msg before signing.
+ In this case, msg must be a precalculated hash (256 bit).
+ If rawmsg is False, the secp256k1 lib will hash the message as part
+ of the ECDSA-SHA256 signing algo.
+ If usenonce is not None, its value is passed to the secp256k1 library
+ sign() function as the ndata value, which is then used in conjunction
+ with a custom nonce generating function, such that the nonce used in the ECDSA
+ sign algorithm is exactly that value (ndata there, usenonce here). 32 bytes.
+ Return value: the calculated signature.'''
+ if rawmsg and len(msg) != 32:
+ raise Exception("Invalid hash input to ECDSA raw sign.")
+ if rawpriv:
+ compressed, p = read_privkey(priv)
+ newpriv = secp256k1.PrivateKey(p, raw=True, ctx=ctx)
+ else:
+ newpriv = secp256k1.PrivateKey(priv, raw=False, ctx=ctx)
+ if usenonce and len(usenonce) != 32:
+ raise ValueError("Invalid nonce passed to ecdsa_sign: " + str(usenonce))
+
+ sig = newpriv.ecdsa_sign(msg, raw=rawmsg)
+ return newpriv.ecdsa_serialize(sig)
+
+@hexbin
+def ecdsa_raw_verify(msg, pub, sig, usehex, rawmsg=False):
+ '''Take the binary message msg and binary signature sig,
+ and verify it against the pubkey pub.
+ If rawmsg is True, no sha256 hash is applied to msg before verifying.
+ In this case, msg must be a precalculated hash (256 bit).
+ If rawmsg is False, the secp256k1 lib will hash the message as part
+ of the ECDSA-SHA256 verification algo.
+ Return value: True if the signature is valid for this pubkey, False
+ otherwise. '''
+ if rawmsg and len(msg) != 32:
+ raise Exception("Invalid hash input to ECDSA raw sign.")
+ newpub = secp256k1.PublicKey(pubkey=pub, raw=True, ctx=ctx)
+ sigobj = newpub.ecdsa_deserialize(sig)
+ return newpub.ecdsa_verify(msg, sigobj, raw=rawmsg)
+
+def estimate_tx_size(ins, outs, txtype='p2pkh'):
+ '''Estimate transaction size.
+ Assuming p2pkh:
+ out: 8+1+3+2+20=34, in: 1+32+4+1+1+~73+1+1+33=147,
+ ver:4,seq:4, +2 (len in,out)
+ total ~= 34*len_out + 147*len_in + 10 (sig sizes vary slightly)
+ '''
+ if txtype == 'p2pkh':
+ return 10 + ins * 147 + 34 * outs
+ else:
+ raise NotImplementedError("Non p2pkh transaction size estimation not" +
+ "yet implemented")
DIR diff --git a/bitcoin/secp256k1_transaction.py b/bitcoin/secp256k1_transaction.py
t@@ -0,0 +1,452 @@
+#!/usr/bin/python
+import binascii, re, json, copy, sys
+from bitcoin.secp256k1_main import *
+from _functools import reduce
+import os
+
+is_python2 = sys.version_info.major == 2
+
+### Hex to bin converter and vice versa for objects
+def json_is_base(obj, base):
+ if not is_python2 and isinstance(obj, bytes):
+ return False
+
+ alpha = get_code_string(base)
+ if isinstance(obj, string_types):
+ for i in range(len(obj)):
+ if alpha.find(obj[i]) == -1:
+ return False
+ return True
+ elif isinstance(obj, int_types) or obj is None:
+ return True
+ elif isinstance(obj, list):
+ for i in range(len(obj)):
+ if not json_is_base(obj[i], base):
+ return False
+ return True
+ else:
+ for x in obj:
+ if not json_is_base(obj[x], base):
+ return False
+ return True
+
+
+def json_changebase(obj, changer):
+ if isinstance(obj, string_or_bytes_types):
+ return changer(obj)
+ elif isinstance(obj, int_types) or obj is None:
+ return obj
+ elif isinstance(obj, list):
+ return [json_changebase(x, changer) for x in obj]
+ return dict((x, json_changebase(obj[x], changer)) for x in obj)
+
+# Transaction serialization and deserialization
+
+
+def deserialize(tx):
+ if isinstance(tx, str) and re.match('^[0-9a-fA-F]*$', tx):
+ #tx = bytes(bytearray.fromhex(tx))
+ return json_changebase(
+ deserialize(binascii.unhexlify(tx)), lambda x: safe_hexlify(x))
+ # http://stackoverflow.com/questions/4851463/python-closure-write-to-variable-in-parent-scope
+ # Python's scoping rules are demented, requiring me to make pos an object
+ # so that it is call-by-reference
+ pos = [0]
+
+ def read_as_int(bytez):
+ pos[0] += bytez
+ return decode(tx[pos[0] - bytez:pos[0]][::-1], 256)
+
+ def read_var_int():
+ pos[0] += 1
+
+ val = from_byte_to_int(tx[pos[0] - 1])
+ if val < 253:
+ return val
+ return read_as_int(pow(2, val - 252))
+
+ def read_bytes(bytez):
+ pos[0] += bytez
+ return tx[pos[0] - bytez:pos[0]]
+
+ def read_var_string():
+ size = read_var_int()
+ return read_bytes(size)
+
+ obj = {"ins": [], "outs": []}
+ obj["version"] = read_as_int(4)
+ ins = read_var_int()
+ for i in range(ins):
+ obj["ins"].append({
+ "outpoint": {
+ "hash": read_bytes(32)[::-1],
+ "index": read_as_int(4)
+ },
+ "script": read_var_string(),
+ "sequence": read_as_int(4)
+ })
+ outs = read_var_int()
+ for i in range(outs):
+ obj["outs"].append({
+ "value": read_as_int(8),
+ "script": read_var_string()
+ })
+ obj["locktime"] = read_as_int(4)
+ return obj
+
+
+def serialize(txobj):
+ #if isinstance(txobj, bytes):
+ # txobj = bytes_to_hex_string(txobj)
+ o = []
+ if json_is_base(txobj, 16):
+ json_changedbase = json_changebase(txobj,
+ lambda x: binascii.unhexlify(x))
+ hexlified = safe_hexlify(serialize(json_changedbase))
+ return hexlified
+ o.append(encode(txobj["version"], 256, 4)[::-1])
+ o.append(num_to_var_int(len(txobj["ins"])))
+ for inp in txobj["ins"]:
+ o.append(inp["outpoint"]["hash"][::-1])
+ o.append(encode(inp["outpoint"]["index"], 256, 4)[::-1])
+ o.append(num_to_var_int(len(inp["script"])) + (inp["script"] if inp[
+ "script"] or is_python2 else bytes()))
+ o.append(encode(inp["sequence"], 256, 4)[::-1])
+ o.append(num_to_var_int(len(txobj["outs"])))
+ for out in txobj["outs"]:
+ o.append(encode(out["value"], 256, 8)[::-1])
+ o.append(num_to_var_int(len(out["script"])) + out["script"])
+ o.append(encode(txobj["locktime"], 256, 4)[::-1])
+
+ return ''.join(o) if is_python2 else reduce(lambda x, y: x + y, o, bytes())
+
+# Hashing transactions for signing
+
+SIGHASH_ALL = 1
+SIGHASH_NONE = 2
+SIGHASH_SINGLE = 3
+SIGHASH_ANYONECANPAY = 0x80
+
+def signature_form(tx, i, script, hashcode=SIGHASH_ALL):
+ i, hashcode = int(i), int(hashcode)
+ if isinstance(tx, string_or_bytes_types):
+ return serialize(signature_form(deserialize(tx), i, script, hashcode))
+ newtx = copy.deepcopy(tx)
+ for inp in newtx["ins"]:
+ inp["script"] = ""
+ newtx["ins"][i]["script"] = script
+ if hashcode & 0x1f == SIGHASH_NONE:
+ newtx["outs"] = []
+ for j, inp in enumerate(newtx["ins"]):
+ if j != i:
+ inp["sequence"] = 0
+ elif hashcode & 0x1f == SIGHASH_SINGLE:
+ if len(newtx["ins"]) > len(newtx["outs"]):
+ raise Exception(
+ "Transactions with sighash single should have len in <= len out")
+ newtx["outs"] = newtx["outs"][:i+1]
+ for out in newtx["outs"][:i]:
+ out['value'] = 2**64 - 1
+ out['script'] = ""
+ for j, inp in enumerate(newtx["ins"]):
+ if j != i:
+ inp["sequence"] = 0
+ if hashcode & SIGHASH_ANYONECANPAY:
+ newtx["ins"] = [newtx["ins"][i]]
+ else:
+ pass
+ return newtx
+
+def txhash(tx, hashcode=None):
+ if isinstance(tx, str) and re.match('^[0-9a-fA-F]*$', tx):
+ tx = changebase(tx, 16, 256)
+ if hashcode:
+ return dbl_sha256(from_string_to_bytes(tx) + encode(
+ int(hashcode), 256, 4)[::-1])
+ else:
+ return safe_hexlify(bin_dbl_sha256(tx)[::-1])
+
+
+def bin_txhash(tx, hashcode=None):
+ return binascii.unhexlify(txhash(tx, hashcode))
+
+
+def ecdsa_tx_sign(tx, priv, hashcode=SIGHASH_ALL, usenonce=None):
+ sig = ecdsa_raw_sign(
+ txhash(tx, hashcode),
+ priv,
+ True,
+ rawmsg=True,
+ usenonce=usenonce)
+ return sig + encode(hashcode, 16, 2)
+
+
+def ecdsa_tx_verify(tx, sig, pub, hashcode=SIGHASH_ALL):
+ return ecdsa_raw_verify(
+ txhash(tx, hashcode),
+ pub,
+ sig[:-2],
+ True,
+ rawmsg=True)
+
+# Scripts
+
+
+def mk_pubkey_script(addr):
+ # Keep the auxiliary functions around for altcoins' sake
+ return '76a914' + b58check_to_hex(addr) + '88ac'
+
+
+def mk_scripthash_script(addr):
+ return 'a914' + b58check_to_hex(addr) + '87'
+
+# Address representation to output script
+
+
+def address_to_script(addr):
+ if addr[0] == '3' or addr[0] == '2':
+ return mk_scripthash_script(addr)
+ else:
+ return mk_pubkey_script(addr)
+
+# Output script to address representation
+
+
+def script_to_address(script, vbyte=0):
+ if re.match('^[0-9a-fA-F]*$', script):
+ script = binascii.unhexlify(script)
+ if script[:3] == b'\x76\xa9\x14' and script[-2:] == b'\x88\xac' and len(
+ script) == 25:
+ return bin_to_b58check(script[3:-2], vbyte) # pubkey hash addresses
+ else:
+ if vbyte in [111, 196]:
+ # Testnet
+ scripthash_byte = 196
+ else:
+ scripthash_byte = 5
+ # BIP0016 scripthash addresses
+ return bin_to_b58check(script[2:-1], scripthash_byte)
+
+
+def p2sh_scriptaddr(script, magicbyte=5):
+ if re.match('^[0-9a-fA-F]*$', script):
+ script = binascii.unhexlify(script)
+ return hex_to_b58check(hash160(script), magicbyte)
+
+
+scriptaddr = p2sh_scriptaddr
+
+
+def deserialize_script(script):
+ if isinstance(script, str) and re.match('^[0-9a-fA-F]*$', script):
+ return json_changebase(
+ deserialize_script(binascii.unhexlify(script)),
+ lambda x: safe_hexlify(x))
+ out, pos = [], 0
+ while pos < len(script):
+ code = from_byte_to_int(script[pos])
+ if code == 0:
+ out.append(None)
+ pos += 1
+ elif code <= 75:
+ out.append(script[pos + 1:pos + 1 + code])
+ pos += 1 + code
+ elif code <= 78:
+ szsz = pow(2, code - 76)
+ sz = decode(script[pos + szsz:pos:-1], 256)
+ out.append(script[pos + 1 + szsz:pos + 1 + szsz + sz])
+ pos += 1 + szsz + sz
+ elif code <= 96:
+ out.append(code - 80)
+ pos += 1
+ else:
+ out.append(code)
+ pos += 1
+ return out
+
+
+def serialize_script_unit(unit):
+ if isinstance(unit, int):
+ if unit < 16:
+ return from_int_to_byte(unit + 80)
+ else:
+ return bytes([unit])
+ elif unit is None:
+ return b'\x00'
+ else:
+ if len(unit) <= 75:
+ return from_int_to_byte(len(unit)) + unit
+ elif len(unit) < 256:
+ return from_int_to_byte(76) + from_int_to_byte(len(unit)) + unit
+ elif len(unit) < 65536:
+ return from_int_to_byte(77) + encode(len(unit), 256, 2)[::-1] + unit
+ else:
+ return from_int_to_byte(78) + encode(len(unit), 256, 4)[::-1] + unit
+
+
+if is_python2:
+
+ def serialize_script(script):
+ if json_is_base(script, 16):
+ return binascii.hexlify(serialize_script(json_changebase(
+ script, lambda x: binascii.unhexlify(x))))
+ return ''.join(map(serialize_script_unit, script))
+else:
+
+ def serialize_script(script):
+ if json_is_base(script, 16):
+ return safe_hexlify(serialize_script(json_changebase(
+ script, lambda x: binascii.unhexlify(x))))
+
+ result = bytes()
+ for b in map(serialize_script_unit, script):
+ result += b if isinstance(b, bytes) else bytes(b, 'utf-8')
+ return result
+
+
+def mk_multisig_script(*args): # [pubs],k or pub1,pub2...pub[n],k
+ if isinstance(args[0], list):
+ pubs, k = args[0], int(args[1])
+ else:
+ pubs = list(filter(lambda x: len(str(x)) >= 32, args))
+ k = int(args[len(pubs)])
+ return serialize_script([k] + pubs + [len(pubs)]) + 'ae'
+
+# Signing and verifying
+
+
+def verify_tx_input(tx, i, script, sig, pub):
+ if re.match('^[0-9a-fA-F]*$', tx):
+ tx = binascii.unhexlify(tx)
+ if re.match('^[0-9a-fA-F]*$', script):
+ script = binascii.unhexlify(script)
+ if not re.match('^[0-9a-fA-F]*$', sig):
+ sig = safe_hexlify(sig)
+ if not re.match('^[0-9a-fA-F]*$', pub):
+ pub = safe_hexlify(pub)
+ hashcode = decode(sig[-2:], 16)
+ modtx = signature_form(tx, int(i), script, hashcode)
+ return ecdsa_tx_verify(modtx, sig, pub, hashcode)
+
+
+def sign(tx, i, priv, hashcode=SIGHASH_ALL, usenonce=None):
+ i = int(i)
+ if (not is_python2 and isinstance(re, bytes)) or not re.match(
+ '^[0-9a-fA-F]*$', tx):
+ return binascii.unhexlify(sign(safe_hexlify(tx), i, priv))
+ if len(priv) <= 33:
+ priv = safe_hexlify(priv)
+ pub = privkey_to_pubkey(priv, True)
+ address = pubkey_to_address(pub)
+ signing_tx = signature_form(tx, i, mk_pubkey_script(address), hashcode)
+ sig = ecdsa_tx_sign(signing_tx, priv, hashcode, usenonce=usenonce)
+ txobj = deserialize(tx)
+ txobj["ins"][i]["script"] = serialize_script([sig, pub])
+ return serialize(txobj)
+
+
+def signall(tx, priv):
+ # if priv is a dictionary, assume format is
+ # { 'txinhash:txinidx' : privkey }
+ if isinstance(priv, dict):
+ for e, i in enumerate(deserialize(tx)["ins"]):
+ k = priv["%s:%d" % (i["outpoint"]["hash"], i["outpoint"]["index"])]
+ tx = sign(tx, e, k)
+ else:
+ for i in range(len(deserialize(tx)["ins"])):
+ tx = sign(tx, i, priv)
+ return tx
+
+
+def multisign(tx, i, script, pk, hashcode=SIGHASH_ALL):
+ if re.match('^[0-9a-fA-F]*$', tx):
+ tx = binascii.unhexlify(tx)
+ if re.match('^[0-9a-fA-F]*$', script):
+ script = binascii.unhexlify(script)
+ modtx = signature_form(tx, i, script, hashcode)
+ return ecdsa_tx_sign(modtx, pk, hashcode)
+
+
+def apply_multisignatures(*args):
+ # tx,i,script,sigs OR tx,i,script,sig1,sig2...,sig[n]
+ tx, i, script = args[0], int(args[1]), args[2]
+ sigs = args[3] if isinstance(args[3], list) else list(args[3:])
+
+ if isinstance(script, str) and re.match('^[0-9a-fA-F]*$', script):
+ script = binascii.unhexlify(script)
+ sigs = [binascii.unhexlify(x) if x[:2] == '30' else x for x in sigs]
+ if isinstance(tx, str) and re.match('^[0-9a-fA-F]*$', tx):
+ return safe_hexlify(apply_multisignatures(
+ binascii.unhexlify(tx), i, script, sigs))
+
+ txobj = deserialize(tx)
+ txobj["ins"][i]["script"] = serialize_script([None] + sigs + [script])
+ return serialize(txobj)
+
+
+def is_inp(arg):
+ return len(arg) > 64 or "output" in arg or "outpoint" in arg
+
+
+def mktx(*args):
+ # [in0, in1...],[out0, out1...] or in0, in1 ... out0 out1 ...
+ ins, outs = [], []
+ for arg in args:
+ if isinstance(arg, list):
+ for a in arg:
+ (ins if is_inp(a) else outs).append(a)
+ else:
+ (ins if is_inp(arg) else outs).append(arg)
+
+ txobj = {"locktime": 0, "version": 1, "ins": [], "outs": []}
+ for i in ins:
+ if isinstance(i, dict) and "outpoint" in i:
+ txobj["ins"].append(i)
+ else:
+ if isinstance(i, dict) and "output" in i:
+ i = i["output"]
+ txobj["ins"].append({
+ "outpoint": {"hash": i[:64],
+ "index": int(i[65:])},
+ "script": "",
+ "sequence": 4294967295
+ })
+ for o in outs:
+ if isinstance(o, string_or_bytes_types):
+ addr = o[:o.find(':')]
+ val = int(o[o.find(':') + 1:])
+ o = {}
+ if re.match('^[0-9a-fA-F]*$', addr):
+ o["script"] = addr
+ else:
+ o["address"] = addr
+ o["value"] = val
+
+ outobj = {}
+ if "address" in o:
+ outobj["script"] = address_to_script(o["address"])
+ elif "script" in o:
+ outobj["script"] = o["script"]
+ else:
+ raise Exception("Could not find 'address' or 'script' in output.")
+ outobj["value"] = o["value"]
+ txobj["outs"].append(outobj)
+
+ return serialize(txobj)
+
+
+def select(unspent, value):
+ value = int(value)
+ high = [u for u in unspent if u["value"] >= value]
+ high.sort(key=lambda u: u["value"])
+ low = [u for u in unspent if u["value"] < value]
+ low.sort(key=lambda u: -u["value"])
+ if len(high):
+ return [high[0]]
+ i, tv = 0, 0
+ while tv < value and i < len(low):
+ tv += low[i]["value"]
+ i += 1
+ if tv < value:
+ raise Exception("Not enough funds")
+ return low[:i]
DIR diff --git a/bitcoin/transaction.py b/bitcoin/transaction.py
t@@ -0,0 +1,490 @@
+#!/usr/bin/python
+import binascii, re, json, copy, sys
+from bitcoin.main import *
+from _functools import reduce
+
+### Hex to bin converter and vice versa for objects
+
+
+def json_is_base(obj, base):
+ if not is_python2 and isinstance(obj, bytes):
+ return False
+
+ alpha = get_code_string(base)
+ if isinstance(obj, string_types):
+ for i in range(len(obj)):
+ if alpha.find(obj[i]) == -1:
+ return False
+ return True
+ elif isinstance(obj, int_types) or obj is None:
+ return True
+ elif isinstance(obj, list):
+ for i in range(len(obj)):
+ if not json_is_base(obj[i], base):
+ return False
+ return True
+ else:
+ for x in obj:
+ if not json_is_base(obj[x], base):
+ return False
+ return True
+
+
+def json_changebase(obj, changer):
+ if isinstance(obj, string_or_bytes_types):
+ return changer(obj)
+ elif isinstance(obj, int_types) or obj is None:
+ return obj
+ elif isinstance(obj, list):
+ return [json_changebase(x, changer) for x in obj]
+ return dict((x, json_changebase(obj[x], changer)) for x in obj)
+
+# Transaction serialization and deserialization
+
+
+def deserialize(tx):
+ if isinstance(tx, str) and re.match('^[0-9a-fA-F]*$', tx):
+ #tx = bytes(bytearray.fromhex(tx))
+ return json_changebase(
+ deserialize(binascii.unhexlify(tx)), lambda x: safe_hexlify(x))
+ # http://stackoverflow.com/questions/4851463/python-closure-write-to-variable-in-parent-scope
+ # Python's scoping rules are demented, requiring me to make pos an object
+ # so that it is call-by-reference
+ pos = [0]
+
+ def read_as_int(bytez):
+ pos[0] += bytez
+ return decode(tx[pos[0] - bytez:pos[0]][::-1], 256)
+
+ def read_var_int():
+ pos[0] += 1
+
+ val = from_byte_to_int(tx[pos[0] - 1])
+ if val < 253:
+ return val
+ return read_as_int(pow(2, val - 252))
+
+ def read_bytes(bytez):
+ pos[0] += bytez
+ return tx[pos[0] - bytez:pos[0]]
+
+ def read_var_string():
+ size = read_var_int()
+ return read_bytes(size)
+
+ obj = {"ins": [], "outs": []}
+ obj["version"] = read_as_int(4)
+ ins = read_var_int()
+ for i in range(ins):
+ obj["ins"].append({
+ "outpoint": {
+ "hash": read_bytes(32)[::-1],
+ "index": read_as_int(4)
+ },
+ "script": read_var_string(),
+ "sequence": read_as_int(4)
+ })
+ outs = read_var_int()
+ for i in range(outs):
+ obj["outs"].append({
+ "value": read_as_int(8),
+ "script": read_var_string()
+ })
+ obj["locktime"] = read_as_int(4)
+ return obj
+
+
+def serialize(txobj):
+ #if isinstance(txobj, bytes):
+ # txobj = bytes_to_hex_string(txobj)
+ o = []
+ if json_is_base(txobj, 16):
+ json_changedbase = json_changebase(txobj,
+ lambda x: binascii.unhexlify(x))
+ hexlified = safe_hexlify(serialize(json_changedbase))
+ return hexlified
+ o.append(encode(txobj["version"], 256, 4)[::-1])
+ o.append(num_to_var_int(len(txobj["ins"])))
+ for inp in txobj["ins"]:
+ o.append(inp["outpoint"]["hash"][::-1])
+ o.append(encode(inp["outpoint"]["index"], 256, 4)[::-1])
+ o.append(num_to_var_int(len(inp["script"])) + (inp["script"] if inp[
+ "script"] or is_python2 else bytes()))
+ o.append(encode(inp["sequence"], 256, 4)[::-1])
+ o.append(num_to_var_int(len(txobj["outs"])))
+ for out in txobj["outs"]:
+ o.append(encode(out["value"], 256, 8)[::-1])
+ o.append(num_to_var_int(len(out["script"])) + out["script"])
+ o.append(encode(txobj["locktime"], 256, 4)[::-1])
+
+ return ''.join(o) if is_python2 else reduce(lambda x, y: x + y, o, bytes())
+
+# Hashing transactions for signing
+
+SIGHASH_ALL = 1
+SIGHASH_NONE = 2
+SIGHASH_SINGLE = 3
+# this works like SIGHASH_ANYONECANPAY | SIGHASH_ALL, might as well make it explicit while
+# we fix the constant
+SIGHASH_ANYONECANPAY = 0x81
+
+
+def signature_form(tx, i, script, hashcode=SIGHASH_ALL):
+ i, hashcode = int(i), int(hashcode)
+ if isinstance(tx, string_or_bytes_types):
+ return serialize(signature_form(deserialize(tx), i, script, hashcode))
+ newtx = copy.deepcopy(tx)
+ for inp in newtx["ins"]:
+ inp["script"] = ""
+ newtx["ins"][i]["script"] = script
+ if hashcode == SIGHASH_NONE:
+ newtx["outs"] = []
+ elif hashcode == SIGHASH_SINGLE:
+ newtx["outs"] = newtx["outs"][:len(newtx["ins"])]
+ for out in range(len(newtx["ins"]) - 1):
+ out.value = 2**64 - 1
+ out.script = ""
+ elif hashcode == SIGHASH_ANYONECANPAY:
+ newtx["ins"] = [newtx["ins"][i]]
+ else:
+ pass
+ return newtx
+
+# Making the actual signatures
+
+
+def der_encode_sig(v, r, s):
+ """Takes (vbyte, r, s) as ints and returns hex der encode sig"""
+ #See https://github.com/vbuterin/pybitcointools/issues/89
+ #See https://github.com/simcity4242/pybitcointools/
+ s = N - s if s > N // 2 else s # BIP62 low s
+ b1, b2 = encode(r, 256), encode(s, 256)
+ if bytearray(b1)[
+ 0] & 0x80: # add null bytes if leading byte interpreted as negative
+ b1 = b'\x00' + b1
+ if bytearray(b2)[0] & 0x80:
+ b2 = b'\x00' + b2
+ left = b'\x02' + encode(len(b1), 256, 1) + b1
+ right = b'\x02' + encode(len(b2), 256, 1) + b2
+ return safe_hexlify(b'\x30' + encode(
+ len(left + right), 256, 1) + left + right)
+
+
+def der_decode_sig(sig):
+ leftlen = decode(sig[6:8], 16) * 2
+ left = sig[8:8 + leftlen]
+ rightlen = decode(sig[10 + leftlen:12 + leftlen], 16) * 2
+ right = sig[12 + leftlen:12 + leftlen + rightlen]
+ return (None, decode(left, 16), decode(right, 16))
+
+
+def txhash(tx, hashcode=None):
+ if isinstance(tx, str) and re.match('^[0-9a-fA-F]*$', tx):
+ tx = changebase(tx, 16, 256)
+ if hashcode:
+ return dbl_sha256(from_string_to_bytes(tx) + encode(
+ int(hashcode), 256, 4)[::-1])
+ else:
+ return safe_hexlify(bin_dbl_sha256(tx)[::-1])
+
+
+def bin_txhash(tx, hashcode=None):
+ return binascii.unhexlify(txhash(tx, hashcode))
+
+
+def ecdsa_tx_sign(tx, priv, hashcode=SIGHASH_ALL):
+ rawsig = ecdsa_raw_sign(bin_txhash(tx, hashcode), priv)
+ return der_encode_sig(*rawsig) + encode(hashcode, 16, 2)
+
+
+def ecdsa_tx_verify(tx, sig, pub, hashcode=SIGHASH_ALL):
+ return ecdsa_raw_verify(bin_txhash(tx, hashcode), der_decode_sig(sig), pub)
+
+# Scripts
+
+def mk_pubkey_script(addr):
+ # Keep the auxiliary functions around for altcoins' sake
+ return '76a914' + b58check_to_hex(addr) + '88ac'
+
+
+def mk_scripthash_script(addr):
+ return 'a914' + b58check_to_hex(addr) + '87'
+
+# Address representation to output script
+
+
+def address_to_script(addr):
+ if addr[0] == '3' or addr[0] == '2':
+ return mk_scripthash_script(addr)
+ else:
+ return mk_pubkey_script(addr)
+
+# Output script to address representation
+
+
+def script_to_address(script, vbyte=0):
+ if re.match('^[0-9a-fA-F]*$', script):
+ script = binascii.unhexlify(script)
+ if script[:3] == b'\x76\xa9\x14' and script[-2:] == b'\x88\xac' and len(
+ script) == 25:
+ return bin_to_b58check(script[3:-2], vbyte) # pubkey hash addresses
+ else:
+ if vbyte in [111, 196]:
+ # Testnet
+ scripthash_byte = 196
+ else:
+ scripthash_byte = 5
+ # BIP0016 scripthash addresses
+ return bin_to_b58check(script[2:-1], scripthash_byte)
+
+
+def p2sh_scriptaddr(script, magicbyte=5):
+ if re.match('^[0-9a-fA-F]*$', script):
+ script = binascii.unhexlify(script)
+ return hex_to_b58check(hash160(script), magicbyte)
+
+
+scriptaddr = p2sh_scriptaddr
+
+
+def deserialize_script(script):
+ if isinstance(script, str) and re.match('^[0-9a-fA-F]*$', script):
+ return json_changebase(
+ deserialize_script(binascii.unhexlify(script)),
+ lambda x: safe_hexlify(x))
+ out, pos = [], 0
+ while pos < len(script):
+ code = from_byte_to_int(script[pos])
+ if code == 0:
+ out.append(None)
+ pos += 1
+ elif code <= 75:
+ out.append(script[pos + 1:pos + 1 + code])
+ pos += 1 + code
+ elif code <= 78:
+ szsz = pow(2, code - 76)
+ sz = decode(script[pos + szsz:pos:-1], 256)
+ out.append(script[pos + 1 + szsz:pos + 1 + szsz + sz])
+ pos += 1 + szsz + sz
+ elif code <= 96:
+ out.append(code - 80)
+ pos += 1
+ else:
+ out.append(code)
+ pos += 1
+ return out
+
+
+def serialize_script_unit(unit):
+ if isinstance(unit, int):
+ if unit < 16:
+ return from_int_to_byte(unit + 80)
+ else:
+ return bytes([unit])
+ elif unit is None:
+ return b'\x00'
+ else:
+ if len(unit) <= 75:
+ return from_int_to_byte(len(unit)) + unit
+ elif len(unit) < 256:
+ return from_int_to_byte(76) + from_int_to_byte(len(unit)) + unit
+ elif len(unit) < 65536:
+ return from_int_to_byte(77) + encode(len(unit), 256, 2)[::-1] + unit
+ else:
+ return from_int_to_byte(78) + encode(len(unit), 256, 4)[::-1] + unit
+
+
+if is_python2:
+
+ def serialize_script(script):
+ if json_is_base(script, 16):
+ return binascii.hexlify(serialize_script(json_changebase(
+ script, lambda x: binascii.unhexlify(x))))
+ return ''.join(map(serialize_script_unit, script))
+else:
+
+ def serialize_script(script):
+ if json_is_base(script, 16):
+ return safe_hexlify(serialize_script(json_changebase(
+ script, lambda x: binascii.unhexlify(x))))
+
+ result = bytes()
+ for b in map(serialize_script_unit, script):
+ result += b if isinstance(b, bytes) else bytes(b, 'utf-8')
+ return result
+
+
+def mk_multisig_script(*args): # [pubs],k or pub1,pub2...pub[n],k
+ if isinstance(args[0], list):
+ pubs, k = args[0], int(args[1])
+ else:
+ pubs = list(filter(lambda x: len(str(x)) >= 32, args))
+ k = int(args[len(pubs)])
+ return serialize_script([k] + pubs + [len(pubs)]) + 'ae'
+
+# Signing and verifying
+
+
+def verify_tx_input(tx, i, script, sig, pub):
+ if re.match('^[0-9a-fA-F]*$', tx):
+ tx = binascii.unhexlify(tx)
+ if re.match('^[0-9a-fA-F]*$', script):
+ script = binascii.unhexlify(script)
+ if not re.match('^[0-9a-fA-F]*$', sig):
+ sig = safe_hexlify(sig)
+ hashcode = decode(sig[-2:], 16)
+ modtx = signature_form(tx, int(i), script, hashcode)
+ return ecdsa_tx_verify(modtx, sig, pub, hashcode)
+
+
+def sign(tx, i, priv, hashcode=SIGHASH_ALL):
+ i = int(i)
+ if (not is_python2 and isinstance(re, bytes)) or not re.match(
+ '^[0-9a-fA-F]*$', tx):
+ return binascii.unhexlify(sign(safe_hexlify(tx), i, priv))
+ if len(priv) <= 33:
+ priv = safe_hexlify(priv)
+ pub = privkey_to_pubkey(priv)
+ address = pubkey_to_address(pub)
+ signing_tx = signature_form(tx, i, mk_pubkey_script(address), hashcode)
+ sig = ecdsa_tx_sign(signing_tx, priv, hashcode)
+ txobj = deserialize(tx)
+ txobj["ins"][i]["script"] = serialize_script([sig, pub])
+ return serialize(txobj)
+
+
+def signall(tx, priv):
+ # if priv is a dictionary, assume format is
+ # { 'txinhash:txinidx' : privkey }
+ if isinstance(priv, dict):
+ for e, i in enumerate(deserialize(tx)["ins"]):
+ k = priv["%s:%d" % (i["outpoint"]["hash"], i["outpoint"]["index"])]
+ tx = sign(tx, e, k)
+ else:
+ for i in range(len(deserialize(tx)["ins"])):
+ tx = sign(tx, i, priv)
+ return tx
+
+
+def multisign(tx, i, script, pk, hashcode=SIGHASH_ALL):
+ if re.match('^[0-9a-fA-F]*$', tx):
+ tx = binascii.unhexlify(tx)
+ if re.match('^[0-9a-fA-F]*$', script):
+ script = binascii.unhexlify(script)
+ modtx = signature_form(tx, i, script, hashcode)
+ return ecdsa_tx_sign(modtx, pk, hashcode)
+
+
+def apply_multisignatures(*args):
+ # tx,i,script,sigs OR tx,i,script,sig1,sig2...,sig[n]
+ tx, i, script = args[0], int(args[1]), args[2]
+ sigs = args[3] if isinstance(args[3], list) else list(args[3:])
+
+ if isinstance(script, str) and re.match('^[0-9a-fA-F]*$', script):
+ script = binascii.unhexlify(script)
+ sigs = [binascii.unhexlify(x) if x[:2] == '30' else x for x in sigs]
+ if isinstance(tx, str) and re.match('^[0-9a-fA-F]*$', tx):
+ return safe_hexlify(apply_multisignatures(
+ binascii.unhexlify(tx), i, script, sigs))
+
+ txobj = deserialize(tx)
+ txobj["ins"][i]["script"] = serialize_script([None] + sigs + [script])
+ return serialize(txobj)
+
+
+def is_inp(arg):
+ return len(arg) > 64 or "output" in arg or "outpoint" in arg
+
+
+def mktx(*args):
+ # [in0, in1...],[out0, out1...] or in0, in1 ... out0 out1 ...
+ ins, outs = [], []
+ for arg in args:
+ if isinstance(arg, list):
+ for a in arg:
+ (ins if is_inp(a) else outs).append(a)
+ else:
+ (ins if is_inp(arg) else outs).append(arg)
+
+ txobj = {"locktime": 0, "version": 1, "ins": [], "outs": []}
+ for i in ins:
+ if isinstance(i, dict) and "outpoint" in i:
+ txobj["ins"].append(i)
+ else:
+ if isinstance(i, dict) and "output" in i:
+ i = i["output"]
+ txobj["ins"].append({
+ "outpoint": {"hash": i[:64],
+ "index": int(i[65:])},
+ "script": "",
+ "sequence": 4294967295
+ })
+ for o in outs:
+ if isinstance(o, string_or_bytes_types):
+ addr = o[:o.find(':')]
+ val = int(o[o.find(':') + 1:])
+ o = {}
+ if re.match('^[0-9a-fA-F]*$', addr):
+ o["script"] = addr
+ else:
+ o["address"] = addr
+ o["value"] = val
+
+ outobj = {}
+ if "address" in o:
+ outobj["script"] = address_to_script(o["address"])
+ elif "script" in o:
+ outobj["script"] = o["script"]
+ else:
+ raise Exception("Could not find 'address' or 'script' in output.")
+ outobj["value"] = o["value"]
+ txobj["outs"].append(outobj)
+
+ return serialize(txobj)
+
+
+def select(unspent, value):
+ value = int(value)
+ high = [u for u in unspent if u["value"] >= value]
+ high.sort(key=lambda u: u["value"])
+ low = [u for u in unspent if u["value"] < value]
+ low.sort(key=lambda u: -u["value"])
+ if len(high):
+ return [high[0]]
+ i, tv = 0, 0
+ while tv < value and i < len(low):
+ tv += low[i]["value"]
+ i += 1
+ if tv < value:
+ raise Exception("Not enough funds")
+ return low[:i]
+
+# Only takes inputs of the form { "output": blah, "value": foo }
+
+
+def mksend(*args):
+ argz, change, fee = args[:-2], args[-2], int(args[-1])
+ ins, outs = [], []
+ for arg in argz:
+ if isinstance(arg, list):
+ for a in arg:
+ (ins if is_inp(a) else outs).append(a)
+ else:
+ (ins if is_inp(arg) else outs).append(arg)
+
+ isum = sum([i["value"] for i in ins])
+ osum, outputs2 = 0, []
+ for o in outs:
+ if isinstance(o, string_types):
+ o2 = {"address": o[:o.find(':')], "value": int(o[o.find(':') + 1:])}
+ else:
+ o2 = o
+ outputs2.append(o2)
+ osum += o2["value"]
+
+ if isum < osum + fee:
+ raise Exception("Not enough money")
+ elif isum > osum + fee + 5430:
+ outputs2 += [{"address": change, "value": isum - osum - fee}]
+
+ return mktx(ins, outputs2)
DIR diff --git a/config.cfg_sample b/config.cfg_sample
t@@ -0,0 +1,37 @@
+
+## Electrum Personal Server configuration file
+## Comments start with #
+
+[wallets]
+## Add addresses to this section
+
+# These are just random addresses I found on a blockchain explorer
+
+# A key can be anything
+addr = 1DuqpoeTB9zLvVCXQG53VbMxvMkijk494n
+# A comma separated list is also accepted
+my_test_addresses = 1KKszdQEpXgSY4EvsnGcGEzbQFmdcFwuNS,1EuEVUtQQ8hmuMHNdMdjLwjpkm6Ef7RYVk
+# And space separated
+more_test_addresses = 3Hh7QujVLqz11tiQsnUE5CSL16WEHBmiyR 1Arzu6mWZuXGTF9yR2hZhMgBJgu1Xh2bNC 1PXRLo1FQoZyF1Jhnz4qbG5x8Bo3pFpybz
+
+[bitcoin-rpc]
+host = localhost
+port = 8332
+user = bitcoinrpc
+password = password
+
+#how often in seconds to poll for new transactions when electrum not connected
+poll_interval_listening = 30
+#how often in seconds to poll for new transactions when electrum is connected
+poll_interval_connected = 5
+
+[electrum-server]
+#0.0.0.0 to accept connections from any IP
+#127.0.0.1 to accept from only localhost
+#recommended you accept localhost only and connect with a ssh tunnel
+host = 127.0.0.1
+port = 50002
+
+[misc]
+#not implemented yet
+print_debug = false
DIR diff --git a/jsonrpc.py b/jsonrpc.py
t@@ -0,0 +1,59 @@
+#jsonrpc.py from https://github.com/JoinMarket-Org/joinmarket/blob/master/joinmarket/jsonrpc.py
+#copyright # Copyright (C) 2013,2015 by Daniel Kraft <d@domob.eu> and phelix / blockchained.com
+
+import base64
+import http.client
+import json
+
+class JsonRpcError(Exception):
+ def __init__(self, obj):
+ self.code = obj["code"]
+ self.message = obj["message"]
+
+class JsonRpcConnectionError(JsonRpcError): pass
+
+class JsonRpc(object):
+ def __init__(self, host, port, user, password):
+ self.host = host
+ self.port = port
+ self.authstr = "%s:%s" % (user, password)
+ self.queryId = 1
+
+ def queryHTTP(self, obj):
+ headers = {"User-Agent": "electrum-personal-server",
+ "Content-Type": "application/json",
+ "Accept": "application/json"}
+ headers["Authorization"] = "Basic %s" % base64.b64encode(
+ self.authstr.encode()).decode()
+ body = json.dumps(obj)
+ try:
+ conn = http.client.HTTPConnection(self.host, self.port)
+ conn.request("POST", "", body, headers)
+ response = conn.getresponse()
+ if response.status == 401:
+ conn.close()
+ raise JsonRpcConnectionError(
+ "authentication for JSON-RPC failed")
+ # All of the codes below are 'fine' from a JSON-RPC point of view.
+ if response.status not in [200, 404, 500]:
+ conn.close()
+ raise JsonRpcConnectionError("unknown error in JSON-RPC")
+ data = response.read()
+ conn.close()
+ return json.loads(data.decode())
+ except JsonRpcConnectionError as exc:
+ raise exc
+ except Exception as exc:
+ raise JsonRpcConnectionError("JSON-RPC connection failed. Err:" +
+ repr(exc))
+
+ def call(self, method, params):
+ currentId = self.queryId
+ self.queryId += 1
+ request = {"method": method, "params": params, "id": currentId}
+ response = self.queryHTTP(request)
+ if response["id"] != currentId:
+ raise JsonRpcConnectionError("invalid id returned by query")
+ if response["error"] is not None:
+ raise JsonRpcError(response["error"])
+ return response["result"]
DIR diff --git a/run-server.bat b/run-server.bat
t@@ -0,0 +1,3 @@
+@echo off
+python3 server.py
+pause
+\ No newline at end of file
DIR diff --git a/server.py b/server.py
t@@ -0,0 +1,634 @@
+#! /usr/bin/python3
+
+#add a feature where it prints the first 3 addresses from a deterministic
+# wallet, so you can check the addresses are correct before importing them
+# into the node
+
+#or deterministic wallets
+#should figure out what do regarding gap limits, when to import more addresses
+# and how many addresses to start with
+# maybe have a separate list of later addresses and if one of them get
+# requested then import more
+
+#TODO try to support ssl
+#doesnt support ssl yet you you must run ./electrum --nossl
+#https://github.com/spesmilo/electrum/commit/dc388d4c7c541fadb9869727e359edace4c9f6f0
+#maybe copy from electrumx
+#https://github.com/kyuupichan/electrumx/blob/35dd1f61996b02a84691ea71ff50f0900df969bc/server/peers.py#L476
+#https://github.com/kyuupichan/electrumx/blob/2d7403f2efed7e8f33c5cb93e2cd9144415cbb9f/server/controller.py#L259
+
+#merkle trees cant be used if bitcoin core has pruning enabled, this will
+# probably requires new code to be written for core
+#another possible use of merkleproofs in wallet.dat
+# https://github.com/JoinMarket-Org/joinmarket/issues/156#issuecomment-231059844
+
+#using core's multiple wallet feature might help, should read up on that
+
+#now that the rescanblockchain rpc call exists in 0.16 which allows specifying
+# a starting height, that will cut down the time to rescan as long as the user
+# has saved their wallet creation date
+
+#one day there could be a nice GUI which does everything, including converting
+# the wallet creation date to a block height and rescanning
+'''
+<belcher> now that 0.16 has this rpc called rescanblockchain which takes an optional start_height, i wonder what the most practical way of converting date to block height is
+<belcher> thinking about the situation where you have a mnemonic recovery phrase + the date you created it, and want to rescan
+<belcher> binary search the timestamps in the block headers i guess, then subtract two weeks just in case
+<wumpus> belcher: binary search in something that is not strictly increasing seems faulty
+<belcher> yes true, so maybe binary search to roughly get to the right block height then linear search +- a few blocks
+<wumpus> belcher: though my gut feeling is that subtracting the two weeks would fix it
+<belcher> when people write down the wallet creation date they probably wont be precise, you could get away with writing only the year and month i bet
+<wumpus> as the mismatch is at most 2 hours
+<Sentineo> wumpus: 2 hours for the clock scew allowed by peers? (when they throw away a block which is older than 2 hours from their actual time)?
+<wumpus> Sentineo: that's what I remember, I might be off though
+<Sentineo> I am not sure if it s 2 or 4 :D
+<Sentineo> lazyness :)
+<wumpus> in any case it is a bounded value, which means binary search might work within that precision, too lazy to look for proof though :)
+'''
+
+##### good things
+
+# well placed to take advantage of dandelion private tx broadcasting
+# and broadcasting through tor
+
+import socket, time, json, datetime, struct, binascii, math, pprint
+from configparser import ConfigParser, NoSectionError
+from decimal import Decimal
+
+from jsonrpc import JsonRpc, JsonRpcError
+import util
+import bitcoin as btc
+
+ADDRESSES_LABEL = "electrum-watchonly-addresses"
+
+VERSION_NUMBER = "0.1"
+
+BANNER = \
+"""Welcome to Electrum Personal Server
+gitub.com/whatever
+
+Monitoring {addr} addresses
+
+Connected bitcoin node: {useragent}
+Peers: {peers}
+Uptime: {uptime}
+Blocksonly: {blocksonly}
+Pruning: {pruning}
+"""
+
+##python has demented rules for variable scope, so these
+## global variables are actually mutable lists
+subscribed_to_headers = [False]
+bestblockhash = [None]
+last_known_recent_txid = [None]
+
+#log for checking up/seeing your wallet, debug for when something has gone wrong
+def debugorlog(line, ttype):
+ timestamp = datetime.datetime.now().strftime("%H:%M:%S,%f")
+ print(timestamp + " [" + ttype + "] " + line)
+
+def debug(line):
+ debugorlog(line, "DEBUG")
+
+def log(line):
+ debugorlog(line, " LOG")
+
+def send_response(sock, query, result):
+ query["result"] = result
+ query["jsonrpc"] = "2.0"
+ sock.sendall(json.dumps(query).encode('utf-8') + b'\n')
+ debug('<= ' + json.dumps(query))
+
+def send_update(sock, update):
+ update["jsonrpc"] = "2.0"
+ sock.sendall(json.dumps(update).encode('utf-8') + b'\n')
+ debug('<= ' + json.dumps(update))
+
+def on_heartbeat_listening(rpc, address_history, unconfirmed_txes):
+ debug("on heartbeat listening")
+ check_for_updated_txes(rpc, address_history, unconfirmed_txes)
+
+def on_heartbeat_connected(sock, rpc, address_history, unconfirmed_txes):
+ debug("on heartbeat connected")
+ is_tip_updated, header = check_for_new_blockchain_tip(rpc)
+ if is_tip_updated:
+ log("Blockchain tip updated")
+ if subscribed_to_headers[0]:
+ update = {"method": "blockchain.headers.subscribe",
+ "params": [header]}
+ send_update(sock, update)
+ updated_scripthashes = check_for_updated_txes(rpc, address_history,
+ unconfirmed_txes)
+ for scrhash in updated_scripthashes:
+ if not address_history[scrhash]["subscribed"]:
+ continue
+ history_hash = util.get_status_electrum( ((h["tx_hash"], h["height"])
+ for h in address_history[scrhash]["history"]) )
+ update = {"method": "blockchain.scripthash.subscribe", "params":
+ [scrhash, history_hash]}
+ send_update(sock, update)
+
+def on_disconnect(address_history):
+ subscribed_to_headers[0] = False
+ for srchash, his in address_history.items():
+ his["subscribed"] = False
+
+def handle_query(sock, line, rpc, address_history):
+ debug("=> " + line)
+ try:
+ query = json.loads(line)
+ except json.decoder.JSONDecodeError as e:
+ raise IOError(e)
+ method = query["method"]
+
+ #protocol documentation
+ #https://github.com/kyuupichan/electrumx/blob/master/docs/PROTOCOL.rst
+ if method == "blockchain.transaction.get":
+ try:
+ tx = rpc.call("gettransaction", [query["params"][0]])
+ send_response(sock, query, tx["hex"])
+ except JsonRpcError:
+ debug("Unable to get tx " + query["params"][0])
+ elif method == "blockchain.transaction.get_merkle":
+ #we dont support merkle proofs yet, but we must reply with
+ #something otherwise electrum will disconnect from us
+ #so reply with an invalid proof
+ #https://github.com/spesmilo/electrum/blob/c8e67e2bd07efe042703bc1368d499c5e555f854/lib/verifier.py#L74
+ txid = query["params"][0]
+ reply = {"block_height": 1, "pos": 0, "merkle": [txid]}
+ send_response(sock, query, reply)
+ elif method == "blockchain.scripthash.subscribe":
+ scrhash = query["params"][0]
+ if scrhash in address_history:
+ address_history[scrhash]["subscribed"] = True
+ history_hash = util.get_status_electrum((
+ (h["tx_hash"], h["height"])
+ for h in address_history[scrhash]["history"]))
+ else:
+ log("WARNING: address scripthash not known to us: " + scrhash)
+ history_hash = util.get_status_electrum([])
+ send_response(sock, query, history_hash)
+ elif method == "blockchain.scripthash.get_history":
+ scrhash = query["params"][0]
+ if scrhash in address_history:
+ history = address_history[scrhash]["history"]
+ else:
+ history = []
+ log("WARNING: address scripthash history not known to us: "
+ + scrhash)
+ send_response(sock, query, history)
+ elif method == "blockchain.headers.subscribe":
+ subscribed_to_headers[0] = True
+ new_bestblockhash, header = get_current_header(rpc)
+ send_response(sock, query, header)
+ elif method == "blockchain.block.get_header":
+ blockhash = rpc.call("getblockhash", [query["params"][0]])
+ header = get_block_header(rpc, blockhash)
+ send_response(sock, query, header)
+ elif method == "blockchain.block.get_chunk":
+ RETARGET_INTERVAL = 2016
+ index = query["params"][0]
+ tip_height = rpc.call("getblockchaininfo", [])["headers"]
+ #logic copied from kyuupichan's electrumx get_chunk() in controller.py
+ next_height = tip_height + 1
+ start_height = min(index*RETARGET_INTERVAL, next_height)
+ count = min(next_height - start_height, RETARGET_INTERVAL)
+ #read count number of headers starting from start_height
+ result = bytearray()
+ the_hash = rpc.call("getblockhash", [start_height])
+ for i in range(count):
+ header = rpc.call("getblockheader", [the_hash])
+ #add header hex to result
+ h1 = struct.pack("<i32s32sIII", header["version"],
+ binascii.unhexlify(header["previousblockhash"])[::-1],
+ binascii.unhexlify(header["merkleroot"])[::-1],
+ header["time"], int(header["bits"], 16), header["nonce"])
+ result.extend(h1)
+ if "nextblockhash" not in header:
+ break
+ the_hash = header["nextblockhash"]
+ send_response(sock, query, binascii.hexlify(result).decode("utf-8"))
+ elif method == "blockchain.transaction.broadcast":
+ try:
+ result = rpc.call("sendrawtransaction", [query["params"][0]])
+ except JsonRpcError as e:
+ result = e.message
+ debug("tx broadcast result = " + str(result))
+ send_response(sock, query, result)
+ elif method == "blockchain.estimatefee":
+ estimate = rpc.call("estimatesmartfee", [query["params"][0]])
+ feerate = 0.0001
+ if "feerate" in estimate:
+ feerate = estimate["feerate"]
+ send_response(sock, query, feerate)
+ elif method == "blockchain.relayfee":
+ networkinfo = rpc.call("getnetworkinfo", [])
+ send_response(sock, query, networkinfo["relayfee"])
+ elif method == "server.banner":
+ networkinfo = rpc.call("getnetworkinfo", [])
+ blockchaininfo = rpc.call("getblockchaininfo", [])
+ uptime = rpc.call("uptime", [])
+ send_response(sock, query, BANNER.format(
+ addr=len(address_history),
+ useragent=networkinfo["subversion"],
+ peers=networkinfo["connections"],
+ uptime=str(datetime.timedelta(seconds=uptime)),
+ blocksonly=not networkinfo["localrelay"],
+ pruning=blockchaininfo["pruned"]))
+ elif method == "server.donation_address":
+ send_response(sock, query, "bc1q5d8l0w33h65e2l5x7ty6wgnvkvlqcz0wfaslpz")
+ elif method == "server.version":
+ send_response(sock, query, ["ElectrumPersonalServer "
+ + VERSION_NUMBER, VERSION_NUMBER])
+ elif method == "server.peers.subscribe":
+ send_response(sock, query, []) #no peers to report
+ else:
+ log("*** BUG! Not handling method: " + method + " query=" + str(query))
+
+def get_block_header(rpc, blockhash):
+ rpc_head = rpc.call("getblockheader", [blockhash])
+ header = {"block_height": rpc_head["height"],
+ "prev_block_hash": rpc_head["previousblockhash"],
+ "timestamp": rpc_head["time"],
+ "merkle_root": rpc_head["merkleroot"],
+ "version": rpc_head["version"],
+ "nonce": rpc_head["nonce"],
+ "bits": int(rpc_head["bits"], 16)}
+ return header
+
+def get_current_header(rpc):
+ new_bestblockhash = rpc.call("getbestblockhash", [])
+ header = get_block_header(rpc, new_bestblockhash)
+ return new_bestblockhash, header
+
+def check_for_new_blockchain_tip(rpc):
+ #TODO might not handle more than one block appearing, might need to
+ # use a "last known block" similar to the transaction code
+ new_bestblockhash, header = get_current_header(rpc)
+ is_tip_new = bestblockhash[0] != new_bestblockhash
+ bestblockhash[0] = new_bestblockhash
+ return is_tip_new, header
+
+def create_server_socket(hostport):
+ server_sock = socket.socket()
+ server_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+ server_sock.bind(hostport)
+ log("Listening on " + str(hostport))
+ return server_sock
+
+def run_electrum_server(hostport, rpc, address_history, unconfirmed_txes,
+ poll_interval_listening, poll_interval_connected):
+ log("Starting electrum server")
+ while True:
+ try:
+ server_sock = create_server_socket(hostport)
+ server_sock.settimeout(poll_interval_listening)
+ while True:
+ try:
+ server_sock.listen(1)
+ sock, addr = server_sock.accept()
+ break
+ except socket.timeout:
+ on_heartbeat_listening(rpc, address_history,
+ unconfirmed_txes)
+ server_sock.close()
+ sock.settimeout(poll_interval_connected)
+ log('Electrum connected from ' + str(addr))
+ recv_buffer = bytearray()
+ while True:
+ try:
+ recv_data = sock.recv(4096)
+ if not recv_data or len(recv_data) == 0:
+ raise EOFError()
+ recv_buffer.extend(recv_data)
+ lb = recv_buffer.find(b'\n')
+ if lb == -1:
+ continue
+ while lb != -1:
+ line = recv_buffer[:lb].rstrip()
+ recv_buffer = recv_buffer[lb + 1:]
+ lb = recv_buffer.find(b'\n')
+ handle_query(sock, line.decode("utf-8"), rpc,
+ address_history)
+ except socket.timeout:
+ on_heartbeat_connected(sock, rpc, address_history,
+ unconfirmed_txes)
+ except (IOError, EOFError) as e:
+ if isinstance(e, EOFError):
+ log("Electrum wallet disconnected")
+ else:
+ log("IOError: " + repr(e))
+ on_disconnect(address_history)
+ time.sleep(0.2)
+ try:
+ server_sock.close()
+ except IOError:
+ pass
+
+def get_input_and_output_scriptpubkeys(rpc, txid):
+ gettx = rpc.call("gettransaction", [txid])
+ txd = btc.deserialize(gettx["hex"])
+ output_scriptpubkeys = [sc['script'] for sc in txd['outs']]
+ input_scriptpubkeys = []
+ for ins in txd["ins"]:
+ try:
+ wallet_tx = rpc.call("gettransaction", [ins["outpoint"][
+ "hash"]])
+ except JsonRpcError:
+ #wallet doesnt know about this tx, so the input isnt ours
+ continue
+ script = btc.deserialize(str(wallet_tx["hex"]))["outs"][ins[
+ "outpoint"]["index"]]["script"]
+ input_scriptpubkeys.append(script)
+ return output_scriptpubkeys, input_scriptpubkeys, txd
+
+def generate_new_history_element(rpc, tx, txd):
+ if tx["confirmations"] == 0:
+ unconfirmed_input = False
+ total_input_value = 0
+ for ins in txd["ins"]:
+ utxo = rpc.call("gettxout", [ins["outpoint"]["hash"],
+ ins["outpoint"]["index"], True])
+ if utxo is None:
+ utxo = rpc.call("gettxout", [ins["outpoint"]["hash"],
+ ins["outpoint"]["index"], False])
+ if utxo is None:
+ debug("utxo not found(!)")
+ #TODO detect this and figure out how to tell
+ # electrum that we dont know the fee
+ total_input_value += int(Decimal(utxo["value"]) * Decimal(1e8))
+ unconfirmed_input = unconfirmed_input or utxo["confirmations"] == 0
+ debug("total_input_value = " + str(total_input_value))
+
+ fee = total_input_value - sum([sc["value"] for sc in txd["outs"]])
+ height = -1 if unconfirmed_input else 0
+ new_history_element = ({"tx_hash": tx["txid"], "height": height,
+ "fee": fee})
+ else:
+ blockheader = rpc.call("getblockheader", [tx['blockhash']])
+ new_history_element = ({"tx_hash": tx["txid"],
+ "height": blockheader["height"]})
+ return new_history_element
+
+def sort_address_history_list(his):
+ unconfirm_txes = list(filter(lambda h:h["height"] == 0, his["history"]))
+ confirm_txes = filter(lambda h:h["height"] != 0, his["history"])
+ #TODO txes must be "in blockchain order"
+ # the order they appear in the block
+ # it might be "blockindex" in listtransactions and gettransaction
+ #so must sort with key height+':'+blockindex
+ #perhaps check if any heights are the same then get the pos only for those
+ #a better way to do this is to have a separate dict that isnt in history
+ # which maps txid => blockindex
+ # and then sort by key height+":"+idx[txid]
+ his["history"] = sorted(confirm_txes, key=lambda h:h["height"])
+ his["history"].extend(unconfirm_txes)
+ return unconfirm_txes
+
+def check_for_updated_txes(rpc, address_history, unconfirmed_txes):
+ updated_srchashes1 = check_for_unconfirmed_txes(rpc, address_history,
+ unconfirmed_txes)
+ updated_srchashes2 = check_for_confirmations(rpc, address_history,
+ unconfirmed_txes)
+ updated_srchashes = updated_srchashes1 | updated_srchashes2
+ for ush in updated_srchashes:
+ his = address_history[ush]
+ sort_address_history_list(his)
+ if len(updated_srchashes) > 0:
+ debug("new tx address_history =\n" + pprint.pformat(address_history))
+ debug("unconfirmed txes = " + pprint.pformat(unconfirmed_txes))
+ debug("updated_scripthashes = " + str(updated_srchashes))
+ else:
+ debug("no updated txes")
+ return updated_srchashes
+
+def check_for_confirmations(rpc, address_history, unconfirmed_txes):
+ confirmed_txes_srchashes = []
+ debug("check4con unconfirmed_txes = " + pprint.pformat(unconfirmed_txes))
+ for uc_txid, srchashes in unconfirmed_txes.items():
+ tx = rpc.call("gettransaction", [uc_txid])
+ debug("uc_txid=" + uc_txid + " => " + str(tx))
+ if tx["confirmations"] == 0:
+ continue #still unconfirmed
+ log("A transaction confirmed: " + uc_txid)
+ confirmed_txes_srchashes.append((uc_txid, srchashes))
+ block = rpc.call("getblockheader", [tx["blockhash"]])
+ for srchash in srchashes:
+ #delete the old unconfirmed entry in address_history
+ deleted_entries = [h for h in address_history[srchash][
+ "history"] if h["tx_hash"] == uc_txid]
+ for d_his in deleted_entries:
+ address_history[srchash]["history"].remove(d_his)
+ #create the new confirmed entry in address_history
+ address_history[srchash]["history"].append({"height":
+ block["height"], "tx_hash": uc_txid})
+ updated_srchashes = set()
+ for tx, srchashes in confirmed_txes_srchashes:
+ del unconfirmed_txes[tx]
+ updated_srchashes.update(set(srchashes))
+ return updated_srchashes
+
+def check_for_unconfirmed_txes(rpc, address_history, unconfirmed_txes):
+ MAX_TX_REQUEST_COUNT = 256
+ tx_request_count = 2
+ max_attempts = int(math.log(MAX_TX_REQUEST_COUNT, 2))
+ for i in range(max_attempts):
+ debug("listtransactions tx_request_count=" + str(tx_request_count))
+ ret = rpc.call("listtransactions", ["*", tx_request_count, 0, True])
+ ret = ret[::-1]
+ if last_known_recent_txid[0] == None:
+ recent_tx_index = len(ret) #=0 means no new txes
+ break
+ else:
+ txid_list = [(tx["txid"], tx["address"]) for tx in ret]
+ recent_tx_index = next((i for i, (txid, addr)
+ in enumerate(txid_list) if
+ txid == last_known_recent_txid[0][0] and
+ addr == last_known_recent_txid[0][1]), -1)
+ if recent_tx_index != -1:
+ break
+ tx_request_count *= 2
+
+ #TODO low priority: handle a user getting more than 255 new
+ # transactions in 15 seconds
+ debug("recent tx index = " + str(recent_tx_index) + " ret = " + str(ret))
+ # str([(t["txid"], t["address"]) for t in ret]))
+ if len(ret) > 0:
+ last_known_recent_txid[0] = (ret[0]["txid"], ret[0]["address"])
+ debug("last_known_recent_txid = " + str(last_known_recent_txid[0]))
+ assert(recent_tx_index != -1)
+ if recent_tx_index == 0:
+ return set()
+ new_txes = ret[:recent_tx_index][::-1]
+ debug("new txes = " + str(new_txes))
+ #tests: finding one unconfirmed tx, finding one confirmed tx
+ #sending a tx that has nothing to do with our wallets
+ #getting a new tx on a completely empty wallet
+ #finding a confirmed and unconfirmed tx, in that order, then both confirm
+ #finding an unconfirmed and confirmed tx, in that order, then both confirm
+ #send a tx to an address which hasnt been used before
+ obtained_txids = set()
+ updated_scripthashes = []
+ for tx in new_txes:
+ if "txid" not in tx or "category" not in tx:
+ continue
+ if tx["category"] not in ("receive", "send"):
+ continue
+ if tx["txid"] in obtained_txids:
+ continue
+ obtained_txids.add(tx["txid"])
+ output_scriptpubkeys, input_scriptpubkeys, txd = \
+ get_input_and_output_scriptpubkeys(rpc, tx["txid"])
+
+ matching_scripthashes = []
+ for spk in (output_scriptpubkeys + input_scriptpubkeys):
+ scripthash = util.script_to_scripthash(spk)
+ if scripthash in address_history:
+ matching_scripthashes.append(scripthash)
+ if len(matching_scripthashes) == 0:
+ continue
+ updated_scripthashes.extend(matching_scripthashes)
+ new_history_element = generate_new_history_element(rpc, tx, txd)
+ log("Found new unconfirmed tx: " + str(new_history_element))
+ for srchash in matching_scripthashes:
+ address_history[srchash]["history"].append(new_history_element)
+ if new_history_element["height"] == 0:
+ if tx["txid"] in unconfirmed_txes:
+ unconfirmed_txes[tx["txid"]].append(srchash)
+ else:
+ unconfirmed_txes[tx["txid"]] = [srchash]
+ return set(updated_scripthashes)
+
+def build_address_history_index(rpc, wallet_addresses):
+ log("Building history index with " + str(len(wallet_addresses)) +
+ " addresses")
+ st = time.time()
+ address_history = {}
+ for addr in wallet_addresses:
+ scripthash = util.address_to_scripthash(addr)
+ address_history[scripthash] = {'addr': addr, 'history': [],
+ 'subscribed': False}
+ wallet_addr_scripthashes = set(address_history.keys())
+ #populate history
+ #which is a blockheight-ordered list of ("txhash", height)
+ #unconfirmed transactions go at the end as ("txhash", 0, fee)
+ # 0=unconfirmed -1=unconfirmed with unconfirmed parents
+
+ BATCH_SIZE = 1000
+ ret = list(range(BATCH_SIZE))
+ t = 0
+ count = 0
+ obtained_txids = set()
+ while len(ret) == BATCH_SIZE:
+ ret = rpc.call("listtransactions", ["*", BATCH_SIZE, t, True])
+ debug("listtransactions skip=" + str(t) + " len(ret)=" + str(len(ret)))
+ t += len(ret)
+ for tx in ret:
+ if "txid" not in tx or "category" not in tx:
+ continue
+ if tx["category"] not in ("receive", "send"):
+ continue
+ if tx["txid"] in obtained_txids:
+ continue
+ obtained_txids.add(tx["txid"])
+
+ #obtain all the addresses this transaction is involved with
+ output_scriptpubkeys, input_scriptpubkeys, txd = \
+ get_input_and_output_scriptpubkeys(rpc, tx["txid"])
+ output_scripthashes = [util.script_to_scripthash(sc)
+ for sc in output_scriptpubkeys]
+ sh_to_add = wallet_addr_scripthashes.intersection(set(
+ output_scripthashes))
+ input_scripthashes = [util.script_to_scripthash(sc)
+ for sc in input_scriptpubkeys]
+ sh_to_add |= wallet_addr_scripthashes.intersection(set(
+ input_scripthashes))
+ if len(sh_to_add) == 0:
+ continue
+
+ new_history_element = generate_new_history_element(rpc, tx, txd)
+ for scripthash in sh_to_add:
+ address_history[scripthash][
+ "history"].append(new_history_element)
+ count += 1
+
+ unconfirmed_txes = {}
+ for srchash, his in address_history.items():
+ uctx = sort_address_history_list(his)
+ for u in uctx:
+ if u["tx_hash"] in unconfirmed_txes:
+ unconfirmed_txes[u["tx_hash"]].append(srchash)
+ else:
+ unconfirmed_txes[u["tx_hash"]] = [srchash]
+ debug("unconfirmed_txes = " + str(unconfirmed_txes))
+ if len(ret) > 0:
+ #txid doesnt uniquely identify transactions from listtransactions
+ #but the tuple (txid, address) does
+ last_known_recent_txid[0] = (ret[-1]["txid"], ret[-1]["address"])
+ else:
+ last_known_recent_txid[0] = None
+ debug("last_known_recent_txid = " + str(last_known_recent_txid[0]))
+
+ et = time.time()
+ log("Found " + str(count) + " txes. Address history index built in "
+ + str(et - st) + "sec")
+ debug("address_history =\n" + pprint.pformat(address_history))
+
+ return address_history, unconfirmed_txes
+
+def import_watchonly_addresses(rpc, addrs):
+ log("Importing " + str(len(addrs)) + " watch-only addresses into the"
+ + " Bitcoin node after 5 seconds . . .")
+ debug("addrs = " + str(addrs))
+ time.sleep(5)
+ for a in addrs:
+ rpc.call("importaddress", [a, ADDRESSES_LABEL, False])
+ #TODO tell people about the `rescanblockchain` call which allows a range
+ log("Done.\nIf recovering a wallet which already has existing " +
+ "transactions, then\nrestart Bitcoin with -rescan. If your wallet " +
+ "is new and empty then just restart this script")
+
+def main():
+ try:
+ config = ConfigParser()
+ config.read(["config.cfg"])
+ config.options("wallets")
+ except NoSectionError:
+ log("Non-existant configuration file `config.cfg`")
+ return
+ rpc = JsonRpc(host = config.get("bitcoin-rpc", "host"),
+ port = int(config.get("bitcoin-rpc", "port")),
+ user = config.get("bitcoin-rpc", "user"),
+ password = config.get("bitcoin-rpc", "password"))
+ #TODO somewhere here loop until rpc works and fully sync'd, to allow
+ # people to run this script without waiting for their node to fully
+ # catch up sync'd when getblockchaininfo blocks == headers, or use
+ # verificationprogress
+ printed_error_msg = False
+ while bestblockhash[0] == None:
+ try:
+ bestblockhash[0] = rpc.call("getbestblockhash", [])
+ except TypeError:
+ if not printed_error_msg:
+ log("Error with bitcoin rpc, check host/port/username/password")
+ printed_error_msg = True
+ time.sleep(5)
+ wallet_addresses = []
+ for key in config.options("wallets"):
+ addrs = config.get("wallets", key).replace(' ', ',').split(',')
+ wallet_addresses.extend(addrs)
+ wallet_addresses = set(wallet_addresses)
+ imported_addresses = set(rpc.call("getaddressesbyaccount",
+ [ADDRESSES_LABEL]))
+ if not wallet_addresses.issubset(imported_addresses):
+ import_watchonly_addresses(rpc, wallet_addresses - imported_addresses)
+ else:
+ address_history, unconfirmed_txes = build_address_history_index(
+ rpc, wallet_addresses)
+ hostport = (config.get("electrum-server", "host"),
+ int(config.get("electrum-server", "port")))
+ run_electrum_server(hostport, rpc, address_history, unconfirmed_txes,
+ int(config.get("bitcoin-rpc", "poll_interval_listening")),
+ int(config.get("bitcoin-rpc", "poll_interval_connected")))
+
+main()
DIR diff --git a/util.py b/util.py
t@@ -0,0 +1,365 @@
+
+import bitcoin as btc
+import hashlib, binascii
+from math import ceil, log
+
+## stuff copied from electrum's source
+
+def to_bytes(something, encoding='utf8'):
+ """
+ cast string to bytes() like object, but for python2 support
+ it's bytearray copy
+ """
+ if isinstance(something, bytes):
+ return something
+ if isinstance(something, str):
+ return something.encode(encoding)
+ elif isinstance(something, bytearray):
+ return bytes(something)
+ else:
+ raise TypeError("Not a string or bytes like object")
+
+def sha256(x):
+ x = to_bytes(x, 'utf8')
+ return bytes(hashlib.sha256(x).digest())
+
+def bh2u(x):
+ return binascii.hexlify(x).decode('ascii')
+
+def script_to_scripthash(script):
+ """Electrum uses a format hash(scriptPubKey) as the index keys"""
+ h = sha256(bytes.fromhex(script))[0:32]
+ return bh2u(bytes(reversed(h)))
+
+def address_to_scripthash(addr):
+ script = btc.address_to_script(addr)
+ return script_to_scripthash(script)
+
+#the 'result' field in the blockchain.scripthash.subscribe method
+# reply uses this as a summary of the address
+def get_status_electrum(h):
+ if not h:
+ return None
+ status = ''
+ for tx_hash, height in h:
+ status += tx_hash + ':%d:' % height
+ return bh2u(hashlib.sha256(status.encode('ascii')).digest())
+
+bfh = bytes.fromhex
+hash_encode = lambda x: bh2u(x[::-1])
+hash_decode = lambda x: bfh(x)[::-1]
+
+def Hash(x):
+ x = to_bytes(x, 'utf8')
+ out = bytes(sha256(sha256(x)))
+ return out
+
+def hash_merkle_root(merkle_s, target_hash, pos):
+ h = hash_decode(target_hash)
+ for i in range(len(merkle_s)):
+ item = merkle_s[i]
+ h = Hash(hash_decode(item) + h) if ((pos >> i) & 1) else Hash(
+ h + hash_decode(item))
+ return hash_encode(h)
+
+## end of electrum copypaste
+
+
+def script_to_address(script):
+ #TODO bech32 addresses
+ #TODO testnet, although everything uses scripthash so the address vbyte doesnt matter
+ return btc.script_to_address(script, 0x00)
+
+def calc_tree_width(height, txcount):
+ return (txcount + (1 << height) - 1) >> height
+
+#follow the flags down into the tree, building up the datastructure
+def decend_merkle_tree(hashes, flags, height, txcount, pos):
+ flag = next(flags)
+ print("f=" + str(flag) + " height=" + str(height) + " txc=" +
+ str(txcount) + " pos=" + str(pos) + " width=" +
+ str(calc_tree_width(height, txcount)))
+ if height > 0:
+ #non-txid node
+ if flag:
+ left = decend_merkle_tree(hashes, flags, height-1, txcount, pos*2)
+ #bitcoin has a rule that if theres an odd number of nodes in
+ # the merkle tree, the last hash is duplicated
+ if pos*2+1 < calc_tree_width(height-1, txcount):
+ right = decend_merkle_tree(hashes, flags, height-1,
+ txcount, pos*2+1)
+ else:
+ right = left
+ #TODO decend down one branch and hash it up, place in right
+ return (left, right)
+ else:
+ hs = next(hashes)
+ hs = hs[:4] + '...' + hs[-4:]
+ print(hs)
+ return hs
+ else:
+ #txid node
+ hs = next(hashes)
+ hs = hs[:4] + '...' + hs[-4:]
+ print(hs)
+ if flag:
+ return "tx:" + str(pos) + ":" + hs
+ else:
+ return hs
+
+def deserialize_core_format_merkle_proof(hash_list, flag_value, txcount):
+ tree_depth = int(ceil(log(txcount, 2)))
+ hashes = iter(hash_list)
+ #one-liner which converts the flags value to a list of True/False bits
+ flags = (flag_value[i//8]&1 << i%8 != 0 for i in range(len(flag_value)*8))
+ try:
+ root_node = decend_merkle_tree(hashes, flags, tree_depth, txcount, 0)
+ return root_node
+ except StopIteration:
+ raise ValueError
+
+#recurse down into the tree, adding hashes to the result list in depth order
+def expand_tree_electrum_format(node, result):
+ left, right = node
+ if isinstance(left, tuple):
+ expand_tree_electrum_format(left, result)
+ if isinstance(right, tuple):
+ expand_tree_electrum_format(right, result)
+ if not isinstance(left, tuple):
+ result.append(left)
+ if not isinstance(right, tuple):
+ result.append(right)
+
+#https://github.com/bitcoin/bitcoin/blob/master/src/merkleblock.h
+#https://github.com/breadwallet/breadwallet-core/blob/master/BRMerkleBlock.c
+def convert_core_to_electrum_merkle_proof(proof):
+ proof = binascii.unhexlify(proof)
+ pos = [0]
+ def read_as_int(bytez):
+ pos[0] += bytez
+ return btc.decode(proof[pos[0] - bytez:pos[0]][::-1], 256)
+ def read_var_int():
+ pos[0] += 1
+ val = btc.from_byte_to_int(proof[pos[0] - 1])
+ if val < 253:
+ return val
+ return read_as_int(pow(2, val - 252))
+ def read_bytes(bytez):
+ pos[0] += bytez
+ return proof[pos[0] - bytez:pos[0]]
+
+ pos[0] = 80
+ txcount = read_as_int(4)
+ hash_count = read_var_int()
+ hashes = [binascii.hexlify(read_bytes(32)[::-1]).decode()
+ for i in range(hash_count)]
+ flags_count = read_var_int()
+ flags = read_bytes(flags_count)
+
+ print(hashes)
+ print([flags[i//8]&1 << i%8 != 0 for i in range(len(flags)*8)])
+ print(txcount)
+
+ root_node = deserialize_core_format_merkle_proof(hashes, flags, txcount)
+ print(root_node)
+ hashes_list = []
+ expand_tree_electrum_format(root_node, hashes_list)
+
+ #remove the first or second element which is the txhash
+ tx = hashes_list[0]
+ if hashes_list[1].startswith("tx"):
+ tx = hashes_list[1]
+ assert(tx.startswith("tx"))
+ hashes_list.remove(tx)
+ #if the txhash was duplicated, that is included in electrum's format
+ if hashes_list[0].startswith("tx"):
+ hashes_list[0] = tx.split(":")[2]
+ pos, txid = tx.split(":")[1:3]
+ pos = int(pos)
+ blockhash = binascii.hexlify(btc.bin_dbl_sha256(proof[:80])[::-1])
+ result = {"pos": pos, "merkle": hashes_list, "txid": txid,
+ "blockhash": blockhash.decode()}
+ return result
+
+merkle_test_vectors = [
+ {'coreproof':
+ "0300000026e696fba00f0a43907239305eed9e55824e0e376636380f00000000000" +
+ "000004f8a2ce51d6c69988029837688cbfc2f580799fa1747456b9c80ab808c1431" +
+ "acd0b07f5543201618cadcfbf7330300000b0ff1e0050fed22ca360e0935e053b0f" +
+ "e098f6f9e090f5631013361620d964fe2fd88544ae10b40621e1cd24bb4306e3815" +
+ "dc237f77118a45d75ada9ee362314b70573732bce59615a3bcc1bbacd04b33b7819" +
+ "198212216b5d62d75be59221ada17ba4fb2476b689cccd3be54732fd5630832a94f" +
+ "11fa3f0dafd6f904d43219e0d7de110158446b5b598bd241f7b5df4da0ebc7d30e7" +
+ "748d487917b718df51c681174e6abab8042cc7c1c436221c098f06a56134f9247a8" +
+ "12126d675d69c82ba1c715cfc0cde462fd1fbe5dc87f6b8db2b9c060fcd59a20e7f" +
+ "e8e921c3676937a873ff88684f4be4d015f24f26af6d2cf78335e9218bcceba4507" +
+ "d0b4ba6cb933aa01ef77ae5eb411893ec0f74b69590fb0f5118ac937c02ccd47e9d" +
+ "90be78becd11ecf854d7d268eeb479b74d137278c0a5017d29e90cd5b35a4680201" +
+ "824fb0eb4f404e20dfeaec4d50549030b7e7e220b02eb2105f3d2e8bcc94d547214" +
+ "a9d03ff1600",
+ 'electrumproof':
+ {'pos': 5, 'merkle': [
+ '4b3162e39eda5ad7458a11777f23dc15386e30b44bd21c1e62400be14a5488fd',
+ 'e01932d404f9d6af0d3ffa114fa9320863d52f7354bed3cc9c686b47b24fba17',
+ 'e24f960d6261330131560f099e6f8f09feb053e035090e36ca22ed0f05e0f10f',
+ '681cf58d717b9187d448770ed3c7eba04ddfb5f741d28b595b6b44580111ded7',
+ 'a12bc8695d676d1212a847924f13566af098c02162431c7ccc4280ababe67411',
+ '7a9376361c928efee7209ad5fc60c0b9b28d6b7fc85dbe1ffd62e4cdc0cf15c7',
+ '33b96cbab4d00745bacebc18925e3378cfd2f66af2245f014dbef48486f83f87',
+ 'ec8be70bd9e947cd2cc037c98a11f5b00f59694bf7c03e8911b45eae77ef01aa',
+ 'b04f82010268a4355bcd909ed217500a8c2737d1749b47eb8e267d4d85cf1ed1',
+ '9d4a2147d594cc8b2e3d5f10b22eb020e2e7b7309054504deceadf204e404feb'],
+ 'txid':
+ 'da1a2259be752dd6b5162221989181b7334bd0acbbc1bca31596e5bc32375770',
+ 'blockhash':
+ "000000000000000014491e51be24278716c24d12ec0dbadf8c5f04f7f1846f5a"}
+ },
+ {"coreproof":
+ "0100000053696a625fbd16df418575bce0c4148886c422774fca5fcab8010000000" +
+ "000001532bfe4f9c4f56cd141028e5b59384c133740174b74b1982c7f01020b90ce" +
+ "05577c67508bdb051a7ec2ef942f000000076cde2eb7efa90b36d48aed612e559ff" +
+ "2ba638d8d400b14b0c58df00c6a6c33b65dc8fa02f4ca56e1f4dcf17186fa9bbd99" +
+ "0ce150b6e2dc9e9e56bb4f270fe56fde6bdd73a7a7e82767714862888e6b759568f" +
+ "b117674ad23050e2931197494d457efb72efdb9cb79cd4a435724908a0eb31ec7f7" +
+ "a67ee03837319e098b43edad3be9af75ae7b30db6f4f93ba0fdd941fdf70fe8cc38" +
+ "982e03bd292f5bd02f28137d343f908c7d6417379afe8349a257af3ca1f74f623be" +
+ "6a416fe1aa96a8f259983f2cf32121bce203955a378b3b44f132ea6ab94c7829a6c" +
+ "3b360c9f8da8e74027701",
+ "electrumproof":
+ {'pos': 9, 'merkle': [
+ '6fe50f274fbb569e9edce2b650e10c99bd9bfa8671f1dcf4e156caf402fac85d',
+ 'aded438b099e313738e07ea6f7c71eb30e8a902457434acd79cbb9fd2eb7ef57',
+ '81f202bdf592d23be08289c38cfe70df1f94dd0fba934f6fdb307bae75afe93b',
+ 'b6336c6a0cf08dc5b0140b408d8d63baf29f552e61ed8ad4360ba9efb72ede6c',
+ '59f2a896aae16f416abe23f6741fcaf37a259a34e8af797341d6c708f943d337',
+ '748edaf8c960b3c3a629784cb96aea32f1443b8b375a9503e2bc2121f32c3f98'],
+ 'txid':
+ 'd494741931290e0523ad747611fb6895756b8e886248716727e8a7a773dd6bde',
+ "blockhash":
+ "000000000000028113c80cc4be7058ab80a7767329d0253558d81d709f62ca40"}
+ },
+ {"coreproof":
+ "000000206365d5e1d8b7fdf0b846cfa902115c1b8ced9dd49cb1780000000000000" +
+ "000001032e829e1f9a5a09d0492f9cd3ec0762b7facea555989c3927d3d975fd407" +
+ "8c7718495a45960018edd3b9e0160a00000dfe856a7d5d77c23ebf85c68b5eb303d" +
+ "85e56491ed6d204372625d0b4383df5a44d6e46d2db09d936b9f5d0b53e0dbcb3ef" +
+ "b7773d457369c228fd1ce6e11645e366a58b3fc1e8a7c916710ce29a87265a6729a" +
+ "3b221b47ea9c8e6f48707b112b8d67e5cfb3db5f88b042dc49e4e5bc2e61c28e1e0" +
+ "fbcba4c741bb5c75cac58ca04161a7377d70f3fd19a3e248ed918c91709b49afd37" +
+ "60f89ed2fefbcc9c23447ccb40a2be7aba22b07189b0bf90c62db48a9fe37227e12" +
+ "c7af8c1d4c22f9f223530dacdd5f3ad850ad4badf16cc24049a65334f59bf28c15c" +
+ "ecda1a4cf3f2937bd70ee84f66569ce8ef951d50cca46d60337e6c697685b38ad21" +
+ "7967bbe6801d03c44fcb808cd035be31888380a2df1be14b6ff100de83cab0dce25" +
+ "0e2b40ca3b47e8309f848646bee63b6185c176d84f1546a482e7a65a87d1a2d0d5a" +
+ "2b683e2cae0520df1e3525a71d71e1f551abd7d238c3bcb4ecaeea7d5988745fa42" +
+ "1a8604a99857426957a2ccfa7cd8df145aa8293701989dd207505923fcb33984394" +
+ "4ce3d21dc259bcda9c251ed90d4e55af2cf5b15432050084f513ac74c0bdd4b6046" +
+ "fb70100",
+ "electrumproof":
+ {'pos': 330, 'merkle': [
+ '23f2f9224c1d8cafc7127e2237fea948db620cf90b9b18072ba2abe72b0ab4cc',
+ 'a08cc5ca755cbb41c7a4cbfbe0e1281ce6c25b4e9ec42d048bf8b53dfb5c7ed6',
+ '37293fcfa4a1cdce158cf29bf53453a64940c26cf1ad4bad50d83a5fddac0d53',
+ 'b812b10787f4e6c8a97eb421b2a329675a26879ae20c7116c9a7e8c13f8ba566',
+ '1d80e6bb677921ad385b6897c6e63703d646ca0cd551f98ece6965f684ee70bd',
+ 'a30cb4e250e2dcb0ca83de00f16f4be11bdfa280838831be35d08c80cb4fc403',
+ 'e34516e1e61cfd28c26973453d77b7efb3bc0d3eb5d0f5b936d909dbd2466e4d',
+ '3e682b5a0d2d1a7da8657a2e486a54f1846d175c18b663ee6b6448f809837eb4',
+ 'a4f53d38b4d025263704d2d61e49565ed803b35e8bc685bf3ec2775d7d6a85fe',
+ 'a821a45f7488597deaaeecb4bcc338d2d7ab51f5e1711da725351edf2005ae2c',
+ '94439833cb3f92057520dd8919709382aa45f18dcda7cf2c7a95267485994a60',
+ 'b6d4bdc074ac13f58400053254b1f52caf554e0dd91e259cdabc59c21dd2e34c'],
+ 'txid':
+ '4734c2c9bcef2fed890f76d3af499b70918c91ed48e2a319fdf3707d37a76141',
+ "blockhash":
+ "00000000000000000035c1e0b8f6c7886a5d41b685c4f0094a5b91759a5fe235"}
+ }
+]
+
parazyd.org:70 /git/electrum-personal-server/commit/99e2ff189e133a58ed70741aa85e145e8bfd8a5c.gph:3447: line too long