sjors / libwally-swift

Swift wrapper for LibWally, a collection of useful primitives for cryptocurrency wallets

LibWally Swift

Opinionated Swift wrapper around LibWally, a collection of useful primitives for cryptocurrency wallets.

Supports a minimal set of features based on v0.8.8. See also original docs.

• Core Functions
  • base58 encode / decode
• Crypto Functions
  • sign ECDSA, convert to DER
• Address Functions
  • Parse to scriptPubKey
  • Generate from scriptPubKey #7 (wishlist, done for SegWit)
  • Derive
  • WIF
  • Detect bech32 typos #4 (wishlist)
  • bech32 and bech32m
• BIP32 Functions
  • Derive scriptPubKey #6 (wishlist)
• BIP38 Functions
• BIP39 Functions
• Descriptor functions
  • Parse and canonicalize
  • Convert to address
  • Convert to scriptPubKey
• Script Functions
  • Serialize scriptPubKey
  • Determine scriptPubkey type
• PSBT functions
  • Parse and serialize (base64 / binary)
  • Check completeness and extract transaction
• Transaction Functions
  • Compose and sign transaction
  • Calculate fee

Items marked with wishlist are not (yet) available upstream.

Multisig as well as Elements specific functions such as confidential addresses are not implemented.

Works with iOs 11+ on 64-bit devices and the simulator.

Usage

Derive address from a seed:

let mnemonic = BIP39Mnemonic("abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about")
let masterKey = HDKey(mnemonic.seedHex("bip39 passphrase"))!
masterKey.fingerprint.hexString
let path = "m/44'/0'/0'"
let account = try! masterKey.derive(path)
account.xpub
account.address(.payToWitnessPubKeyHash)

Derive address from an xpub:

let account = HDKey("xpub6ASuArnXKPbfEwhqN6e3mwBcDTgzisQN1wXN9BJcM47sSikHjJf3UFHKkNAWbWMiGj7Wf5uMash7SyYq527Hqck2AxYysAA7xmALppuCkwQ")
let receivePath = "0/0"
key = account.derive(receivePath)
key.address(.payToPubKeyHash) # => 1JQheacLPdM5ySCkrZkV66G2ApAXe1mqLj

Parse an address:

var address = Address("bc1q6zwjfmhdl4pvhvfpv8pchvtanlar8hrhqdyv0t")
address?.scriptPubKey # => 0014d09d24eeedfd42cbb12161c38bb17d9ffa33dc77
address?.scriptPubKey.type # => .payToWitnessPubKeyHash

Create and sign a transaction:

let txId = "400b52dab0a2bb5ce5fdf5405a965394b43a171828cd65d35ffe1eaa0a79a5c4"
let vout: UInt32 = 1
let amount: Satoshi = 10000
let witness = Witness(.payToWitnessPubKeyHash(key.pubKey))
let input = TxInput(Transaction(txId)!, vout, amount, nil, witness, scriptPubKey)!
transaction = Transaction([input], [TxOutput(destinationAddress.scriptPubKey, amount - 110)])
transaction.feeRate // Satoshi per byte
let accountPriv = HDKey("xpriv...")
let privKey = try! accountPriv.derive("0/0")
transaction.sign([privKey])
transaction.description # transaction hex

See also the included Playground and tests.

Install

As part of a project.

### CocoaPods

Add to your Podfile:

pod 'LibWally', :git => 'https://github.com/Sjors/LibWally-Swift.git', :tag => 'v0.0.3', :submodules => true

and then run:

pod install --verbose

Build

For development.

Install dependencies:

brew install gnu-sed automake libtool

Clone the repository, including submodules:

git clone https://github.com/Sjors/libwally-swift.git --recurse-submodules

Xcode will build libwally-core for you, which may take a few minutes.

When making changes to libwally-core, keep in mind that cleaning the build folder in Xcode will not trigger a recompile. You need to call git clean -dfx inside CLibWally/libwally-core.

libwally-swift's Issues

P2SH-Segwit and Legacy Single Signature signing occasionally failing

I am using LibWally-Swift to sign for multisig and single signature transactions/psbt's.

Frequently I get an error for P2SH-Segwit transactions when broadcasting them:

non-mandatory-script-verify-flag (signature must be zero for failed check(multi)sig operation

And occasionally for Legacy I get an error when broadcasting as well:

mandatory-script-verify-flag-failed (operation not valid with the current stack size) (code 16)

In the readme it states Compose and sign (SegWit) transaction. Does that imply non segwit transactions do not have signing functionality? I am confused because in the tests it has examples for signing all three types of transactions.

As a side note if it is a multisig transaction all three formats work fine as do single sig Native Segwit transactions.

Of course I may be doing something wrong... However if you have any obvious advice for me it would be much appreciated. Again thank you for the wonderful library.

Use LibWally 0.8.7 / 0.8.8 API improvements

See suggested improvements here: ElementsProject/libwally-core#349 (comment)

Non extended private key derivation?

Is it possible to derive a normal private key instead of only extended private keys?

Infer address from scriptPubKey

When a wallet composes a transaction, or needs to interpret an existing transaction, it's useful to be able to convert its scriptPubKey outputs to an address.

Start with wally_scriptpubkey_get_type. This returns the WALLY_SCRIPT_TYPE_ type:

#define WALLY_SCRIPT_TYPE_UNKNOWN   0x0
#define WALLY_SCRIPT_TYPE_OP_RETURN 0x1
#define WALLY_SCRIPT_TYPE_P2PKH     0x2
#define WALLY_SCRIPT_TYPE_P2SH      0x4
#define WALLY_SCRIPT_TYPE_P2WPKH    0x8
#define WALLY_SCRIPT_TYPE_P2WSH     0x10
#define WALLY_SCRIPT_TYPE_MULTISIG  0x20

libwally-core would need to add parser functions for each of those types, in particular WALLY_SCRIPT_TYPE_P2PKH, WALLY_SCRIPT_TYPE_P2SH, WALLY_SCRIPT_TYPE_P2WPKH and WALLY_SCRIPT_TYPE_P2WSH .

See ElementsProject/libwally-core#118

Call CLibWally Functions directly

Is there a way to call CLibWally functions directly?

Ability to access pubkey data directly?

I would like to display the hex of derived pubkeys to the user so that they may verify the derived pubkeys are what they expect them to be when for example importing a seed (along with the address and wif). When I try and access the raw data of the pubkey like so:

key.pubKey.data

I get an error:

'data' is inaccessible due to 'internal' protection level

Is there a way I can disable the internal protection level so that I may access the data directly?

PSBT.finalize() causing a crash

I have found a bug that is only effecting a very specific type of transaction, I am able to consistently replicate this.

Finalizing single signature PSBT's with more then two inputs are causing LbWally-Swift to crash here:

public mutating func finalize() -> Bool {
        var psbt = UnsafeMutablePointer<wally_psbt>.allocate(capacity: 1)
        psbt.initialize(to: self.wally_psbt)
        defer {
            psbt.deallocate()
        }
        guard wally_finalize_psbt(psbt) == WALLY_OK else {
            return false
        }
        return true
    }

Here is an example culprit PSBT:

cHNidP8BALACAAAAA4H3/ONLmdp/hMvfX/4K8HbeuxN1D1Cl5YxD/XOs0VJuAAAAAAD9////iU1RvtTUo4/2Sf4s9FBMphOVdKPiYajZc57+P+XpKegBAAAAAP3///+SVXjLEcHD16XjRJERJRugRJHAvkibNUQ684Q8/1j+OwEAAAAA/f///wEudAAAAAAAACIAIMNsUgb+DfidqgmB+jkSHBjNfTSYC3YXjt3Clr8rFn5jAAAAAAABAR8QJwAAAAAAABYAFLANUblgpSofl8eylI7CRPE62QzFIgICt4h4iljY72O3vP+cyWkfgw11OV7HHar8cmYOFtBn7SZHMEQCIAbIiwMClMlCr+6ye+iYn7w//DaGNFoQqxx9jYMft1+HAiAYGUu2t/iyix0waHQu6z0AE7w8e3UWwm56Z4r11rgZIAEiBgK3iHiKWNjvY7e8/5zJaR+DDXU5XscdqvxyZg4W0GftJhhJD/gBVAAAgAEAAIAAAACAAAAAAA0AAAAAAQEfECcAAAAAAAAWABRVOL2EqfdQecYd+gMRYno4ybOPAiICAoI5knNM8b7LjSQkzMoJZuEUfYQEtxCVOYSSnWyyOugWRzBEAiBXePGlvNZIQEHzwqkuGuuNWXjDwDaZfSb8NrCKqKSqPgIgb6VJmX61zQ7w3JmAoqRg/Aht29cxZ7xEI02UUe94TB0BIgYCgjmSc0zxvsuNJCTMyglm4RR9hAS3EJU5hJKdbLI66BYYSQ/4AVQAAIABAACAAAAAgAAAAAAMAAAAAAEBHxAnAAAAAAAAFgAUm1khoZrGf9yKKICK6hDsIuU5egQiAgNr3T6Q5LQBEdbt1MpH7wnC/1vlEXVBLP75YlsPkk/jOkcwRAIgKeakN8YyoxRy3cW1SseGxH4umXPdhgnHOkciMR0z4cwCICy7kmTQBEoQQ8Q7kHp+TD+62aTp8eo+AsQJl8s6TS/2ASIGA2vdPpDktAER1u3UykfvCcL/W+URdUEs/vliWw+ST+M6GEkP+AFUAACAAQAAgAAAAIAAAAAACwAAAAAA

I bypass LibWally-Swift and use Bitcoin Core to sign the PSBT without issue so I do not think there is anything wrong with the PSBT or its signatures.

When I setup a breakpoint and step through it the following error prints out on the crash point:

Printing description of psbt:
expression produced error: error: /var/folders/3d/jzth48_15_g1mrhgr2pkkwcm0000gn/T/expr16-427a63..swift:1:92: error: use of undeclared type 'CLibWally'
Swift._DebuggerSupport.stringForPrintObject(Swift.UnsafePointer<Swift.UnsafeMutablePointer<CLibWally.wally_psbt>>(bitPattern: 0x114c80e50)!.pointee)

Multisig transactions with any number of inputs works fine, single sig transactions less then 3 inputs works fine. Which is strange. Perhaps related to #30 ?

Thanks again for the great library. If there is anything else I can do to help solve the issue just let me know.

Add Liquid functions

How can I add Liquid functions like asset_blinding_key_from_seed?
I already tried to remove the "--disable-elements" from the build-libwally.sh.
I have added the wally_elements.h in the module.modulemap but I can't access the functions.
How can I access the elements functions?

Cross-compile libwally-core for device

I'm still looking for the correct ./configure incantation to cross compile libwally-core to arm64.

Error: Can't exec "aclocal" in "Build libwally-core for device and simulator and combine into a single library..." step

What I'm trying to do

I'm trying to use libwally-swift in my swift project using cocoapods and I'm facing an issue.

What else I tried to do

So I tried to clone and build libwally-swift separately and still facing same error.

When I run this command

$ ./build-libwally.sh -dsc

I get this error

Build libwally-core for device and simulator and combine into a single library...
Can't exec "aclocal": No such file or directory at /usr/local/Cellar/autoconf/2.69/share/autoconf/Autom4te/FileUtils.pm line 326.
autoreconf: failed to run aclocal: No such file or directory
sed: ./tools/build-aux/ltmain.sh: No such file or directory
sed: ./tools/build-aux/ltmain.sh: No such file or directory
sed: ./src/secp256k1/build-aux/ltmain.sh: No such file or directory
sed: ./src/secp256k1/build-aux/ltmain.sh: No such file or directory

---

Useful debugging info:

I've followed the installation guidelines

I ran the below commands successfully

$ brew install gnu-sed
$ git clone https://github.com/blockchain/libwally-swift.git --recurse-submodules

---

• I checked this file /usr/local/Cellar/autoconf/2.69/share/autoconf/Autom4te/FileUtils.pm and it exists.

• I made sure that my user is the owner and have exec permission to everything inside /usr/local/Cellar/autoconf/

• I checked line 326 that have the issue and here is the containing function

---

Mac Catalyst build?

I had assumed I would easily be able to build this on Mac catalyst as the framework seems to be macOS + iOS compatible in Xcode. But immediately ran into a few roadblocks.. Unfortunately I’m not very experienced at compiling libraries and hacking away at them to build them for Mac catalyst. Any idea how one may achieve this? Should it work out of the box or is there some heavy lifting that needs doing in order to achieve that?

I understand i can easily use it explicitly for macOS but building one codebase for both would be really nice :-)

Switch secp256k1 to correct commit after libwally-core build

follow-up of #69

we want to switch back to the submodule HEAD so it doesn't end up in a dirty state.

Avoid dirty libwally-core submodule after build

Since #54 the build script switches from libsecp256k1-zkp to vanilla libsecp256k1 (and matching the commit used by Bitcoin Core). This uses libwally-core's --enable-standard-secp configure flag (introduced in ElementsProject/libwally-core#301).

The problem is that this switch leaves the libwally-core submodule dirty.

One possible solution would be to patch libwally-core to use libsecp256k1 from a custom directory. In that case we could include it directly as a submodule in libwally-swift and ignore libwally-core's libsecp256k1-zkp submodule.

Not sure how involved this is, cc @jgriffiths.

Perhaps there's another approach that doesn't require an upstream change. The script could do some magic with symlinks, but meh...

Use libwally's BIP32 path parsing code

#64 (comment)

Function(s) to use:

• https://wally.readthedocs.io/en/latest/bip32/#c.bip32_key_from_parent_path_alloc

Travis

Probably best done using the macOS build environment.

Unable to build for Playground

Hopefully this works again after #61.

Fix libwally-core linker warning

Probably needs a tweak to the build script: https://github.com/blockchain/libwally-swift/blob/master/build-libwally.sh

Add Swift Package Manager Support

Essentially what is wip in #73

I recently found out that Swift 5.4 added extensible build tools to Swift Package Manager, which will allow us to support deterministic builds with Swift Package Manager, since we can use it to call our external build script to build the C static libs.

For some reason this isn't documented in the official docs for the package manifest. 🤷

Restrict public vars to use private-only setters

See e.g. https://github.com/blockchain/libwally-swift/pull/14/files#r363318690

BIP32: scriptPubKey getter for HDKey

Currently (pending #5) you can get the address for a HDKey instance, but in order to get the scriptPubKey, you have to parse the address. This adds complexity when generating a change output, which a wallet shouldn't need to represent as an address (at that stage).

Depends on upstream ElementsProject/libwally-core#117

Travis replacement

Travis builds stopped working, because afaik they no longer offer free builds for open source projects and their main product is overpriced. I'm happy to pay ~$10 per month.

I could also run a CI system on my own Ubuntu server or iMac. If anyone has recommendations (assuming this can be done securely). Maybe Apple's new Xcode SAAS can be of use? I'm really behind on these things.

Reduce boiler plate c-bridge code

For example in order to call bip39_mnemonic_from_bytes and convert Data to String:

precondition(entropy.data.count <= MAX_WORDS)
var bytes = UnsafeMutablePointer<UInt8>.allocate(capacity: MAX_WORDS)
let bytes_len = entropy.data.count

var output: UnsafeMutablePointer<Int8>?
defer {
    wally_free_string(output)
}
entropy.data.copyBytes(to: bytes, count: entropy.data.count)

precondition(bip39_mnemonic_from_bytes(nil, bytes, bytes_len, &output) == WALLY_OK)

if let words_c_string = output {
    let words = String(cString: words_c_string)
    self.init(words)
} else {
    return nil
}

This code is currently not reused in any way, even though libwally-core only uses a limited number of input and output data types.

One approach could be similar to how libwally-core Python tests work, with a mapping of C to Python: https://github.com/ElementsProject/libwally-core/blob/master/src/test/util.py

• ('bip39_mnemonic_from_bytes', c_int, [c_void_p, c_void_p, c_ulong, c_char_p_p])

The Swift Package Manager should be useful ingredient. See e.g. this blog post, which suggests creating a "basic wrapper" and then "create a second Swift wrapper: a wrapper around the wrapper, providing a more natural Swift implementation".

creating multisig transactions possible?

I am attempting to build a multisig transaction using libwally-swift with a p2wsh input/output but getting stuck when specifying the Witness type for my input as it seems to be limited to p2wpkh and p2sh-p2wpkh e.g. single sig types only.

Is there something I am missing? Is it just a limitation of libwally-swift at the moment that only single sig transactions can be created?

Thanks,
Fontaine

Use wally_secp_randomize before signing

Although Bitcoin signatures are deterministic thanks to RFC6979, adding entropy to libwally using wally_secp_randomize helps it protect against side-channel attacks.

https://wally.readthedocs.io/en/latest/core/#c.wally_secp_randomize

Witness input amount type UInt64 is causing a rounding bug

I have a testnet witness input:

"witness_utxo" =                 {
                    amount = "0.009506239999999999";
                    scriptPubKey =                     {
                        address = tb1qtrj390qt7xn3vkl64e4ce6mzkyqtky5njqrumc;
                        asm = "0 58e512bc0bf1a7165bfaae6b8ceb62b100bb1293";
                        hex = 001458e512bc0bf1a7165bfaae6b8ceb62b100bb1293;
                        type = "witness_v0_keyhash";
                    };
                };

In my code I convert the input amount to satoshis with:

let amount = UInt64((witness_utxo["amount"] as! Double) * 100000000)

Which of course converts 0.009506239999999999 to 950624.
I then use LibWally-Swift to sign the input with (summarizing the code for brevity):

let witness = Witness(.payToWitnessPubKeyHash(key!.pubKey))
let input = TxInput(Transaction(txid)!, vout, amount, nil, witness, scriptPubKey)!
var transaction = Transaction(inputsToSign, outputsToSend)
let signedTx = transaction.sign(privKeys)

The error I receive when trying to broadcast is:

error =     {
        code = "-26";
        message = "non-mandatory-script-verify-flag (Signature must be zero for failed CHECK(MULTI)SIG operation) (code 64)";
    }

Which apparently is caused by input amounts not matching precisely as per this bitcoin stackexchange question.

Any ideas how I can prevent this? Is this a testnet issue? I was not even aware you can get input amounts sub satoshi like this. Is it possible to supply LibWally with a double instead of UInt64 so the input amount matches?

Libwally Pods are not getting installed

I had added below line into podfile

"pod 'LibWally', :git => 'https://github.com/blockchain/LibWally-Swift.git', :tag => 'v0.0.3', :submodules => true"

and when trying to install this using "pod install" command, I am getting below error

Combined device & simulator library no longer works

./build-libwally.sh -dsc should combine the simulator and device libraries, but this no longer works:

Related: #66 and #62

Hopefully #61 fixes this.

Error building libwally-swift with [email protected]

When running./build-libwally.sh -dsc / pod install, I am getting the following errors:

/Applications/Xcode.app/Contents/Developer/usr/bin/make  all-recursive
Making all in secp256k1
gcc -DHAVE_CONFIG_H -I. -I./src -O2  -std=c89 -pedantic -Wno-long-long -Wnested-externs -Wshadow -Wstrict-prototypes -Wundef -Wno-overlength-strings -Wall -Wno-unused-function -Wextra -Wcast-align -Wconditional-uninitialized -fvisibility=hidden -g -O2 -c src/gen_context.c -o gen_context.o
gcc -O2  -std=c89 -pedantic -Wno-long-long -Wnested-externs -Wshadow -Wstrict-prototypes -Wundef -Wno-overlength-strings -Wall -Wno-unused-function -Wextra -Wcast-align -Wconditional-uninitialized -fvisibility=hidden -g -O2  gen_context.o -o gen_context
./gen_context
  CC       src/libsecp256k1_la-secp256k1.lo
./libtool: line 151: -o: command not found
  CCLD     libsecp256k1.la
./libtool: line 151: -o: command not found
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: no output file specified (specify with -o output)

After digging into it, it looks like an issue with a recent update to libtool that affected our ability to build libwally-core (ElementsProject/libwally-core#320).

For anyone encountering this issue, I have a workaround:

brew unlink libtool
curl https://raw.githubusercontent.com/Homebrew/homebrew-core/0fbd6e24c4122e18ade1ec6c5916cb21de14f352/Formula/libtool.rb -o libtool.rb
brew install libtool.rb

This essentially unlinks your current install of [email protected], pulls an old formula of libtool (2.4.6_4), and installs that instead. That should fix the build until libwally-core and libwally-swift is updated.

To undo the workaround above,

brew unlink libtool
rm -rf /opt/homebrew/Cellar/libtool/2.4.6_4
brew install libtool

P.S: huge fan of your Bitcoin Explained podcast! Thanks for your work on that and this library! 🙏🏼

Feature request: Expose outputs to Transaction

It would be useful to be able to access the outputs of a raw transaction like so Transaction("rawtxhex").outputs just as you can access the inputs.

Avoid spurious rebuilding of libwally-core

It currently rebuilds all of libwally-core every time I run a test.

`PSBT.sign(privKey)` seems to only work on multisig transactions

I may be doing it wrong but I am using the PSBT.sign(privKey) function for multi-sig transactions and it works every time. However for single sig transactions I have to fall back to the the traditional way of signing raw transactions. I would very much like to enable users to be able to scan any PSBT and have the app sign it if it can.

Just curious if this is by design or if I am doing something wrong here?

EDIT: To clarify when I say it does not work what I mean is the PSBT will never be finalized or complete, it may actually be signing but the finalization/completion never returns true. I have confirmed with certainty the private key I am trying to use to sign is the correct private key and that there is only one input to sign.

Bitcoin Core 0.20.1 seems to break the psbt signer

Please see bitcoin/bitcoin#19650

libwally-core build warnings

As of #69 Xcode is handling the building of libwally-core. This reveals a number of warnings:

We should ideally resolve these upstream rather than suppress the warnings. That is, assuming these are indeed upstream bugs and not some configuration issue on our end.

Detect bech32 typo position(s)

See upstream ElementsProject/libwally-core#116

It would enable this:

Support for Swift Package Manager

Will LibWally be installable via the Swift Package Manager in the future?

`PSBT.input.origins` returns nil for a complete psbt

Just wondering how I may access the origins info on the inputs of a completed psbt? It seems once fully signed the origins info always returns nil.

Use Result type

A value that represents either a success or a failure, including an associated value in each case

https://developer.apple.com/documentation/swift/result

xcframework with signature

Is there a way to get the LibWally.xcframework with a valid signature? Because when I get the framework via Cocapods the framework is not signed.

Use framework breaks printing with LLDB

When importing libwally-swift as a framework via Cocoapods, we get this error when trying to do po in LLDB

error: expression failed to parse:
error: virtual filesystem overlay file '/.../LibWally/IntermediateBuildFilesPath/LibWally.build/Release-iphonesimulator/LibWally.build/all-product-headers.yaml' not found
error: virtual filesystem overlay file '/.../LibWally/IntermediateBuildFilesPath/LibWally.build/Release-iphonesimulator/LibWally.build/all-product-headers.yaml' not found

error: couldn't IRGen expression. Please check the above error messages for possible root causes.
error: couldn't IRGen expression. Please check the above error messages for possible root causes.

The cause is fairly well documented in the following LLVM issues:
apple/llvm-project#4467
apple/llvm-project#4464

And a great summary can be found here:
https://steipete.com/posts/couldnt-irgen-expression