Bitcoin for the Befuddled (2015)
A. HELLO MONEY! A SIMPLE JAVASCRIPT PROGRAM
One factor that makes Bitcoin so exciting is that anyone can write computer programs to link directly into the Bitcoin network and perform financial transactions. This appendix explains how to write a simple JavaScript program that watches a wallet and indicates when bitcoins have been sent to the wallet. In Appendix B, we’ll create some more advanced programs in Java that expand on this idea.
The Meaning of “Easy”
Writing programs that manipulate money in Bitcoin is easy. However, as we walk through this process, it might not seem very easy to you.
But imagine, instead, if we built our app or service using a traditional e-commerce system. Roughly, here’s what those steps would look like:
1. Incorporate your business to receive a DUNS1 number.
2. Get a corporate account at your bank.
3. Have your account verified by a merchant services provider (Intuit, PayPal, Stripe, Apple, etc.).
4. Set up an account via the provider with your DUNS number to get access keys.
5. Acquire proprietary library software from the provider (you’ll probably need to update this library regularly to maintain your software).
6. Redirect your customers to a special provider to authorize payments.
7. Write your code.
8. Have your finished app reviewed by the payment provider.
Here is the main step needed to get a Bitcoin-based, e-commerce system up and running:
1. Write your code.
Having to complete just this one task is what we mean when we use the term easy when programming with Bitcoin.
Three Ways to Write Bitcoin Software
To write Bitcoin-enabled software, you can use three different approaches:
1. Use a merchant service.
2. Connect to a local Bitcoin wallet program (typically the original Bitcoin wallet application based on Satoshi’s initial code, named bitcoind).
3. Create a program that inserts itself directly into the Bitcoin network.
Using approach #1, you would connect to a web API over the Internet. That API would be supplied by a third-party company to manage your bitcoins. Companies that supply these APIs for sending and receiving bitcoins are typically called merchant services.
For example, if your website sells widgets and you want people to buy your widgets with bitcoins, using a merchant service may be a simple, nofuss way to get your website working quickly. An additional benefit of a merchant service is that it may also offer services to automatically convert Bitcoin to/from other currencies as part of a transaction. Many companies supply these APIs, and BitPay is a common choice. But you can find a more extensive list of companies on the official Bitcoin foundation website.2
However, this approach has some drawbacks. First, these vendor APIs arguably go against the spirit of Bitcoin, because using an API middleman makes you beholden to an external party; serious Bitcoiners prefer to retain 100 percent control over their money. Second, these APIs are mostly designed only for common use cases, so they don’t allow the flexibility and innovative app development that we want to foster with this book. Third, the design of these APIs tends to change frequently, making it difficult to provide you with current information. For these reasons, we won’t spend much time discussing vendor APIs in this book.
Approach #2 involves connecting to a local Bitcoin wallet and basically running the wallet on autopilot. Two of the Bitcoin wallet programs based on Satoshi’s original code—Bitcoin Core and bitcoind—can be remote-controlled using a special protocol called JSON-RPC, as we will discuss shortly. Because these two programs are the gold standard in Bitcoin wallets, it is very enticing to utilize them for custom programs. Programs written to automate Bitcoin Core and bitcoind are easy to understand, and libraries exist in all popular programming languages, simplifying their use. You can use this approach in JavaScript, Ruby, PHP, Python, and many other languages. Unfortunately, the main drawback of using a local Bitcoin wallet program and controlling it from your own code is that this process often clunky and inefficient, as you’ll see in the first programming example.
NOTE
Bitcoin Core and bitcoind share the same code. This common code is called the Bitcoin reference client, or just Bitcoin. It was the first implementation of the Bitcoin protocol and eventually was split into two variants: Bitcoin Core has a friendly graphical interface (UI), and bitcoind is a more stripped-down version with a text-based interface.
Approach #3 involves going right to the metal and inserting your Bitcoin-enabled program directly into the Bitcoin network. You can do this in languages such as Java, C++, or Go or any language that has a fully implemented Bitcoin client library available. This approach is more robust and less resource-intensive than approach #2, but it is more technically complex.
However, if you’re a programmer who believes in the Bitcoin ethos, being able to write an app that is a real first-class citizen in the Bitcoin network—a true Bitcoin node participating in the Bitcoin system—is motivating. The reason is that absolutely no limitations restrict what your app can do (as long as your app obeys the rules of the network.) In Appendix B, we’ll write some programs that use this approach.
General Security Notes on Bitcoin Programming
An important disclaimer we need to mention here is that in this appendix, we’ll write programs that send and receive only a few pennies. The techniques and example discussed are useful for learning the basic concepts of Bitcoin programming but are in no way appropriate for writing programs that manipulate substantial sums of money. If you plan on writing serious Bitcoin applications, you’ll need to do the following:
1. Learn the basic concepts from the sample programs in this chapter.
2. Use this knowledge to study and understand the underlying source code for the Bitcoin libraries used in this chapter.
3. Follow the forums used by the developers and other library users to stay current with any security risks involved when using these libraries.
Most important, be aware that we’re using community-maintained source code in our examples; if a clever black hat hacker3 manages to insert some rogue code into the official library repositories, he or she can steal all your money. Even if you understand the library code perfectly, you run the risk of jeopardizing the safety of your money. For example, as you’re downloading this library code from the Internet, a black hat hacker has many opportunities to perform a man-in-the-middle attack4 and insert rogue code into a doctored version of the library that is incorporated into your program. As a result, the hacker can steal all your money.
Additionally, as mentioned in earlier chapters, hackers can steal your bitcoins in many other ways that aren’t specific to Bitcoin programming. In a few years, if the current popularity of Bitcoin continues, we suspect most computer viruses will include code that immediately empties any Bitcoin wallets they find.
The bottom line is that you need to understand the grave risks your money might be exposed to if you plan on creating more advanced Bitcoin programs; you’ll be able to safely protect your money only if you have a comprehensive and firm understanding of Bitcoin technology that goes beyond the introduction we give in this chapter. Please proceed with caution!
Some Upbeat Notes on Bitcoin Security
Now that you’ve endured some fire and brimstone to prevent you from losing money, we’ll point out a few facts that should make you feel confident about Bitcoin security:
• The core Bitcoin network has so far maintained a record of almost perfect security. Most of the security risks involved in using bitcoins are due to careless acts that people executed on their own computers and could have been avoided.
• Despite the risks we’ve discussed so far in this chapter, established programming practices exist that mitigate all of these risks. You will be able to write secure Bitcoin software if you study this book, follow general guidelines on secure software development, and keep up-to-date on the latest security concerns discussed on Bitcoin development forums.
• As you are learning to write programs that use bitcoins, make sure you use a computer that doesn’t contain Bitcoin wallets with substantial amounts of bitcoins in them. By doing so, you can learn to write Bitcoin software and avoid any dangers: You can’t accidentally lose bitcoins (or have them stolen) on your development machine if you store your coins elsewhere.
Writing Your First Bitcoin Program in JavaScript
By convention, the first program a programmer writes when learning a new technology is a Hello World program: It’s the simplest program possible in that it just prints the message Hello World! on the screen. Essentially, it indicates to the novice that your initial code seems to be working at the outset.
However, the Bitcoin programs we need to write must perform two main tasks: receive money and send money. In this chapter, we’ll focus solely on receiving money and write a program called Hello Money!. In Appendix B we’ll write a Bye-Bye Money program to send money.
Why Use JavaScript?
JavaScript is arguably the most well-known programming language in use today, because most developers who build software for the Web must learn it sooner or later (it is the only language that can natively run in a standard web browser). However, the JavaScript code we create will not run in a web browser: This is because web browsers have strong protections called cross-domain restrictions that shield them against communicating with external programs and websites. These restrictions limit what you can do with JavaScript within a web browser, making it difficult to interact with any external wallets (which is no surprise, because you wouldn’t want your wallet emptied by visiting a malicious website).
However, currently you can also run JavaScript on a server, as is commonly done using the Node.js library. When run on a server, no such restrictions apply, and we can write simple programs that run in a console and can interact with Bitcoin Core and bitcoind.
Bitcoin Core vs. Bitcoind
As discussed earlier, the programming techniques described in this chapter will work with either of the wallet apps, Bitcoin Core or bitcoind. The only substantive difference between the two programs is that Bitcoin Core has a graphical UI; basically, it is just bitcoind with a frontend attached to it. Because of the friendliness of the graphical UI, we’ll use Bitcoin Core for the example in this chapter. However, bitcoind is slightly less resource-intensive and can more easily run on computer servers via a terminal console. Consequently, it’s best for you to use bitcoind if you actually deploy a finished program that uses the concepts in this chapter.
Preparing Your Machine for JavaScript Bitcoin Programming
You’ll need a few tools to ready your machine before you begin programming, so let’s get them now. The instructions provided here assume a Windows development environment; skip ahead to “For Mac Hackers” on page 219 or “For Linux Folks” on page 219 if you’re using either of those platforms.
Installing Node.js
You first need to download an installer for Node.js from the http://nodejs.org/ website. When you run the installer for Node.js, it will also install the Node Package Manager (npm) that we’ll rely on next.
Installing node-bitcoin
Now you’ll need a JavaScript library that connects to Bitcoin Core and bitcoind. We’ll use the node-bitcoin library. To install node-bitcoin, go to a command prompt and enter npm install bitcoin. This command invokes the (previously installed) Node Package Manager, which downloads everything in the library in a simple step.
Starting Bitcoin Core
If you haven’t already, download Bitcoin Core from its official website at http://bitcoin.org/en/download.
Next, you need to fire up Bitcoin Core in server mode. The wallet app will open an extra socket that we’ll use to connect to the wallet from our JavaScript program. To do this on Mac and Linux machines, go to the Bitcoin Core installation directory from the console and run./Bitcoin-Qt -server. On Windows machines, open a command prompt, go to the C:\Program Files (x86)\Bitcoin directory, and enter bitcoin-qt.exe -server.
The first time you run Bitcoin Core in server mode, it will ask you to create a file named bitcoin.conf in a specific location and add a user ID and password to this file. Follow the directions as suggested because we’ll use the userid and password shortly. Then, restart Bitcoin Core.
If you already have Bitcoin Core (previously called Bitcoin-Qt) installed but are following this tutorial for the first time, then you will need to locate your previously created bitcoin.conf file. For Windows users you might want to look in C:\Users\<username>\AppData\Roaming\Bitcoin\bitcoin.conf. For Mac users, try /Users/<username>/Library/ApplicationSupport/Bitcoin/bitcoin.conf. For Linux users, try /home/<username>/ .bicoin/bitcoin.conf.
NOTE
Gotcha for Windows users: If you try to create the bitcoin.conf file with Windows Notepad, be aware that Notepad will (oh so helpfully) tack on a .txt extension, which Windows will also (oh so helpfully) keep completely hidden from you for your own protection (a prime example of why most hackers these days try to avoid using Windows). To circumvent this problem, put quotes around the name bitcoin.conf when you enter it into the Save As dialog in Notepad.
At this point, as is usual for a Bitcoin wallet, Bitcoin Core will take several hours downloading the blockchain before it’s ready for the next steps—a perfect time for an extended lunch break!
For Mac Hackers
If you’re using a Mac and are comfortable using a terminal, you can run specific commands to automatically search for, download, and install the required programs.
On Mac, download homebrew, a command-line tool that will handle the entire process for you:
# Get homebrew and if you haven't already
ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"
brew tap phinze/homebrew-cask
brew install brew-cask
# Get node.js and bitcoin stuff
brew cask install bitcoin
brew install nodejs npm
npm install bitcoin
# Run bitcoin-qt in server mode
~/Applications/Bitcoin-Qt.app/Contents/MacOS/./Bitcoin-Qt -server
For Linux Folks
If you’re using flavors of Debian Linux, such as Ubuntu, you’ll just use the PPA feature to install the libraries:
sudo add-apt-repository ppa:bitcoin/bitcoin
sudo apt-get update
sudo apt-get install nodejs npm bitcoin-qt
npm install bitcoin
bitcoin-qt -server
With a working Bitcoin Core server, we’re now ready to start programming.
Hello Money!
Okay, let’s write our first Bitcoin app. Simply type the following complete program into a file named hellomoney.js:
var bitcoin = require("bitcoin");
var client = new bitcoin.Client({
host: 'localhost',
port: 8332,
user: 'myUsername',
pass: 'myPassword'
});
var previousBalance = 0;
function mainLoop() {
client.getBalance('*', 0, function (err, balance) {
if (err) {
console.log(err);
} else if (balance > previousBalance) {
console.log("Hello Money! New balance: " + balance);
previousBalance = balance;
} else {
console.log("Nothing's changed.");
}
});
}
setInterval(mainLoop(), 5000);
Before we start running the app, let’s analyze what the code does line by line.
Part 1: Initializing the Connection with Bitcoin Core
var bitcoin = require("bitcoin");➊
var client = new bitcoin.Client(➋
{
host: 'localhost',
port: 8332,
user: 'myUsername',➌
pass: 'myPassword'➍
});
var previousBalance = 0;➎
The first line indicates that we’re using the node-bitcoin library ➊. Next, we establish a connection to the Bitcoin Core server that we set up ➋. Because we’ll be running the app on the same machine that Bitcoin Core is installed on, our host is set to 'localhost'. By default, the bitcoin server will run on port 8332.
Important: On the next two lines ➌➍, input the user ID and password that you entered into the bitcoin.conf file. Replace the placeholders shown here with your own. Then, we create a variable to keep track of the previous balance in our program ➎, which we start at zero.
Part 2: The Main Loop
Now we’ll write a loop that checks the balance in our wallet and reports to us when it has changed. The basic steps follow:
1. Ask Bitcoin Core for the current balance in its wallet.
2. If the balance is higher than the previous balance, print a message, and update the previous balance to the current balance.
3. Set a timer that completes the entire procedure again every five seconds.
The following mainLoop function carries out the preceding steps:
function mainLoop() {
client.getBalance('*', 0, function (err, balance) {➊
if (err) {
console.log(err);➋
} else if (balance > previousBalance) {➌
console.log("Hello Money! New balance: " + balance);➍
previousBalance = balance;➎
} else {
console.log("Nothing's changed.");
}
});
}
setInterval(mainLoop, 5000);➏
First, the function asks Bitcoin Core for the balance ➊. As we do this, we create a callback function, which reads function(err,balance) {}. A callback function is called at some future date. In this instance, it will be called when we receive the resulting balance from Bitcoin Core.
NOTE
If you’ve never worked with Node.js before, you may need some time to learn how to read this type of programming idiom. The philosophy in Node.js is that whenever your program has to wait on an external process (in this case the Bitcoin Core program), it makes you create callback functions, as opposed to stopping the program dead as you wait for a result to arrive. This is called asynchronous programming and enables your program to do other tasks while waiting for data to come in.
When the balance becomes available, we first check whether an error has occurred and display it if one has ➋. Next, we check whether the new balance is higher than the previous balance ➌. If so, we print a message ➍ and update the previous balance ➎. Finally, we create a timer that runs every 5000 milliseconds with the setInterval function to repeatedly call mainLoop ➏.
The Bitcoin Core JSON-RPC API
The mainLoop function operates by talking with Bitcoin Core. This talking protocol is formatted as JSON-RPC, or JavaScript Object Notation – Remote Procedure Call. Simply put, JSON-RPC describes a structure with which two computers (or two programs on a single computer) can write messages to each other in a way that’s easy for computer programmers to integrate into their code.
In our small app, we use only a single command to communicate with Bitcoin Core—the client.getBalance function. But many more commands are available that let you manipulate Bitcoin addresses, modify the wallet, analyze transactions, and complete most other tasks you’d want to do in a Bitcoin program. To learn which functions are available in the JSON-RPC interface, browse to the official API function list at https://en.bitcoin.it/wiki/Original_Bitcoin_client/API_calls_list.
Running the Hello Money! App
To try out our shiny new Hello Money! program, run the following command in a console from the directory containing hellomoney.js:
> node hellomoney.js
Next, simply send 0.001 bitcoins to the wallet managed by Bitcoin Core from another wallet. To do this, you’ll need to use one of the public addresses, which you can find by clicking Receive in Bitcoin Core. The way we’ve written the app, it will count money that has arrived in your wallet only if it has received a confirmation on the blockchain. (In the next section, we’ll write a Hello Money! program that detects money immediately on arrival, without requiring confirmations.)
Because confirmations typically take about 10 minutes, you’ll need to wait a bit before you see the following:
> node hellomoney.js
Hello Money! New balance: 0.001
Congratulations! You’ve written a program that can watch a Bitcoin wallet and detect when money has been sent to it. If you then send additional money, the app will create messages to report on these new deposits as well.
Limitations of Writing Bitcoin Programs That Use JSON-RPC
By using JSON-RPC, we were able to write a program in JavaScript in no time that could report money sent to a Bitcoin wallet. However, if you’re an experienced programmer (or have an advanced understanding of the Bitcoin protocol), you’ll identify many problems with our example app.
One problem is that the app uses polling to detect when money has arrived. Polling constantly checks the status of Bitcoin Core by interrogating the software every five seconds. This process is similar to asking: “Do you have money yet? Do you have money now? What about now?” As you might imagine, repeatedly asking the same question like this not efficient.
If you’re writing a production-quality application, it’s therefore best to write it using pushing. Pushing involves specifying that our app is waiting for money and that it would like to be notified when money has arrived. But because of the way our connection with Bitcoin Core is set up, enabling pushing is rather difficult. After all, Bitcoin Core is primarily meant to be a wallet app in which no pushing to external programs is needed. Automating Bitcoin Core with JSON-PRC is a bit of a hack, and for this reason, we may run into limitations, such as needing to use inefficient polling.
NOTE
In more recent versions of Bitcoin Core and bitcoind, a feature called walletNotify allows for pushing, but the way it works is rather complicated—enough so that we recommend you avoid using it. Instead, we would suggest you use BitcoinJ instead of Bitcoin Core for pushing, as we will discuss in Appendix B.
Another problem with our app is that we’re just checking a crude balance amount to determine when money was sent to our wallet. However, the Bitcoin blockchain can sometimes trigger situations that cause the balance of a wallet to jump in ways that are unexpected and could make our app produce incorrect results.
For instance, suppose our wallet receives some money with one confirmation, but the blockchain suddenly forks, unconfirming the transaction that was tied to that money and causing the wallet to suddenly lose much of the confirmed money. Then, within the same five-second window, someone sends us more money. In this case, the balance in the wallet could actually decrease, even though we received some new money, and the Hello Money! message will never be triggered.
This is a very rare scenario, but if you’re writing an app that manipulates large quantities of money, such incorrect behavior is intolerable. The solution to this problem is to use client.listTransactions (instead of client.getBalance). Then review these transactions to determine whether any new transactions involved sending money to the wallet in the last five seconds, which is what our app is looking for. However, this is too complicated to add to our simple Hello Money! program.
An additional problem is that Bitcoin Core (and bitcoind) requires large amounts of blockchain data and computational resources to run, even though our little app doesn’t need most of this power. Is there any way of reducing these resource requirements? As you’ll see in Appendix B, we can do this by writing a program that runs directly on the Bitcoin network.