About The Book
In the year 2017, Bitcoin touched a market capitalization of over 100 billion dollars. In the year 2014, one Bitcoin could buy about 500 dollars, just three years later one Bit coin buys 16,000 dollars. The Initial Coin Offering or a ICO is becoming the preferred method of raising money.
Bitcoin, Ethereum, Blockchain are the most difficult technologies to understand. That s why most people including technology folks cannot understand the future direction of these technologies. The only way to understand anything complex is by going back to the basics. This is what we do in this book. We explain every byte of the Bit coin block chain that is downloaded on your computer. Only by going back to your roots can you understand anything complex.
Most of the code in this book is written in Python as today it is the easiest language to use. The Bitcoin Source is written only in C++. Most of the important Bit coin data structures are only documented in code, a bare knowledge of reading and not writing C++ will help. Finally, the official client for Ethereum is written in the programming language Go.
This book is written for a programmer. We use code and not words to describe a Blockchain is. We believe that all kinds of people including non-technology folks will need some programming to grasp the basic concepts of the Blockchain. There is no other way to understand this technology.
Finally, we end with the biggest use of Smart Contracts which is raising money using a ICO. Our primary focus is on Bitcoin s and Blockchains and not on Ethereum and Smart Contracts which comprises only 4 chapters.
International Currency transfers are very expensive today. The the advent of the Lighting Network and side chains, the Bitcoin blockchain can scale to a level where it can handle transactions faster than any credit card transaction. You cannot use Bitcoins to buy coffee today.
One of the recent bigger innovations of Blockchain technology is the Initial Coin Offering or a ICO. This will enable millions of people to invest in companies using Blockchain technology. Vijay Mukhi s work will help us understand the technologies under the hood that makes it happen
Chapter 01. Basics of the Bitcoin Block Header - This chapter sets the tone of the book. Using Python code, we understand some of the members of the Bitcoin Block Header using the existing Bitcoin blocks. We also prove that Bitcoin has an arguable Christian influence.
Chapter 02. Transactions - Basics - A Bitcoin block mainly stores transactions. Here, we gradually move forward to explain how a Bitcoin transaction is structured into inputs and outputs. A Bitcoin block is mainly made up of transactions.
Chapter 03. Computing the Merkle Hash - In this chapter, we have our first encounter with the cryptographic and mathematical constructs used in Bitcoin, the Merkle Tree or Hash. In place of the SHA-256 hash, we calculate the individual hash vales of each transaction and then use Merkle to return a single hash representing all the transactions in a block.
Chapter 04. Bitcoin Addresses - This chapter explains how a Bitcoin address is created along with a checksum. We use the standard library functions and then write our own code to generate a Bitcoin address.
Chapter 05. Vanity Bitcoin Addresses - An entire chapter is dedicated to creating vanity Bitcoin address which can be viewed on the Bitcoin blockchain for eternity.
Chapter 06. Difficulty and Nonce - The details of two block header fields, nonce and difficulty which were left unexplained earlier are elaborated here. These fields are used by miners to mine a block. The difficulty field ascertains if the miners have successfully mined a block.
Chapter 07. Storing Bitcoin Transactions using SQL - The highlight of this chapter is SQL. Using SQL, we store Bitcoin transactions in a Sql database called PostgreSQL. Plus, we verify some basic Bitcoin facts like does the difficulty change every 2016 blocks and more.
Chapter 08. Transactions - Inputs and Outputs - The chapter focuses on understanding the role of inputs and outputs in a Bitcoin transaction. Our first gentle introduction to scripting and signatures.
Chapter 09. Hiding Data in the blockchain - The Blockchain contains more information besides the Bitcoin transactions. It contains emails, pdf files, gif files etc. It still is a mystery as to how all this data got into a pristine Blockchain.
Chapter 10. Signing Transactions - Signing a Bitcoin transaction is what separates the men from the boys. The chapter reveals the fact that a real Bitcoin transaction with multiple inputs is signed by a certain private key P. A public key generated by the private key P confirms later that this private key P signed the transaction without ever having access to the private key P.
Chapter 11. Roll your own transaction - This chapter is where we put our all our knowledge to practice. We create a raw transaction and send it to a miner to mine. The catch here is that we are not using code written by the Bitcoin Core developers.
Chapter 12. Client and Server - This is the smallest chapter in our book. It focuses on the three types of networks that the Bitcoin server can operate on, mainnet, testnet and regtest.
Chapter 13. Notaries and OP_RETURN - This chapter exposes how we can now legally place any data of our choice in a blockchain. A new script opcode called OP_RETURN is introduced for this purpose.
Chapter 14. Pay to Script Hash or Multi-Sig Bitcoin addresses - Bitcoin addresses starting with 3 can get very complex. These Bitcoin addresses are called multi-sig and need more than one private key to sign a transaction if they are to be unlocked and stored in the outputs.
Chapter 15. Basic Networking - This chapter and the next chapter explain the Bitcoin networking protocol which is used to download transactions and blocks from miners. Miners, in general, are very pretentious. They will not send you any transaction or blocks unless you talk to them in a certain way. The chapter demonstrates the methodology and the communication protocol between the Bitcoin miners using some shortcuts that give you grief.
Chapter 16. More Networking - The networking saga continues. The program is enhanced to handle any data sent by a miner, including a 1MB block of transaction data. Further, in the chapter, a series of raw transaction bytes are sent across to a miner without using the send transaction method.
Chapter 17. Hashes SHA0 and SHA1 - The program in this chapter replicates or overrides one of the standard cryptographic library functions. We write a function that creates a sha0/1 hash value instead of using the standard library function. We then check the output with the library function to ensure that the calculated hash value is the same.
Chapter 18. Hashes - Sha-256 and RipeMD-160 - This chapter gives an insight into the technique used when computing both the SHA-256 and RIPEMD-160 hash. None of these hashes contain code we are proud of or happy with, especially the RIPEMD-160 hash. We are not cryptographers.
Chapter 19. ECC with Sage - Part 1 - A public key is just another point on the Elliptic curve. With this knowledge, we sign a transaction using a private key and then verify a signature with only the public key. In conclusion, a random number comes handy when you do not want the world to know your private key.
Chapter 20. ECC with Sage - Part 2 - This chapter signals the end of our cryptography journey. Here we calculate a public key and signature by using code that does not rely on Sage or a any Python library.
Chapter 21. Sending our own transaction - A milestone reached. We write a program to create all the bytes for a transaction and send it over. What s more, our code also signs the raw transaction before putting it on wire.
Chapter 22. Sending one transaction without using library functions - This chapter is a step forward. The program now has only our code replacing the standard library functions to create a transaction. Using this approach, we can create every byte of the transaction to be sent to a miner.
Chapter 23. Index folder - The chapter marks our journey into the undocumented internals of the Bitcoin world. An in-depth study of the index folder where every transaction is stored and the dat files where every block resides. We check over 197 million key-value pairs for consistency.Chapter 24. UTXO Dataset - The heart and soul of the Bitcoin blockchain is the UTXO set or those transaction hashes that have at least one unspent output. Simple rule: No UTXO set, no blockchain. This is the only practical approach for Bitcoin to keep track of those transaction outputs that are unspent, in a very efficient way. We do the newer UTXO set in another chapter.
Chapter 25. Wallets - All transactions related to the newly created or existing Bitcoin addresses are stored in a Bitcoin Wallet. We get a thorough understanding of all different keys present in the Berkley DB database wallet.dat.
Chapter 26. Rev/Undo files - After the blk files, the rev or undo files take up the maximum amount of hard disk space. The UTXO set deals with outputs whereas the undo blocks store inputs in a highly-compressed format. This chapter validates the data in each undo block.
Chapter 27. peers.dat and banlist.dat - The chapter looks at the two eluded files, banlist.dat and peers.dat, found in the Bitcoin folder. The banlist file works as advertised. It is the peers file that decides using cryptography, which peers we can connect to. Once connected, we can safely download the blockchain.
Chapter 28. Miners, blocks and more - This chapter focuses on miners and blocks. The initial part is focused on fields like chainwork and mediantime which are not stored on disk. Then, we determine who mined a block. We end by scanning through 200 million transactions.
Chapter 29. fee_estimates.dat - We pay miners to carry our Bitcoin transactions. Someone has to tell our wallet how much to pay a miner. This estimation of fees or change is where the file fee_estimates.dat comes in.
Chapter 30. Building the Bitcoin Source code - A chapter we wear on our sleeves. The original Bitcoin source code is compiled and built but with a small twist. The book file sizes are reduced to 512 KB, instead of 128MB and more. This chapter is what is responsible for this book being written.
Chapter 31. Testing Bitcoin for bugs - We enter the la-la lands of writing Bitcoin tests which can be written in C++ or Python. Our choice will always be the low-level C++ tests. This chapter carries out a test that checks if a certain transaction is well-formed.
Chapter 32. Ethereum Solidity - A few chapters are dedicated to Ethereum, the best known Blockchain after Bitcoin. Ethereum is also called the world computer as it runs programs written by others on computers all over the world. The biggest drawback with Bitcoin is that it has a limited programming vocabulary.
Chapter 33. Ethereum Leveldb keys and GOLANG - Ethereum like Bitcoin stores key-value pairs in a level dB database but it does not use blk or rev files. Likewise, Ethereum account balances are also stored in this giant state machine that is made up of only key value pairs. As of October 2107, there are over 300 million such key value pairs. This chapter gives an understanding into the different key value pairs.
Chapter 34. Ethereal Unravelling the State Machine - This chapter divulges into the differences between the UTXO data set and the Ethereum state machine. A Bitcoin UTXO set stores no account balances, which is not true for Ethereum. Then again, staying true to our style, we have attempted to display the balance of our Ethereum address using some very unstable code.
Chapter 35. Bitcoin Cash vs Bitcoin Core vs Segwit2x - Bitcoin Cash a new hard fork, actually has blocks whose sizes are close to 8MB. The heart of this chapter is segregated witnesses, something that will enable the settlement layer within the Bitcoin blockchain. We also throw in a few words of wisdom on segwit2x and why in our view it will not be bigger than Bitcoin Core.
Chapter 36. Bitcoin Core 0.15, UTXO and more - Bitcoin Core 0.15 released in September 2017. It brings in lots of goodies to the table, mostly unseen. Thankfully the UTXO set was rewritten to make it simpler but faster. We also cover smaller stuff like replay attacks and what is the actual size of a block and the structure of the mempool file and how Bitcoin creates a caching hash and ending with pruning the Blockchain.
Chapter 37. Transactions and Blocks- Error Checks - This is a revision of what type of error checks go into confirming a Bitcoin transaction. We learn what is dust and how does one calculate it. We also learn that the size of the signature and public key in the inputs cannot exceed 1650 bytes. Finally, we calculate how we compute the sigops or signature opcodes present in a transaction.
Chapter 38. ICO and Smart Contract Security - Bitcoin Core 0.15 released in September 2017. It brings in lots of goodies to the table, mostly unseen. Thankfully the UTXO set was rewritten to make it simpler but faster. We also cover smaller stuff like replay attacks and what is the actual size of a block and the structure of the mempool file and how Bitcoin creates a caching hash and ending with pruning the blockchain.
Chapter 39. What is a Bitcoin and a Blockchain - In our journey, all the way through this book, we tried defining a Bitcoin and a blockchain. We may be able to hammer out an agreement on the definition of a Bitcoin, but not on blockchain, not in the present nor in the future . So, everyone has the right to say their product is based on blockchain technologies. It comes as no surprise that there is an umbrella of blockchains in this realm.
Chapter 40. AI and Blockchain- Never The Twain Shall Meet - Artificial Intelligence is the biggest blockbuster technology that will change the solar system. Blockchain comes in a close second. We will actually build some basic AI models using a python library called Keras. We found no use case where blockchain and AI technologies could work in sync. Our biggest grouse with AI is that it is not a black box but a black hole which nobody understands.
About The Author
Vijay Mukhi will be 61 before the year 2018 leaves us which makes him an elder statesman. He has basically done two things in his life. Teaching in his own computer training institute aptly called Vijay Mukhi s Computer Institute or VMCI. At the same time wrote lots of books, over 80 of them ranging from traditional programming languages like C++ , Autodesk Animator .
His books have been translated into language like Japanese and Portuguese. All the books Vijay writes are inspired by the way he teaches. In his classes, he gets people who do understand the subject matter he teaches. He thus starts from scratch and explains every program line by line. Every line of code is worth a thousand words. Vijay Mukhi is not a software developer but a teacher at heart.
His hobby is working with law enforcement and the industry in catching the cyber-criminal.