0: Welcome to the Jungle
A seismic shift is happening under your feet at this very moment. The institutions that facilitate daily life from degree granting, to voting, to financing home loans, all rely on your trust in them to accurately keep track of enormous amounts of information.
This process of organizing information precedes the digital age by thousands of years. When our most distant ancestors started working collectively to run Bison off cliffs, they leveraged our greatest strength as a species by organizing information together to achieve a larger goal.
As the layers of complexity mounted with the transition to agriculture, some form of hierarchy was needed to keep track of things. Thus the “ledger” was born: written records of credits and debits that allowed infinitely more specialization to occur.
But who controls the ledger? And how can people trust that those in charge would fairly record each transaction without swaying the ledger to benefit themselves?
Throughout this book, we hope to illuminate how distributed ledger technology can usher in a new era of transparency and fairness into our systems by fundamentally rewriting the back-end architecture that underpins our society.
It boggles the mind to think about how significantly ledgers effect our daily lives. What country are you a citizen of? How much money is in your bank account? Where did your groceries come from?
In the 21st century, answers to these fundamental questions are by and large stored somewhere on someone else’s database. When we shifted from paper to computers, we largely shifted existing patterns of processing information into an infinitely more powerful medium, yet one that is still just as susceptible to forgery and destruction.
For instance, you may notice that vintage paper spreadsheets look remarkably similar to modern digital spreadsheets. While digital spreadsheets are capable of performing computations only dreamed of in the paper age, they are even more susceptible to tampering than their paper predecessors.
Physical erasure marks might leave some evidence of an important line item being removed from a budget on a paper ledger, while digital erasure marks can be much harder to spot.
In the first computer age, centralized power structures were able eat the world by leveraging new found computational power to hoard data inside of central database silos. Unfortunately for the many disparate stakeholders in the system from users, to shareholders, to regulators, to citizens of far flung locales, the very nature of centralized systems do not lend themselves naturally to checks and balances. Unless many different trusted eyes have access to the same unadulterated information repository, problems will continue to occur inside of opaque black boxes.
Since the development of the Bitcoin protocol in 2009, we will make the case that a second computer age has been ushered in. While bitcoin invented nothing radically new from a computer science perspective, it combined existing technologies in a new way to create the first provable “trust” system. In less than ten years, this trust system has somehow convinced total strangers from all around the world to wantonly burn billions of dollars worth of electricity in an effort to create this secured shared commons. As a by product of trying to get rich, Bitcoin miners have secured a distributed network of interconnected databases, where everyone can agree on a common set of facts without needing to share any common values.
While such a feat is in itself worth investigating for the pure audacity, we must stay vigilant knowing this new age does not automatically usher in some kumbaya panacea. Like all new industries, pirates and hucksters are abound. Despite the occasional charlatans, this new age gives anyone the power to contribute to global open database. In this new kind of database, no authority has the power to corrupt the ledgers we rely on to prove who we are, who we communicate with, or what we create.
Ministry of Truth
We will begin our journey to understand what this “distributed ledger” concept is about at the humble college Registrar’s office. Here we will find our first proto “distributed application”.
A registrar has three fundamental tasks it is trusted to do properly.
Receive grades from professors
Convert those grades into a grade point average
Record those grades for all time in a secure ledger
Or more generally:
Receive Data -> Process Data -> Record Data
Data is any piece of information (represented digitally). Examples include variables filled with either floating point values (like GPA) or strings of characters (like name):
B+ = 3.33
Intro to Chemistry Course Code = 101
Student ID = Alice1
Process is where the data is converted into a result using some kind of logic. In computer speak these could be logical IF, THEN, ELSE, AND, OR, etc. type statements:
IF GPA >= 2.5
IF Credit Hours >= 120
IF Chemistry C101 Requirement = Taken
Then -> issue degree to Alice1
Record is the storage of the processed data that becomes ultimate source of truth about what courses Alice took, and how well she did in them.
While the university may focus on leading edge research or student-professor ratios to sell themselves on the market, the legitimacy of the institution rests with the often overlooked and seemingly mundane registrar’s office.
If students, employers, and other institutions cannot trust that applicants really earned their degrees, then what value does the institution have?
With potentially hundreds of thousands of euros/dollars/dinars/satoshis on the line for each degree granted, it is amazing how rarely malicious actors attempt to co-opt the registrar's ledger. Trust of the ledger is so ingrained in our collective psyche, when a Frank Abagnale occasionally bursts into the popular consciousness, we have to make a movie about it.
Four main factors influence the susceptibility of any centralized ledger to attack.
The incentives for system insiders (administrators) to corrupt the ledger
The incentives for system outsiders (users) to corrupt the ledger
Need for the system to inter-operate with other systems
Fluidity of the data (or how frequently the data needs to be updated)
The university has a massive incentive to provide accurate records of student performance. As the product they sell is the intangible “degrees” they provide, it is paramount people trust that the degrees granted are valid.
While the university itself has a strong incentive to maintain the integrity of the ledger, fallibility exists in administrators and faculty which could be bribed to change grades.
While the vast majority of students are honest, there is an ever-present temptation to corrupt the ledger to benefit oneself. Incentives for students are to earn the highest grades possible so they can secure the best opportunities post-graduation.
There is also the outside chance a hacker will attempt to infiltrate the registrar to create a fictitious degree within the institution. The reward for doing so is extremely asymmetric, with little to lose and much to gain.
Despite the incentives in place for insiders to accept bribes, and outsiders to attempt to modify the ledger for their benefit, registrars by and large are secure systems.
Registrars do not need to rely on other systems to work properly. Access to the ledger is limited solely to local administrators that collect data from a decentralized network of teachers. There is no single point of failure among the teachers, as each one would need to be corrupted individually to sway the results of the student’s degree.
When you request an official transcript to send to an employer or other higher education institution, it is often still done via physical mail as the official letterhead and seal is difficult to forge.
Sending transcripts digitally exposes the centralized digital ledger to some risk, as any open communication with the outside world not protected through an air gapping measure is in theory vulnerable.
However, it is not difficult to perform a simple encryption cipher on the transcript once a line of trust is established between any two parties.
The sending institution simply:
encrypts the student’s transcript into a random string of letters and numbers
provides the receiving institution with a nonce to confirm the communication channel is secure
then sends the encrypted data over the newly created secure channel with a key to unlock on the other end
Because the channel is encrypted, if an attacker attempts to view the data they would only see random strings of letters and numbers. Even more importantly, if an outside attacker attempts to intercept the message and change the results by even a single character, the results will not match and the receiver will know the message is compromised.
Note: this book will try very hard to stop before explaining how these concepts work at a raw computer science level. If you are curious how the pistons fire and the fuel is mixed, there are many insightful resources online that will be linked to.
Of course, this is a balancing act that will surely confuse some readers at some point. The sections that we think will cause some non-techies to become more confused than helped will be demarcated with ------------------- above and below.
So far we have a system that only insiders (administrators) can interact with, and only select outsiders (users like students or employers) can view.
In theory, the system is impervious to attack as long as access is not compromised internally (by administrators) or externally (by hackers) who gain access inside the system through a vulnerability in the network.
Unlike a bank account where the ledger of credit and debits is in constant flux, once a course is taken the results are locked in a fixed state forever. (Eg. Alice1 getting a B+ in Chemistry in May 1979 is an “immutable” fact)
No amount of coercing should be able to modify this fact once it is recorded into the ledger. This allows the registrar to record the transaction in many places including in psychical books stored securely under lock and key.
Even if the digital database is corrupted by hackers after the grades are recorded, a physical backup copy of the ledger exists that will show the truth. Thus, the only way to truly tamper with the registrar is to not only corrupt the digital database, but also tamper with every printed copy of the ledger that exists.
Ultimately, the security of the registrar's office boils down to the four primary issues addressed above:
Administrators are a small group of trusted individuals (internal hacks)
Administrators do not expose their network to outsiders (external hacks)
Administrators do not communicate with outsiders (interoperability)
The data itself is simple in that it does not need to dynamically change once recorded, so can be more easily backed up into redundant copies (fluidity)
Our registrar example was a sneaky way of explaining the basic techniques distributed ledgers use to remain secure. Registrar's have some basic ingredients of distributed ledger, but lack the formal mathematics that can unequivocally "prove" records have not been corrupted.
What if instead of needing to look in dusty books to validate that records match, all records were stored in thousands of databases around the world that all updated simultaneously each time new transactions were recorded?
The Massachusetts Institutive of Technology realized this issue, and began encoding student’s grades into a blockchain in 2017. In the MIT blockchain system:
Teacher A sends a transaction saying student Alice1 received a B+ in Chemistry 101,
A unique record identifying data fields such as Teacher A, Alice1, B+, and Chemistry 101 are recorded into the next block of transactions.
Once recorded in the block, ledgers from all around the world register a copy of the transaction in their ledgers.
For the system to work, all copies of the ledger must agree on the same history of all previous blocks. If a ledger has a different history, then its transactions are rejected as invalid. In this world, history cannot be tampered with without breaking the entire system.
But if all records are public, can't anyone look up sensitive information about anyone else?
Even though a public record of the transcript exists, they are encrypted so the general public cannot simply look up that Alice received a B+ in Chemistry (unless she wants them to).
Instead they would see something like this:
A1babab sent c2dcdcd to e3fefef Where:
A1babab represents Alice
C2dcdcd represents the transcript
E3fefef represents the employer
For Alice to prove her credentials, Alice acting ALONE (eg. NOT through a registrar middleman) would send her results to the employer, who alone could decrypt them.
In traditional databases, administrators are able to Read/Write/Modify/Delete data with the right credentials.
In essence a distributed ledger works just like a regular database except users are only able to read & write. No modifications or deletions are possible as the corrupted ledger would not match all of the others.
In the distributed ledger world, the entire Registrar’s office has been turned into an automated peer-to-peer system!
The Registrar "App" works something like:
Step 1: Instead of submitting the grades to the registrar’s office, the teacher instead submits the grades directly to a distributed ledger.
Step 2: Instead of calling the Registrar’s office, spending 10 dollars, and waiting days for the transcript to be sent over.. Alice can send her transcript instantly at almost no cost by sending a transaction from her account to her prospective employer's account directly.
Step 3: The employer receiving Alice’s transcript knows with certainty she really took the courses she said she did and earned her degree. No one other than Alice and the employer knows the contents of the transmission, only that address A sent something encrypted to address B.
Does this mean that the entire university is going to be replaced, and all hierarchies are going away?
Of course not.
A trusted authority still needs to issue credentials to the teachers so they can issue grades. Functions like the admissions process, tenuring of faculty, naming new buildings, etc. are still controlled by an administration.
The key difference is the operating system they use to post decisions is now auditable and immutable, rather than a fallible black box.
dApps are Apps
A registrar's office is in essence an "Application" (App) no different from the flashlight function on your smartphone, or the accounting software you use to do your taxes. Applications simply provide value by solving a problem with programmatic logic.
Thus the notion of a "Distributed Application" or (dApp) revolution presented in the next chapter is NOT a revolution in application logic, but rather a revolution in the record keeping system that applications interact with.
Instead of being run by one potentially fallible computer (or cloud of computers with a central authority), dApps are run by a shared network of distributed ledgers that allow any two parties to interact with each other where no middleman can block or modify the transaction.
As we will find out, this is not a panacea that will automatically solve all of the world's problems. In fact, it will create many new problems.
While fake news and hate speech can be censored and removed from centralized social media platforms, the core logic that would underpin a distributed ledger based social network leads to alarming new outcomes.
Fake news, hate speech, and all things unsavory in the eye of any individual beholder cannot be censored
Fake news will be recorded for all time in an un-corruptible format along with real news, gray news, and all shades in between.
On the flip side:
If politicians used distributed ledger based email for all official communications, a private email server debacle would be impossible.
Censorship of incriminating records could never be fully expunged even if every physical copy is confiscated.
Distributed ledgers change nothing about the front end of our digital experience, (distributed Twitter can be cloned to look identical to centralized Twitter) but has the potential to change everything about the databases that underpin the backend of our modern world.
The tradeoff in this new world is all information from college degrees (to expunged records of falsely accused sex offenders) will be preserved in digital ink, even if we invent new systems to filter how the data is presented.
Creating the “un-burnable” Library of Alexandria
When analyzing potential distributed ledger platforms to use as integrity engines for enterprise applications, or indirectly as investments, one crucial question must always be asked...
Is the project trying to create an un-burnable Library of Alexandria?
If the answer is yes, the project stands a non-zero chance of succeeding.
If the answer is no, run quickly.
The Library of Alexendria was the finest repository of collective human knowledge in the ancient world. Unfortunately, the technology did not exist to create immutable redundant backups of each book in the library, and centuries of hard won mathematics, philosophy, and history were lost in a great fire.
Central databases, much like the Library of Alexandria, can go up in digital flames with a single press of the delete key. Existentially worse than a total loss, databases can be subtly corrupted with no digital fingerprints left behind to show the original truth.
From Orwellian newspeak, to fake news, to the deletion of property records, we desperately need true digital ink to write our histories in, or be subject to the next generation of despots who will continue to manipulate what we see and hear.
As we will uncover throughout the book, distributed ledgers exist to create immutable redundant records of account. Not to get rich, not to revolutionize E-commerce (though we will investigate that use case), but to preserve data and limit the power middlemen have over our daily lives.
Unlocking vast wealth is a mere externality of vastly increasing the efficiency of the middleman value chain. With trust implicit in the system and friction to perform transactions of any complexity approaching zero, people will trade more, earn more, and grow more.
During the original dot com bubble, Sand Hill road exploded with bright eyed MBAs desperate to cash in on the first internet gold rush. E-commerce-for-X was the de rigeur ticket out of corporate hell, and into the pantheon of the entrepreneurial greats like Steve Jobs and Jeff Bezos. Digitization would obliterate high overhead brick-and-mortar incumbents in whatever niche they chose. Pet shops, nail salons, and funeral homes, would all be slain as their venture capital fueled warships carved out massive chunks of market share.
In a space with so much crushing competition from centralized incumbent survivors of the original dot com bubble, and fellow distributed ledger competitors (there are 2000+ token projects and counting) searching for teams driven by a deeper purpose than self-enrichment cannot be overstated.