Massively scalable ‘blockchainified’ databases (BigchainDB) … next internet phase … only 100 public databases to organise ‘Big Data’ and every ‘thing’ globally
Massively scalable 'blockchainified' databases are the next phase of the internet. In its current form, Blockchain is limited in terms of size, scalability and latency and is not a world changing technology. The next incarnation of Blockchain will not be Blockchain, but Blockchainified databases which overcome these limitations and are a profound technology that can scale and change the world. As few as 100 blockchainified databases (by sector or country) could organise (not necessarily store) every 'thing' around the world. This is a Zero Marginal Cost technology which means they could be deployed at practically no cost and could become a free service globally.
This article introduces the applications for Blockchainified databases and then provides a comprehensive overview of the technology. Wisdom Networks bring Wisdom to 'things'. Blockchainified databases store and transact 'things'. The Appendix below contains screenshots and a comprehensive overview of the 'things' in multiple sectors across society.
Massively scalable ‘blockchainified’ databases (BigchainDB) … next internet phase … only 100 public databases to organise every ‘thing’ globally
Blockchain is a distributed database that has gained notoriety in one if its application as a cryptocurrency or Bitcoin. The early incarnation of the Blockchain is, however, severely limited (size, latency and scalability) that renders it unusable in its current form. The next generation of Blockchain is not blockchain at all. The next generation begins with Massively Scalable Databases that are used as the foundation of the world’s largest web sites and adds Blockchain functionality. The outcome is a Blockchain enabled massively scalable database. This database will operate just like the internet. As computers are added to the collective internet, the networks scales and support every increasing users. As database servers (or computers) are added to the collective Blockchainified database, the collective capacity and reach of the database scales. This means that one database could store all the data for an entire country or global sector. 50 wisdom networks could organise the world. Similarly, 100 Blockchain-enabled massively scalable databases could organise society’s ‘things’ around the world in every country and sector.
Blockchainified databases can be configured in a variety of ways. A community model (similar to the internet) would mean that (1) no one node or organisation controls owns the data or the network of databases, and (2) secure exchange of information between 2 parties independent of central or facilitating nodes. Other models could involve private ownership of community ‘things’. It is possible that (like the internet), the storage and organisation of things becomes a global and collective endeavour.
Blockchainified databases reduce society complexity exponentially by consolidating every 'thing' in one distributed network and simultaneously increase transparency, accountability and engagement with all citizens exponentially. Public blockchainified databases would be even more profound and could provide a single network of society's 'things' at Zero Marginal Cost. Data previously lost and isolated in vertical silos become available to all in a public, distributed horizontal network accessible to all.
The community is crowd creating evolution from Industrial Economy to Network Society using concurrent processes. Community crowd created internet (collective connectivity). Community crowd created world wide web (collective information). Community crowd created social networks (collective community). Everyone knows everyone else. Community crowd creates open data (collective things) for every one. Industrial Economy fragments and centralises things in silos. Blockchainified databases overcomes these limitations and stores and transacts society's things in a distributed database that provides collective accessibility, transparency and governance across society. Everyone has access to every thing. Community can crowd creates wisdom (collective knowledge and prioritised action). The Industrial Economy is inefficient and ineffective due to its physical technologies that can't scale and the underlying tools of centralisation and fragmentation. Wisdom Networks overcome these limitations and, in conjunction with other Zero Marginal Cost technologies, can organise collective collaboration, collective knowledge and collective wisdom for every 'thing' in sectors, countries and the Global Village.
Zero Marginal Cost technologies enable the Network Society and Global Village. Zero Marginal Cost techologies include internet (collective connectivity), wisdom networks (collective collaboration, knowledge and wisdom), mobile devices (collective universal access), blockchain-enabled massively scalable databases (collective 'things'). Wisdom Networks bring Wisdom to 'things'. Blockchainified databases store and transact 'things'.
BigchainDB: A Scalable Blockchain Database
For your convenience, we will extract the key parapagraphs from a profound whitepaperb by BigchainDB on blockchain enabled massively scalable databases.
We introduce a concept called blockchain pipelining, which is key to scalability when adding blockchainlike characteristics to the distributed DB. We present a thorough description of BigchainDB, an analysis of latency, and preliminary experimental results. The paper concludes with a description of use cases. (Page 1)
A Public BigchainDB
Toward a decentralised application stack
The initial excitement surrounding Bitcoin stemmed from its use as a token of value, for example as an alternative to government-issued currencies. As people learned more about the underlying blockchain technology, they extended the scope of the technology itself (e.g. smart contracts), as well as applications (e.g. intellectual property). With this increase in scope, single monolithic 'blockchain' technologies are being re-framed and refactored into building blocks at four levels of the stack: 1. Applications; 2. Decentralized computing platforms ('blockchain platforms'); 3. Decentralized processing ('smart contracts') and decentralized storage (file systems, databases), and decentralized communication 4. Cryptographic primitives, consensus protocols, and other algorithms. (Page 2)
Centralisation can provide technology efficiency and economies of scale and agility. However these come at the expense of fragility. Even the distributed, synchronised cloud platforms like AWS, Azure, Digital Ocean, come with risks of single-operator systems.
Natural communities are resilient to extinction because they can accommodate the loss of small numbers without losing the functioning of the community as a whole. Communities function as a 'mesh' of interweaving relationships and capabilities with a common vision or purpose. Wisdom Networks are working on the implementation of mesh networks where multiple paths, for process and data, are available and survive the loss of some nodes. The internet works in much the same way but is significantly more fragile - the whole of the current (2016) internet is based around 9 main networks. Wisdom Network participants will create dynamic mesh networks, sometimes bypassing infrastructure providers entirely, to create a robust, always on, network of the future.
The distributed network is made useful with the distributed stack that puts the network to work doing useful things for communities.
Blockchains and Databases
The need for scale
Wisdom Networks are designed from the ground up to be massively scalable to meet the needs of large population groups and globally distributed communities. Such networks will have millions of people, and each person could be participating in multiple networks. The WN database, of whatever form, will process billions of transactions per minute- if Twitter is a 'flood' then Wisdom Network is a data Tsunami ….. 24x7.
BigchainDB: Blockchain meets Big Data
We built BigchainDB on top of an enterprise-grade distributed DB, from which BigchainDB inherits high throughput, high capacity, low latency, a full-featured efficient NoSQLquery language, and permissioning. Nodes can be added to increase throughput and capacity.
BigchainDB has the traditional blockchain benefits of decentralized control, immutability, and creation & transfer of assets. The decentralized control is via a federation of nodes with voting permissions, that is, a super-peer P2P network . The voting operates at a layer above the DB’s built in consensus. Immutability / tamper-resistance is via an ordered sequence of blocks where each block holds an ordered sequence of transactions; and a block’s hash is over its transactions and related data, and the previous block’s hash; that is, a block chain. Any entity with asset-issuance permissions can issue an asset; any entity with asset-transfer permissions and the asset’s private key may transfer the asset. This means hackers or compromised system admins cannot arbitrarily change data, and there is no single-point-of-failure risk. Scalable capacity means that legally binding contracts and certificates may be stored directly on the blockchain DB. The permissioning system enables configurations ranging from private enterprise blockchain DBs to open, public blockchain DBs.
Layering BlockChain principles on an 'eventually consistent' database or data layer means that the type of datastore used for bigdata can be something attached to a WN node. The data flowing through the network can be forked out to a massive scale repository for BigData tools to work on the whole pool of data. The analysis and insights can then be fed back into the Wisdom Network as another form 'collective wisdom' that further enhances the behaviour of the underlying WN itself as well as influencing the behaviour of the people and organisations participating in the network.
Forms of AI could be applied to the network both on the streams of ideas, decisions, data as well as using BigData analysis. In addition human moderators could monitor their own WN sectors and provide guidance in ways that machines and AI could not.
BigchainDB in the decentralization ecosystem
Figure 1 illustrates how BigchainDB can be used in a fully decentralized setting, or as a mild extension from a traditional centralized computing context. BigchainDB is complementary to decentralized processing / smart contracts (e.g. Ethereum VM  or Enigma ), decentralized file systems (e.g. IPFS ), and communication building blocks (e.g. email). It can be included in higher-level decentralized computing platforms (e.g. Eris/Tendermint ). It can be used side-by-side with identity protocols, financial asset protocols (e.g. Bitcoin ), intellectual property asset protocols (e.g. SPOOL ), and glue protocols (e.g. pegged sidechains , Interledger ). Scalability improvements to smart contracts blockchains will help fully decentralized applications to better exploit the scalability properties of BigchainDB. BigchainDB works in compliance with more centralized computing blocks too. One use case is where decentralizing just storage brings the majority of benefit. Another use case is where scalability needs are greater than the capabilities of existing decentralized processing technologies; in this case BigchainDB provides a bridge to an eventual fullydecentralized system.
Bitcoin Scalability Issues
Throughput. The Bitcoin network processes just 1 transaction per second (tps) on average, with a theoretical maximum of 7 tps. It could handle higher throughput if each block was bigger, though right now making blocks bigger would lead to size issues (see Capacity and network bandwidth, below). This throughput is unacceptably low when compared to the number of transactions processed by Visa (2,000 tps typical, 10,000 tps peak), Twitter (5,000 tps typical, 15,000 tps peak), advertising networks (500,000 tps typical), trading networks, or email networks (global email volume is 183 billion emails/day or 2,100,000 tps). An ideal global blockchain, or set of blockchains, would support all of these multiple high-throughput uses.
Latency. Each block on the Bitcoin blockchain takes 10 minutes to process. For sufficient security, it is better to wait for about an hour, giving more nodes time to confirm the transaction. By comparison, a transaction on the Visa network is approved in seconds at most. Many financial applications need latency of 30 to 100 ms.
Capacity and network bandwidth. The Bitcoin blockchain is about 50 GB; it grew by 24 GB in 2015. It already takes nearly a day to download the entire blockchain. If throughput increased by 2,000x to Visa levels, the additional transactions would result in database growth of 3.9 GB/day or 1.42 PB/year. At 150, 000 tps, the blockchain would grow by 214 PB/year (yes, petabytes). If throughput were 1M tps, it would completely overwhelm the bandwidth of any node’s connection, which is counterproductive to the democratic goals of Bitcoin. (Page 6)
BigChainDB - High level description
Decentralized control is achieved via a DNS-like federation of nodes with voting permissions. Other nodes can connect to read, and propose transactions; this makes it a super-peer P2P network. The voting operates at a layer above the DB’s built in consensus. Quorum is a majority of votes. (Page 13)
Choice of distributed database
In the future, we envision a variety of distributed databases being 'blockchainified' according to the approach of this paper. Every relational database, document store and graph store on the planet might someday have a blockchain version.(Page 31-32)
The key to usability will be in having mechanisms that allow for enablement of user initiated events in the WN that are not then tied up waiting for data consistency in the entire network. The principle required is a form of 'fire and forget' where as long as the nearest node gets the transaction ok and validates it, then the WN user can continue to work.
Public vs. Private Bitcoin
A Public BigchainDB - Example permissioning
At the core of a public BigchainDB are the 'Caretakers': organizations with an identity that has a 'Voting Node' role. An identity with that role can vote to approve or reject transactions, and can vote whether to assign the 'Voting Node' role to another identity. (Note: an organization can have more than one identity, so a Caretaker could have two or more identities with a 'Voting Node' role.) To start, the public BigchainDB will have five identities with the Voting Node role: three held by ascribe and two held by other organizations chosen by ascribe. That is, the public BigchainDB will start with three Caretakers: ascribe and two others. From there, additional Caretakers will be selected and added to the federation by existing Caretakers. Caretakers will have divergent interests to avoid collusion, but must have one thing in common: they must have the interests of the Internet at heart. In choosing Caretakers, there will be a preference to organizations that are non-profit or building foundational technology for a decentralized Internet; and for diversity in terms of region, language, and specific mandate.
The right organizational structure will be critical to the success of a public BigchainDB. Governance issues have plagued the Bitcoin blockchain. We can take these as lessons in the design of a public BigchainDB. We are consulting with lawyers, developers, academics, activists, and potential Caretakers to develop a strong, stable system that is transparent enough to be relied on and flexible enough to meet the needs of the network. Ultimately, the public BigchainDB will operate entirely independently under its own legal entity. It will choose its own Caretakers and set its own rules—but it will always work toward the long-term goal of a free, open, and decentralized Internet.(Page 46-47)
Blockchain and Wisdom Networks are different, but both organise 'things'
Blockchain and Wisdom Networks are very different, but do the same thing. They manage things in society. The blockchain is a means to manage distributed, non-repudiable transactions and crowd creates the resulting register of ‘things’. Blockchain manages flows and stocks. The wisdom network focuses organises ‘things’ (data, information, community, collaboration, knowledge, wisdom) to reveal wisdom and focus the wisdom of crowds on outcomes.
Block Chain and Wisdom Networks are complementary. Wisdom Networks add value to things in blockchain by organising wisdom around every 'thing'. The Blockchain provides a source of ‘things’ to link to and organise.
Zero Marginal Cost Networks (internet, blockchain, wisdom networks etc) displace millions of ‘Industrial Economy’ fragmented technologies and hierarchies that manage ‘things’ by organisation, sector or country across the world at zero cost and with global reach. For example, one 'Equity Market' block chain could crowd create transactions and registers for securities transactions worldwide. One Equity Market Wisdom network could manage the global equity market. One Blockchain could manage land titles globally. Blockchain is another type of global, horizontal Zero Marginal Cost network that manages ‘things’ globally.
Each sector may consist of 50 things. Each country may consist of a few hundred things. Some of these things will be managed by Blockchain technology at either a global country level. The use of Blockchain depends on each ‘thing’. Land and securities need Blockchain. Holes in the road ‘things’ that need to be fixed do not.
Blockchain and Wisdom Networks are part of the inevitable evolutionary ‘More with less’ evolution that means less adminstrative and people are required within the services sector. Like previous Industrial Revolutions, the Third Industrial Revolution will result in a jump in productivity and significant decline in employment to achieve the same outcomes.
What does wisdom actually look like? Screenshots!
Wisdom Networks organise society's 'things' within networks in the Internet of Everything (IOE) via people and their mobile devices. Please find some screenshot examples below of 'things' and 'books' in sectors. Please note the tabs at the top of the screen allow a person to switch between various sectors. Each sectors contains different 'things', but the underlying method of linking 'things' into 'books' and sharing them remains the same. One method of organising past, present and future means a dramatic reduction in complexity across society and a scalable solution to organise society and the global village at zero marginal cost.
Sector pages and presentations contain further screenshots and workflow diagrams for each sector.
1 consolidated method to exchange 'wisdom'
Each sector is a subset of the Network Society. Within a Network Society, sectors and countries use a common method ( things = books = sectors = countries = global village) to exchange wisdom (data, information, community, collaboration, knowledge wisdom). There are exponential benefits from using a common method of exchanging wisdom across individuals, families, sectors, countries, regions and the global village, rather than fragmenting wisdom and sectors using an Industrial Economy model. Network Society development model enables citizens to actively participate in sector governance and service delivery within the community, rather than citizen non-participation because sector governance and service delivery is isolated in hierarchies outside the community.
NOTE: Society already uses the 'things' into 'books' method to organise their social life. Billions of people have been trained in this method. It just needs to be enabled in other sectors.
- Health Wisdom doctors, nurses and patient crowd creating health (via HealthBook)
- Education Wisdom: teachers, parents and students crowd creating education ( via EducationBook)
- Equity Market Wisdom: companies, advisers and investors crowd creating IPO's, ventures, trade sales and innovation
- Democracy Wisdom: politicians and citizens crowd creating real-time citizen participation and democracy (via PoliticiansBook and Network Democracy
- Government Wisdom: politicians, bureacrats, citizens and industries crowd creating governance and services outcomes
- Governance Wisdom: governance is about directing effort to prioritised, merit based outcomes. Wisdom Networks reveal perfect knowledge and optimise action action.
- Management Wisdom: directors (direct), managers (manage and review) and executives (execute) crowd create management and leadership within an organisation with a 3 role flat management structure
- Regional integration: people across regions can exchange wisdom about any "thing" in any sector
- Global: any person anywhere around the world can participate, contribute or seek humanitarian assistance from any sector in any Network Society
Hardware complexity to Software simplicity
The complexity of physics can be simplified to e=mc2. The complexity of society can be simplified to Things = Books = Sectors = Countries = Global Village. This formula represents the design of software that organises Network Society. An Industrial Economy divides people using hierarchies within countries and globally using physical (ie hardware) tools to organise society. Network Society unites people with few networks in sectors, countries and Network Democracy around the world using virtual (ie software) tools to organise society. The primary means for organising society becomes networks of people organised by software at zero marginal cost with immediate outcomes, rather than isolated, physical hierarchies using expensive infrastructure with uncertain outcomes over long timeframes. The 'Global Village' becomes a mesh of countries and sectors. Each country is unified by software as '1 Country'. All countries are unified by software as '1 Global Village'. Network Society simply comes 'Over the Top' (OTT) of society and virtually organises every 'thing' in the cloud.