Computer to computer communication is an integral part of modern life. Each time you browse to a website, update your operating system or an application, or send an e-mail to a friend, you are depending on computers to communicate reliably.
Having a good understanding of how computers communicate is really fundamental to many topics within Bitcoin. This is so, for example, if you have concerns about the privacy of your transactions and storage arrangements, or if you are keen on setting up your own Bitcoin server.
This primer covers the basics of computer communication. It first provides some background information on the main set of standards for modern computer communication: the Internet protocol suite. Next, it offers insight into how the protocols work to move messages between computers with an example as well as with discussion of the Internet protocol model. Finally, we will discuss the topology of computer networks, particularly that of peer to peer networks.
The Internet is a global network with billions of desktop computers, laptops, phones, servers, printers, and other types of devices at its endpoints. The connectivity between all these endpoints depends on three main factors:
- A vast physical communication infrastructure
- Commercial agreements between the many owners and users of this infrastructure
- A set of communication protocols known as the Internet protocol suite (also known as the TCP/IP protocol suite)
Physically, the Internet is essentially just a vast network of copper, coaxial, and primarily fiber-optic cables with supporting equipment such as routers, repeaters, and switches. In addition, some very small portion of the Internet relies on connections via wireless equipment.
The Internet is separated into chunks called autonomous systems. These are formally defined portions of the Internet’s infrastructure with one controlling administrative entity. Autonomous system numbers which uniquely identify each autonomous system are distributed by the Internet Assigned Numbers Authority (IANA), a standards organization that also oversees various other tasks related to management of the Internet.
The main autonomous systems within the Internet are all administered by telecommunications companies. Some of these systems span several continents or even the entire globe. Smaller autonomous systems are also administered by other types of companies as well as organizations such as universities and governments.
Sometimes a particular administrator might only control one autonomous system. In other cases, it may control more than one.
The administrator of an autonomous system has the right to determine a policy on which data is transported across the autonomous system and how (that is, via what routes and with what priority).1
But while the administrative entity has the rights to use the infrastructure of the autonomous system, it does not necessarily own all that infrastructure outright. Frequently, the infrastructure will be leased from other organizations. It is also possible for a piece of Internet infrastructure to be leased to multiple autonomous system administrators for shared usage.
What makes the Internet work is that all these autonomous systems are connected to each other, both by physical infrastructure and commercial agreements. Physically, two autonomous systems can be connected via one or more private peering connections. They can also be connected via Internet exchange points, which are hubs where the infrastructure of many autonomous systems interconnect.
Commercial agreements between autonomous systems is what ensures that data can be transported across them. These agreements can either be peering agreements—where two autonomous systems agree to transport each other’s data either for a price or for free—or transit agreements—where one autonomous system agrees to carry the data of another for a price.
Both the commercial agreements and the physical infrastructure that constitute the Internet are fairly decentralized in their organization: the Internet is really a product of individual organizations making choices on commercial partnerships and infrastructure investments, and not subject to much centralized steering.
Sometimes there are hiccups in these arrangements. In 2008, for example, Sprint stopped peering with all of Cogent’s autonomous systems for three days, which temporarily cut off 3.3% of global Internet addresses from the rest of the Internet.2 Overall, however, the decentralized physical and commercial organization that underpins the Internet works remarkably well in keeping much of the world connected.
We will not delve into the commercial agreements that make the Internet any further. We will briefly return to the physical infrastructure that makes the Internet in the last section.3 Until, then, I will focus on the last key element of the Internet: the Internet protocol suite. This is the main topic that will help you begin to grasp computer communication.
So what exactly is the Internet protocol suite?
Suppose that you decide to communicate with a neighbor in the evening by emitting pulses of light from flashlights. A bit unusual admittedly, but perhaps you are both kids in the 1950s before the modern era of cell phones and computers.
In order for this communication approach to be successful, you will need to apply a set of rules to interpret the meaning of particular pulses and patterns of pulses from each other’s flashlights.
If all you needed were “yes”, “no”, and “I don’t know” communication signals, you might, for instance, decide on the following elementary protocol:
- Holding the flashlight on for three seconds means “I don’t know”
- Holding the flashlight on for two seconds means “yes”
- Holding the flashlight on for one second means “no”
This elementary protocol will not make for very sophisticated discussions between you and your neighbor. If you needed more complex communication, you might use a more sophisticated protocol such as morse code.
Similarly, the billions of endpoints around the world that are part of the Internet continuously send and receive electrical, light, and radio signals. These signals are carried across the globe by a vast intermediating infrastructure. All these endpoints and their intermediary infrastructure can only cooperate successfully because they share a particular set of rules regarding the meaning of these signals, namely the rules from the Internet protocol suite.
Though this set of protocols embodies much more complexity than, say, a system with morse code for flash lights, the essence of their operation is really just the same: signals are transmitted and received, and these have consequences for computers due to shared communication protocols.
The Internet protocol suite does not contain all the protocols that might be used in Internet communication. For instance, it does not contain the Bitcoin protocol, which is set of standards that allows communication between Bitcoin nodes on blocks, transactions, and so on.
Instead, the Internet protocol suite contains all the basic protocols necessary for Internet communication. Higher-level protocols, such as the Bitcoin protocol, rely on these basic protocols for handling the transport of data across the Internet between computers and specific applications on those computers.
While it might be called the Internet protocol suite, the suite really functions as the core set of standards for computer communication in general.
For instance, suppose you just connect two computers via a cable, and neither of them are connected to the Internet. In a technical sense, you could really implement any basic communication protocols you wanted, as long as both computers were configured to be able to interpret them.
Yet, the Internet protocol suite will generally also be utilized for handling all, or at least the bulk of the communication work in this type of situation. It is not often that anyone will have a need for rolling out their own set of basic communication protocols.
The Internet protocol suite is nowadays developed and managed jointly by specialists from universities, research institutes, companies, and governments around the world, under the coordination of the Internet Engineering Task Force. However, the basic architecture of these protocols and how they operate together to enable computer communication stems from the research efforts in building ARPANET in the 1960s and 1970s, the very first operating network of computers.
ARPANET was created by the Advanced Research Projects Agency, an agency of the US Department of Defense.4 In the 1960s, computers were few and far between, and each of these computers tended to specialize in particular applications and data. ARPA funded the ARPANET project primarily so that people conducting defense research or otherwise working in the defense industry could more easily and cost-efficiently employ the capabilities of multiple computers.5, 6
You will commonly hear that ARPANET was built in order to have a resilient communication network that could resist foreign attacks, particularly from the Soviet Union. Research into computer networks in the early 1960s by Paul Baran from the Rand Corporation indeed had this particular focus.7 However, resiliency against foreign invaders seems at best to only have been a very a distant concern in the creation of ARPANET.8
The first message on ARPANET, “lo,” was sent in 1969 between computers at Stanford and UCLA. The intention was to type “login”, but the connection was interrupted. This was not the first time that two computers communicated.9 So in that regard, ARPANET was not particularly revolutionary.
Instead, what made ARPANET revolutionary was that it constituted the first practical implementation of a network with more than two computers with continually open communication lines. ARPANET eventually combined with many other independent networks to form what we nowadays call the Internet.
The key innovation in network communication from ARPANET that has been adopted by the Internet protocol suite was packet switching. The concept had first been independently suggested by Paul Baran from the RAND Corporation and Donald Davies from the British National Physical Laboratory earlier in the 1960s.10 But ARPANET was the first practical implementation of packet switching.
So what is packet switching exactly?
Rather than sending any particular communication, such as a request for the contents of a website, in one single chunk, computers that operate according to the Internet protocol suite divide communications into packets. These packets are, then, reassembled by the computer at the other end.11 (Exceptions to this segmentation only occur when the amount of data being transferred is very small, such as for “address resolution protocol” requests).
An alternative approach to computer communication is to always send all data for a particular communication in a single chunk. A web browser request for the content of a website, for example, could just be sent as one single block of data equipped with the necessary metadata for the destination server. This means that for the duration of transmission, a communication channel would have to be dedicated to these two computers. (Modern cables can be split into multiple communication channels, so other data would still be able to pass through other channels.)
Various historical computer networks indeed adopted this type of approach. The traditional telephone system also essentially works in this manner. A connection channel is established between two telephones that cannot be used by anyone else, until the telephone conversation has ended.
Packet switching, however, has several substantial advantages in data transmission. Some of the main advantages are as follows:
- Channel efficiency. Sending all information as a single chunk would require reserving an entire communication channel until the transmission has completed, a practice known as circuit switching. This wastes substantial space, however, as your communication would not consistently utilize the communication channel. With packet switching, the channel can be utilized much more effectively.
- Error management. If there is any error in transmission with a single chunk of data, you need to resend all the data to the destination computer. In a packet switching network, you can resend only those packets that did not arrive successfully (as for a website), or even choose not to resend these data packets at all if it’s not strictly necessary (as usually happens with video streaming).
- Obstruction management. While transmitting data in packets, you can direct them via different routes if the route you were using becomes very busy.
- Router efficiency. Sending data as single packets would require the routers in the intermediary Internet infrastructure to first collect all the relevant data in a communication, before they can send it on. That means, they would have to be able to store a substantial amount of data in secondary memory. With packet switching, however, routers can store most data in primary memory. This means that they can work much more efficiently and are more economical.
Why is the suite of protocols for Internet communication called the Internet protocol suite? It is actually named after the key protocol within the suite, namely the Internet protocol. The alternative name, the TCP/IP protocol suite, stems from the importance of both the Internet protocol (IP) and the transmission control protocol (TCP) within the suite.
Note that even a minimal implementation of the Internet protocol suite on a communication endpoint will contain many more protocols than just the Internet and transmission control protocols.
To form some understanding of how the Internet protocol suite works, it is best to work through a high-level example.
Suppose that you open your web browser and type “https://www.Bitcoin.org” into the address bar. You press “enter.” You browser translates this as follows: “Please request the contents of the website www.Bitcoin.org, and load them into my web browser upon receipt. The data for the website should be formatted according to the https protocol”.
How does a message that you want the contents for this website actually end up at the right web server? And how do we ensure that it is processed by the right application on the web server?
Those questions are addressed on a high level in the next two sections. I first turn to how the communication message is processed within your own computer. I, then, turn to how the message travels through your home network and the rest of the Internet to arrive at the destination server to be processed by the right application.
To start, in order to send any communications to the server of Bitcoin.org, just having its domain name (i.e., www.Bitcoin.org) is not sufficient. That is just a nickname easy to remember for humans. Computers do not actually use these names in sending around communications.
Instead, your computer will need to know an associated IP address in order to route your data to Bitcoin.org’s web server. General purpose computers such as your desktop or laptop all have some minimal routing capabilities. Most of the work in Internet communication, however, is performed by dedicated intermediary devices known as routers (the name stemming from their primary function).
Sometimes your computer may already know an IP address for Bitcoin.org, perhaps because you recently visited that page. Normally, however, you will have to obtain the information from a domain name server. This type of server stores one or more corresponding IP addresses for every domain name in its memory, and allows users of the Internet to query them.
You router will have a domain name server application installed on it. If this application does not know the IP address of a domain name such as www.Bitcoin.org, it will typically connect to a domain name server owned by your Internet service provider (ISP). The information is, then, returned to your domain name server application which forwards the information to your computer. (In case your ISP’s domain name server also does not have the information, it will fetch it for you from a domain name server elsewhere.)
The domain name system—a hierarchically organized collection of servers that translate domain names to IP addresses—is in place because storing all information on domain names and associated IP addresses on your own computer or a local server is practically impossible. The number of domains is vast, new ones are continuously being added, old ones are continuously deleted, domains may have multiple IP addresses (such as Google.com), and IP addresses can also change. You need a dedicated system to manage all that information effectively.
Alternatively, you could have also typed an IP address for Bitcoin.org directly into your web browser to avoid using the domain name system. So technically the DNS system is not needed. Conducting all web communication in this manner, however, would be fairly impractical.
IP addresses are difficult for most humans to remember. The Bitcoin.org IP address from my location, for example, is as follows: 138.68.248.245. The impracticality of plainly using IP addresses is the fact that they can also change for websites. (Domain names can also change, but do so much less frequently than IP addresses.) In order for humans to be able to use the Web, the domain name system, or something like it, seems fairly indispensable.
Once your computer has the right IP address, probably by utilizing the domain name system, the data you entered into your browser, “https://www.Bitcoin.org,” is processed in four main steps by your computer.
To start, your entry specifies that you want to use the https protocol (which stands for hypertext transfer protocol with transport layer security) to communicate with the web server. The https protocol is just the http protocol—the standard for communicating web content such as HTML and CSS—secured with cryptography.
The https specification in your browser entry will result in a handshake between your computer and the server that results in a shared cryptographic keys for secure communication.
After the handshake, your web browser will create a request that can be understood by a web server application such as Apache or NGINX. The request specifies that you would like to receive the HTML data belonging to www.Bitcoin.org (other types of data belonging to the web page is received in separate requests).
Protocols such as https and http are known as application layer protocols. They allow two applications on different computers to communicate. There are many different application layer protocols.
Receiving and sending e-mails via your outlook inbox, for instance, requires communication with a distant e-mail server application via the pop, imap, and smtp protocols. (E-mail services accessed through your web browser, instead, also use http or https.) For another example, logging into a remote computer can be done through the telnet or ssh protocols (as telnet is unencrypted, however, using that protocol is not typically a very good idea).
After your web browser has created the request for Bitcoin.org’s HTML data, your computer will split the request into multiple packets. Each of these packets contains some part of the data that together describe the request. That is usually called their payload.
At this point, your computer will start adding metadata to the each packet’s payload, in order to successfully transport it to the web server. The first pieces of metadata that are added will primarily help the Bitcoin.org web server understand
- the order in which the packets for the request should be reassembled;
- how to handle data packets that were not received;
- which source and end applications are relevant to the communication (your web browser and its web server application respectively); and
- which application layer protocol is used in the communication.
The protocol that handles this portion of your request for the contents of Bitcoin.org’s website is known as the transport control protocol (TCP). The main metadata added by the protocol are sequence numbers, and source and destination ports.
Sequence numbers help the receiving computer understand if any packets are still missing from the transmission and in which order to reassemble them (so it is crucial to handling tasks (1) and (2) above). If needed, the receiving computer can request certain packets of your computer again on the basis of these sequence numbers.
Information about which application is needed on the web server, the protocol being used, and to what application on the original computer any communications need to be returned (so tasks (3) and (4) above), is primarily handled via specifying source and destination logical ports.
Logical ports are not to be confused with physical ports, which are interfaces on our computer that allow us to connect devices, such as USB ports and RJ45 ports. They both receive the name “ports” because they are locations at which data comes into our computers. But they have a distinct role in computer communications.
To understand logical ports better, let’s return to our example. Each data packet associated with your request for Bitcoin.org’s website would have the same destination logical port, typically 443 for an https request.
At the destination web server, the web server application will be connected to that port, at least for processing https requests (it might be connected to other logical ports for processing requests with different types of protocols). So any data received with this port specification by your computer will be forwarded to the web server application. The latter will, then, re-assemble the packets and process the payload with the https protocol, because of the specific port number.
Bitcoin.org’s website only works with https, as is the case for most modern websites. But had it also been configured to accept http requests, its web server would probably also have been listening on port 80, not just port 443.
Ports such as 80 (http) and 443 (https) are called well-known ports or system ports. Well-known ports are each associated with a particular protocol.
In addition to the destination port, TCP also requires that each packet associated with your request is given the same source port. This source port is allocated automatically by your computer from a group of available port numbers. Any data eventually returned by the web server regarding your request will enter your computer on that specified source port. The type of port that functions as the source port here is known as an ephemeral port.
In this example, the source port will be associated with a particular tab of your browser application. This insures that any data does not travel to the wrong tab, the wrong instance of your browser application, or to some other application altogether.
Ephemeral ports are typically closed after a single request by a browser. So after you have received the HTML content, your computer will probably assign a new ephemeral port for the next step in retrieving all the content on the Bitcoin.org website (such as CSS content and images).
Why is your source port not the same well-known port, 443, as the destination port? Well-known ports are really reserved for particular server applications that need to continuously accept requests according to certain protocols. These ports cannot just change all the time, otherwise it would create trouble for clients attempting to find the web server applications. This is not a concern with your browser sessions, or other types of client applications.
TCP is known as a transport layer protocol. The other major transport layer protocol you need to know is the user datagram protocol (UDP).
Unlike TCP, UDP only specifies source and destination logical ports. It has no rules for order of assembly or how to handle data packets that are not received in a communication. Though most browser activity employs TCP, such as when you go to www.Bitcoin.org, UDP is, for example, utilized for live video streaming.
After adding the necessary metadata from the TCP protocol to the payload of each packet in the request that is going to Bitcoin.org—most notably sequence numbers, and source and destination logical ports—your computer will start adding metadata from the Internet protocol (a type of network layer protocol). The main pieces of metadata here are the source and destination IP addresses.
What exactly are IP addresses? Within your computer, you will probably have one or more network adapters (also called by various other names such as network interface adapters or network controllers). These allow you to be connected to other computers, either via an ethernet cable or wirelessly.
When actively connected (either to a wireless access point or via a cable), a network adapter in your computer will have a unique IP address. Hence, it is possible for one computer to be associated with multiple IP addresses that can be used for Internet communication.
Starting with the source IP address on each data packet in our example, it, thus, specifies a device on a your computer to which all the website data must be returned by the server.12 The destination IP address is that of some network adapter on a web server.
We will discuss exactly what happens to the packets once they leave your computer in the next section. But it needs to be noted here that IP addressing becomes a bit complicated, because the source address put on the packets by your computer is not typically the source address the web server will see. So what exactly is happening during transportation?
Originally, the intention behind the Internet protocol was that each computer would have its own IP address. These IP addresses would then provide an indication of where computers are located within the Internet.
Over time, however, this became infeasible with the main version of the Internet protocol everyone was using (version 4). This particular version only allows around 4.7 billion addresses, which is far too little in our connected world. In addition, each computer nowadays typically has at least two network adapters.
Several responses in the history of development of the Internet have ameliorated the problems with the limited number of IPv4 addresses. Perhaps most significant was to create a distinction between public and local (or private) IP addresses.
Typically, modern home networks will have only one public IP address. This public IP address is specifically for the Internet-facing end of your router. It provides an indication of the autonomous system in which your home network is located. By extension, it typically also gives an approximate geographical location of your home network.
All your local network adapters will, then, have local IP addresses. Each of these adapters will be able to communicate with the local network-facing end of your router also operating under a local IP address.
Local IP addresses can be selected from a standardized range. This range cannot be used for any public IP addresses.
Local IP addresses provide no indication with regards to where your home network is located; many other home networks will use the same local IP addresses as on your network. This is, however, inconsequential because local IP addresses are not used for communicating within the Internet. Any public IP address, by contrast, will be unique.
As for autonomous system numbers, allocation of public IP addresses is coordinated by IANA with the support of various regional organizations.
So what more or less happens to your packets regarding your request for the HTML content of Bitcoin.org is as follows. Once a packet with your local IP address arrives at your router, it will strip off that local IP address as the source and replace it with the router’s own public IP address. Only then is the packet sent on to the web server.
Your router will have in place a system so that it can forward packets back to your particular network adapter when it receives them from the web server. I will explain this process in more detail in the section “Travelling through the cloud”.
Your original request for Bitcoin.org’s server has now been split into multiple packets and supplemented with metadata from the transport control protocol and the Internet protocol. The last step before the data with the request for Bitcoin.org leaves your computer is to ensure that your data can travel between it and the next hop.
If your local network connection is via a cable, travel to the next hop will be specified according to the ethernet standards. If your connection is wireless, travel to the next hop will be specified according to the IEEE 802.11 standards. The protocols in these standards are known as data link layer protocols.
The key metadata within both the ethernet and IEEE 802.11 standards are the source and destination MAC addresses (MAC stands for media access control address). A MAC address is a globally unique number associated with network adapters.13 It may be encoded by software or physically printed. A MAC address is, therefore, also called a physical address.
Each of your packets will be provided with a source MAC address, that of your network adapter, and that of the MAC address of the next link to the final destination. In a typical home network, this will be a MAC address of your router.
We will explain the purpose of MAC addresses in communication more in the next section.
This concludes our discussion of the data actually sent out from your computer. We did not discuss all the metadata and their functions in assembling packets. But above at least provides a rough idea of what happens within your computer.
Any packet of your request to Bitcoin.org that has been provided with the proper metadata—including sequence numbers, source and destination logical ports, source and destination IP addresses, and source and destination MAC addresses—is ready to leave your computer.
What exactly happens from here on out in your home network depends on how it is set up. But it would be fairly typical to proceed as follows.
To start, each packet will travel from your computer to your home network’s router. Usually, this is an all-in-one device that can perform the function of a switch, wireless access point, router, modem, and firewall. We call these devices “routers,” but in fact they are usually devices that do much more than just route network traffic. In more sophisticated networks, these different functions will tend to be performed by dedicated devices.
If your computer is connected via a cable to your “router”, each of your data packets will first actually travel to the switch. A switch has the ability to memorize any connected computers and devices by their MAC address, and it can, thus, efficiently establish stable communications between them.14 From the switch it will be passed on to the actual router component of your home “router”. If you are on a wireless connection, a wireless access point performs a similar function.
Note that if you were sending data to another computer on your local network, the data would pass through the switch or wireless access point towards the other computer.
The MAC address specified on any of the Bitcoin.org packets that leaves your computer will probably belong to a network interface directly connected to the router component of your home router. It is, however, possible for communication to become more complicated.
Dedicated switches, for example, can sometimes have their own MAC address, in which case that will be the first destination MAC address of your packets, not that of your router.
As soon as your router sees a packet with Bitcoin.org as a destination (via its IP address), it will create an entry in its network address translation table. On the one side, the entry will contain the local IP address of your network adapter and the ephemeral port your browser tab will listen on for the data. On the other side, the entry will contain a logical port on the public-facing side of the router. Each packet that, then, passes through the router from your computer will be changed as follows:
- The router will strip each packet of its source and destination MAC address. This is necessary to get the data packet to the next hop.
- In addition, the router will strip the source IP address and logical port on each packet. It will, then, list its own public IP address as well as the logical port specified in the network address translation table.
- Finally, the router will add its own MAC address and that of the next hop to the packet. This destination MAC address is determined via its routing table on the basis of the web server’s public IP address.
As a user, you can really reserve any of the more than 65,000 ports on your local system for an application to receive data via port forwarding. It is, however, recommended to use the well-known port number if there is one associated with the protocol (between 0 and 1023), or otherwise an available user port (between 1024 and 49151). Typically such a user port will be recommended with the application. Higher ports are typically advised to be reserved for ephemeral ports or customized services.
Finally, your router sends the data into the cloud, typically using a phone line, coaxial cable, or fiber-optic cable, first passing through a modem to transform the signals so that they can successfully travel through these media.
These are the main steps taken to send data into the cloud by a typical home network. A few additional steps may be involved, of course, such as data passing through a firewall. But these are the main steps you should understand.
So what happens in the cloud? To put it simply, the “cloud” is essentially just a vast network of switches; routers; repeaters and amplifiers; and copper, coaxial, and fiber-optic cables (and, to a much smaller extent, equipment for wireless connections). Until your packet arrives at the destination IP address, your data packet roughly travels as follows through the Internet:
- It arrives at a router
- The router strips the source and destination MAC address (changing network and transport layer metadata is no longer required, because we are only dealing with public IP addresses)
- The router consults its routing table to see where the packet should go next on the basis of the destination public IP address
- The router adds its own MAC address as the new source address, and the destination router’s MAC address as the new destination address
- As the router is connected to multiple routers, the packet will first travel through a switch that forwards it to the router with the corresponding MAC address
- It, then, ends up at the next hop, and the process starts all over again
Eventually, each of your packets will arrive at the public IP address specified for the web server. They may all take the same route, but packet switching of course also allows them to use different routes.
So far, I have suggested that the destination public IP address actually belongs to the web server with the HTML content for Bitcoin.org. If true, the incoming packets from your request will be re-assembled by the web server application listening to port 443. It will, then, fetch the HTML data for the Bitcoin.org domain and send it back to our computer in a similar way (or, if it does not have the data, a response to this effect).
I’m not actually sure, however, that the destination public IP address for Bitcoin.org actually contains the HTML content directly. Certainly with websites that have high traffic, such as Google.com, the destination IP address will belong to a server that frequently further distributes traffic to other servers.
While that would add some complexity to the storyline, you should have a rough notion of what would happen in this case. The kinds of transformations with regards to ports and IP addresses that occur in your home network would, namely, occur here as well.
It’s mind boggling how quickly data travels across the Internet through the multitude of steps we just described. For you, any request to Bitcoin.org’s server will probably travel through a connection on our global network of fiber-optic submarine communication cables.15 Yet, loading the main page usually takes only a few seconds at most (and for many of us, less than a second).
One common question people have when they learn about these protocols: Why do the network adapters in my computer and the Bitcoin.org web server as well as my own and intermediary routers, all need two types of addresses in communication? That is, why do they need both MAC addresses and IP addresses?
The short answer is that a MAC address is like a name for a networking device, while an IP address is an indication of its location. And combining both of these addresses makes it much more effective to organize communication between computers.
Imagine, for example, that computers only had MAC addresses. Could you actually use those addresses to route traffic through the Internet?
A MAC address only identifies a network device. But the location of that device could move. For instance, when you take your laptop or phone to the local café, the location of your Wi-Fi adapter changes even though its MAC address stays the same.
In addition, while public IP addresses have hierarchy (mainly in that certain prefixes belong to particular autonomous systems), there is no hierarchy with MAC addresses. So while routers on the Internet can implement forwarding rules on the basis of IP ranges, MAC addresses would have to be allocated forwarding rules on an individual basis. And those forwarding rules would constantly have to change on the basis of where the network adapters are located. All this would make routing Internet traffic incredibly inefficient, if not infeasible.
On the flip side, only having IP addresses would also present complications in a number of ways. For one example, home routers usually have a server application installed called the DHCP server (or dynamic host protocol server). Most devices on a home network will request a local IP address from this server when they come online (rather than have one permanently assigned). Sometimes, however, you might want to reserve a particular IP address for a particular adapter.
But how could the DHCP server reserve an IP address for a particular adapter, if it has no name by which to identify that particular adapter? You probably could find a workaround to this problem, but it is much easier just to have unique names for our adapters. (Note that these names are rather “unique enough,” as it is hard to guarantee that they are never duplicated.)
All in all, it is difficult to imagine how you could organize computer communication without both MAC and IP addresses.
Expositions of the Internet protocol suite will often focus discussion on the Internet protocol model, or TCP/IP model, in order to provide a deeper understanding of its protocols.16
What exactly is the Internet protocol model? It is a model that subdivides the protocols needed for network communication into four subtypes.17 The protocols in each of these subtypes, broadly relate to a similar type of function that is performed by our computers and devices for network communication.
As these subtypes have a certain hierarchical relationship, the model organizes them into the four layers that we have already identified in the discussion: the application layer, the transport layer, the network layer, and the data link layer.18 The application layer is generally said to be the highest level layer, while the data link layer is generally said to be the lowest level layer.
Let’s start with understanding the hierarchical relationship. The essence is as follows: while a data packet at a particular layer in the model need not necessarily have passed through a higher-level layer, it must always pass through any remaining lower level layers in order for communication to have any chance of occurring successfully. So unlike in the discussion of our HTML request to Bitcoin.org’s server in the previous sections, computer communications do not have to be processed by all layers in the model.
An ARP request is an example of a communication that is only processed by the data link layer. A computer sends a very simple, single data packet to another computer or device. The payload only asks for IP address information, while the only metadata is from the data link layer. This message does not involve any application, transport, or network layer metadata.
A ping request (this works via the Internet control message protocol) is an example communication that is only processed by the data link and network layers. This is also a very simple message that just seeks to establish connectivity between two computers.
By contrast, what could never happen is that a successful communication is processed by a higher layer in the model, but not one of the lower layers. A ping request, for example, could not be sent without being processed by the data link layer. Similarly, a web request could never be processed without passing through the transport, network, and data link layer. This is the essence of the hierarchy in the Internet protocol model.
Although the Internet protocol model is usually focused on organizing the Internet protocol suite, you will sometimes here people speak of certain layer devices, standards, or software applications.
For instance, a switch is often referred as a data link layer device, while a router is often referred to as a network layer device. This indicates the highest layer on which communications data is processed on those devices. A switch, at least typically, does not process any metadata belonging to the network, transport, and application layer (though it can forward all that data). A router, at least typically, does not process any metadata belonging to the transport and application layer (though it can forward all that data).
Though the layers in the model are broadly defined, we can summarize the respective functions of their associated protocols as follows:
- Data link layer: all the protocols which help establish the communication links between directly connected computers and devices
- Network layer: all the protocols that support establishing communication links between distantly connected computers and devices
- Transport layer: all the protocols that support successful transport from the application on one computer to the application on another computer
- Application layer: all the protocols that support communication between applications, so that any transported data can be successfully rendered for end-users
Importantly, the Internet protocol model is not set in stone. Technologies and standards continuously develop. Hence, the protocols that associated with each layer can change over time.
In this last section, we will return to the topic of the architecture of the Internet, focusing specifically on peer to peer networks.
Think of your home computer network or that of a small office. We often describe such local area networks—that is, networks that are located in a very limited geographical area—as either client-server or peer to peer. These labels describe how the network typically operates on a functional level.
A client-server network generally has two main characteristics in its operation. First, in the sharing of network resources, some computers clearly function as servers and some clearly as clients. Second, these servers are managed by a central administrator (either a person or a team).
By contrast, in a typical peer to peer network computers do not operate according to such a strong hierarchy. Computers in the network tend to function both as clients and servers in the provision of network resources.
So, for instance, a small office network might have a dedicated server which stores the files of all employees, a server which handles commonly used applications, an e-mail server, and so on. Employees will, then, connect to these servers from their own computers for accessing these resources. Say the servers are also managed by a local network administrator. This type of small office network would typically be called a client-server network.
Your home network, by contrast, probably does not have any servers and, by extension, anyone managing them. Instead, you likely just have a number of computers and other devices that operate relatively independently from each other. Any resource sharing between these computers—say, for instance, accessing an application located on one computer from one of the other computers—is incidental. This type of home network is typically called a peer to peer network.
The distinction between client-server and peer to peer networks is not binary. Even very hierarchical networks will tend to sometimes have computers interacting in a peer to peer fashion and vice-versa. Hence, any network will fall in between these two extremes in how they operate on a functional level.
In addition, the label most appropriate for a local area network can change. A home network which is originally a peer to peer network, for instance, might better be dubbed a client-server network once you start working with centrally managed e-mail, file, and application servers.
The Internet is neither really a client-server network or a peer to peer network. What we can say, however, is that some of our Internet interactions occur within a client-server architecture, while others take place within a peer to peer architecture.
Using the World Wide Web is clearly a good example of an Internet activity governed by a client-server architecture. A network of servers provides web resources to end-users. Each of these servers is owned and managed by an organization or private person. E-mail is also an example of an activity that is commonly executed within a client-server architecture (unless the sender and recipient both run their own e-mail servers).
Our peer to peer interactions on the Internet mostly occur within geographically distributed peer to peer networks. In fact, when we speak of “peer to peer networks”, we are typically referring to these latter networks specifically, rather than local area peer to peer networks.
Just as within peer to peer local area networks, the computers in these distributed peer to peer networks operate with relatively flattened hierarchies. But they differ in two distinct ways:
- Instead of being connected locally via ethernet cables or a Wi-Fi connection, distributed peer to peer networks are connected via the Internet. These connections are typically more fluid than in a home network, as computers are continuously moving in and out of these larger networks.
- In addition, distributed peer to peer networks are specifically designed to serve resources to network participants. In a home network, peer to peer networks are usually appealing for their cost-efficiency and convenience, not necessarily for sharing resources.
You can participate in a distributed peer to peer network by downloading an application that implements certain communication protocols and standards. As these applications rely on many computers to work successfully, they are typically called distributed applications. Bitcoin, of course, is an example of a distributed application.
The most popular distributed peer to peer networks are file sharing networks such as μTorrent, Vuze, and eMule. These types of file sharing networks usually operate according to the BitTorrent protocol.
This protocol turns all files into torrents, which includes the data of the original file split into many separate fragments as well as a metadata file about those fragments. Any peer that wants the original file will first download the metadata file and, then, start grabbing the fragments. While downloading the file, the peer is known as a downloader.
While downloading, the peer can allow others to upload fragments of the file it already possesses. After a downloader has the entire file, it can continue to allow others to upload the file from its system, becoming what is known as a seeder.19
Peers do not operate on an exactly equal footing in these file sharing networks. Any peer who downloads substantially more than they upload, for example, is typically known as a leecher.20 Yet, by design, these peer to peer file sharing networks operate in markedly little hierarchy.
The BitTorrent protocol discussed above was created precisely in reaction to the shutdown of Napster and similar more centralized peer to peer networks. Bitcoin was inspired partially by this history of file sharing21 to respond in similar fashion to the failures of many earlier alternative currency systems such as E-gold and Ecash. In a discussion forum of the P2P Foundation, Satoshi offered the following reflection:
A lot of people automatically dismiss e-currency as a lost cause of all the companies that failed since the 1990’s. I hope it’s obvious it was only the centrally controlled nature of those systems that doomed them. I think this is the first time we’re trying a decentralized, non-trust-based system.22
Before moving on, we need to wrap up one point of frequent confusion. Just because a network is peer to peer does not mean that you have no reliance on third parties at all. This is the case even for very decentralized peer to peer networks such as Bitcoin. For an obvious example, anyone that uses a peer to peer network will usually have to rely on the Internet, which has a vast array of communication equipment owned and operated by others, including routers, switches, cables, and satellites.
Importantly, you should always take great care to figure out the details when you hear words like “peer to peer” and “decentralized.” First, these are broad terms that really do not tell you enough about the specifics of a network to assess its properties. Second, these terms have, unfortunately, become favored buzz words for a great number of very unscrupulous people.
1. Formally, an autonomous system is a collection of Internet prefixes under the ownership of a single administrative entity which maintains a clearly defined routing policy to the rest of the Internet. IETF, Network Working Group, “Guidelines for creation, selection, and registration of an autonomous system,” March 1996 (https://tools.ietf.org/html/rfc1930).
2. See Andrew Blum, Tubes: A Journey to the Center of the Internet (Harper Collins, New York: 2012), p. 123.
3. There are many good sources for further learning about the physical topology and commercial agreements that make the Internet. An nice introduction is given by Andrew Blum, ibid.
4. ARPA is nowadays called the Defense Advanced Research Projects Agency (DARPA).
5. By this time, accessing computers remotely via terminals was already common practice. That, however, on its own would not have been a practical approach to using the capabilities of different computers. Suppose you were working on an application, for instance, and needed some data from another computer. You would still have to ensure that the two computers could communicate, as your terminal would have had no real abilities to intermediate, such as by storing the data directly.
6. For a discussion, see Katie Hafner and Matthew Lyon, Where Wizards Stay Up Late: The origins of the Internet, Simon & Schuster (New York: 1996), esp. p. 40f. See also the discussion between Charles Severance and Leonard Kleinrock, “The first two packets on the Internet,” March 4 (2014), available at (https://www.youtube.com/watch?v=uY7dUJT7OsU). Leonard Kleinrock was one of the main early researchers on the ARPANET project.
7. See Rand Corporation, “Paul Baran and the origins of the Internet”, available at https://www.rand.org/about/history/baran.html.
8. See Katie Hafner and Matthew Lyon, ibid., for instance, on p. 10. See also Charles Severance and Leonard Kleinrock, ibid.
9. See, for example, Katie Hafner and Matthew Lyon, ibid., p. 45 for a mention of earlier experiments with computer communication.
10. See Katie Hafner and Matthew Lyon, ibid., pp. 52–67.
11. The term “data packet” sometimes also has a more specific meaning, namely as any data unit with network layer data (mainly the IP addresses of the two computers communicating). A data unit on the ethernet link layer is, then, usually called a frame, while any data unit on the transport layer is, then, called a segment or datagram.
12. To be exact, your computer does not only have IP addresses that are associated with network interface adapters. It also has other types of IP addresses, including, for example, a loop-back IP address. Any data sent to this address, usually 127.0.0.1, is returned to your own computer. It is used for testing purposes. So, to be precise, your computer cannot just have multiple IP address simultaneously, it can have multiple IP addresses associated with different network interface adapters.
13. This is roughly true. MAC addresses may sometimes be used for multiple devices. As long as MAC addresses are unique enough, however, it will not cause any problems for data traveling over the Internet.
14. For the switch, this is wired into the hardware. A device known as a bridge can perform the same function via software.
15. Originally, these types of cable were made from copper. You will still find old, unused copper cables buried in our seas and oceans, as they are often not removed after being decommissioned.
16. Sometimes, people will speak of the Internet protocol suite as both a set of protocols and a model to describe them. I find this a poor use of language, however, as a “suite” really refers to a set of something, such as a hotel suite or a software suite.
17. There are also versions that employ five layers, rather than four. The fifth layer usually refers to the physical connections between nodes on the Internet, such as copper cables, fiber-optic cables, and radio signals. The OSI model is an alternative model that specifies how network communication can exist. The Internet protocol model is more a reflection of modern standards than the OSI model.
18. The original model was loosely defined by the Internet Engineering Task Force in “Requirements for Internet hosts – communication layers,” RFC 1122, October 1989 (https://tools.ietf.org/html/rfc1122#page-26).
19. The term “peer” is sometimes also used to refer specifically to a downloader.
20. This term “leecher” is sometimes also used to refer specifically to a downloader.
21. See, for example, the Cryptography Mailing List, “Bitcoin P2P e-cash paper,” November 7 (2008), available at: https://www.metzdowd.com/pipermail/cryptography/2008-November/014823.html.
22. Satoshi Nakamoto, “Bitcoin open source implementation of a P2P currency,” February 15 (2009), available at: http://p2pfoundation.ning.com/forum/topics/bitcoin-open-source.