Ahoy, me hearty crew ! Welcome to a series of articles dedicated to networking and network programming beginners. If you’re not a beginner, no problem : everyone needs a concentrated source of basic information once in a while, we all have doubts, that’s human. If you’re not a programmer, you’re welcome too : it can’t hurt to understand what your computer does behind your back (I mean, besides watching cat videos).
I’m writing this because I intend to introduce you to network programming on a variety of systems and devices, and I don’t want to type the same things twice if it can be avoided. Consider this my avoidance. Seriously, though, you too can point people to this page when you’re tired of trying to explain your work to friends, family and cats.
I’ll start with explaining as simply and precisely as I can some of the core concepts of networking. We’ll start from the “big picture” and make our way down into the heart of madness.
Networking : what is it ?
It ain’t about computers sending each other pictures of their genital peripherals or photos of the electricity they had for lunch. That would be social networking, a form of communism (based on the Fox News Criterion : it has the word “social” in it).
Computer networking, at its core, is about moving data from point A to point B. A and B can be any type of computer, separated by any distance (both physically and logically). It’s this universality which makes networking complicated. But it gets a lot easier once you start to think of moving data as moving anything else. Let’s use explosives for an analogy.
Network Protocol Stacks
Imagine for a moment you’re Commander in Chief and you need to send some uppity dictator the message that you really dislike him.
If he is next door to you, you can just toss some explosives through the window. If he’s in the next city, however, you’ll need to put those explosives into a missile in order to reach him. If he’s in another country, you’ll need to put that missile onto a jet fighter to get your explosives that far. And if he’s on another continent, you’ll need to put that jet fighter on an aircraft carrier for your explosives to achieve global reach.
You end-up with a stack of delivery systems, where each level encapsulates the one directly below itself.
Network protocols stack the same way. This is generally represented by the fabled OSI model. OSI here stands for Open Systems Interconnection. And it’s the model that is open, not the systems (someone forgot to tell the NSA).
The Internet Protocol Suite
The OSI model is the most generic there is, however you’re probably here because you’re interested in programming something to do with the Internet and/or embedded systems. Be it robots, IoT devices or ad hoc applications, the model that really suits your needs is the Internet Protocol Suite.
This model defines five protocol layers. Remember the explosives analogy ? Well every level of encapsulation represents a layer in the Internet Protocol Suite.
I will now go through each of them top-to-bottom, because it seems most logical. I don’t care if you disagree, this is my show.
Good news : there’s actually nothing to learn here ! At this level we’re dealing with your software’s data, the stuff you want to transmit and/or receive. The explosives. In military parlance, explosives are a payload. That is also how we call our data. Maybe because the Internet was originally a DARPA project.
The application layer is essentially variables of your program : strings, integers, etc… There’s no standard protocol at this level. That makes sense since this is the application layer and you’re the one writing the application.
Like throwing grenades through a window, you can transmit data at the application level : that’s basically what you do when you give variables as arguments to a function. Needless to say, nothing goes onto the network.
This is where you put your explosive payload into a missile, to extend its reach. Here we put our payload data into a transport packet.
There are many types of missiles depending on what you launch them with (plane, ship, bazooka, etc…) and what you launch them at (plane, ship, tank, etc…). Transport packets carry similar information : the source and destination ports.
Port numbers are arbitrary but not random : they are expressed on 16 bits and some values are reserved to identify specific service types (web, e-mail, etc…). If you need a port for purposes no known port serves, try and choose one that won’t clash with anything else. You can check the list of reserved ports on Wikipedia. Basically, every port between 49,152 and 65,535 will always be fair game.
As with the application layer, it’s possible to transfer packets at this level, for example between two different programs running on the same computer. Here too, nothing actually ends-up on your network cables : the packets just move around in your computer’s RAM.
If you need to shoot your missile further than its range allows, you load it onto a jet fighter. Our transport packet’s jet fighter is the IP packet and it’s pretty much as tiny as Tom Cruise. Cue Kenny Loggins’ Danger Zone.
IP stands for Internet Protocol, the unimaginatively-named protocol for the Internet’s Internet Layer. I’m guessing Captain Obvious was head of DARPA when they came up with that one.
A jet has a home base from which it takes off and it also has a target. Both have unique coordinates. The IP packets’ coordinates system is the IP addressing scheme. Each IP packet contains a source and destination IP address, and each identifies a unique machine on the network.
IP addresses are neither random nor arbitrary : they follow specific rules. That’s why your local network’s IP addresses start with 192.168. Thanks to those rules, the many routers that make up the internet know where to send (route) each packet.
Jets have small fuel tanks. If you’re going to “install Democracy” somewhere really far away, you’d better put your jet, with its missile, with its explosive payload, aboard an aircraft carrier and send it all as close as possible to those you want to “help” so violently. One problem, though : unlike planes, aircraft carriers can’t sail in straight lines. They prefer to avoid land, enemy navies and the occasional sea monster.
Likewise, the Internet is a vast ocean fraught with obstacles and perils. While there is a shortest path your IP packet could take, there’s no practical way to know if some routers along that path are down or congested. Unpredictable detours may be required. That uncertainty is one reason the Internet is traditionally represented as a cloud (get it ?).
Your IP packet’s aircraft carrier is the Ethernet frame. Just like ships, frames sail in straight lines from one router to the next. When they get to a router, it decides where to send them next. It’s like Sean Connery tracing the next straight line on his map as he sails his stolen Russian submarine towards the USA : plans must be adapted on the go.
Frames have a mechanism that allows for such adaptation : each frame carries source and destination MAC addresses. Unlike IP addresses and ports, a frame’s destination MAC address is updated each time that frame traverses a router. Think of it as the next instruction of your car’s GPS.
This is what your frame looks like as it sails though the Interwebs :
Each layer brings overhead : additional computation and bandwidth requirements that do not serve a purpose as far as the application is concerned. Layering limits that overhead.
The fifth and final layer in the Internet Protocol Suite is the “PHY” layer. Good news : as far as application programming goes, there’s nothing for you to learn here. This layer is where the cables and chips live. It’s the water that floats the aircraft carrier. The air beneath the jet fighter’s wing.
Embedded programmers sometimes code on this layer but it is unrelated to moving network packets. Most of the time, it’s about initializing chips. For instance you would use out-of-band interfaces like MDIO (a form of I²C) to setup an Ethernet PHY.
Networking may not be easy or obvious, but it is logical and efficient. At any rate, I have no doubt you’re smart enough to get the gist of it. I can guarantee that in a little while you’ll become very familiar with protocol layers and what they do for you. You won’t become a networking expert (that’s probably a life-long endeavor) but you’ll be started down that path.
How far you go down that path should really depend only on what you want to achieve with networking. It’s very easy to get lost in details of specific technologies or protocols. Remember : networking is the sum total of all communication protocols spoken by all networked systems ever (not just PC’s) and all networked programs ever. In complexity this is comparable to the sum total of all languages spoken by humans ever and their local variants. For the same reasons you don’t need to speak all languages in the world, neither do your applications need to know every protocol in the world.
Next time I’ll start going into the details of the Application Layer : just because it’s not a standard protocol doesn’t mean there isn’t a lot to talk about.