Data communications by example for developers

In this section I discuss communication protocols; we start with a grounding on the basics and move through to some worked examples. To start with let’s look at some of the simplest cases and understand how communications between computer systems work in practise.

There are many methods of communicating between two computers, be they embedded, mobile, desktop or server. Protocols range from the low level SPI and I2C right through to stream based approaches such as sockets over a network. Conversely, probably the easiest one to learn first is TCP/IP sockets, as this is available out of the box in most programming environments without the need to install extra software or hardware. If you can use a web browser on the device, it has support for TCP/IP sockets! For now let's take a look at communications without considering any particular protocol.

Probably the simplest case is protocols like vt100 that just send and receive character data. They allow the user to interact with a remote computer using a command line shell. Character data from the keyboard is sent to the remote system and any results are sent back to be displayed on our monitor. This is the simplest (and one of the oldest) forms of communication. Below is an example image, it shows the user inputting the text CMD and pressing enter. The remote system then responds and the response is sent back to the user.

Character based communications

Systems such as the one shown in the image above can only work on stream based protocols. Stream protocols guarantee the order that data arrives, and further they ensure that no data is lost. There are obvious overheads to providing such a stream, but it makes solutions where ordering is important much easier to write. Notice above that the communication is shown as two streams of data, that is because many such solutions are essentially two way (or full duplex).

We can see in the diagram above that we write information to the output channel and expect results back on the input channel. There is no special processing of the results as they are just pieces of text in both directions. There are many protocols that use this approach - common ones include the ssh protocol and telnet.

Message based communication

Message based communication is the next step up. It is based on blocks of data in known formats being transmitted between systems.

Let’s take a simple example of email messages sent between two people. If we have common messages, then we could easily understand each other. Let’s say for example we had an email message that indicated there was a party at a given place and time. If we knew the format, we could easily extract the data from it when it arrived:

Subject: party
Time: hh:mm
Place: postcode
End of message

Upon receiving this message we would immediately know there was a party and would look for the time and place. This is a simple example of a protocol. 

In order for systems to understand each other there needs to be a common format at the wire level. Normally, the pieces of information we send are called messages; each message is preceded by its length so that if we receive an unexpected message we can still continue processing. If the protocol does not guarantee completeness then we also need to send a crc as well to ensure correctness.

Messages normally have a message type, so for example in the above example we may have a type of PartyMessage. Each message has an agreed set of fields, in the case of party message it was time and place.

There are two main modes of communication between systems, stream based and packet based. Stream based communication guarantees the order in which things arrive, so you get the data in exactly the form it was sent. Examples are sockets and rs232. An example of sockets is http protocol.

Packet based systems make no guarantees about ordering or even delivery at all. They send and receive packets of up to a certain size. Examples are UDP and multicast messaging.

Either solution can transmit message based data but there is the obvious drawback of possible data loss in the case of packet based solutions. Sockets themselves are provided on top of the packet based UDP protocol, but the socket layer provides the ordering.

Next: Using sockets for a character based stream where I build a simple teletype server.