TFTP Synchronization Extensions (TFUP): Client and Server Programs in C with Unix Sockets

Author: James Northway
james@jamesnorthway.net

Introduction

This work provides an example of how the TFTP protocol can be augmented with additional features, and provides such an implementation. The extensions described here add facilities to list and synchronize files between a local and remote host, in addition to the existing read and write operations provided by TFTP. To differentiate the extended protocol from the Trivial File Transfer Protocol (TFTP), it will be referred to as the Trivial File Update Protocol (TFUP). TFUP is an extension of TFTP as per RFC 783, as previously implemented. The TFTP packet types and reliability mechanisms are still present. As such, the TFUP server retains interoperability with TFTP clients.

Existing protocols that are comparable include rcp/scp which can transfer a file between a local and remote host, and vice versa, unencrypted in the case of rcp and via SSH in the case of scp. The differences being, these protocols do not include a directory listing command or a synchronisation command. They can merely overwrite an existing file in each direction.

Rsync can synchronise files between remote computers but is dependent on authentication between the hosts, as does scp and rcp. TFUP lacks this, making it more trivial and suitable for applications where authentication is unnecessary.

This report will review the protocol and the design of a proposed network application implementing TFUP, and then show the protocol in action with such a program. It is assumed the reader is familiar with the earlier TFTP implementation report, which covers some program components not discussed here.

The Trivial File Update Protocol

The TFUP protocol incorporates all packet types of TFTP; read request, write request, data, acknowledgement and error. It adds two new packet types; timestamp request, and timestamp, as well as adding a new prescribed behaviour for directory listing and read requests for this listing.

The new packet type, timestamp request, is sent from the client to the server. The server responds with a timestamp packet or an error packet if the file is not found. The timestamp request packet has an operation code (op code) of 6, and has the following structure:

 2 bytes    n bytes     1 byte
|op code | file name | null byte |

The new packet type timestamp is sent from the server to the client in reply to a timestamp request, if the file is present in the server’s current working directory. The timestamp packet has an operation code of 7, and has the following structure:

 2 bytes    n bytes     1 byte	    n bytes     1 byte
|op code | file name | null byte | timestamp | null byte |

The packets keep with the TFTP convention of terminating strings with a null byte, and using a two byte op code. The chosen op codes for the new TFUP packet types do not duplicate any existing TFTP packet op code. This allows TFTP interoperability to be preserved by a server program which provides a TFUP service.

The timestamp is included in the packet as a string of n bytes to address the year 2038 limitation by providing for any timestamp that will fit the remaining space; after op code, file name and null bytes have been accounted for within a 516 byte TFTP packet.

New behaviours of the client and server are prescribed; the client must be able to persist after a single transaction, as it can now initiate a second transaction based on the result of the first. The server, on receiving a read request for the file name .tfup_rlist, will regenerate that file with the current directory listing, including timestamps, before sending it to the client in the same manner as TFTP.

Synchronization logic is implemented in the client to maintain the server’s role as a listener and responder. The logic applied by the client is as follows:

The client supports the following commands:

Command Description
rlist Requests the file .tfup_rlist from the server and display its contents on screen upon receipt. The .tfup_rlist file must consist of one line per file, with a null byte separating the file name from the timestamp. It must only include files in the server’s current working directory, that is, the directory in which the server program was invoked, and exclude directories and hidden files.
llist Displays a list of files and their timestamps in the current working directory on the local computer.
rupdate filename Requests a timestamp for that file name from the server, upon receipt of a TFUP timestamp or TFTP error packet, the client will apply the aforementioned synchronization logic.

Changes to the Existing TFTP Client and Server Applications

The overall design of the program, including the four modules; file.c, fsm.c, netudp.c and tftp.c, has only been modified slightly, with a change from the tftp.c file name to the tfup.c name to denote the extended protocol. These modules are shared between fsyncsrv.c, the server program, and fsynccli.c, the client program. These programs are still controlled by finite state machines, however, the client state machine has been redesigned, and now resembles the server state machine more closely. The reason for this change is that the client now needs to action consecutive commands, and therefore conduct any number of arbitrary transactions initiated via the user interface before terminating, as opposed to being invoked to perform a single transaction. It also must support the new update logic.

A number of new functions have been added to the following modules:

file.c now contains three new functions:

fsm.c has a new client_fsm function due to the redesigned client state machine.

netudp.c has had no new additions.

tftp.c has been renamed tfup.c and changes include:

Two new op codes have also been defined in the header file tfup.h.

Minor alterations and corrections may have been made to any of the existing functions to fit the new protocol or correct programming errors.

Design of the New TFUP Network Application

The Server

TFUP Server State Diagram
Figure 1, Server States

The TFUP server state machine is identical to the TFTP server implementation. However, actions taken within those states have been expanded to respond to timestamp requests and requests for the directory list file.

New functions used in the server program:

The Client

TFUP Client State Diagram
Figure 2, Client States

The client uses a subset of the server states utilising the same modules and functions. The client has the reset and standby states omitted due to termination at the completion or abandonment of a transaction. The client therefore has three states; send, wait and receive. Unlike the server, the client will form a read or write request immediately upon being executed, without transitioning through the receive state first. After the initial packet to start the transaction, any other packets will be formed in the receive state. The client receives user input as command line arguments and does not wait for any input from the user while running.

The client will continue the Send -> Wait -> Receive cycle until an error occurs, the timeout limit is exceeded in the wait state, the last acknowledgement for a get is sent, or the last acknowledgement for a put is received. Once the transaction is completed or abandoned, the client will terminate.

Implementation

The client and server have been written in C to compile with the GNU C Compiler on Debian Linux versions 5 and 6. They do not utilise any third party libraries, only those included with glibc. A makefile is provided which will compile the client as fsynccli, and the server as fsyncsrv, or both (all) if none specified. ‘make clean’ will delete the binaries created previously. Both the client and server will operate from the current directory when they are invoked. This need not be the directory the executable resides in. Like the TFTP implementation, User Datagram Protocol (UDP) is used as the messaging protocol for transferring the TFUP packets.

Demonstration of the Client and Server

The output shown will be the process of compiling and running the server and client in separate directories on the local computer. Rlist and llist will be invoked on the client. A file that does not exist locally (client side), does not exist remotely (server side), is newer on the client and newer on the server will be synchronised. Finally, an attempt will be made to synchronise a file that does not exist on the client or server. The client and server files will be in clientfiles and serverfiles directories that do not contain the binaries. Rlist and llist will be shown before and after the file updates.

Client shell input/output

james@CSCcli:~/test$ ls
clientfiles  serverfiles  synccli  syncsrv
james@CSCcli:~/test$ cd synccli
james@CSCcli:~/test/synccli$ make client
gcc -ansi -Wall -D_POSIX_C_SOURCE fsynccli.c file.c fsm.c netudp.c tfup.c -o fsynccli

james@CSCcli:~/test/synccli$ cd ../clientfiles
james@CSCcli:~/test/clientfiles$ ls
mobydick.txt  only_on_client.txt  short.txt

james@CSCcli:~/test/clientfiles$ ../synccli/fsynccli 192.168.1.115 3333
fsynccli> rlist
Received block: 1

short.txt 1382864936
only_on_server.txt 1382865012
mobydick.txt 1382864952

Done.
fsynccli> llist

short.txt 1382864960
only_on_client.txt 1382864991
mobydick.txt 1382863992

fsynccli> rupdate only_on_server.txt
Local file does not exist. Downloading from server.
Received block: 1
Done.

fsynccli> rupdate only_on_client.txt
Remote file does not exist. Uploading to server.
Transferred block: 1
Done.

fsynccli> rupdate short.txt
Timestamp for local file is  1382864960
Timestamp for remote file is 1382864936
Local file is newer. Uploading to server.
Transferred block: 1
Done.

fsynccli> rupdate mobydick.txt
Timestamp for local file is  1382863992
Timestamp for remote file is 1382864952
Remote file is newer. Downloading from server.
Received block: 2453
Done.

fsynccli> rlist
Received block: 1

short.txt 1382865809
only_on_client.txt 1382865728
only_on_server.txt 1382865012
mobydick.txt 1382864952

Done.
fsynccli> llist

short.txt 1382864960
only_on_client.txt 1382864991
only_on_server.txt 1382865656
mobydick.txt 1382865961

fsynccli> rupdate exists_nowhere.txt
Remote file does not exist. Uploading to server.
No such file or directory
Local file does not exist either.

Failed.
fsynccli> exit
james@CSCcli:~/test/clientfiles$

Server shell input/output

james@CSCsrv:~/test$ ls
clientfiles  serverfiles  synccli  syncsrv
james@CSCsrv:~/test$ cd syncsrv
james@CSCsrv:~/test/syncsrv$ make server
gcc -ansi -Wall -D_POSIX_C_SOURCE fsyncsrv.c file.c fsm.c netudp.c tfup.c -o  
fsyncsrv

james@CSCsrv:~/test/syncsrv$ cd ../serverfiles
james@CSCsrv:~/test/serverfiles$ ls
mobydick.txt  only_on_server.txt  short.txt

james@CSCsrv:~/test/serverfiles$ ../syncsrv/fsyncsrv 3333

***TFUP Server***
Read request for directory listing.
Transferred block: 1

***TFUP Server***
Timestamp request for only_on_server.txt

Done.

***TFUP Server***
Read request for only_on_server.txt
Transferred block: 1

***TFUP Server***
Timestamp request for only_on_client.txt
No such file or directory

Done.

***TFUP Server***
Write request for only_on_client.txt
Received block: 1
Done.

***TFUP Server***
Timestamp request for short.txt

Done.

***TFUP Server***
Write request for short.txt
Received block: 1
Done.

***TFUP Server***
Timestamp request for mobydick.txt

Done.

***TFUP Server***
Read request for mobydick.txt
Transferred block: 2453

***TFUP Server***
Read request for directory listing.
Transferred block: 1

***TFUP Server***
Timestamp request for exists_nowhere.txt
No such file or directory

Done.

Experiment

The experiment will be conducted between a local and remote computer over the internet. For the test, fsyncsrv will be running on a machine in Florida, USA with a 100Mbps port speed, and fsynccli will be run on a machine with a residential ADSL connection in QLD, Australia. The test will involve conducting the same tasks as the usage demonstration over the internet. After each test ls –l will be run to show the file changes taking place as opposed to rlist and llist. This avoids using the TFUP programs tested to validate their own behaviour.

Test 1: rlist and llist (Completed successfully)

Client:

james@CSClocal:~/test/clientfiles$ ls -l
total 1240
-rw-r--r-- 1 james james 1255665 2013-10-27 19:26 mobydick.txt
-rw-r--r-- 1 james james      47 2013-10-27 19:09 only_on_client.txt
-rw-r--r-- 1 james james      21 2013-10-27 21:43 short.txt
james@CSClocal:~/test/clientfiles$ ../synccli/fsynccli 199.167.30.180 3333
fsynccli> rlist
Received block: 1

mobydick.txt 1382874062
only_on_server.txt 1382865012
short.txt 1382865808

Done.
fsynccli> llist

short.txt 1382874199
only_on_client.txt 1382864991
mobydick.txt 1382865961

Server:

james@CSCinet:~/serverfiles$ ls -l
total 1236
-rw-r--r-- 1 james james 1255665 Oct 27 07:41 mobydick.txt
-rw-r--r-- 1 james james      47 Oct 27 05:10 only_on_server.txt
-rw-r--r-- 1 james james      21 Oct 27 05:23 short.txt
james@CSCinet:~/serverfiles$ ../tfup/fsyncsrv 3333


***TFUP Server***
Read request for directory listing.
Transferred block: 1

Test 2: File does not exist locally (Completed successfully)

Client:

fsynccli> rupdate only_on_server.txt
Local file does not exist. Downloading from server.
Received block: 1
Done.
fsynccli> exit
james@CSClocal:~/test/clientfiles$ ls -l
total 1244
-rw-r--r-- 1 james james 1255665 2013-10-27 19:26 mobydick.txt
-rw-r--r-- 1 james james      47 2013-10-27 19:09 only_on_client.txt
-rw-r--r-- 1 james james      47 2013-10-27 21:51 only_on_server.txt
-rw-r--r-- 1 james james      21 2013-10-27 21:43 short.txt
james@CSClocal:~/test/clientfiles$

Server:

***TFUP Server***
Timestamp request for only_on_server.txt

Done.


***TFUP Server***
Read request for only_on_server.txt
Transferred block: 1

***TFUP Server***

Test 3: File does not exist remotely (Completed successfully)

Client:

fsynccli> rupdate only_on_client.txt
Remote file does not exist. Uploading to server.
Transferred block: 1
Done.
fsynccli>

Server:

***TFUP Server***
Timestamp request for only_on_client.txt
No such file or directory

Done.


***TFUP Server***
Write request for only_on_client.txt
Received block: 1
Done.


***TFUP Server***
^C
james@CSCinet:~/serverfiles$ ls -l
total 1240
-rw-r--r-- 1 james james 1255665 Oct 27 07:41 mobydick.txt
-rw-r--r-- 1 james james      47 Oct 27 07:57 only_on_client.txt
-rw-r--r-- 1 james james      47 Oct 27 05:10 only_on_server.txt
-rw-r--r-- 1 james james      21 Oct 27 05:23 short.txt
james@CSCinet:~/serverfiles$

Test 4: File is more recently modified on the client (Completed successfully)

Client:
james@CSClocal:~/test/clientfiles$ touch short.txt
james@CSClocal:~/test/clientfiles$ ls -l
total 1244
-rw-r--r-- 1 james james 1255665 2013-10-27 19:26 mobydick.txt
-rw-r--r-- 1 james james      47 2013-10-27 19:09 only_on_client.txt
-rw-r--r-- 1 james james      47 2013-10-27 21:51 only_on_server.txt
-rw-r--r-- 1 james james      21 2013-10-27 22:03 short.txt
james@CSClocal:~/test/clientfiles$ ../synccli/fsynccli 199.167.30.180 3333
fsynccli> rupdate short.txt
Timestamp for local file is  1382875383
Timestamp for remote file is 1382865808
Local file is newer. Uploading to server.
Transferred block: 1
Done.
fsynccli>

Server:

james@CSCinet:~/serverfiles$ ls -l
total 1240
-rw-r--r-- 1 james james 1255665 Oct 27 07:41 mobydick.txt
-rw-r--r-- 1 james james      47 Oct 27 07:57 only_on_client.txt
-rw-r--r-- 1 james james      47 Oct 27 05:10 only_on_server.txt
-rw-r--r-- 1 james james      21 Oct 27 05:23 short.txt
james@CSCinet:~/serverfiles$ ../tfup/fsyncsrv 3333


***TFUP Server***
Timestamp request for short.txt

Done.


***TFUP Server***
Write request for short.txt
Received block: 1
Done.


***TFUP Server***
^C
james@CSCinet:~/serverfiles$ ls -l
total 1240
-rw-r--r-- 1 james james 1255665 Oct 27 07:41 mobydick.txt
-rw-r--r-- 1 james james      47 Oct 27 07:57 only_on_client.txt
-rw-r--r-- 1 james james      47 Oct 27 05:10 only_on_server.txt
-rw-r--r-- 1 james james      21 Oct 27 08:02 short.txt
james@CSCinet:~/serverfiles$

Test 5: File is more recently modified on the server, and is larger (Completed successfully)

Client:

james@CSClocal:~/test/clientfiles$ ls -l
total 1244
-rw-r--r-- 1 james james 1255665 2013-10-27 19:26 mobydick.txt
-rw-r--r-- 1 james james      47 2013-10-27 19:09 only_on_client.txt
-rw-r--r-- 1 james james      47 2013-10-27 21:51 only_on_server.txt
-rw-r--r-- 1 james james      21 2013-10-27 22:03 short.txt
james@CSClocal:~/test/clientfiles$ ../synccli/fsynccli 199.167.30.180 3333
fsynccli> rupdate mobydick.txt
Timestamp for local file is  1382865961
Timestamp for remote file is 1382874062
Remote file is newer. Downloading from server.
Received block: 2453
Done.
fsynccli> exit
james@CSClocal:~/test/clientfiles$ ls -l
total 1244
-rw-r--r-- 1 james james 1255665 2013-10-27 22:16 mobydick.txt
-rw-r--r-- 1 james james      47 2013-10-27 19:09 only_on_client.txt
-rw-r--r-- 1 james james      47 2013-10-27 21:51 only_on_server.txt
-rw-r--r-- 1 james james      21 2013-10-27 22:03 short.txt
james@CSClocal:~/test/clientfiles$

Server:

james@CSCinet:~/serverfiles$ ls -l
total 1240
-rw-r--r-- 1 james james 1255665 Oct 27 07:41 mobydick.txt
-rw-r--r-- 1 james james      47 Oct 27 07:57 only_on_client.txt
-rw-r--r-- 1 james james      47 Oct 27 05:10 only_on_server.txt
-rw-r--r-- 1 james james      21 Oct 27 08:02 short.txt
james@CSCinet:~/serverfiles$ ../tfup/fsyncsrv 3333


***TFUP Server***
Timestamp request for mobydick.txt

Done.


***TFUP Server***
Read request for mobydick.txt
Transferred block: 2453

***TFUP Server***

Test 6: File does not exist locally or remotely (Completed successfully)

Client:

james@CSClocal:~/test/clientfiles$ ../synccli/fsynccli 199.167.30.180 3333
fsynccli> rupdate does_not_exist.txt
Remote file does not exist. Uploading to server.
No such file or directory
Local file does not exist either.

Failed.
fsynccli> exit
james@CSClocal:~/test/clientfiles$ ls -l
total 1244
-rw-r--r-- 1 james james 1255665 2013-10-27 22:16 mobydick.txt
-rw-r--r-- 1 james james      47 2013-10-27 19:09 only_on_client.txt
-rw-r--r-- 1 james james      47 2013-10-27 21:51 only_on_server.txt
-rw-r--r-- 1 james james      21 2013-10-27 22:03 short.txt

Server:

***TFUP Server***
Timestamp request for does_not_exist.txt
No such file or directory

Done.


***TFUP Server***
^C
james@CSCinet:~/serverfiles$ ls -l
total 1240
-rw-r--r-- 1 james james 1255665 Oct 27 07:41 mobydick.txt
-rw-r--r-- 1 james james      47 Oct 27 07:57 only_on_client.txt
-rw-r--r-- 1 james james      47 Oct 27 05:10 only_on_server.txt
-rw-r--r-- 1 james james      21 Oct 27 08:02 short.txt

Conclusion

This project has produced a potential protocol with extended functionality that has been derived from, and retains server side backwards compatibility with, the TFTP protocol. A client and server program has been produced which was demonstrated to synchronise files over the internet on the basis of latest modification time. Due to the constraints of the TFTP data transfer method, such as a small packet size, and waiting on acknowledgement of single packets, the transfer of files of 1MB or more was, anecdotally, slower than protocols commonly used today. The transfer method is, however, reliable and finite state machines seem an able method of controlling network communications.

Links

TFUP Client and Server Source Tarball MIT License Readme

The TFTP Protocol (Revision 2) RFC 783
TFTP Client and Server Programs in C with Unix Sockets
User Datagram Protocol RFC 768

Copyright (c) 2013, 2014, 2015 James Northway.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Last revised 5/07/2015. james@jamesnorthway.net