Data synchronisation protocol

Until the protocol is not developed, RescoManager will have to be installed in a server, and all the users will need to access, enter and modify data on-line. This requires them to have Internet connectivity at all times.

Once the data synchronisation protocol has been developed, users can work off-line as well as on-line, with the data being synchronised as soon as two computers with the software installed enter into range.

Description:

1. Types of synchronisation

The protocol should allow for background synchronisation between nodes. Options to be used should be based on:

• delayed tolerant networks, or
• asynchronous email protocoles, used a few years ago when Internet was less reliable.

The project to be developed must be capable of synchronising an SQL database between 2 or more computers without intervention from the user.

It will be developed in Python http://www.python.org) using Django  (http://www.djangoproject.com)

2. ORM data

The system must be able to migrate different types of databases: sqlite3, (http://sqlite.org), mysql (http://www.mysql.org), etc.

The registers contain a UUID field (unique identifier, RFC 4122) which will be created using the python library (uuid) for this purpose.

The nodes will function offline and synchronise with their main node and their secondary nodes when they go online. There will be no connection between nodes most of the time.

3. Methods of synchronisation

• A node will act as the client and server simultaneously.
• The operative platform will have main nodes that will contain 100% of the data.
• The main nodes will synchronise some data between them.
• The secondary nodes will contain part of the data from the main node and provide their data to other secondary nodes.
• The communication layer should be thought of as a stream, so that it can be sent online and offline.

4. Security

• Synchronisation will be carried out using HTTP protocol with user authorisation and password.
• The user will be authorised through an authentication function.
• The objective of this function is to personalise the method and the type of authorisation.

5. Access to parent nodes

Each node will be linked to its parent node through a URL that will contain the access and content of the data to be synchronised.

6. Data integrity

The data replaced must remain in tables to trace all updates carried out.

7. Conflict resolution

An optimistic conflict resolution system must be used, as in the majority of SGBD.

8. Data filter to be updated

The content of the data to be updated will be defined by SQL sentences in two cases:

1. Updates of whole tables: system tables etc.
2. Partial updates of tables and their related tables: information relevant to the node.

9. Error management and register

An thorough register must be made of the actions performed in the database and in the communication process.

10. Resource monitoring

Production will be carried out in diverse environments and platforms, some of them very limited (OLPC). Therefore, the process must function while monitoring the free space on the disc and minimise the use of RAM memory.

Justification: make possible for RescoManager to run off-line