HDN In Outline

As per the glossary of terms, HDN is formal shorthand for writing out data state deltas to delta files. It was first used in RAM Primacy HDB data object repositories, as a means of storing data. In its original form, HDN described new data objects, changes in data objects and data object deletions, but nothing beyond the realm of data objects. It was realized that by mirroring deltas as they arose to other physical machines, repository data could be backup in real time. It was also realized that mirroring could enable server redundancy strategies, but only if all data resources in applications were covered. Accordingly, delta notation was extended to cover all forms of data resources found in applications.

As per the glossary of terms, HDN is formal shorthand for writing out data state deltas to delta files. It was first used in RAM Primacy HDB data object repositories, as a means of storing data. In its original form, HDN described new data objects, changes in data objects and data object deletions, but nothing beyond the realm of data objects. It was realized that by mirroring deltas as they arose to other physical machines, repository data could be backup in real time. It was also realized that mirroring could enable server redundancy strategies, but only if all data resources in applications were covered. Accordingly, delta notation was extended to cover all forms of data resources found in applications.

There is a naming convention which relates to where deltas are in the system. This is as follwos:-

In file deltas the data repository is known, so only the object id (address within the repository), and the member values are specified. Origin and notification deltas must further qualify the delta with the data resource id. Transmission deltas must be fully qualified, so app id, resource id, object id and member values. In addition, transmission deltas must also identify the server upon which the the data state change arose. The server id is that within the cluster (always A, B, C or D). This is needed to terminate delta circulation. For simplicity the accepted practice is to include the server, app, and resource ids in all deltas except file deltas.

Data Resource Types and Consequent Delta Formats

Note that in the formats described below, the server id is denoted as 's' and the app id as 'a'. Resource ids are denoted as either b, c, d, f, i or r, depending on resource type.

Persistant Collections: These are delta file backed memory resident collections, i.e. serial datcrons. The RAM component can be a set, a 1:1 map or a 1:many map. In the case of sets, only values exist so the available operations are ADD (+) and DELETE (-). Accordingly, the file deltas are of the form [+=]value, while the deltas outside delta files are of the form s.a.c[+-]value.

In the 1:1 map case the entries are key-object pairs, with the key being unique. The available operations are ADD (a new key-object pair), DELETE (an existing key-object pair), and MODIFY (the object part of an existing key-object pair). Deltas that ADD or MODIFY are both of the form s.a.c.Key=Value. Deltas that DELETE are of the form s.a.c.Key-.

In the 1:Many map case the entries are key-object pairs, but the key is not required to be unique. This extends the available operations. ADD either adds a new key and object or it adds a new object to an existing key, so the delta is of the form s.a.c.Key+Value. MODIFY must specify both key and object in order to uniquely identify the target key-object pair, so the delta is of the form s.a.c.Key.old_value=new_value. DELETE must specify both key and object if a single key-object pair is deleted, but only the key if all key-object pairs of the same key are deleted. The delta forms are respectively s.a.c.Key.value- and s.a.c.Key-.

HDB Indexes: Idsets:

Binary Datum Repositories: These are clearly part of the application and are accordingly assigned delta ids. However binary datum repositories do not write deltas. As binary datum are only values (of a data object member), the task of writing deltas falls to the applicable data object repository.

Data object repositories: These are by far the most important data resource for obvious reasons! Data object repositories offer four operations namely INSERT, UPDATE, DELETE and FETCH, of which the first three will change the data state and thus give rise to a delta. In addition to the server, app and repository ids, the deltas must describe a complete new object in the case of INSERT, and what has been set, added to or removed in each changed member in the case of UPDATE. In the case of DELETE, the delta only needs to state the object id.

In an INSERT the resulting delta has the form s.a.r.objectId[member assignments]. Member assignments are of the form mbrId=value. Multiple member assignments are produced where the member has multiple members. Where the member data type is a (sub)class, the assigned value is the subclass object id, whose issuance would have resulted in a delta. Delta notation itself, is not hierarchical.

In an UPDATE the resulting delta has the form s.a.r.objectId[member adjustments]. Member adjustments are of the form mbrId=value where the member is single value, but where members support multiple values, the adjustments are either of the form mbrId+value or mbrId-value. In a DELETE the resulting delta has the form s.a.r.objectId=0.

Values in a delta are always atomic. Where the member data type is binary, the value stored in the repository will be the address of the binary datum in the applicable binary datum repository. In the delta however, the value will be the binary datum itself. Deltas usually occupy a single line but where strings span multiple lines, HadronZoo Multi-line Format is used.

Application Specific Files and Directories, and Delta Realms

App Specific Files: These are fixed path files an application can maintain for any purpose. The file content is considered to be application data so can be modified at will. However the file itself is considered an intrinsic part of the application. As such, deleting or renaming the file would amount to an application change, something that is outside the scope of the delta regime. This limits the possible deltas on content change, to APPEND (+) and REPLACE (=). The deltas are thus of the form s.a.f[+=]value, where value is the new or additional file content.

App Specific Directories: These are directories an application can maintain for any purpose. Here the directory is intrinsic so cannot be deleted or moved, but the directory content (subdirectories and files), are application data, subject to the full set of file operations. The available operations are thus, APPEND (+), REPLACE (=) and DELETE (-). The deltas are of the form s.a.d.fname.op.value, where fname is either the filename or relative path of the file to the directory. There is no need for a CREATE operator as named files are created if they does not currently exist.

Note that currently, application specific files and directories are not 'mainstream' deltas and are treated separately by the DS.