diff options
| -rw-r--r-- | doc/merge.txt | 31 |
1 files changed, 31 insertions, 0 deletions
diff --git a/doc/merge.txt b/doc/merge.txt new file mode 100644 index 0000000..28757a7 --- /dev/null +++ b/doc/merge.txt @@ -0,0 +1,31 @@ +-*- markdown -*- + +The merge process +================= + +- merge-fetch maintains a single file 'fetched' referring to a given + entry in 'logorder', indicating which entries are fetched and + sequenced so far. + +- merge-backup reads 'fetched' and pushes these entries to secondary + merge nodes, maintaining one file per secondary, + 'backup.<secondary>', indicating how many entries that have been + copied to and verified at the secondary in question. + +- merge-sth writes a new 'sth' file by reading the + 'backup.<secondary>' files into a list, picking a new tree size by + sorting the list (in falling order) and indexing it with the + 'backupquorum' config option. If the new tree size is smaller than + what the old 'sth' file says, no new STH is created. + +- merge-dist distributes 'sth' and missing entries to frontend nodes. + +TODO +==== + +- Run the three pieces in parallell. + +- Improve merge-fetch by parallellising it using one process per +storage node writing to a "queue info" file (storage-node, hash) and a +single "queue handling process" reading queue files and writing to the +'fetched' file. |