Add documentation.pmerge

1 files changed, 31 insertions, 0 deletions

diff --git a/doc/merge.txt b/doc/merge.txt
new file mode 100644
index 0000000..28757a7
--- /dev/null
+++ b/doc/merge.txt
@@ -0,0 +1,31 @@
+-*- markdown -*-
+
+The merge process
+=================
+
+- merge-fetch maintains a single file 'fetched' referring to a given
+  entry in 'logorder', indicating which entries are fetched and
+  sequenced so far.
+
+- merge-backup reads 'fetched' and pushes these entries to secondary
+  merge nodes, maintaining one file per secondary,
+  'backup.<secondary>', indicating how many entries that have been
+  copied to and verified at the secondary in question.
+
+- merge-sth writes a new 'sth' file by reading the
+  'backup.<secondary>' files into a list, picking a new tree size by
+  sorting the list (in falling order) and indexing it with the
+  'backupquorum' config option. If the new tree size is smaller than
+  what the old 'sth' file says, no new STH is created.
+
+- merge-dist distributes 'sth' and missing entries to frontend nodes.
+
+TODO
+====
+
+- Run the three pieces in parallell.
+
+- Improve merge-fetch by parallellising it using one process per
+storage node writing to a "queue info" file (storage-node, hash) and a
+single "queue handling process" reading queue files and writing to the
+'fetched' file.