Draft preview
Implementing decentralized timestamps: pipelines, state machines, and verification
How to engineer an automated, zero-cost OpenTimestamps pipeline: solving canonicalization, managing asynchronous state, and enabling offline verification.
In the first part of this series (Proving existence and authorship), we established the theoretical principles of decentralized timestamps: anchoring document hashes into the Bitcoin blockchain using OpenTimestamps (OTS) Merkle aggregation. We also established that for written essays and technical specifications, managing detached private key signatures introduces an unnecessary two-step verification penalty for readers. Instead, embedding the author's identity directly into validated frontmatter (author: Giovanni Filardo) and anchoring the exact raw bytes achieves airtight protection against plagiarism and prior-art theft with friction-free, single-step verification.
Moving from cryptographic theory to a functioning software pipeline, however, exposes a fundamental systems engineering challenge: an impedance mismatch in time.
Modern build tools, compilation scripts, and continuous deployment systems run in seconds. They are designed to be fast, deterministic, and repeatable. In contrast, distributed blockchain consensus is fundamentally asynchronous: public calendar servers aggregate Merkle trees over several hours, and Bitcoin block confirmations arrive at variable intervals governed by Poisson processes.
If you attempt to bridge these two systems naively, your build process will quickly suffer from hanging pipelines, non-deterministic outputs, and brittle cryptographic proofs.
While the principles explored here apply to any automated document pipeline—such as software release digests, legal document workflows, or regulatory archives—this article uses a static website generator as a concrete, end-to-end case study. We examine how to build an automated, zero-cost stamping pipeline that maintains clean separation of concerns.
The CI/CD anti-pattern
When automating timestamp generation, a common initial instinct is to insert the stamping command directly into a continuous integration (CI/CD) deployment pipeline:
Running stamping operations inside CI/CD creates three severe failure modes:
- The latency trap: When a file is stamped (
ots stamp document.txt), the calendar server returns an incomplete proof receipt within seconds. However, that proof is not fully anchored until the calendar's Merkle root is mined into a Bitcoin block—a process that takes between two and six hours. A build runner cannot hang indefinitely waiting for L1 settlement. - Network dependencies and rate limits: Calling external calendar servers during every CI execution introduces external network dependencies into what should be an isolated compilation step. Temporary network partitions or calendar server rate limits will cause deployment builds to fail intermittently.
- Loss of build determinism: If an automated build queries an external API or injects variable blockchain timestamps during compilation, identical source commits will produce different binary outputs. This breaks reproducible builds.
The architectural rule is straightforward: Stamping and upgrading are asynchronous, out-of-band operational cycles; the build and compilation pipeline must remain strictly synchronous, offline, and read-only.
The canonicalization trap vs. exact byte hashing
Before hashing any digital document, you must answer a deceptively simple question: exactly what bytes are being stamped?
A common instinct in publishing systems is to attempt "smart" normalization:
- Stripping YAML/JSON frontmatter so that taxonomy or tag changes don't alter the body hash.
- Stripping trailing whitespace from lines.
- Converting line endings (
\r\nto\n).
While intuitive, this normalization introduces the canonicalization trap:
- Markdown syntax corruption: In standard Markdown, two trailing spaces at the end of a line indicate a hard line break (
<br>). Stripping trailing whitespace mutates the formatting and semantics of the document. - Verification brittleness: If you normalize with an ad-hoc regular expression, anyone who wishes to verify the proof offline in the future must execute that exact same regex logic. If their script strips whitespace slightly differently, the computed SHA-256 hash will fail to match the
.otsproof. - Loss of metadata integrity: The publication date, series tags, and article title in the frontmatter are themselves substantive claims of the published work. Normalizing them away removes them from cryptographic protection.
The robust solution is exact byte hashing:
- Hash the source file exactly as committed.
- If you correct a typo or update frontmatter tags three months later, that change constitutes a legitimate new revision.
- Rather than pretending the revised document is identical to the original, the pipeline generates a new
.otsreceipt for the new version while preserving the historical proof for the previous version. The original proof remains 100% valid for the original bytes forever.
The local manifest state machine
To bridge the gap between instant local edits and multi-hour blockchain confirmation, the repository maintains a lightweight local index via a JSON file: timestamps-manifest.json.
The manifest schema
The manifest tracks the content hash, versioned proof path, and confirmation status for every document:
{
"version": 1,
"documents": {
"system-architecture-spec": {
"latest_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"status": "confirmed",
"proof_file": "proofs/system-architecture-spec/v1.ots",
"bitcoin_block": 870142,
"stamped_at": "2024-11-13T10:48:00Z",
"confirmed_at": "2024-11-13T14:15:00Z",
"revisions": []
}
}
}
The four operational states:
UNTRACKED: A newly authored document. The manifest has no record for this identifier.PENDING: The initial stamp has been created. The fileproofs/<slug>/<version>.otscontains the calendar server's cryptographic receipt, but the Merkle root has not yet been confirmed in a Bitcoin block.CONFIRMED: An upgrade operation has fetched the final Merkle branch connecting the file's hash to a confirmed Bitcoin block header. The.otsfile is now fully self-contained and verifiable offline.REVISED: When an author updates the content, the file hash changes. The manifest preserves the old hash and.otsproof in an append-onlyrevisionslist (retaining verifiable prior art for the original version) and transitions the new version into a freshPENDINGcycle.
Operational workflow and tooling
Rather than burdening deployment runners, provenance is managed locally through two targeted scripts: stamp and upgrade.
1. The stamping cycle
Before committing a new or revised document, run the local stamp script:
node scripts/provenance/stamp.mjs content/posts/my-spec.md
The script executes the following deterministic steps:
- Computes the SHA-256 digest of the raw source file bytes.
- Inspects
timestamps-manifest.json:- If the hash matches an existing
CONFIRMEDrecord, it reports that the document is already anchored and exits without network calls. - If the hash is new or modified, it calls the OpenTimestamps CLI:
ots stamp content/posts/my-spec.md - Moves the generated
content/posts/my-spec.md.otsinto the versioned proof directory (proofs/my-spec/<version>.ots).
- If the hash matches an existing
- Records the state as
PENDINGin the manifest with the current timestamp.
2. The upgrade cycle
A few hours later—or via a scheduled local task—run the upgrade command:
ots upgrade proofs/**/*.ots
The OpenTimestamps client contacts the calendar servers, checks whether the aggregated Merkle root has been included in a mined Bitcoin block, and serializes the complete Merkle path back into the .ots file.
The script then extracts the confirmed Bitcoin block height and updates the manifest status to CONFIRMED.
git add content/ proofs/ timestamps-manifest.json
git commit -m "docs: anchor architecture spec to Bitcoin block #870142"
Repository hygiene and storage footprint
A practical concern when incorporating cryptographic proofs into software repositories is repository bloat. Many decentralized implementations store bulky transaction receipts, binary blocks, or runtime dependencies that inflate repository size over time.
OpenTimestamps avoids this entirely:
- Detached proof storage: Proof receipts live in a separate
proofs/directory rather than being concatenated into source files. - Microscopic storage footprint: A finalized
.otsproof file is approximately 400 to 700 bytes in size. Stamping an entire library of 1,000 comprehensive specifications or essays adds less than 600 kilobytes of binary data to your repository—smaller than a single uncompressed screenshot. - Git attributes: To prevent Git from modifying line endings inside binary proof files, declare them in
.gitattributes:proofs/**/*.ots binary
The verification protocol
The defining strength of this architecture is that verification is completely self-contained. Anyone can verify the anteriority and authenticity of a document without an account, without an API key, and without contacting any third-party service.
When you execute ots verify, the client evaluates the proof through two distinct stages:
- Stage 1 (Off-Chain Merkle Path): The
.otsfile contains a serialized recipe of operations (sibling hashes and concatenation rules). The verifier computes the SHA-256 digest of your local document, combines it step-by-step with intermediate sibling hashes from the receipt, and calculates the calendar server's aggregated Merkle root. If a single byte of the document was modified, the calculated root will fail to match. - Stage 2 (On-Chain Bitcoin Proof): The upgraded receipt continues the cryptographic path into Bitcoin. It proves how the calendar Merkle root was embedded into a transaction's
OP_RETURNscript, and how that transaction connects through the block's internal transaction tree to themerkle_rootrecorded in the 80-byte Bitcoin block header. - The Proof-of-Work Check: The client verifies that the block header meets Bitcoin's Proof-of-Work difficulty target and belongs to the valid blockchain. Because Bitcoin headers are compact (under 70 MB for the entire chain since 2009), this check can be performed completely offline on a local machine.
Reproducible verification: from raw bytes to block explorer
To understand how an auditor verifies your proof, it helps to walk through the journey step-by-step. The defining feature of OpenTimestamps is that an auditor never has to trust the author, the hosting server, or even OpenTimestamps itself.
1. Compute the local file hash
The auditor begins by calculating the SHA-256 digest of the downloaded document:
shasum -a 256 my-document.md
# Example output:
# 16efa7abadaf4f86969bacbf0f7db2acce22d88927760d01d82b13e71903281b
2. Inspect the Merkle audit path
The .ots file is a serialized recipe of cryptographic operations (concatenations and hashes) rather than a simple digital signature. Using the OpenTimestamps CLI:
npx --package=opentimestamps ots-cli.js info v1.ots
The output reveals the entire path connecting the document to Bitcoin consensus:
File sha256 hash: 16efa7abadaf4f86969bacbf0f7db2acce22d88927760d01d82b13e71903281b
Timestamp:
append 432c9ad4330a567a
sha256
prepend c81dd1e2e31c1f8487185dc055accd5c0fe38a368ddb8978b664fb789ec08ed5
sha256
...
# transaction id 8a45961925c0a76f953fa9c4c741bfdf3d46074341a94d129c87ebfc6e28254f
...
verify BitcoinBlockHeaderAttestation(975431)
# Bitcoin block merkle root affcd985cdea192f1fba9fa3cfb3be5b00b8ac359552aa91685ebff7882fbb72
Notice the key anchor points in this recipe:
- The file hash: Matches the digest calculated in Step 1.
- The operations: Each
appendandprependprovides the "sibling hash" needed to climb up the calendar's Merkle tree without needing to download anyone else's documents. - The transaction ID: Identifies the Bitcoin transaction containing the calendar's commitment.
- The Bitcoin block header: Confirms the block height (
#975431) and the block's transaction Merkle root.
3. Why inclusion in the Merkle root cannot be faked
A natural question is: How does an auditor know their specific hash was truly included in that Bitcoin transaction root?
The answer lies in the collision and pre-image resistance of SHA-256:
- The verifier starts with the document hash $H_0$ and applies each operation in sequence: $$H_1 = \text{SHA256}(H_0 \mathbin{\Vert} \text{sibling}_1)$$ $$H_2 = \text{SHA256}(\text{sibling}_2 \mathbin{\Vert} H_1)$$
- If an attacker changed even a single byte or punctuation mark in the document, the initial hash would completely change (the avalanche effect). Every subsequent intermediate hash would diverge, and the computed root would fail to match the Bitcoin transaction.
- Because SHA-256 cannot be reversed, it is computationally impossible (requiring on the order of $2^{128}$ to $2^{256}$ operations) to find any other document that evaluates to the same root through that operation sequence.
4. Audit on an independent block explorer
To verify without any specialized OpenTimestamps tools, open any independent public Bitcoin explorer (such as mempool.space or blockstream.info):
- Search for the transaction ID from the proof:
https://mempool.space/tx/8a45961925c0a76f953fa9c4c741bfdf3d46074341a94d129c87ebfc6e28254f - Inspect the outputs: an
OP_RETURNscript contains the exact 32-byte Merkle root output by your proof's calendar branch. - Open the block header for that transaction:
https://mempool.space/block/975431 - Verify that the block's
Merkle Rootmatches the final hash computed by the proof (affcd985cdea...).
5. Automated zero-trust verification
To perform this entire verification in one automated step against your own local Bitcoin full node (bitcoind):
npx --package=opentimestamps ots-cli.js verify \
-b http://user:pass@127.0.0.1:8332 \
v1.ots -f my-document.md
Alternatively, anyone can drag-and-drop the markdown file and .ots proof into the web verifier at opentimestamps.org to run the same cryptographic validation in their browser.
Crucially, because the author's identity is declared directly in the frontmatter (author: Giovanni Filardo) and committed into the raw hashed bytes, verification requires only a single command. The auditor does not need to retrieve a public key from an external keyserver, import an X.509 certificate, or run a detached signature check. The text, metadata, and attribution are all verified simultaneously against Bitcoin consensus.
The auditor does not need to trust the author, the host web server, or any commercial certificate authority. The proof is mathematically self-evident, anchored to the energy and consensus of the Bitcoin network.
Conclusion
By treating decentralized timestamps as generic cryptographic anchors, we achieve a reliable, low-overhead method for proving digital priority:
- Decouple authorship from financial transactions: Use asymmetric key signatures or Git commit provenance to prove identity without managing cryptocurrency wallets or paying gas fees.
- Isolate build systems from asynchronous latency: Handle stamping and upgrading as out-of-band operational cycles, keeping build pipelines deterministic, fast, and offline.
- Anchor exact bytes and version revisions: Stamp exact immutable files or Git commits to avoid fragile normalization, and preserve historical proofs across revisions.
- Enable trustless verification: Provide detached proofs that any third party can verify independently using open-source tools.