Forgetting, in a system built on remembering

Everlock is built around a simple promise:

History is part of the storage.

A write does not silently replace the previous value.

It creates another commit.

That makes it possible to inspect how something changed, read an older version, replicate the history, or recover from many kinds of mistakes without adding a separate versioning system.

It also creates a problem that every history-preserving system eventually has to answer.

What happens when the history becomes too large to keep forever?

For Everlock, retention is awkward precisely because it pushes against one of the reasons Git was chosen in the first place.

Growing forever is still a policy

If the repository only ever grows, then the retention policy is effectively:

keep everything forever

That may be fine for a while.

It may even be the right default for some stores.

But a busy site, mail store, file share, or other long-lived repository accumulates data.

Eventually the difference between "we have not implemented retention" and "we intentionally retain everything" matters operationally.

The difficult part is that the obvious cleanup technique can destroy another property Everlock cares about.

Stable history.

Rewriting history solves one problem by creating another

Suppose a repository has:

A -> B -> C -> D -> E

and a retention policy decides that only D and E should remain visible.

One option is to rewrite the retained history onto a new root:

D' -> E'

The old commits can then become unreachable and eventually disappear.

This gives a clean new repository.

It also changes commit ids.

The rewritten D' is not D.

The rewritten E' is not E.

For source code, rewriting history is already disruptive.

For Everlock, where repositories are application storage and may have been cloned, backed up, or replicated elsewhere, it is a much larger semantic change.

A backup from last month may now appear unrelated to the current repository even though it contains the same logical files.

That is too high a price for ordinary retention.

Shallow history keeps identities stable

The other model is to keep the existing commit ids and establish a boundary.

The repository still contains:

A -> B -> C -> D -> E

but clients are only served from a chosen point onward.

Conceptually:

A -> B -> C || D -> E
            ^
       shallow boundary

D remains D.

E remains E.

Existing clones still know the same commits.

The retained part of history has not been replayed into a new identity.

This is the direction Everlock chose.

Retention cuts history shallow rather than rewriting it.

The design is less powerful.

That is intentional.

Stable ids matter more than perfect cleanup

The important property of a Git commit is not just that it contains some state.

Its object id names that exact state and history.

Other repositories may refer to it.

Backups may contain it.

Logs may mention it.

A remote clone may have it as part of its own graph.

Once Everlock treats Git history as application history, those ids become part of the storage semantics.

Changing them to make retention tidier would mean the retention job can rewrite the identity of the past.

That is a dangerous capability to give routine maintenance.

So the policy becomes:

retention may hide old history
retention may not rename retained history

That is a useful constraint.

Shallow retention also has a cost

There is a capability that history rewriting gives you and shallow retention does not.

If old history has been dropped from the server's reachable view, the server cannot reconstruct objects it no longer has merely by moving the shallow boundary.

The missing objects have to come from somewhere.

Fortunately, Git's content-addressed model gives this a clean recovery path.

If an older clone still has the dropped commits, pushing those objects back restores the exact same objects.

The hashes have not changed.

The old history slots back into the graph it originally belonged to.

That is another reason stable ids are valuable.

Recovery is based on content identity rather than conversion.

The first design was much larger

The initial retention implementation had more machinery.

There was a truncation engine.

A configurable step size.

A backup namespace.

A retention mode selector.

A dry-run flag.

A manual command:

/git repo truncate

Each piece had a reason.

If a human can trigger a destructive operation, then previewing it is useful.

If several truncation strategies exist, a mode selector is needed.

If rewriting might go wrong, a backup namespace looks reassuring.

The pieces reinforce one another.

Then review removed the dangerous operation.

And most of the surrounding machinery became unnecessary with it.

Removing one command removed a system

The manual truncate command disappeared.

Without that command, the rewrite engine no longer had a caller.

Without rewriting, the backup namespace no longer had a purpose.

Without multiple retention modes, the mode enum disappeared.

Without a manual destructive operation, the dry-run plumbing became unnecessary.

Roughly 350 lines followed.

What remained was much smaller:

one retention window per repository
one scheduled maintenance job

Configuration became:

[repos.my-project]
retention = "90d"

That is intentionally boring.

Retention is policy, not an operator ritual.

Maintenance that has no decisions should be automatic

Garbage collection followed the same reasoning.

The original system had a command for it:

/git repo gc

But pruning unreachable objects is not usually a decision a human needs to make interactively.

If the repository is healthy, maintenance should happen.

If it is damaged, the system should identify the repository and fail loudly.

So the explicit command was removed and garbage collection became a daily job.

The same idea applies to retention.

If the policy says 90 days, the system does not need an administrator to periodically remember a command.

Configuration expresses the decision.

Automation executes it.

Inspection is different from action

Removing the manual truncation command does not remove the need to understand what retention will do.

That requirement simply belongs in a different operation.

Everlock keeps the ability to inspect the effect through:

/git repo size <name> retention=90d

That reports without acting.

This separation is useful:

configuration -> policy
scheduled job -> action
inspection    -> explanation

The previous design bundled explanation into a dangerous action through a dry-run flag.

The smaller design gives each concern its own place.

Configuration formats deserve long-term thinking

The retention work also exposed a small configuration-design choice.

A compact representation could have been:

[repos]
my-project = "90d"

Instead, Everlock uses a table per repository:

[repos.my-project]
retention = "90d"

Today that is more verbose.

But it means another repository-specific setting can be added later without changing the shape of the file.

For example:

[repos.my-project]
retention = "90d"

can naturally become:

[repos.my-project]
retention = "90d"
another_setting = "..."

without converting a scalar into a table.

The choice costs almost nothing now and avoids a format migration later.

Older binaries may meet newer configuration

Everlock also auto-updates, which creates another subtle compatibility case.

A binary may occasionally encounter configuration written by a newer version.

That means repository-specific fields should not make older versions refuse to start merely because they do not understand a setting they can safely ignore.

So configuration fields are optional and use Serde defaults.

The principle is simple:

unknown future setting
        |
        v
old binary keeps parsing

That does not solve every compatibility problem.

But it avoids turning harmless configuration growth into an unnecessary startup failure.

Retention contradicts the core promise

Features that extend an architecture are often easier than features that push against it.

Adding more history to a history-preserving system is easy.

Deleting history is not.

That is why retention deserves more design attention than a normal maintenance command.

The system should not quietly undermine its own strongest property just because disk usage eventually becomes inconvenient.

Everlock's answer is deliberately narrower than the most capable retention system it could have built.

No routine history rewriting.

No changing commit ids.

No operator-facing truncation machinery.

Just a retention window and shallow boundaries.

Less capability can be the safer feature

It is tempting to judge a design by how many modes it supports.

Rewrite mode.

Shallow mode.

Backup mode.

Manual mode.

Dry-run mode.

Each mode looks like flexibility.

But every additional destructive option increases the number of states operators and code have to reason about.

In this case, the strongest design decision was not adding a better truncation engine.

It was deleting the one that violated the property Everlock cared about most.

The result is less flexible.

It is also easier to explain:

Retention may shorten the visible history, but it does not rewrite the history we keep.

That is a much stronger invariant than a menu of strategies.

Forgetting should preserve the shape of what remains

Everlock begins with a system designed to remember.

Retention introduces forgetting.

The question is therefore not merely:

How do we remove old data?

It is:

How do we remove old data without changing the identity of the data we keep?

Shallow history provides that boundary.

The remaining commits stay the same.

Existing clones remain related.

Old backups continue to refer to the same objects.

And if dropped objects have to be restored from another copy, Git's content addressing gives them their old identity back.

That is a useful way to think about retention in any system built around immutable history:

Forget as little as necessary, and do not rewrite the memory you decided to keep.

For configuration details, see the retention reference.