Re: [PATCH] split: add content-defined chunking to --bytes
From:
Leonid Evdokimov
Subject:
Re: [PATCH] split: add content-defined chunking to --bytes
Date:
Fri, 1 May 2026 18:30:21 +0300
On Mon, Apr 27, 2026 at 11:33:32AM +0100, Pádraig Brady wrote:
> > Question #1: Does it make sense to add the CDmv patch to split(1),
> > or is this becoming too specific to the Git use case?
> >
> > My intuition is that it may be too specialized, but I may be mistaken.
> > All in all, I've originally seen CDC from a Git-specific angle as well.
>
> Yes it's probably best to not include CDmv in split,
> but it would be worth documenting in info as a possibility.
I've missed that "external" CDmv is impossible to implement accurately
with --line-bytes as the end of chunk for --line-bytes is "the first
SEPARATOR after the CDC cut-point". So, the chunk filename based on
$(tail -c $WINDOW line-bytes-chunk) is not necessary "stable" across
file modifications. It's still statistically okay-ish,
but the guarantee is slightly weaker than I originally thought.
However, "external" CDmv idea is absolutely fine with --bytes
as the cut-point is right after CDC window.
I don't think it's the reason to reconsider the decision on CDmv not
belonging to split(1). I'm just highlighting my incorrect assumption.
> Copyright assignment has stalled,
> but will hopefully resume after new emails sent.
🙏*
*) just testing Unicode handling :-)
--
WBRBW, Leonid Evdokimov, https://darkk.net.ru tel:+79816800702
PGP: 6691 DE6B 4CCD C1C1 76A0 0D4A E1F2 A980 7F50 FAB2