coreutils
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[PATCH] split: add content-defined chunking to --bytes


From: Leonid Evdokimov
Subject: [PATCH] split: add content-defined chunking to --bytes
Date: Mon, 2 Mar 2026 15:32:10 +0300

Hello,

Here is the patch to implement CDC in `split --bytes`. I'm submitting
it for review before proceeding with adding CDC to --line-bytes.

I've tested the patch on x86_64, ppc64be and Apple M1, with gcc and clang.

Texinfo documentation is currently missing.

I'm mostly unsure about the following:

1) autoconf and l10n logic being right, as I'm not familiar with AC/AM
and gettext.

2) embedding PCG RNG into make-buz-table. Is there a better way to
accomplish the goal and is it a better way needed?

3) licensing/authorship headers. There might be guidelines I'm missing.

4) right place for getcachelinesize(). Should it be a separate file
and/or part of gnulib?

5) busy-loop of randperm_new() on random-source being stream of 0xFF.
On one hand, that's a "bug" in randint_choose() and randpem_bound(),
on the other hand - one may say that it's just a foot-shooting case.

6) moving `+1` byte allocation to be specific for lines_split(). I've
not run asan build to test correctness. +1 is there for 35 years and,
seems, lines_split() is the only user of that extra byte, but my eye
might miss something.

7) 40 MiB limit for 32-bit CDC hashes, it's tempting to say "42 MB".
Should we? :-)

I've tried to add enough comments to make the code easy to understand,
but I can add more if that's helpful as the memory is still fresh.

The patch patch is also available at github:
https://github.com/coreutils/coreutils/compare/master...darkk:coreutils:cdc

-- 
WBRBW, Leonid Evdokimov, https://darkk.net.ru tel:+79816800702
PGP: 6691 DE6B 4CCD C1C1 76A0  0D4A E1F2 A980 7F50 FAB2

Attachment: 0001-split-add-content-defined-chunking-to-bytes-SIZE.patch
Description: Text Data

Attachment: 0002-maint-split-drop-duplicated-code.patch
Description: Text Data


reply via email to

[Prev in Thread] Current Thread [Next in Thread]