Re: [PATCH] tests: protect ulimit -v determination with a timeout
From:
Pádraig Brady
Subject:
Re: [PATCH] tests: protect ulimit -v determination with a timeout
Date:
Wed, 15 Jan 2025 15:55:21 +0000
User-agent:
Mozilla Thunderbird Beta
On 15/01/2025 11:59, Pádraig Brady wrote:
On 15/01/2025 07:33, Bernhard Voelker wrote:
Hi Padraig,
On 1/15/25 02:36, Pádraig Brady wrote:
This avoids a test hang seen with bash on Solaris 11
+ # Wrap with timeout as on some systems (like Solaris 11)
+ # fork() can return EAGAIN under vm constraints,
+ # causing bash at least to loop retrying the fork
ulimit_supported is mainly used by get_min_ulimit_v_ for a given
coreutils tool, while the above now uses $SHELL in between:
isnt't that usually much larger than our tools, and hence the
resulting minimum ulimit_v will be wrong then?
A valid consideration, but I think that should be OK
as the `ulimit -v` is only applied to the last subshell,
and so shouldn't behave significantly differently.
I.e. we're doing, timeout -> $SHELL -> ulimit; not
ulimit -> timeout -> $SHELL
Actually you're right, but in the opposite way :)
With the $SHELL invocation we get a clean shell
and thus the limits returned can be too _small_.
In testing with the `timeout $SHELL` the limit returned was 3000,
while with the original ulimit in a subshell it was around 5000.
So we can't use the timeout $SHELL mechanism really I think.
Now bash doesn't actually loop forever in this constrained case.
It uses an increasing backoff before retrying the fork, and
limits the retries to 5, which adds up to about 15s,
which isn't much worse than the 10s timeout anyway.
I also noticed that the `trap '' SEGV` _or_ the `ulimit -c 0`
are required to trigger this on bash 5.2.32(1)-release (sparc-sun-solaris2.11):
$ date; (ulimit -c 0; ulimit -v 3000 && src/cat --version); date
Wednesday, January 15, 2025 at 2:43:54 PM CET
-bash: fork: retry: Resource temporarily unavailable
-bash: fork: retry: Resource temporarily unavailable
-bash: fork: retry: Resource temporarily unavailable
-bash: fork: retry: Resource temporarily unavailable
-bash: fork: Resource temporarily unavailable
Terminated
Wednesday, January 15, 2025 at 2:44:09 PM CET
$ date; (ulimit -v 3000 && src/cat --version); date
Wednesday, January 15, 2025 at 2:44:17 PM CET
ld.so.1: cat: fatal: /lib/64/libc.so.1: Cannot allocate memory
Killed
Wednesday, January 15, 2025 at 2:44:17 PM CET
In fact once any extra command is used it adds the extra resource requirement:
$ (true; ulimit -v 12000 && src/cat --version)
-bash: fork: retry: Resource temporarily unavailable
$ (ulimit -v 4000 && src/cat --version)
cat (GNU coreutils) 9.5.239-a836
Now it's tempting to merge the ulimit calls like:
$ (ulimit -c 0 -v 4000 && src/cat --version)
cat (GNU coreutils) 9.5.239-a83
but then we might underestimate, and we do have usage in scripts of
(something && ulimit -v $vm && ...), so it's probably best to
leave that as is.
Looking at the bash code, it hardcodes the retries to taking about 16 seconds,
so I'm thinking our only recourse is to bail if we detect this slow case.
The attached does that, and I've verified it runs fine on Solaris 11 and Fedora
41 at least.
cheers,
Pádraig