gh-157742: Check for signals every 64th iteration in the int arithmetic loops

[eendebakpt]

x_mul() checks for signals on every row, x_divrem() on every quotient digit and the decimal string conversion on every digit. PyErr_CheckSignals() costs about as much as a hundred digit operations, so for small operands the checks cost more than the arithmetic.

SIGCHECK() now takes the loop index and checks on every 64th iteration, the same pattern _sre uses.

Benchmark	main	PR
a * b (10 x 10 digits)	159 ns	97.2 ns: 1.64x faster
a * b (60 x 60 digits)	2.61 us	2.22 us: 1.18x faster
a * b (500 x 500 digits)	99.1 us	79.7 us: 1.24x faster
a // b (1000 // 2 digits)	13.8 us	10.3 us: 1.34x faster
a // b (1000 // 500 digits)	204 us	not significant
pow(3, 5000)	10.7 us	8.38 us: 1.28x faster
math.factorial(2000)	97.7 us	81.6 us: 1.20x faster
str(7**1000)	8.07 us	7.34 us: 1.10x faster

Benchmark script

import pyperf

runner = pyperf.Runner()
def ints(na, nb):
    return (f"import random; r = random.Random(1); "
            f"a = r.getrandbits({na*30-1}) | (1 << {na*30-2}); "
            f"b = r.getrandbits({nb*30-1}) | (1 << {nb*30-2})")
runner.timeit("a * b (10 x 10 digits)", "a * b", setup=ints(10, 10))
runner.timeit("a * b (60 x 60 digits)", "a * b", setup=ints(60, 60))
runner.timeit("a * b (500 x 500 digits)", "a * b", setup=ints(500, 500))
runner.timeit("a // b (1000 // 2 digits)", "a // b", setup=ints(1000, 2))
runner.timeit("a // b (1000 // 500 digits)", "a // b", setup=ints(1000, 500))
runner.timeit("pow(3, 5000)", "pow(3, 5000)")
runner.timeit("math.factorial(2000)", "factorial(2000)",
              setup="from math import factorial")
runner.timeit("str(7**1000)", "str(x)", setup="x = 7**1000")

Generated with Claude Code

• Issue: Improve performance of pidigits and int arithmetic #157742

[skirpichev]

Could you test pidigits on this pr?

[eendebakpt]

Could you test pidigits on this pr?

pidigits is in the noise (although on a quiet machine one can measure the improvement I believe). With the same script as above one can see that also the multiplication of 64-bits ints gains.

Benchmark	main (run 1 / run 2)	PR (run 1 / run 2)	Result
a * b (1 x 2 digits)	25.3 / 25.2 ns	23.2 / 23.2 ns	1.09x faster
a * b (2 x 1 digits)	25.5 / 25.7 ns	22.9 / 22.8 ns	1.12x faster
a * b (3 x 1 digits)	25.1 / 25.5 ns	23.4 / 23.3 ns	1.08x faster
pidigits	155 / 155 ms	155 / 155 ms	not significant

[skirpichev]

LGTM

[skirpichev]

This affects also string conversion.

[eendebakpt]

Yes, but the gain is not really worth mentioning. For small number of digits the overhead dominates, for large number of digits the bigint algorithm in _pylong takes over.

[skirpichev]

On your benchmarks: 1.10x speedup, no? Looks too big for noise.

[eendebakpt]

I benchmarked again on another system, not a factor 1.04 so it is not noise. But conversion to string is mostly relevant for small strings and the gain there very small (due to construction of the str object), For very large objects another code path is used (_pylong).

[picnixz]

For systems under duress, how long should we wait when doing a ctrl+c for Python to exit properly (is it visible?)

Note to myself and for others in general but did we audit other parts of the code where we spent long calculations without checking for signals? I never really considered this problem but I wonder if hashlib in general should not check that when doing hashing per large blocks. I never worked with the check signal API so I do not know when this applies as well (if we are not holding the GIL should we check for signals?)

[eendebakpt]

Note to myself and for others in general but did we audit other parts of the code where we spent long calculations without checking for signals? I never really considered this problem but I wonder if hashlib in general should not check that when doing hashing per large blocks. I never worked with the check signal API so I do not know when this applies as well (if we are not holding the GIL should we check for signals?)

The signal checks are not done everywhere consistently.

It is not present in x_add (but does not really need it either, it linear in the input). For math.gcd it is not there, so on my system math.gcd(100k digits) is blocked for 25 seconds (both main and the PR).

For x_mul the situation is more complex: for normal multiplications the number of signal checks is too high (hence this PR). For a 60 x 60 digit multiplication on main we have a check every 70 ns, with the first version of the PR no checks, for the entire computation of 3 us, which seems reasonable. For a larger number of digits the karatsuba path is followed, which does not have signal checks inside k_mul. karatsuba calls multiplications for smaller sizes, but whether they hit the i==63 check in this PR depends on the size. For a = (1 << 6_000_000) - 1 and b = a - 1 we have a computation of 3.4 seconds, with no signal checks from this PR.

I updated the PR to add a signal check to k_mul itself. That helps for karatsuba and some other cases as well. On main pow(a, 2**300000, m), m = 60 digits was blocking for 3 seconds, in the PR for only 2 ms.

A parallel approach is to improve the performance of the signal check itself. That is done in #157748.