[Python] How come the performance of numpy.linalg.solve has changed drastically between...

I noticed that numpy.linalg.solve has vastly different performance since v2.0.0, on both Linux and Windows (haven't tested it on macOS). Basically, for reasonable matrices (up to about 1800x1800) the new versions are much faster (up to 10x faster of my Linux machine, I got even larger speedups on Windows), then for larger ones (up to about 8800x8800) the 1.26.4 version is faster, and for even larger ones the 2.0.0 version again becomes faster.

I wonder what happened? Is that due to an update of the underlying library?

If there were some modifications that impacted performance, I would have expected something to appear in the release notes, but there wasn't anything of the sort.

I ran a small benchmark using perfplot:

import matplotlib.pyplot as plt
import numpy as np
import perfplot


def main():
plot = False

if not plot:
out = perfplot.bench(
setup=lambda n: np.random.rand(n + 1, n),
kernels=[lambda a: np.linalg.solve(a[1:], a[0])],
n_range=[2**k for k in range(15)],
)

np.savetxt("n.txt", out.n_range)
np.savetxt(f"timings_{np.__version__.replace(".", "_")}.txt", out.timings_s)
else:
n = np.loadtxt("n.txt")
timings_1_26_4 = np.loadtxt("timings_1_26_4.txt")
timings_2_0_0 = np.loadtxt("timings_2_0_0.txt")

plt.plot(n, timings_1_26_4, label="1.26.4")
plt.plot(n, timings_2_0_0, label="2.0.0")
plt.legend()
plt.show()

plt.plot(n, timings_1_26_4 / timings_2_0_0, label="1.26.4 / 2.0.0")
plt.plot([0, 2**14], [1.0, 1.0])
plt.legend()
plt.show()


if __name__ == "__main__":
main()


I ran it once with numpy==1.26.4 and once with numpy==2.0.0, and then a third time with plot=True to just plot.

Here is the overall result (x-axis is number of unknowns of the linear system, y-axis is time it takes to solve in seconds): Performance comparison of numpy.linalg.solve between v1.26.4 and v2.0.0.

Here is a zoom up to n=200: Zoom on the beginning of the plot.

And here is the ratio time for 1.26.4 / time for 2.0.0 (the horizontal line is y=1 to see which is faster): Ratio of the timings for versions 1.26.4 and 2.0.0. When the curve is above the horizontal line, it means that v2.0.0 is faster.

In the last image, one can see that there is a peak at about n=35 where we have 10x.

Continue reading...