Skip to content

CI: build the bundled libgit2 in Release mode for wheels - #1497

Open
OrHayat wants to merge 3 commits into
libgit2:masterfrom
OrHayat:ci/release-build-for-wheels
Open

OrHayat wants to merge 3 commits into
libgit2:masterfrom
OrHayat:ci/release-build-for-wheels

Conversation

@OrHayat

@OrHayat OrHayat commented Sep 23, 2026

Copy link
Copy Markdown

The libgit2 bundled in the published wheels is built unoptimized, so every non-Windows wheel on PyPI is ~2x slower than it needs to be.
The same issue was in windows build and it was fixed (098cbe3, Oct 2025): build.ps1 now passes cmake --build --config=Release, so the shipped git2.dll is optimized.

OrHayat and others added 2 commits September 23, 2026 11:42
build.sh defaults BUILD_TYPE to Debug and neither cibuildwheel environment set
it, so every wheel built through it shipped an unoptimized libgit2.
build.sh reuses a cached ci/ when the three version numbers in versions.txt
match, so a cached Debug build satisfies a Release build and the rebuild is
skipped.
@jdavid

jdavid commented Sep 23, 2026

Copy link
Copy Markdown
Member

Thanks, do you want to do the same for libssh2?

@OrHayat

OrHayat commented Sep 23, 2026

Copy link
Copy Markdown
Author

done
all was found when i wrote pretty complex python script that tried to find branches that were squashed merged
by comparing patch-ids.
it was MUCH slower. then regular git
after debugging i found those 3 "issues"

Problem Where Effect on the script
GIL held during both calls pygit2 tree.c, diff.c 8 threads = 1.06x, thread pool useless
Unoptimized (-O0) libgit2 in wheel pygit2 build.sh (PR 1497) every diff ~2x slower
Tree diff doesn't skip unchanged dirs libgit2 (PR 6863) cost = repo size, not change size
Setup 1 thread 8 threads Speedup
pygit2 stock, Python 3.14 (GIL) 2.43s 2.31s 1.06x
pygit2 + GIL release, 3.14 (GIL) 2.25s 0.48s 4.66x
pygit2 stock, Python 3.14t (no GIL) 2.22s 0.48s 4.61x

3.14t experiment worked but it. probably need changes like add lock per repository or sometihng like that
or checking the code of libgit2 and making IT thread safe

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants