Intermittent Segaults with PNG Inputs on Windows builds w/ OpenCV 5.0

[Breakthrough]

Description

I've seen the Windows build flaking at a ~2% rate, and think it's started with OpenCV 5.0. I don't see any source changes that might explain this, however the issue seems to be in the prebuilt FFmpeg binary that is included in the package. Example: https://github.com/Breakthrough/PySceneDetect/actions/runs/35548340704/job/106178294407

D:\a\_temp\02bf8af7-1216-404a-bfb2-8f1ba7760c24.sh: line 2:   660 Segmentation fault         python -X dev -m pytest -vv
pytest exit code: 139

We should file an upstream bug for this in the actual OpenCV repo once we can get a more reliable reproduction.

Reproduction

On release mode it happens less frequently, but more often when running the process with a debugger attached. In production, the crash rate seems to depend on the prior state of the world (specifically, which video was decoded first). The crash also seems to only occur when OPENCV_LOG_LEVEL is set to ERROR.

Workarounds

• set OPENCV_LOG_LEVEL=FATAL
• Set the capture backend explicitly, and use cv2.CAP_IMAGES for PNGs. By default, it uses cv2.CAP_ANY which tries ffmpeg first, loading it into the process.

Tasks

• Add workaround (81c414c)
• Consider adding an argument to the VideoStreamCv2 constructor to allow explicitly setting the OpenCV capture API, can mirror upstream default of cv2.CAP_ANY. (Alternatively: should we allow **kwargs instead?)
• Create self contained reproduction and file upstream report: 5.x: intermittent opencv-python crash on process exit after VideoCapture fails to find a decoder (regression in 4.14.0) opencv/opencv#30017
• Monitor CI runs for the next few weeks and see if the issue reoccurs (verify 81c414c worked)

[Breakthrough]

Reproduction:

"""opencv-python 5.0.0.93 on Windows crashes intermittently with 0xC0000005 at exit.
The issue occurs after cv2.VideoCapture allows the FFmpeg plugin to probe PNG input.
4.13.0.92 does not seem to be affected.

Reproduction:

    pip install opencv-python==5.0.0.93
    python opencv_exit_crash_repro.py

Each trial is a fresh process that decodes a few video frames, opens a PNG sequence, reads a
frame, and exits normally. The crash is intermittent (about 1 in 5 runs), so we run many trials.

Setting OPENCV_LOG_LEVEL=FATAL makes the crash go away.
"""

import os
import subprocess
import sys
import tempfile

import cv2
import numpy as np

TRIALS = 40


def make_media(folder: str):
    writer = cv2.VideoWriter(
        os.path.join(folder, "video.mp4"), cv2.VideoWriter_fourcc(*"mp4v"), 25.0, (640, 480)
    )
    rng = np.random.default_rng(0)
    for i in range(50):
        frame = rng.integers(0, 255, (480, 640, 3), dtype=np.uint8)
        writer.write(frame)
        if i < 10:
            cv2.imwrite(os.path.join(folder, f"frame{i + 1:03d}.png"), frame)
    writer.release()


def trial(folder: str, api: int):
    video = cv2.VideoCapture(os.path.join(folder, "video.mp4"))
    for _ in range(5):
        assert video.read()[0]
    # With CAP_ANY the FFmpeg plugin probes the PNGs first and fails ("Could not find decoder
    # for codec_id=61") before CV_IMAGES opens them. That failed probe is what causes the crash.
    images = cv2.VideoCapture(os.path.join(folder, "frame%03d.png"), api)
    assert images.read()[0] and images.getBackendName() == "CV_IMAGES"
    # Exit normally. The crash happens after this point, while the plugin DLL unloads.


if __name__ == "__main__":
    if len(sys.argv) == 3:
        trial(sys.argv[1], int(sys.argv[2]))
        sys.exit(0)
    print(f"opencv-python {cv2.__version__}, python {sys.version.split()[0]}")
    with tempfile.TemporaryDirectory() as folder:
        make_media(folder)
        for name, api in (("CAP_ANY (default)", cv2.CAP_ANY), ("CAP_IMAGES", cv2.CAP_IMAGES)):
            codes = [
                subprocess.run(
                    [sys.executable, __file__, folder, str(api)], stderr=subprocess.DEVNULL
                ).returncode
                for _ in range(TRIALS)
            ]
            crashes = sum(code & 0xFFFFFFFF == 0xC0000005 for code in codes)
            other = sum(code != 0 for code in codes) - crashes
            print(f"{name:18s}: {crashes}/{TRIALS} crashed at exit, {other} other failures")