Memory leaking in rewrite_images?

[kuhnroyal]

Description of the bug

I noticed memory building up in my application and traced this to the rewrite_images call, if I remove this, everything is fine.
Not sure if I miss some cleanup? I tried pymupdf.TOOLS.store_shrink(100) but that didn't help.

How to reproduce the bug

import gc
from pathlib import Path

import pymupdf


def main() -> None:
    dpi_threshold: int = 150
    dpi_target: int = 100
    quality: int = 50

    input_pdf: Path = Path("input.pdf")
    output_pdf: Path = Path("output.pdf")

    for i in range(10):
        with pymupdf.open(input_pdf) as doc:
            doc.rewrite_images(
                dpi_threshold=dpi_threshold,
                dpi_target=dpi_target,
                quality=quality,
            )

            save_opts = {
                "garbage": 4,  # Maximum garbage collection
                "deflate": True,  # Use deflate compression
                "clean": True,  # Clean up redundant objects
                "pretty": False,  # Don't pretty-print (saves space)
                "ascii": False,  # Don't use ASCII encoding (saves space)
                "expand": 0,  # Don't expand content streams
                "linear": False,  # Don't linearize (can increase size)
                "deflate_images": True,  # Compress images
                "deflate_fonts": True,  # Compress fonts
                "use_objstms": True,  # Use object streams for better compression
                "compression_effort": True,  # Use maximum compression effort
            }

            doc.save(
                output_pdf,
                **save_opts,
            )
            print(f"Done {i}")

    gc.collect()
    pymupdf.TOOLS.store_shrink(1000)
    print("Cleaned")


if __name__ == "__main__":
    main()

My test PDF has 6 pages with full screen images.
I created a memray graph: memray-flamegraph-mem_test.py.99494.html

PyMuPDF version

1.26.4

Operating system

MacOS

Python version

3.13

[JorjMcKie]

Most of this is probably inevitable considering what is happening under the hood when potentially evry single image (large objects!) is being rewritten.
You should execute gc.collect() after shrinking the store.
Compression effort maximum is 100 BTW.

[JorjMcKie]

I made a test using 10 different PDFs and can confirm your observation.
Using this script:

import gc
import os
import sys
import time

import pymupdf
from pymupdf import mupdf
import psutil
import glob

print(f"{pymupdf.version=}")
pymupdf.TOOLS.mupdf_display_errors(False)
"""
Test for memory leaks using PyMuPDF rewrite_images.
"""
gc.set_debug(gc.DEBUG_UNCOLLECTABLE)

files = glob.glob("*.pdf")  # Loop
print("Memory deltas (MB)")
count = len(files)
memory_usage_start = [0] * count
memory_usage_before_gc = [0] * count
memory_usage_stop = [0] * count
dpi_threshold: int = 150
dpi_target: int = 100
quality: int = 50
MB_VALUE = 2**20
t0 = time.time()
for i in range(count):

    # Record initial memory usage
    process = psutil.Process(os.getpid())
    memory_usage_start[i] = process.memory_info().rss / MB_VALUE

    # Load file
    doc = pymupdf.open(files[i])
    doc.rewrite_images(
        dpi_threshold=dpi_threshold,
        dpi_target=dpi_target,
        quality=quality,
    )
    _ = doc.tobytes(garbage=3, deflate=True)
    _ = None
    doc.close()
    doc = None
    mupdf.fz_empty_store()
    print(f"Done {files[i]}")

    # Record memory usage before cleanup
    memory_usage_before_gc[i] = process.memory_info().rss / MB_VALUE
    gc.collect()

    # Record mem final usage
    memory_usage_stop[i] = process.memory_info().rss / MB_VALUE

for i in range(count):
    stop = memory_usage_stop[i]
    start = memory_usage_start[0] if i == 0 else memory_usage_stop[i - 1]
    print(f"memory delta round {i}: {round(stop - start)}")

t1 = time.time()
print("=" * 50)
print(f"Duration: {round(t1 - t0)} sec")

This is the output (Windows 11):

py gc-test.py
pymupdf.version=('1.26.5', '1.26.7', None)
Memory deltas (MB)
Done sdw_2025_1.pdf
Done sdw_2025_10.pdf
Done sdw_2025_2.pdf
Done sdw_2025_3.pdf
Done sdw_2025_4.pdf
Done sdw_2025_5.pdf
Done sdw_2025_6.pdf
Done sdw_2025_7.pdf
Done sdw_2025_8.pdf
Done sdw_2025_9.pdf
memory delta round 0: 259
memory delta round 1: 189
memory delta round 2: 244
memory delta round 3: 190
memory delta round 4: 169
memory delta round 5: 245
memory delta round 6: 182
memory delta round 7: 193
memory delta round 8: 212
memory delta round 9: 202
==================================================
Duration: 147 sec

Unsurprisingly, it is the base library MuPDF that does the image rewriting - not PyMuPDF.
So I am going to file a MuPDF issue.
I am afraid however that this will not receive top priority as circumventing the problem is easy.
Keep you posted.

[JorjMcKie]

MuPDF issue link https://bugs.ghostscript.com/show_bug.cgi?id=708874

[kuhnroyal]

@JorjMcKie Thanks for looking into this. How do you think I can circumvent this? Running GC didn't help much.

[JorjMcKie]

Instead of insisting on rewriting images in the same script, simply execute this as a subprocess, and either wait for ech subprocess to end before you start the image rewriter again with the next PDF ... or even start multiple subprocesses in parallel.

[julian-smith-artifex-com]

The upstream bug has been fixed in mupdf's master and 1.26.x release branches. So it will be fixed in our next release.

We will close this issue when we have released the fix.

[kuhnroyal]

Awesome, thanks!