Regression? Spurious error? in insert_pdf in v1.25.4

[C-Saunders]

Description of the bug

With v1.25.4, calling insert_pdf sometimes causes an IndexError to be raised, but the resulting object seems fine. If insert_pdf is called a second time with the same source doc, no error is raised.

With v1.25.3 I do not see this error.

How to reproduce the bug

import pymupdf

doc = pymupdf.open("...pdf")
new_doc = pymupdf.open()
new_doc.insert_pdf(doc, from_page=1, to_page=len(doc)-1) # raises IndexError: range object index out of range

len(new_doc) # has the correct number of pages
new_doc.save(...) # saved doc looks fine

import pymupdf

doc = pymupdf.open("...pdf")
new_doc = pymupdf.open()
new_doc.insert_pdf(doc, from_page=1, to_page=1) # raises IndexError: range object index out of range

len(new_doc) # == 1

new_doc_2 = pymupdf.open()
new_doc_2.insert_pdf(doc, from_page=1, to_page=1) # does not raise this time

Unfortunately I've been unable to find a PII-free example to share, but I hoped there might be enough detail here to work from. If I'm able to find a good reproduction I will share it, though!

PyMuPDF version

1.25.4

Operating system

MacOS

Python version

3.11

[JorjMcKie]

If I'm able to find a good reproduction I will share it, though!

Yes, please do. Without this, there is not much hope.
As a preliminary information: are the documents (source or target) Form PDFs (i.e. have widgets)?

[C-Saunders]

Yes, the source documents have forms. The destinations are new/empty, exactly like in my example code.

In one example, I have

• first page has form
• second page does not

No error:

doc = pymupdf.open("source.pdf")

new_doc = pymupdf.open()
new_doc.insert_pdf(doc, from_page=0, to_page=0)

new_doc_2 = pymupdf.open()
new_doc_2.insert_pdf(doc, from_page=1, to_page=1)

However this raises the error in v1.25.4, but not in v1.25.3:

doc = pymupdf.open("source.pdf")

new_doc = pymupdf.open()
new_doc.insert_pdf(doc, from_page=1, to_page=1)

[JorjMcKie]

Please provide me with the full error message so I can look at the statement number in the source code.

[JorjMcKie]

However this raises the error in v1.25.4, but not in v1.25.3

This is a matter of course: Support for PDF form fields has been added to the method in version 1.25.4.
To avoid the error altogether specify doc.insert_pdf(..., widgets=False).
Obviously, widgets will then be ignored as it was the case in prior versions.

[evgenii-optifino]

Ignoring widgets helped to me

[JorjMcKie]

@C-Saunders / @evgenii-optifino Please let me have with the full exception message so I can trace back in the source code.

Note: Merging the files did work in any case. Taking care of the widgets in source pages is an internal final step which does any required adjustments in the resulting target document. Failures here do no harm to the merged target.
The error seems to occur when the source is a Form PDF but copied pages happen to not contain any widgets.

[C-Saunders]

@JorjMcKie sorry for the delay here. GitHub didn't notify me about your first reply

File <application code>
  subset_pdf.insert_pdf(
File ".../.venv/lib/python3.11/site-packages/pymupdf/__init__.py", line 4668, in insert_pdf
  self._do_widgets(docsrc, _gmap, from_page=fp, to_page=tp, start_at=sa, join_duplicates=join_duplicates)
File ".../.venv/lib/python3.11/site-packages/pymupdf/utils.py", line 1916, in do_widgets
  src_page = src[src_range[i]]
                  ~~~~~~~~~^^^
IndexError: range object index out of range

[JorjMcKie]

This is just a typo. Should be src_page = src[i].

[slarrain]

Same bug here. Dowgrading to 1.25.3 until new version comes out. Thanks for the fix.

[JorjMcKie]

Same bug here. Dowgrading to 1.25.3 until new version comes out. Thanks for the fix.

The appropriate fallback would be just setting widgets=False.

[jorritfolmer]

Here's a quick test with accompanying in-the-wild PDF to catch this. I'm sure it can be a lot smarter so I'm posting it here instead of creating a PR.

diff --git a/tests/test_4412.py b/tests/test_4412.py
new file mode 100644
index 00000000..fa47dbb4
--- /dev/null
+++ b/tests/test_4412.py
@@ -0,0 +1,31 @@
+import io
+import os
+import pymupdf
+
+scriptdir = os.path.abspath(os.path.dirname(__file__))
+resources = os.path.join(scriptdir, "resources")
+
+def test_4412_1():
+    # This tests whether a page from a PDF containing widgets found in the wild
+    # can be inserted into a new document with default options
+    file_path = os.path.join(resources, "4412.pdf")
+    doc = pymupdf.open(file_path)
+    new_doc = pymupdf.open()
+    buf = io.BytesIO()
+    new_doc.insert_pdf(doc, from_page=106, to_page=106)
+    new_doc.save(buf)
+    assert len(new_doc)==1
+    new_doc.close()
+
+def test_4412_2():
+    # This tests whether a page from a PDF containing widgets found in the wild
+    # can be inserted into a new document with widgets=False
+    file_path = os.path.join(resources, "4412.pdf")
+    doc = pymupdf.open(file_path)
+    new_doc = pymupdf.open()
+    buf = io.BytesIO()
+    new_doc.insert_pdf(doc, from_page=106, to_page=106, widgets=False)
+    new_doc.save(buf)
+    assert len(new_doc)==1
+    new_doc.close()

4412.pdf