ENH: capture and report build errors when rebuilding editable wheels

[dnicolodi]

When editable-verbose is not enabled, redirect the build output to a file. When the build fails, parse this file to look for the build error and append it to the ImportError exception message.

Fixes #820.

Replaces #750.

[rgommers]

I like the idea, and the implementation is nice and concise. A few comments around robustness (encoding, ninja errors, etc.)

[rgommers]

The first line that the break triggers on won't get printed here, right? Would be nice to capture that line as well.

[dnicolodi]

Leaving it out was a conscious decision. That line reads:

FAILED: [code={exitcode}] {target_name}

and I don't think it adds much information: the exit code is not relevant in the vast majority of the cases, and the target name is more often than not completely opaque to the user.

However, if you insist, I can include this line.

[lesteve]

To give an idea in a scikit-learn context, this line would look like this:

FAILED: [code=1] sklearn/tree/_splitter.cpython-314t-x86_64-linux-gnu.so.p/_splitter.c 

I am fine with not having it, since one of the next line points directly at the Cython file (which is more useful than the ninja target I think).

/home/lesteve/dev/scikit-learn/sklearn/tree/_splitter.pyx:846:13: Syntax error in simple statement list

[dnicolodi]

This should be ready to be merged, thus tentatively marking this to be included in the next release.

[dnicolodi]

@rgommers do you want to have another look at this?

[lesteve]

Hi there, thanks a lot @dnicolodi for the PR, this would be very helpful for scientific Python maintainers who are not necessarily meson-python or meson expert (my self included 😅).

I gave it a go with scikit-learn and I am a bit confused because I don't get the Cython complication error in the output. I am probably missing something and I will probably try again in a few days and possibly debug what is happening.

Here is what I am doing in the mean time:

• build scikit-learn in editable mode (without editable verbose) pip install --no-build-isolation --editable .
• Introduce a Cython compilation error in one of our .pyx
• then do `python -c 'import sklearn'. I am expecting a rebuild which fails and the Cython compilation error to show up in the output but it doesn't somehow:

❯ python -c 'import sklearn'                            
Traceback (most recent call last):
  File "/home/lesteve/micromamba/envs/scikit-learn-dev/lib/python3.14t/site-packages/_scikit_learn_editable_loader.py", line 346, in _rebuild
    subprocess.run(self._build_cmd, cwd=self._build_path, env=env, stdout=subprocess.DEVNULL, check=True)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lesteve/micromamba/envs/scikit-learn-dev/lib/python3.14t/subprocess.py", line 577, in run
    raise CalledProcessError(retcode, process.args,
                             output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/home/lesteve/micromamba/envs/scikit-learn-dev/bin/ninja']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import sklearn
  File "/home/lesteve/micromamba/envs/scikit-learn-dev/lib/python3.14t/site-packages/_scikit_learn_editable_loader.py", line 312, in find_spec
    tree = self._rebuild()
  File "/home/lesteve/micromamba/envs/scikit-learn-dev/lib/python3.14t/site-packages/_scikit_learn_editable_loader.py", line 348, in _rebuild
    raise ImportError(f're-building the {self._name} meson-python editable wheel package failed') from exc
ImportError: re-building the scikit-learn meson-python editable wheel package failed

• my understanding is that build/cp314t/meson-logs/meson-python-build-log.txt should contain the Cython error but this file doesn't exist.

And yes, in case you are wondering, I did triple-check that I was using the dev meson-python in this PR branch 😉.

[dnicolodi]

Thanks for testing @lesteve

And yes, in case you are wondering, I did triple-check that I was using the dev meson-python in this PR branch 😉.

And yet, you are not 😉

Traceback (most recent call last):
File "/home/lesteve/micromamba/envs/scikit-learn-dev/lib/python3.14t/site-packages/_scikit_learn_editable_loader.py", line 346, in _rebuild
subprocess.run(self._build_cmd, cwd=self._build_path, env=env, stdout=subprocess.DEVNULL, check=True)

With this PR applied, line 346 should not look like this, actually the subprocess.run() function call should have its arguments split onto two lines. This code is copied into the editable loader for the installed editable package, thus updating meson-python is not enough. The editable package must be re-installed. I don't know if a package manager, when invoked with --no-build-isolation, may fail to detect that one of the build dependencies has been updated and install a cached version of the editable package when no edits have been made to the sources.

[dnicolodi]

I've tested with a package containing Cython code and Cython errors are reported correctly:

>>> import complex.test
Traceback (most recent call last):
  File "/Users/daniele/src/meson-python/.venv/lib/python3.14/site-packages/_complex_editable_loader.py", line 345, in _rebuild
    subprocess.run(self._build_cmd, cwd=self._build_path, env=env, check=True,
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                   stderr=subprocess.STDOUT, stdout=log)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.14/3.14.6/Frameworks/Python.framework/Versions/3.14/lib/python3.14/subprocess.py", line 578, in run
    raise CalledProcessError(retcode, process.args,
                             output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/opt/homebrew/bin/ninja']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    import complex.test
  File "/Users/daniele/src/meson-python/.venv/lib/python3.14/site-packages/_complex_editable_loader.py", line 307, in find_spec
    tree = self._rebuild()
  File "/Users/daniele/src/meson-python/.venv/lib/python3.14/site-packages/_complex_editable_loader.py", line 369, in _rebuild
    raise exc
ImportError: re-building the complex meson-python editable wheel package failed
cython -M --fast-fail -3 /Users/daniele/src/meson-python/tests/packages/complex/test.pyx -o test.cpython-314-darwin.so.p/test.pyx.c

Error compiling Cython file:
------------------------------------------------------------
...
# SPDX-License-Identifier: MIT

def answer():
    return 42

baz
^
------------------------------------------------------------
/Users/daniele/src/meson-python/tests/packages/complex/test.pyx:8:0: undeclared name not builtin: baz
ninja: build stopped: subcommand failed.

[lesteve]

And yet, you are not 😉

It's kind of 😭 and 🎉 at the same time. I got it to work eventually and this looks great, thanks 🙏!

No idea what was going on (and I almost don't want to understand it 🙄) I decided to remove the file that the traceback was pointing to (/home/lesteve/micromamba/envs/scikit-learn-dev/lib/python3.14t/site-packages/_scikit_learn_editable_loader.py), rebuild and it works, I get the Cython compilation error when the rebuilding fails.