Remove external untokenize package. Use tokenize from the Python standard library. - #325
Conversation
|
This is required for Python 3.14. The Tested with docker run -it python:3.14 bash
cd /tmp/
git clone https://github.com/tdenewiler/docformatter.git
cd docformatter
git checkout untokenize-std-lib
pip install tox
tox -e py314All tests are passing. If you stay on |
|
I ran the tox tests on Python 3.10, 3.11, 3.12, 3.13, and 3.14. |
|
The key insight here is that FWIW I recommend using |
|
+1 for removing this unused dependency |
|
@weibullguy Hopefully this is a very clear "merge", and a release to fix 3.14 would be welcome |
|
I'll probably look into the pydocstyle plugin for ruff. It supports a |
Does this plugin format docstrings or just verify them against various standards? |
|
FWIW even with this PR, on 3.14 docformatter adds extraneous newlines. |
* updated v1.7.8 * MNT: Re-rendered with conda-smithy 3.61.2 and conda-forge-pinning 2026.04.28.18.11.5 * Convert from v0 meta.yaml to v1 recipe.yaml syntax * MNT: Re-rendered with conda-smithy 3.61.2 and conda-forge-pinning 2026.04.28.18.11.50 Other tools: - conda-build 25.11.1 - rattler-build 0.63.1 - rattler-build-conda-compat 1.4.13 * Bump python_min from 3.9 to 3.10 Xref PyCQA/docformatter@6a90c74 * Remove untokenize runtime dependency Xref PyCQA/docformatter#325 * Extend python_version in tests as per https://conda-forge.org/docs/maintainer/example_recipes/pure-python/#the-tests-section --------- Co-authored-by: Wei Ji <23487320+weiji14@users.noreply.github.com> Co-authored-by: Carlo <carlodri@users.noreply.github.com>
…uations (#360) A blank line that contains trailing whitespace (e.g. " \n") produces an NL token whose `.line` is not exactly "\n" or "\r\n". The guard in `is_newline_continuation` only excluded the exact-blank forms, so such a line was misclassified as a continuation of the preceding line. This caused `_do_update_token_indices` to rewrite the token's start row back to the previous token's end row, yielding a start position that precedes the previous token's end. `tokenize.untokenize` then raised `ValueError: start (r,c) precedes previous end (r,c)` and docformatter crashed (regression from the stdlib tokenize migration in #325, v1.7.8). Classify any whitespace-only line as blank by testing `token.line.strip()` instead of comparing the raw line against {"\n", "\r\n"}. Genuine continuations (whose stripped line is non-empty) are unaffected. Fixes #355. Co-authored-by: Doyle Rowland <doyle.rowland@reliaqual.com>
No description provided.