Generate a standard conftest.py, mostly for customizing pytest's bombastic test header, which otherwise lets the more informative make output look too pale in comparison.
Fix shellcheck SC2068 (unquoted array expansions), SC2145 (mixed string/array arguments), SC2328 (redirection in command substitution), SC2173 (untrapable signals), and SC2148 (missing shebang) errors across 14 script files.
Also configure scripts/Makefile with --severity=error so that only errors (not warnings or notes) cause check failures. To be tightened by follow-up commits.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev
After a pipeline change, CI now runs "make all" in a repo's root, which uncovers two problems:
1. The help integration test only succeeded as long CI didn't run "make all" before "make test". That way, the checked out repository lacked the generated __init__.py files needed for some modular subcommands to be fully loaded, and hence, the test should have failed. The entire machinery only worked because the subcommands in question are not not essential to building jw-pkg itself: "secrets" and "posix". So, this commit adapts the help integration test to the new reality.
2. Regarding python-tools.sh: Commit 55060486 satisfies yapf in some places of the source code, but in others not anylonger. So patch python-tools.sh's newline handling again.
While not thematically similar, both fixes get baked into one commit to satisfy the requirement that every single commit needs to pass "make clean all check test" individually.
Add support for PY_INIT_SUBMODULES to py-mod.mk. If it is defined in a Makefile including py-mod.mk, the listed submodules will be added to __init__.py and thus included in the list of things that can be imported from a module.
This commit also adds support for --submodules to python-tools.sh for that to happen.
Letting python-tools.sh rewrite symbols is more robust than rewriting an entire __init__.py with PY_INIT_FILTER in the including Makefile. The latter can break in non-obvious ways if python-tools.sh changes __init__.py's format.
Make python-tools.sh support --symbol-filter to remedy that. The option takes an sed script which should expect a string of two non-whitespace tokens: The module from which the symbol is imported, and the name of the symbol in that module. It's output will then be used as the symbol to be exported from __init__.py.
Also, support the PY_SYMBOL_FILTER variable in py-mod.mk. If it's defined, it is used for --symbol-filter.
The __init__.py files as gnerated by python-tools.sh contain multiple issues, fix them:
- Make the machinery fail if the same type name is imported from
different modules
- Support relative imports from .Module import Module instead of
having to use the entire module path as import source
- Import types explicitly re-exported with "as":
from .Module import Module as Module
Otherwise ruff will regard the type as "imported but not used"
- Add "# ruff: noqa: E501" near the top. The import lines can get
long and are beyond manual control (except for renaming the
modules themselves, that is). This can cause ruff to fail, so get
it to accept long lines in __init__.py. The style violation
doesn't make much of a difference in generated code, anyway,
because nobody reads that. Plus what's happening in the code
isn't rocket science, so good style wouldn't help much with
understanding, either.
This promptly digs up two symbol name conflicts lib.pm.dpkg and lib.pm.rpm. Fix them along with this commit to keep it from breaking the build.
This commit adds support for static typechecking with mypy.
Notable additions:
- A new target "check" which does the type checking
- Py-mods.mk, meant to be included from a directory containing python modules
in subdirectories, but not being a python module itself. It makes the all
target depend on check only if PY_RUN_CHECK_AFTER_BUILD is defined and
true. That's because pypy is under heavy development, and the Ubuntu 18.04
version is too old to work for lots of the code.
import my.mod.thing, my.mod.thang wouldn't work, if the modules where not in the same directory hierarchy, e.g. in $(HOME)/blah/my/mod/thing.py and $(HOME)/blub/my/mod/thang.py, not even with PYTHONPATH=$(HOME)/blah:$(HOME)/blub.
This commit beautifies module import path deduction a little. It also adds (disabled) code for importing submodules, which may or may not be enabled by a command-line option in the future.
