Improve Makefile variable caching: PREREQ, JW_PKG_NO_CACHE, subsections #28

Merged

Jan Lindemann merged 6 commits from jan/feature/20260617-py-path-ldlibpath-mk-space-separate-jw-pkg-xxx-path into master 2026-06-17 20:46:13 +02:00 AGit

Conversation 0 Commits 6 Files changed 10 +61 -27

Jan Lindemann commented 2026-06-17 20:38:53 +02:00

Owner

Copy link

This PR adds various improvements regarding how generated Makefile variables are cached

• Generate system search paths from PREREQ and friends:
  Introduce the PREREQ_BUILD and PREREQ_RUN Makefile variables, and make them, together with the already existing variable PREREQ, available in the entire project via defs.mk. This allows to append project names to them, e.g. in local.mk, and have system search paths (PATH, PYTHONPATH, LD_LIBRARY_PATH) extended automatically.

• JW_PKG_NO_CACHE:
  Allow to turn off caching by setting JW_PKG_NO_CACHE to true in the environment or local.mk.

• Space-separated subsections:
  Make $(call ...) macros accept multiple dependency flavours or subsections separated by whitespace.

cache-projects.mk: Remove unused ifndef / endif

make/Makefile is responsible to generate $(TOPDIR)/cache-projects.mk. The variables are taken from .cache-project.mk, with some variables intentionally omitted, but their ifndef / endif blocks remain place. Not harmful but ugly. Make sed range-delete the left-over blocks entirely.

cache.mk: Support JW_PKG_NO_CACHE

cache.mk generates .project-cache.mk, and this commit supports disabling the definitions in the generated cache by setting JW_PKG_NO_CACHE=true.

topdir.mk: Move PREREQ definition into defs.mk

At present, the PREREQ-variable is effectively only used to detect if prerequiste packages haven't run "make all" before make is run in a given package. Also, it's only useful in $(TOPDIR). This commit splits the variable up into PREREQ_BUILD and PREREQ_RUN, and makes the variables available in every Makefile of a package by placing them in defs.mk instead of topdir.mk.

This also fixes a problem that PREREQ was cached before being filled, hence empty. Which effectively wasn't much of a problem, because it was basically unused, but still.

cmds.projects.BaseCmdPkgRelations: Support spaces as delimiter

Some options to the pkg-xxx commands, like flavour, --subsections and --ignore understand a comma as delimiter if multiple option values are specified. The comma character is not very friendly to use in $(call ...) macros, though, so support spaces and pipe characters as well.

cmds.projects.Cmd[Ldlibpath|Exepath]: Support --delimiter

Support a --delimiter option to the ldlibpath and exepath commands. Notable use case are the JW_PKG_XXX_PATH variables, which should use spaces instead of colons.

TODO: Merging those two command modules with BaseCmdPkgRelations would have made introducing this redundancy unnecessary, check if that's a possibility.

py-path|ldlibpath.mk: Space-separate JW_PKG_XXX_PATH

The following variables contain colons as path-separators:

• JW_PKG_PYTHON_PATH
• JW_PKG_EXE_PATH
• JW_PKG_LD_LIBRARY_PATH

This commit makes them use spaces instead, so they can be more easily amended by Makefiles using them. Also define them in a more uniform way, and use the newly introduced PREREQ_RUN variable to fill them, which in turn can also be appended to before that.

This PR adds various improvements regarding how generated Makefile variables are cached - *Generate system search paths from PREREQ and friends*: Introduce the PREREQ_BUILD and PREREQ_RUN Makefile variables, and make them, together with the already existing variable PREREQ, available in the entire project via defs.mk. This allows to append project names to them, e.g. in local.mk, and have system search paths (PATH, PYTHONPATH, LD_LIBRARY_PATH) extended automatically. - *JW_PKG_NO_CACHE*: Allow to turn off caching by setting ```JW_PKG_NO_CACHE``` to ```true``` in the environment or ```local.mk```. - *Space-separated subsections*: Make $(call ...) macros accept multiple dependency flavours or subsections separated by whitespace. #### cache-projects.mk: Remove unused ifndef / endif make/Makefile is responsible to generate $(TOPDIR)/cache-projects.mk. The variables are taken from .cache-project.mk, with some variables intentionally omitted, but their ifndef / endif blocks remain place. Not harmful but ugly. Make sed range-delete the left-over blocks entirely. #### cache.mk: Support JW_PKG_NO_CACHE cache.mk generates .project-cache.mk, and this commit supports disabling the definitions in the generated cache by setting JW_PKG_NO_CACHE=true. #### topdir.mk: Move PREREQ definition into defs.mk At present, the PREREQ-variable is effectively only used to detect if prerequiste packages haven't run "make all" before make is run in a given package. Also, it's only useful in $(TOPDIR). This commit splits the variable up into PREREQ_BUILD and PREREQ_RUN, and makes the variables available in every Makefile of a package by placing them in defs.mk instead of topdir.mk. This also fixes a problem that PREREQ was cached before being filled, hence empty. Which effectively wasn't much of a problem, because it was basically unused, but still. #### cmds.projects.BaseCmdPkgRelations: Support spaces as delimiter Some options to the pkg-xxx commands, like flavour, --subsections and --ignore understand a comma as delimiter if multiple option values are specified. The comma character is not very friendly to use in $(call ...) macros, though, so support spaces and pipe characters as well. #### cmds.projects.Cmd[Ldlibpath|Exepath]: Support --delimiter Support a --delimiter option to the ldlibpath and exepath commands. Notable use case are the JW_PKG_XXX_PATH variables, which should use spaces instead of colons. TODO: Merging those two command modules with BaseCmdPkgRelations would have made introducing this redundancy unnecessary, check if that's a possibility. #### py-path|ldlibpath.mk: Space-separate JW_PKG_XXX_PATH The following variables contain colons as path-separators: - JW_PKG_PYTHON_PATH - JW_PKG_EXE_PATH - JW_PKG_LD_LIBRARY_PATH This commit makes them use spaces instead, so they can be more easily amended by Makefiles using them. Also define them in a more uniform way, and use the newly introduced PREREQ_RUN variable to fill them, which in turn can also be appended to before that.

Jan Lindemann added 6 commits 2026-06-17 20:38:53 +02:00

cache-projects.mk: Remove unused ifndef / endif ... 3833fb0988

make/Makefile is responsible to generate $(TOPDIR)/cache-projects.mk. The variables are taken from .cache-project.mk, with some variables intentionally omitted, but their ifndef / endif blocks remain in place. Not harmful but ugly. Make sed range-delete the left-over blocks entirely.
Signed-off-by: Jan Lindemann <jan@janware.com>

cache.mk: Support JW_PKG_NO_CACHE ... 8712cbc7b5

cache.mk generates .project-cache.mk, and this commit supports disabling the definitions in the generated cache by setting JW_PKG_NO_CACHE=true.
Signed-off-by: Jan Lindemann <jan@janware.com>

topdir.mk: Move PREREQ definition into defs.mk ... 10bde0c05d

At present, the PREREQ-variable is effectively only used to detect if prerequiste packages haven't run "make all" before make is run in a given package. Also, it's only useful in $(TOPDIR). This commit splits the variable up into PREREQ_BUILD and PREREQ_RUN, and makes the variables available in every Makefile of a package by placing them in defs.mk instead of topdir.mk.
This also fixes a problem that PREREQ was cached before being filled, hence empty. Which effectively wasn't much of a problem, because it was basically unused, but still.
Signed-off-by: Jan Lindemann <jan@janware.com>

cmds.projects.BaseCmdPkgRelations: Support spaces as delimiter ... 37d5bcddd4

Some options to the pkg-xxx commands, like flavour, --subsections and --ignore understand a comma as delimiter if multiple option values are specified. The comma character is not very friendly to use in $(call ...) macros, though, so support spaces and pipe characters as well.
Signed-off-by: Jan Lindemann <jan@janware.com>

cmds.projects.Cmd[Ldlibpath|Exepath]: Support --delimiter ... 0af41ac832

Support a --delimiter option to the ldlibpath and exepath commands. Notable use case are the JW_PKG_XXX_PATH variables, which should use spaces instead of colons.
TODO: Merging those two command modules with BaseCmdPkgRelations would have made introducing this redundancy unnecessary, check if that's a possibility.
Signed-off-by: Jan Lindemann <jan@janware.com>

py-path|ldlibpath.mk: Space-separate JW_PKG_XXX_PATH ... 156d073885

The following variables contain colons as path-separators:
- JW_PKG_PYTHON_PATH - JW_PKG_EXE_PATH - JW_PKG_LD_LIBRARY_PATH
This commit makes them use spaces instead, so they can be  more easily amended by Makefiles using them. Also define them in a more uniform way, and use the newly introduced PREREQ_RUN variable to fill them, which in turn can also be appended to before that.
Signed-off-by: Jan Lindemann <jan@janware.com>

Jan Lindemann scheduled this pull request to auto merge when all checks succeed 2026-06-17 20:38:56 +02:00

Jan Lindemann merged commit 156d073885 into master 2026-06-17 20:46:13 +02:00

Jan Lindemann changed title from Improve Makefile variable caching: PREREQ, JW_PKG_NO_CACHE, Space-separated subsections to Improve Makefile variable caching: PREREQ, JW_PKG_NO_CACHE, subsections 2026-06-17 20:47:29 +02:00

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

janware/jw-pkg!28

No description provided.