janware/jw-pkg
1
Code Pull requests Activity Actions
jw-pkg/make/cache.mk

34 lines
997 B
Makefile
Raw Permalink Normal View History

make: Further improve build time This commit sees several improvements to the build performance: - Introduce cache.mk, which creates makefiles caching often used variables, per tree and per project. - Define more variables with := enclosed in condistions, instead of defining them with ?=, because the RHS of ?= is expanded deferredly. - Add more definitions for executables. - Move some more specialized definitions out into specialized makefiles, notably htdocs.mk and tmpl.mk Signed-off-by: Jan Lindemann <jan@janware.com>
2019-02-28 13:22:12 +00:00
# ----- define these variables
# CACHED_FILES = $(TOPDIR)/VERSION
# CACHED_VARS = VERSION
make/[Makefile|*.mk): Improve variable caching This commit aims at improving speed by using better caching. - Makefile, cache.mk: Split .cache.mk up To allow caching of runtime path variables which are project-specific, split .cache.mk up in .cache-project.mk and .cache-projects.mk - ldlibpath.mk: Cache ldlibpath, exepath and pythonpath Place the output of $(call proj_query ldlibpath), $(call proj_query, exepath) and $(call proj_query pythonpath) in JW_PKG_LD_LIBRARY_PATH, JW_PKG_EXE_PATH, and JW_PKG_PYTHON_PATH respectively, and cache the variables in make/.project-cache.mk. - cache.mk: Use = instead of := Recursively expanded variables are nearly as fast as := variables if the assigned value is a fixed string. And sometimes it's not, rightly so, because variables get assigned below, as with JW_PKG_XXX for instance. - cache.mk: Use $(TOPDIR) as variable values Replace absolute references to project's topdir by $(TOPDIR) with sed. As soon as the project queries produce absolute paths, they will be transformed into relative paths which allow the code base to be moved to a different location and still remain functional without a rebuild. Signed-off-by: Jan Lindemann <jan@janware.com>
2026-01-22 14:12:58 +01:00
.PHONY: cache clean-cache clean-cache-projects
make: Further improve build time This commit sees several improvements to the build performance: - Introduce cache.mk, which creates makefiles caching often used variables, per tree and per project. - Define more variables with := enclosed in condistions, instead of defining them with ?=, because the RHS of ?= is expanded deferredly. - Add more definitions for executables. - Move some more specialized definitions out into specialized makefiles, notably htdocs.mk and tmpl.mk Signed-off-by: Jan Lindemann <jan@janware.com>
2019-02-28 13:22:12 +00:00
make/[Makefile|*.mk): Improve variable caching This commit aims at improving speed by using better caching. - Makefile, cache.mk: Split .cache.mk up To allow caching of runtime path variables which are project-specific, split .cache.mk up in .cache-project.mk and .cache-projects.mk - ldlibpath.mk: Cache ldlibpath, exepath and pythonpath Place the output of $(call proj_query ldlibpath), $(call proj_query, exepath) and $(call proj_query pythonpath) in JW_PKG_LD_LIBRARY_PATH, JW_PKG_EXE_PATH, and JW_PKG_PYTHON_PATH respectively, and cache the variables in make/.project-cache.mk. - cache.mk: Use = instead of := Recursively expanded variables are nearly as fast as := variables if the assigned value is a fixed string. And sometimes it's not, rightly so, because variables get assigned below, as with JW_PKG_XXX for instance. - cache.mk: Use $(TOPDIR) as variable values Replace absolute references to project's topdir by $(TOPDIR) with sed. As soon as the project queries produce absolute paths, they will be transformed into relative paths which allow the code base to be moved to a different location and still remain functional without a rebuild. Signed-off-by: Jan Lindemann <jan@janware.com>
2026-01-22 14:12:58 +01:00
CACHE_PROJECT_MK ?= .cache-project.mk
CACHED_VARS_ONLY_PROJECT ?= \
PROJECT \
topdir.mk: Move PREREQ definition into defs.mk At present, the PREREQ-variable is effectively only used to detect if prerequiste packages haven't run "make all" before make is run in a given package. Also, it's only useful in $(TOPDIR). This commit splits the variable up into PREREQ_BUILD and PREREQ_RUN, and makes the variables available in every Makefile of a package by placing them in defs.mk instead of topdir.mk. This also fixes a problem that PREREQ was cached before being filled, hence empty. Which effectively wasn't much of a problem, because it was basically unused, but still. Signed-off-by: Jan Lindemann <jan@janware.com>
2026-06-17 14:08:58 +02:00
PREREQ_BUILD \
PREREQ_RUN \
make/[Makefile|*.mk): Improve variable caching This commit aims at improving speed by using better caching. - Makefile, cache.mk: Split .cache.mk up To allow caching of runtime path variables which are project-specific, split .cache.mk up in .cache-project.mk and .cache-projects.mk - ldlibpath.mk: Cache ldlibpath, exepath and pythonpath Place the output of $(call proj_query ldlibpath), $(call proj_query, exepath) and $(call proj_query pythonpath) in JW_PKG_LD_LIBRARY_PATH, JW_PKG_EXE_PATH, and JW_PKG_PYTHON_PATH respectively, and cache the variables in make/.project-cache.mk. - cache.mk: Use = instead of := Recursively expanded variables are nearly as fast as := variables if the assigned value is a fixed string. And sometimes it's not, rightly so, because variables get assigned below, as with JW_PKG_XXX for instance. - cache.mk: Use $(TOPDIR) as variable values Replace absolute references to project's topdir by $(TOPDIR) with sed. As soon as the project queries produce absolute paths, they will be transformed into relative paths which allow the code base to be moved to a different location and still remain functional without a rebuild. Signed-off-by: Jan Lindemann <jan@janware.com>
2026-01-22 14:12:58 +01:00
PREREQ \
VERSION \
HEX_VERSION \
FULL_NAME \
JW_PKG_LD_LIBRARY_PATH \
JW_PKG_EXE_PATH \
JW_PKG_PYTHON_PATH
Further improve build time: Try to avoid calling projects.py This commit tries to remove the necessity to call projects.py from $(TOPDIR) to speed up recursive builds over all projects yet again. This is a major undertaking. There are two variables which are filled py projects.py in $(TOPDIR): PREREQ and PREREQ_DIRS. Sadly, the latter is a path relative to $(TOPDIR)/make, so this is kind of pointless. Unless the cache is maintained in $(TOPDIR), a thing I tried to avoid. So this commit is only able to cache $(PREREQ), not $(PREREQ_DIRS), which still is a hassle. Introduced defs-dirs.mk for that, to make it accessible to make.mk, and modified all the other parts of the machinery, too. Signed-off-by: Jan Lindemann <jan@janware.com>
2019-02-28 14:48:57 +00:00
all: cache
make: Further improve build time This commit sees several improvements to the build performance: - Introduce cache.mk, which creates makefiles caching often used variables, per tree and per project. - Define more variables with := enclosed in condistions, instead of defining them with ?=, because the RHS of ?= is expanded deferredly. - Add more definitions for executables. - Move some more specialized definitions out into specialized makefiles, notably htdocs.mk and tmpl.mk Signed-off-by: Jan Lindemann <jan@janware.com>
2019-02-28 13:22:12 +00:00
clean: clean-cache
clean-cache:
make/[Makefile|*.mk): Improve variable caching This commit aims at improving speed by using better caching. - Makefile, cache.mk: Split .cache.mk up To allow caching of runtime path variables which are project-specific, split .cache.mk up in .cache-project.mk and .cache-projects.mk - ldlibpath.mk: Cache ldlibpath, exepath and pythonpath Place the output of $(call proj_query ldlibpath), $(call proj_query, exepath) and $(call proj_query pythonpath) in JW_PKG_LD_LIBRARY_PATH, JW_PKG_EXE_PATH, and JW_PKG_PYTHON_PATH respectively, and cache the variables in make/.project-cache.mk. - cache.mk: Use = instead of := Recursively expanded variables are nearly as fast as := variables if the assigned value is a fixed string. And sometimes it's not, rightly so, because variables get assigned below, as with JW_PKG_XXX for instance. - cache.mk: Use $(TOPDIR) as variable values Replace absolute references to project's topdir by $(TOPDIR) with sed. As soon as the project queries produce absolute paths, they will be transformed into relative paths which allow the code base to be moved to a different location and still remain functional without a rebuild. Signed-off-by: Jan Lindemann <jan@janware.com>
2026-01-22 14:12:58 +01:00
rm -f $(CACHE_PROJECT_MK)
cache: $(CACHE_PROJECT_MK)
$(CACHE_PROJECT_MK): $(CACHED_FILES) $(JWBDIR)/make/cache.mk
cache.mk: Support JW_PKG_NO_CACHE cache.mk generates .project-cache.mk, and this commit supports disabling the definitions in the generated cache by setting JW_PKG_NO_CACHE=true. Signed-off-by: Jan Lindemann <jan@janware.com>
2026-06-17 14:10:12 +02:00
@echo 'ifneq ($$(JW_PKG_NO_CACHE),true)' | tee $@.tmp
cache.mk: Don't overwrite defined variables Check with ifndef if a variable is defined before hard defining it in cache.mk. This gives more flexibility when overriding variables with local.mk. Signed-off-by: Jan Lindemann <jan@janware.com>
2026-05-05 16:54:18 +02:00
@echo $(foreach v,$(CACHED_VARS),$v = $(value $(v)) EOL) | \
$(SED) 's/\s\+EOL/\n/g;' | \
$(SED) 's/\s*\(\S\+\)\s*=\s*\(.*\)\s*/ifndef \1\n \1 = \2\nendif\n/g;' | \
make/[Makefile|*.mk): Improve variable caching This commit aims at improving speed by using better caching. - Makefile, cache.mk: Split .cache.mk up To allow caching of runtime path variables which are project-specific, split .cache.mk up in .cache-project.mk and .cache-projects.mk - ldlibpath.mk: Cache ldlibpath, exepath and pythonpath Place the output of $(call proj_query ldlibpath), $(call proj_query, exepath) and $(call proj_query pythonpath) in JW_PKG_LD_LIBRARY_PATH, JW_PKG_EXE_PATH, and JW_PKG_PYTHON_PATH respectively, and cache the variables in make/.project-cache.mk. - cache.mk: Use = instead of := Recursively expanded variables are nearly as fast as := variables if the assigned value is a fixed string. And sometimes it's not, rightly so, because variables get assigned below, as with JW_PKG_XXX for instance. - cache.mk: Use $(TOPDIR) as variable values Replace absolute references to project's topdir by $(TOPDIR) with sed. As soon as the project queries produce absolute paths, they will be transformed into relative paths which allow the code base to be moved to a different location and still remain functional without a rebuild. Signed-off-by: Jan Lindemann <jan@janware.com>
2026-01-22 14:12:58 +01:00
$(SED) 's|$(realpath $(TOPDIR))|$$(TOPDIR)|g' | \
$(SED) 's|$(realpath $(PROJECTS_DIR))|$$(PROJECTS_DIR)|g' | \
$(GREP) . | \
cache.mk: Support JW_PKG_NO_CACHE cache.mk generates .project-cache.mk, and this commit supports disabling the definitions in the generated cache by setting JW_PKG_NO_CACHE=true. Signed-off-by: Jan Lindemann <jan@janware.com>
2026-06-17 14:10:12 +02:00
tee -a $@.tmp
@echo 'endif # ifneq ($$(JW_PKG_NO_CACHE),true)' | tee -a $@.tmp
make/[Makefile|*.mk): Improve variable caching This commit aims at improving speed by using better caching. - Makefile, cache.mk: Split .cache.mk up To allow caching of runtime path variables which are project-specific, split .cache.mk up in .cache-project.mk and .cache-projects.mk - ldlibpath.mk: Cache ldlibpath, exepath and pythonpath Place the output of $(call proj_query ldlibpath), $(call proj_query, exepath) and $(call proj_query pythonpath) in JW_PKG_LD_LIBRARY_PATH, JW_PKG_EXE_PATH, and JW_PKG_PYTHON_PATH respectively, and cache the variables in make/.project-cache.mk. - cache.mk: Use = instead of := Recursively expanded variables are nearly as fast as := variables if the assigned value is a fixed string. And sometimes it's not, rightly so, because variables get assigned below, as with JW_PKG_XXX for instance. - cache.mk: Use $(TOPDIR) as variable values Replace absolute references to project's topdir by $(TOPDIR) with sed. As soon as the project queries produce absolute paths, they will be transformed into relative paths which allow the code base to be moved to a different location and still remain functional without a rebuild. Signed-off-by: Jan Lindemann <jan@janware.com>
2026-01-22 14:12:58 +01:00
mv $@.tmp $@
Copy permalink