Build vs. target targets about nonrec-make

[aostruszka]

Hello again :)

In general there should be no problem with cross-compilation (this was my initial use case) however so far I had no case where in the same tree I would have tools (meant to be used for "build" machine and during build) and code (meant to be cross-compiled to target machine). But let's try to think it over together.
I'll call your two cases as A and B.
I don't know how you have defined rules for BUILD_TARGET but I assume that in general you're using cross-tools (that is you have defined HOST_TARGET in config.mk and in relevant config-*.mk you have given the cross toolchain).

Case A.
Are you sure that it works as you expected? I'm asking since as of now wildcard expansion works like: if there is special char in value then treat all as wildcard and do glob. This has an unintended consequences that when some expected file is missing or it was given wrong name in Rules.mk then it will be left out (see here for a related problem/discussion). So it seems to me that giving generated.c to SRCS should not work as you expected/thought.

I can't speculate much for your case B since I don't know your BUILD_TARGET rules.

However I can show you how I just solved/worked around (you choose :)) your need. I've added couple dummy wrappers (norm-gcc, norm-g++, cross-gcc, cross-g++) which do nothing then just pass arguments to real gcc/g++ (I don't have any cross toolchain ready) so I can see which toolchain is used. In config-default.mk I've changed defaults to "cross-gcc" and "cross-g++" so by default everything is meant for cross compilation. Then I put something like:

=====a/Rules.mk=====
TARGETS := daemon generator

GENERATOR_OBJS := generator1.o generator2.o

SRCS := *.c
SRCS_EXCLUDES := $(GENERATOR_OBJS:.o=.c)

daemon_DEPS = $(OBJS_$d) $(OBJPATH)/generated.o

$(OBJPATH)/generated.c : $(OBJPATH)/generator
    @echo Generating files;$< > $@

generator_DEPS := $(GENERATOR_OBJS)

BUILD_TARGETS += $(addprefix $(OBJPATH)/,generator $(GENERATOR_OBJS))
=====b/Rules.mk=====
TARGETS := tool

SRCS := *.c
tool_DEPS = $(OBJS_$d)

CPPFLAGS_$d := -DTOOL

BUILD_TARGETS += $(OBJPATH)/tool
=====Rules.mk=====
SUBDIRS := a b

# Define it as non-recursive variable
BUILD_TARGETS :=
=====final.mk=====
ifneq ($(strip $(BUILD_TARGETS)),)
  $(BUILD_TARGETS) : CC := norm-gcc
  $(BUILD_TARGETS) : CXX := norm-g++
endif

This seems to do what you wanted (if I understood your need correctly). It has some drawbacks though - main being the fact that in $(OBJDIR) a you'll have mixed outputs: some for "target" and some for "build" machine. This should not be a problem since you know which ones are which so if you intend to deploy this somewhere then I would add those "target" outputs in each directory to INSTALL_LIB/BIN (see 6a1ae99 for an example).

I'm also not sure what you really expect from case B. I have just build the tool but my suspicion is that it is not needed per se but it is needed to do something (generate, modify ...). If this is so then it would fall under case A - I'd then just give a name/alias to this tool in its Rules.mk and refer to it whenever I need to use it.

Let me know what you think.
Best regards
Andrzej

[torpesco]

Thanks. I'll start by commenting on case A first in case I don't get to B tonight.

It does appear to work as expected, yes, but you're correct about generated.c not working in SRCS. I wrote case A from memory and made a mistake. In the working version that I've got, generated.o is in daemon_DEPS. The output does go into an architecture-appropriate output directory.

Some modified output, although this one with a generated header:

mkdir -p /src/nonrec/liba/obj/debug-arm-target
mkdir -p /src/nonrec/liba/obj/debug-x86_64-build
/devenv-arm/bin/gcc -g -Wall -march=armv5 -Werror -O2 -ggdb -MD -MP -I/src/nonrec/liba/obj/debug-arm-target -I/src/nonrec/include -I/devenv-arm/include -DCONFIG_ARM -c -o /src/nonrec/liba/obj/debug-arm-target/b1.o /src/nonrec/liba/b1.c
/devenv-x86_64/bin/gcc -g -Wall -Werror -ggdb -MD -MP -I/src/nonrec/include -I/devenv-x86_64/include -c -o /src/nonrec/liba/obj/debug-x86_64-build/generator.o /src/nonrec/liba/generator.c
/devenv-x86_64/bin/gcc -L/devenv-x86_64/lib -Wl,-R/devenv-x86_64/lib -ggdb /src/nonrec/liba/obj/debug-x86_64-build/generator.o -o /src/nonrec/liba/obj/debug-x86_64-build/generator
/src/nonrec/liba/obj/debug-x86_64-build/generator > /src/nonrec/liba/obj/debug-arm-target/generated.h
/devenv-arm/bin/gcc -g -Wall -march=armv5 -ggdb -MD -MP -I/src/nonrec/include -I/devenv-arm/include -DCONFIG_ARM -c -o /src/nonrec/liba/obj/debug-arm-target/b2.o /src/nonrec/liba/b2.c
/devenv-arm/bin/ld -r -o /src/nonrec/liba/obj/debug-arm-target/liba.o /src/nonrec/liba/obj/debug-arm-target/b1.o /src/nonrec/liba/obj/debug-arm-target/b2.o

[torpesco]

Before I get to case B, some notes on the makefile modifications I've made so far:

• I renamed HOST_ARCH to ARCH (IIRC, to match our old makefiles)

Additional variables/defines (I took a lazy copy & paste approach for now) in skel.mk:

OBJDIR_BUILD := $(OBJDIR_PREFIX)/$(BUILD_MODE)-$(BUILD_ARCH)-build

define build_build_to_real_dir
$(if $(strip $(TOP_BUILD_DIR)),$(patsubst $(TOP_BUILD_DIR)%/$(OBJDIR),$(TOP)%,$(1)),$(patsubst %/$(OBJDIR),%,$(1))) 
endef

define real_to_build_build_dir
$(if $(strip $(TOP_BUILD_DIR)),$(TOP_BUILD_DIR)$(subst $(TOP),,$(1))/$(OBJDIR_BUILD),$(1)/$(OBJDIR_BUILD))
endef

OBJPATH_BUILD = $(call real_to_build_build_dir,$(d))
CLEAN_DIR_BUILD = $(call real_to_build_build_dir,$(subst clean_,,$@))
DIST_CLEAN_DIR_BUILD = $(patsubst %/$(OBJDIR_BUILD),%/$(firstword $(subst /, ,$(OBJDIR_BUILD))),$(call real_to_build_build_dir,$(subst dist_clean_,,$@)))

define tgt_rule_build
[same as tgt_rule - just OBJPATH_BUILD instead of OBJPATH]
endef

...in def_rules.mk:

define build_skeleton
[copy of skeleton, with OBJPATH changed to OBJPATH_BUILD
endef

...in footer.mk:

[CC/CXX/LDFLAGS/ARCH overrides as in my original post.]

ifdef BUILD_TARGETS
[same as 'ifdef TARGETS' block, but with BUILD_TARGETS, OBJPATH_BUILD -- creates BUILD_TARGETS_$(d)]
endif

$(foreach vd,$(d) $(SRCS_VPATH_$(d)),$(eval $(call build_skeleton,$(vd)))) 
$(foreach tgt,$(filter-out $(OBJS_$(d)),$(BUILD_TARGETS_$(d))),$(eval $(call tgt_rule_build,$(tgt))))

tree_$(d) : $(TARGETS_$(d)) $(BUILD_TARGETS_$(d)) $(foreach sd,$(SUBDIRS_$(d)),tree_$(sd))

dir_$(d) : $(TARGETS_$(d)) $(BUILD_TARGETS_$(d))

...in header.mk, BUILD_TARGETS is cleared.

I believe that's it.

[torpesco]

On to case B (though I should have stopped for tonight by now).

I just discovered two odd things. My B is actually a blend of A and B, with a generator.c file that's compiled into generator that generates generated.c, which is in turn compiled into an object file that's linked into tool.

First. I have:

SRCS_VPATH += $(OBJPATH_BUILD)
SRCS += generated.c

Apparently this works because I do not use a wildcard in SRCS in this Rules.mk file, given the need to include specific files from a couple external source directories.

Second: If I add $(OBJS_$(d)) to BUILD_TARGETS, then the correct compiler is used.

As with what you mention about the paths, the OBJS created from the SRCS list go into OBJPATH. I guess that's because of how OBJS_$(d) is assigned in footer.mk. However, generator.o goes into OBJPATH_BUILD, because it's got a relative path and tgt_rule_build adds the OBJPATH_BUILD prefix.

The thing that puzzles me is that I have to add $(OBJS_$(d)) to BUILD_TARGETS.

Ah ha.

OK. Now it doesn't puzzle me. It's because if TARGETS isn't defined, then footer.mk sets TARGETS_$(d) = $(OBJS_$(d)). If I wrap that in an ifndef BUILD_TARGETS, then I don't need $(OBJS_$(d)) to be in BUILD_TARGETS.

In footer.mk:

ifdef TARGETS
[...]
else ifndef BUILD_TARGETS
TARGETS_$(d) := $(OBJS_$(d)) 
endef

The output path for the object files is still OBJPATH... so I'd have to mess with how OBJS_$(d) is assigned.

[torpesco]

For now, I did end up adding BUILD_SRCS, BUILD_SRCS_VPATH, BUILD_OBJS_$(d), etc. The build_to_real_dir call had trouble for getting the directory for things like the directory's INCLUDES_$(d), so I added $(BUILD_TARGETS): @RD := $(d) in that block I listed in my original post.

Not the most elegant thing having duplicated so much to do this. I'll have to think about possible refinements later. One I have done was to make $(OBJPATH) a parameter of skeleton. (I'm still on the previous skeleton define - that's the one change I haven't merged yet from your master branch.)

[torpesco]

Working well enough for me, so I'll close this. I'll have to polish it up into a form I can share in my fork in case it's useful for anyone.

[aostruszka]

Sorry for not getting back but I'm overloaded with my daily tasks.

If I'll get some spare time I'll try to think about this more (as a challenge - I don't predict myself having such a need :)).

Best regards
Andrzej

[torpesco]

Thanks. :)