Hi

In my custom module I have to use file-normalize procedure, unfortunately, it cannot manage any paths which start with a tilde (e.g, ~/foo, ~foo/bar), could you, please, apply the below

--- a/lib/misc.tcl
+++ b/lib/misc.tcl
@@ -77,6 +77,12 @@ proc file-normalize {path} {
        if {$path eq ""} {
            return ""
        }
+       if {[regexp -- {^~[^/]} $path]} {
+           set path [file join /home [string range $path 1 end]]
+       }\
+       elseif {[regexp -- {^~[/]} $path]} {
+           set path [file join [env HOME] [string range $path 2 end]]
+       }
        set oldpwd [pwd]
        if {[file isdir $path]} {
            cd $path

Thanks for autosetup!

To handle dependencies among options, it would be nice to be able to force the value of an option from within auto.def.

if {[opt-bool depends-on-foo]} {
   set-opt-bool foo ;# this here
}

Maybe there's a way to do that already and I just missed it. Thought?

If you say something like

define defaultprefix /opt/mypackage

before the system module gets included, that overrides the default of /usr/local for the configure --prefix option, but ./configure --help still claims --prefix defaults to /usr/local because it has that value hard-coded into its message text. It needs to use the defaultprefix value here instead.

I'd make the change myself, but my Tcl is waaaay rusty. :)

I can tell you that the change needs to be somewhere around line 28 in autosetup/system.tcl.

Checkin [92515ac] introduces an incompatibility in the script file syntax that's still documented the old way on the github.io site.

The project could use a ChangeLog.md file to call out such things as well. I had to go digging through the checkin stream to figure out what happened here on upgrading from 0.6.7.

last time I've read the docs not very careful..

"need to selected" : user => missing 'be' ?
"as the default if is not provided" : developer/reference => missing 'it' ?
"idiosyncracies" : developer/reference => idiosyncrasies
"the the" : developer/reference
"EXEXT" : developer/reference => EXEEXT

“=> description : developer/reference -> missing "

This regex in lib/system.tcl is too loose: ^@(if|else|endif)\s*(.*)

1. It allows zero spaces between @if and the variable name; it should require at least 1
2. It allows junk after @else and @endif

I suggest changing it to ^@(if\s+(.*)|else|endif)$, assuming Tcl's regex engine allows nested grouping and the inner one continues to get treated as "second" as it currently is. If Tcl regexes can't cope with that, then you'll have to keep the (.*) bit at the end of the RE and handle the second problem separately, such as by checking for empty condargs in the else and endif cases.

If you do this, it should be changed after merging my recent PR, if you're going to accept it, because it moves the affected line, which would create a merge conflict if you change it before merging the PR.

https://msteveb.github.io/autosetup/ :
Jim Tcl => http://jim.berlios.de/

http://msteveb.github.io/autosetup/developer/examples/ :
redis => http://redis.io/ - new homepage
Jim Tcl Project => http://jim.berlios.de/

https://msteveb.github.io/autosetup/developer/ :
waf => https://waf.io/ - new homepage
Tcl Scripting => 404
Modules => 404

https://msteveb.github.io/autosetup/why/ :
http://jim.berlios.de/

Debian packaging tools pass --disable-silent-rules to configure, it would be great if autosetup could just ignore those.

I see that make test runs the tests in the testsuite directory, but there are some files that look like tests in the examples/testbed directory. Are those officially considered part of testing or not?

Specifically, I am looking to add tests for a new pkg-config-related function, but I don't know where to put it. The only other testing of pkg-config is in the examples/testbed directory.

The lpop proc introduced in 92515ac needs use util to be visible from make-template. I think this should be transparent from the user's point of view.

In testbed, this is mitigated by the call to autosetup-require-version, which itself calls use util. Is it assumed that a call to autosetup-require-version be there for the whole machinery to work, or would it be better to explicitely use util in autosetup?

Also, small typo in compatiblity here - I'm not opening a separate PR for that ;)

Thanks!

In fc665b9 an typo occurred, so now the example doesn't work.

cc-chared should be cc-shared

See here. This discussion has also been raised on FreeNode/#neomutt earlier.

It looks like sysconfdir and localstatedir should be treated slightly differently than the other paths, when it comes to making them relative to prefix.

To the best of my understanding:

. sysconfdir should honor prefix only if it's not /usr
. localstatedir should never honor prefix

What are your thoughts?

Using the current trunk, as of yesterday:

$ ./autosetup/autosetup-find-tclsh 
/usr/bin/tclsh

# But jimsh0 is here:

$ l autosetup/jimsh0
-rwxrwxr-x 1 stephan stephan 314656 Oct 15  2018 autosetup/jimsh0

But using 0.6.9 it works as expected:

$ ./autosetup/autosetup-find-tclsh 
/home/stephan/fossil/libfossil/autosetup/jimsh0

i haven't looked into what the exact problem is, though.

If CC is specified to a non-default compiler when running ./configure, that same CC is not used when auto-reconfigure kicks in. To get this effect, the auto.def needs something like:

    set cc [opt-val cc]
    if {[string length $cc]} {
        define CC $cc
        define CC_FOR_BUILD $cc
    }

my recommendation would be to either include CC=xxx to the AUTOREMAKE command or to add a set of common flags (to the cc module) which do something like the above for CC and CC_FOR_BUILD (which my example doesn't handle properly because it's presumably for cross-compiling, which i never do).

The msys/mingw environment is very limited. No vfork/waitpid, getcwd() and similar which return paths give backslashes rather than forward slashes and there is no 'cc' in the PATH.

This means that we see:

$ ./autosetup
No installed jimsh or tclsh, building local bootstrap jimsh0
./find-tclsh: line 10: cc: command not found

And :

$ ../../jimsh0 ../../autosetup
Host System...
Build System...
Error: Could not find a C compiler. Tried: gcc
Try: 'autosetup --help' for options

autosetup doesn't place too many demands on the Tcl interpreter, so we can probably add some compatibility support to autosetup to work around these differences.

The shell script parts of configure.in are hard to translate automatically, but a lot of the AC_XXX macros should be easy enough to translate. This will help developers new to autosetup spend less time looking up the command reference.

After a system update which includes clang 6.0, my configure scripts stopped including -fPIC in my C++ code, which leads to building of shared libraries to fail with a "recompile with -fPIC" message. The culprit is:

============
Failed: c++ -g -O2 -c -Werror -Wall -Wextra -Wsign-compare -fPIC -Werror conftest__.c -o conftest__.o
clang: error: treating 'c' input as 'c++' when in C++ mode, this behavior is deprecated [-Werror,-Wdeprecated]
============
The failed code was:
int main(void) {

return 0;
}

The corresponding auto.def test is:

cc-with {-lang c++} {
    cc-check-flags -fPIC
}

The -Werror and friends flags which appear in the logs are injected by a prior auto.def test:

cc-check-flags -Werror -Wall -Wextra -Wsign-compare -fPIC

i've been able to work around this by moving that second test below the C++ -fPIC test, but that is somewhat unsatisfying because i want the C++ compiler to pedantically fail if it doesn't understand -fPIC and it might not if -Werror is missing (it might just warn about the unsupported flag, but still exit with rc 0).

In any case, when compiling a test snippet in C++ mode, the temp file name should probably have a .cpp or .c++ extension rather than .c, since clang complains about that extension assumption. Alternately, perhaps a completely arbitrary extension, like .xxx, might stop it from complaining.

When trying to build an autoconf-based OpenOCD its configure calls configure.gnu in jimtcl/ which in turn execs autosetup's configure. This results in $0 (and hence dir variable) containing full path and so it needs to be quoted inside the backticks for autosetup-find-tclsh invocation.

This -gstabs flag is GNU gcc specific and does not apply to llvm's 'gcc' or clang.

        switch -glob -- [get-define host] {
                *-*-darwin* {
                        # Don't generate .dSYM directories
-->                       lappend cmdline -gstabs
                }
        }
        lappend cmdline $src -o $tmp {*}$opts(-libs)

Cross-referencing to the issue comment I left in the related OpenOCD issues:
esden/summon-arm-toolchain#54

autosetup needs an easy way to search for system libraries/components similar to cmake's FIND_PACKAGE (http://www.vtk.org/Wiki/CMake:How_To_Find_Libraries)

It should take into account:

• Any --with-xyz path specified by the user
• pkg-config if applicable (pay attention to cross-compile issues)
• Common locations

In the documentation, the user options section syntax is described as follows:

options {
    boolopt            => "a boolean option which defaults to disabled"
    boolopt2=1         => "a boolean option which defaults to enabled"
    stringopt:         => "an option which takes an argument, e.g. --stringopt=value"
    stringopt2:=value  => "an option where the argument is optional and defaults to 'value'"
    optalias booltopt3 => "a boolean with a hidden alias. --optalias is not shown in --help"
    boolopt4 => {
        Multi-line description for this option
        which is carefully formatted.
    }
}

Question 1

However, examining the function options-add

...etc

let's checkout this area:

This is the case in which a line in the user options section does contain a colon, and thus the option is a string option.
Looking at line

However, I see no mention of this in the documentation. Is this an undocumented feature?

I also find it non-intuitive that using the with-option-foo:name=default syntax will result in the help text describing the argument to --with-option-foo= as name, but will internally create a variable that holds the value of the argument as
with-option-foo.
My expectation is that that syntax would result in the help for the argument being ?=y? and the internal variable being declared as name with the value of default or whatever the user passes.

Question 2

Similar to my previous question, it seems that this area of that function:

is the implementation of this part of the documented syntax for user options:
tringopt2:=value => "an option where the argument is optional and defaults to 'value'"

This sets what the default value looks like in the help text, but does not actually propagate this value into autosetup(optdefault), as I think the documentation implies.

Further, this line

Is this also a mistake?

(Written here for the lack of a mailing list for autosetup.)

It may be some 10 years too late to use Jam (or, more specifically, the FreeType.org version of it, FT-Jam) as a build tool, but I still find it nice; and its build scripts are concise and to the point. So, here is a plugin for autosetup that provides the glue between an autosetup-driven configuration system and a Jam-driven build. It works for C projects on Linux at least and supports out-of-tree builds, cross-compilation (at least I've succeeded in doing a MinGW cross-compile on Linux) and shared libraries with versioned SONAMEs. Unfortunately, no automake-compliant jam (un)install targets yet, and much more testing on different platforms is needed.

Available in Git: http://github.com/alex-shpilkin/autosetup-jam. A port of the typical autosetup example is included. (I'm not experienced in automake, so no Makefile.am ports yet.)

The question is whether the adopted 'plugin' strategy is really adequate. It means copying the plugin files over to the autosetup development directory; it also involves extending autosetup functionality that wasn't really meant to be. In particular, I had to provide a local.tcl file in order to customize --install and --init stages, and the latter involved copying a considerable chunk of code from init.tcl (the part that creates a configure script).

FAQ:

platform-indepdendent => platform-independent

Autosetup should have a way to say "find a function foo() defined in foo.h and provided in libfoo underneath canonically-structured directory /foo". Autosetup provides all of the pieces to do this, but not a single wrapper function for it all.

I believe this should be a core function because it's a common pattern to have a non-system directory with lib and include subdirs which provide and declare (respectively) a non-standard library interface that a program needs to run. If found, you should have the option to modify CFLAGS, LDFLAGS, and LIBS accordingly or to define a separate set of parallel variables with the necessary flags.

The specific need that sparked this is linking a program to bcm_host.so on a Raspberry Pi, with functions defined in bcm_host.h, both under non-system directory /opt/vc. This is a common pattern, used by many other non-system libraries.

I just wrote the following bit in an auto.def file, but it seems like something like it should be provided by Autosetup so I can make a one-line call to it from my auto.def file.

# Check for libraspberrypi-dev stuff
set old_LIBS [get-define LIBS]      ;# don't want LIBS modded directly
if {![file isdirectory "/opt/vc"]} {
    msg-result "WARNING: Not building on a Pi or libraspberrypi-dev not installed!"
    define PI_CFLAGS ""
    define PI_LFLAGS ""
    define PI_LIBS   ""
} elseif {![cc-with {-cflags "-L/opt/vc/lib"} {
             cc-check-function-in-lib bcm_host_get_peripheral_address bcm_host
           }]} {
    user-error "Found /opt/vc but cannot link to -lbcm_host!"
} elseif {![cc-with {-cflags "-I/opt/vc/include"} {
             cc-check-includes bcm_host.h
           }]} {
    user-error "Found /opt/vc but cannot #include bcm_host.h!"
} else {
    define PI_CFLAGS "-I/opt/vc/include"
    define PI_LFLAGS "-L/opt/vc/lib"
    define PI_LIBS   "-lbcm_host"
}
define LIBS $old_LIBS

As hinted in the code above, it would be nice if there was a way to have the mechanism modify a non-standard set of variables (PI_* in my example) for the case where that Makefile needs these variables in some cases and not others.

Did I overlook an existing mechanism like this in Autosetup, or is this a good feature to add to Autosetup? I can see lots and lots of use for something like this.

A possible extension to this idea is to implement it in terms of pkg-config. To make it work with the above use case, you'd need a way to override PKG_CONFIG_PATH before calling pkg-config --cflags bcm_host and such because this package doesn't install itself to the default search path for pkgconfig. This could just be a higher-level wrapper function on the basic function, taking just the pkgconfig package name and optional PKG_CONFIG_PATH value and providing all of the parameters to the lower-level version.

Caught by a chance:

. string replace Jam 1 1 i
Jam

returns origin string (should return Jim). It seems that must be fixed as

diff --git a/jimsh0.c b/jimsh0.c
index 3ba77d5..07963a0 100644
--- a/jimsh0.c
+++ b/jimsh0.c
@@ -7833,7 +7833,7 @@ Jim_Obj *JimStringReplaceObj(Jim_Interp *interp,
         return NULL;
     }

-    if (last <= first) {
+    if (last < first) {
         return strObjPtr;
     }

"If the doesn’t exist" : developer/reference
"idiosyncracies In general" : developer/reference
"to strip to strip" : developer/reference
"decl" : developer/reference
"given given" : developer/reference
"It the executable is not found" : developer/reference

"@ CC@ is replace" : developer

This isn't really a bug, per se, but is an unfortunate side-effect of how cc-check-xxx works:

$ CC_FOR_BUILD=clang CC=clang ./configure
...
Checking whether the C compiler accepts -Wimplicit-fallthrough=5...yes
...

$ make
...
+ clang ... test.c
warning: unknown warning option '-Wimplicit-fallthrough=5'; did you mean '-Wimplicit-fallthrough'? [-Wunknown-warning-option]

That "mismatch" would seem to be a side-effect of cc-check-xxx necessarily compiling with no warning/error flags.

There is a workaround, and i'm posting this ticket primarily to document that workaround in case anyone else stumbles over this (not because i expect that this can be fixed in autosetup):

cc-with {-cflags {-Wall -Wextra -Werror}} {
    cc-check-flags -Wimplicit-fallthrough=5
}

That (as hoped) will pass on gcc7+ but fail on clang.

(And, for the public record, i HATE the gcc devs for making implicit fall-through a warning offense. One does not simply take a perfectly valid, widely-used 40-year-old feature and suddenly start reporting it via the warning flags. Those of us who habitually compile with -Werror now all hate you guys. We all now get to go "fix" all of our well-behaved source trees to please that handful of people who can't handle switch fall-through properly. Grrrrr.)

This used to work:
--docdir='${prefix}/share/doc/jim'
Now, @DocDir@ will be messed-up in the Makefile.

I don't know if this now how things should be, and the change wasn't more than a minor irritation for me, so I'm not sure if this is a bug report or just complaining. :)

Is this supposed to work in a Makefile template?

@if INCLUDE_IT
"INCLUSION ENABLED"
@include sub.make
@else
"INCLUSION DISABLED"
@endif

I thought it was, and we're relying on it some.. but apparently it doesn't. The test I put in place over here generates the following Makefile

# I have been included
"INCLUSION DISABLED"

So the conditional block works somehow, i.e., I don't get the "INCLUSION ENABLED" string in the Makefile, but apparently the @include directive doesn't care?

When AUTOREMAKE is triggered by a subdirectory's makefile, paths resolved in the auto.def are treated as relative to the current (sub) dir. This causes, e.g., configure to write generated files in the subdir instead of the top dir. My recommendation is to add a cd $builddir && or cd $builddir; to the start of AUTOREMAKE, like i've done in my Makefile.in:

AUTOREMAKE := cd @TOP_SRCDIR@; @AUTOREMAKE@

Currently, the output of any *.in file is rewritten every time the configure script runs, which means if the Makefile has dependencies on any of the output files, those dependents get rebuilt even though nothing will change afterward.

I propose that the piece of autosetup that writes these files out change so that:

1. It writes the output of substituting variables into foo.in to foo.in.tmp instead of to foo.
2. If foo doesn't exist yet, rename foo.in.tmp to foo.
3. If the file size of foo is the same as foo.in.tmp, do a byte-by-byte comparison: if they are the same, then remove foo.in.tmp, else continue to case 4.
4. Remove foo and rename foo.in.tmp to foo: the file contents differ.

I'm willing to do this myself, and I'm just about annoyed enough by the needless rebuilds to do it soon, but I thought I'd file it in case you wanted to tackle it yourself. I'll be more than happy to just pull the result into my package. :)

EDIT: It would also be nice if the Created foo from foo.in message output by autosetup would change to Updated foo from foo.in in case 4 above and to Left foo alone; unchanged in the "then" branch of case 3 above.

assuming the jim0.c in this git repo is the one shipped with autosetup'd packages, 560KB is added to every single shipped package. this is by far more bloated than typical (sane) autoconf stuff.
ideally the interpreter should be much leaner (it takes 45 secs to compile on my 800 mhz arm netbook): but as a first step it could notably reduced in filesize for shipping by applying some source code transformations.

while(<>) {
    my @a = split /\s+/;
    next if !defined($a[0]);
    for my $i(0..scalar(@a)-2) {
        print $a[$i] . " " if length($a[$i]);
    }
    print $a[scalar(@a)-1], "\n";
}

applying this script to remove adjacent whitespace reduces the filesize by 130 kb already.
further improvements could be achieved by using short defines like U for often used words like interp etc and by removing unnecessary spacing (for example between ) and {).
also there seems to be code like tcp/ip stuff that could be stripped away for distribution.
optimally the interpreter source is <100kb and compiles in less than 10 sec on slow devices in the end.

It doesn't appear as though there's a good way to detect whether or not a compiler supports a given flag. Specifically, I'm trying to detect whether or not LTO is supported via the -flto flag.

clang and gcc > 4.6 both do, but before gcc 4.6, it's not supported. I looked in to how I could possibly hook in to cctest to accomplish this, but it doesn't look like there's a clean way to search through errinfo for text output. Thoughts on how to proceed?

$ clang++ -flto -c /tmp/foo.cpp
$ clang++ -flto-enoent -c /tmp/foo.cpp
clang: warning: argument unused during compilation: '-flto-enoent'
Exit 0
$ g++-mp-4.6 -flto -c /tmp/foo.cpp
$ g++-mp-4.6 -flto-enoent -c /tmp/foo.cpp
cc1plus: error: unrecognized command line option '-flto-enoent'
Exit 1

In a project of mine, I've come to write several additional checks for C/C++ compilation. Not that it is hard to write them on one's own, but I thought that they would be useful to autosetup users in general, if put into cc or cc-lib. I am not quite sure about where they should go, but I'll be glad to provide a patch once if it turns out that they are actually worth including.

Below is the autosetup --manual=markdown output for the (separate for now) module containing the checks.

Module: cc-ext

The cc-ext module provides additional checks for C compiler features.

cc-check-flags flag...

Checks whether the given C/C++ compiler flags can be used. Defines feature names prefixed with HAVE_CFLAG and HAVE_CXXFLAG respectively, and appends working flags to -cflags and CFLAGS or CXXFLAGS.

cc-check-standards ver...

Checks whether the C/C++ compiler accepts one of the specified -std=$ver options, and appends the first working one to -cflags and CFLAGS or CXXFLAGS.

cc-check-c11

Checks for several C11/C++11 extensions and their alternatives. Currently checks for _Static_assert, _Alignof, __alignof__, __alignof.

here: http://msteveb.github.io/autosetup/why/
title is "autosetup - "

Currently the user manual intermixes both end user and developer information.

For the casual user, the end usage should be readily accessible while the developer info should be presented separately or later in the manual.

$title

Hiya!

[stephan@cheyenne:~/cvs/autosetup/examples/dropbear]$ ./configure --prefix=$HOME
...
[stephan@cheyenne:~/cvs/autosetup/examples/dropbear]$ make distclean
...
make[2]: Entering directory `/home/stephan/cvs/autosetup/examples/dropbear/libtommath'
rm -f *.bat *.pdf *.o *.a *.obj *.lib *.exe *.dll etclib/*.o demo/demo.o test ltmtest mpitest mtest/mtest mtest/mtest.exe \
    *.idx *.toc *.log *.aux *.dvi *.lof *.ind *.ilg *.ps *.log *.s mpi.c *.da *.dyn *.dpi tommath.tex `find . -type f | grep [~] | xargs` *.lo   *.la
rm -rf .libs
cd etc ; MAKE=make make clean
cd: 1: can't cd to etc
make[3]: Entering directory `/home/stephan/cvs/autosetup/examples/dropbear/libtommath'
...

and from there it endlessly loops in libtommath.

I use @top_srcdir@ in compat/Makefile.in like this SRCDIR= @top_srcdir@/something and I end up with ./something instead of an absolute path (note that . is not even correct, it should be .. for relative)

autosetup needs a test suite to check for regressions.
The test suite needs to be run under both Jim and Tcl.

Autosetup version: 732d2b3 pulled a few hours ago.

i'm not 100% certain this is a bug, but it's bitten me twice now and certainly seems like one:

i'm on a system with no c++ installed. My auto.def is implicitly checking for C++ via:

cc-with {-lang c++} { ... }

And the end result is that CXX gets set to "false" (without the quotes). That, of course, causes make to try to use false as the C++ compiler.

i've worked around this by doing:

set cxxCheck [get-define CXX]
if {false eq $cxxCheck} {
  define CXX ""
}

But that seems really kludgy.

Can you confirm whether "false" is the expected value for a binary which is not found?

I just ran into a bit of a buzz saw trying to use @top_srcdir@ in a project.

The situation is that I have a script tools/foo.in which needs to refer to files stored elsewhere in the source tree. The script is called as tools/foo from the top-level Makefile, so CWD is tools/.., which means you cannot simply prepend @top_srcdir@ to source tree paths in the obvious way:

media_path = "@top_srcdir@/media"

because it turns into ../media, which when interpreted from the CWD of the script tries to look for a ../media directory relative to the top-level source directory, where make is run from. There is no such directory, so it fails.

@srcdir@ doesn't help here: it only shifts the same problem by one directory level.

Building out-of-tree adds a second layer to the problem since these autosetup variables become absolute paths, so the above example turns into /path/to/source/tree/media, so it's now pointing at the directory we intended above: that is, ../media relative to the tools directory, rather than relative to the top source directory! This means you cannot fix the problem so it works with both cases, due to this mix of absolute and relative paths.

The solution is this one-liner in auto.def:

 define abs_top_srcdir [file normalize [get-define top_srcdir]]

This does two things:

1. By doing it in auto.def, the definition of @top_srcdir@ doesn't depend on CWD being equal to the current file's location. CWD is the top of the source tree at this point, by definition.

2. By normalizing the path, we don't have a behavior shift between building in-tree and building out-of-tree. The path refers to the same directory in both cases: the top source directory, as it says on the label.

I realize you can't just redefine the way @srcdir@ and @top_srcdir@ are defined, since other projects will have made their separate accommodations with its current definition, and thus may break if you "fix" their definitions.

I am instead asking that you add the above definition or something like it as an alternative to @top_srcdir@.

When performing "./autosetup --help" in the code repository, no help message is displayed. In fact, no message is displayed at all.

This is true for both the built in repository autosetup file, but also for the examples.

An older version (v0.6.2) seemed to work, as it is what fossil v1.29 used. Running ./configure --help in fossil source repo, prints a nice help message, but doing the same in either the autosetup repository, or with a configure file generated by it, does nothing, regardless of auto.def existance.