ninja-build / ninja Goto Github PK

A bootstrap.bat should be provided for Windows, to reduce dependencies.

Today I had an interesting experience updating ninja; I started w/ a git pull and optimistically tried to ninja ninja; that failed b/c the build.ninja file was out of date. So then I tried ./bootstrap.sh and that build ninja manually ok, but then failed to link ninja b/c of duplicate symbols (between state.o & ninja_jumble.o). But ninja_jumble.o shouldn't have been getting linked at this point! It seems like a libninja.a from a previous build was being appended to instead of replaced, somehow. Running ninja -t clean & re-runnning bootstrap.sh made everything hoopy again (and ninja can rebuild itself directly now, too).

This bug is about build/libninja.a containing a ninja_jumble.o from a previous build during the subsequent build where ninja_jumble.o shouldn't be getting linked.

Console logs pasted below for the steps described above.

$ git pull
remote: Counting objects: 1414, done.
remote: Compressing objects: 100% (455/455), done.
remote: Total 1093 (delta 702), reused 925 (delta 534)
Receiving objects: 100% (1093/1093), 2.14 MiB | 1.10 MiB/s, done.
Resolving deltas: 100% (702/702), completed with 221 local objects.
From git://github.com/martine/ninja
   65eeebd..48c54a3  gh-pages   -> origin/gh-pages
   224f251..3fbb25b  master     -> origin/master
Auto-merging src/build.cc
Merge made by recursive.
 configure.py                                  |   69 +++++--
 doc/manual.asciidoc                           |   68 +++++--
 misc/ninja_syntax.py                          |   13 +
 src/browse.cc                                 |    2 +-
 src/build.cc                                  |   18 +-
 src/build.h                                   |    9 +
 src/build_log.cc                              |   16 +-
 src/build_log.h                               |    5 +-
 src/build_test.cc                             |   30 ++-
 src/clean.cc                                  |    3 +-
 src/clean.h                                   |    9 +-
 src/{ninja_test.cc => disk_interface_test.cc} |  295 +++++++++----------------
 src/eval_env_test.cc                          |  101 +++++++++
 src/graph.cc                                  |   32 ++-
 src/hash_map.h                                |   29 +++
 src/ninja.cc                                  |   69 +++++-
 src/parsers.cc                                |   82 ++++++--
 src/parsers.h                                 |    8 +-
 src/parsers_test.cc                           |   89 +++++++-
 src/stat_cache.cc                             |    4 +-
 src/stat_cache.h                              |    4 +-
 src/{ninja_jumble.cc => state.cc}             |   43 ++--
 src/{ninja.h => state.h}                      |   30 ++-
 src/state_test.cc                             |   41 ++++
 src/string_piece.h                            |   51 +++++
 src/subprocess-win32.cc                       |    6 +-
 src/subprocess.cc                             |    6 +-
 src/test.h                                    |    2 +-
 28 files changed, 779 insertions(+), 355 deletions(-)
 rename src/{ninja_test.cc => disk_interface_test.cc} (72%)
 create mode 100644 src/eval_env_test.cc
 rename src/{ninja_jumble.cc => state.cc} (87%)
 rename src/{ninja.h => state.h} (86%)
 create mode 100644 src/state_test.cc
 create mode 100644 src/string_piece.h

$ ninja ninja 
[2/9/1] CXX build/ninja_jumble.o
FAILED: clang++ -MMD -MF build/ninja_jumble.o.d -g -Wall -Wno-deprecated -fno-exceptions -fvisibility=hidden -pipe -O2  -c src/ninja_jumble.cc -o build/ninja_jumble.o
clang: error: no such file or directory: 'src/ninja_jumble.cc'
clang: error: no input files
build stopped: subcommand failed.

$ ./bootstrap.sh 
Building ninja manually...
Building ninja using itself...
wrote build.ninja.
[0/0/3] LINK ninja
FAILED: clang++ -B/usr/local/gold/bin -Lbuild  -o ninja build/ninja.o -lninja
/usr/local/gold/bin/ld: error: build/libninja.a(state.o): multiple definition of 'State::LookupNode(std::string const&)'
/usr/local/gold/bin/ld: build/libninja.a(ninja_jumble.o): previous definition here
/usr/local/gold/bin/ld: error: build/libninja.a(state.o): multiple definition of 'State::LookupRule(std::string const&)'
/usr/local/gold/bin/ld: build/libninja.a(ninja_jumble.o): previous definition here
/usr/local/gold/bin/ld: error: build/libninja.a(state.o): multiple definition of 'State::kPhonyRule'
/usr/local/gold/bin/ld: build/libninja.a(ninja_jumble.o): previous definition here
/usr/local/gold/bin/ld: error: build/libninja.a(state.o): multiple definition of 'State::AddIn(Edge*, std::string const&)'
/usr/local/gold/bin/ld: build/libninja.a(ninja_jumble.o): previous definition here
/usr/local/gold/bin/ld: error: build/libninja.a(state.o): multiple definition of 'State::AddOut(Edge*, std::string const&)'
/usr/local/gold/bin/ld: build/libninja.a(ninja_jumble.o): previous definition here
/usr/local/gold/bin/ld: error: build/libninja.a(state.o): multiple definition of 'State::AddEdge(Rule const*)'
/usr/local/gold/bin/ld: build/libninja.a(ninja_jumble.o): previous definition here
/usr/local/gold/bin/ld: error: build/libninja.a(state.o): multiple definition of 'State::AddRule(Rule const*)'
/usr/local/gold/bin/ld: build/libninja.a(ninja_jumble.o): previous definition here
/usr/local/gold/bin/ld: error: build/libninja.a(state.o): multiple definition of 'State::GetNode(std::string const&)'
/usr/local/gold/bin/ld: build/libninja.a(ninja_jumble.o): previous definition here
/usr/local/gold/bin/ld: error: build/libninja.a(state.o): multiple definition of 'State::RootNodes(std::string*)'
/usr/local/gold/bin/ld: build/libninja.a(ninja_jumble.o): previous definition here
/usr/local/gold/bin/ld: error: build/libninja.a(state.o): multiple definition of 'State::State()'
/usr/local/gold/bin/ld: build/libninja.a(ninja_jumble.o): previous definition here
/usr/local/gold/bin/ld: error: build/libninja.a(state.o): multiple definition of 'State::State()'
/usr/local/gold/bin/ld: build/libninja.a(ninja_jumble.o): previous definition here
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

$ ./bootstrap.sh
Building ninja manually...
Building ninja using itself...
wrote build.ninja.
[0/0/16] LINK ninja
Done!

Say you've got a build that's fully up to date.

Now run ./configure --foobar, producing a new build.ninja.

The next time you run ninja, it will first notice that the command line in the log doesn't match the new command line we've recorded (with --foobar), and run configure a second time.

My first instinct is to say that the problem lies in that configure is designed to be run by hand as well as by the build, and that we shouldn't trust the logged command in the build log. We can maybe add an attribute to the rule that says as much. It feels like a hack though.

Although the definition of variables with upper-case characters is recognized by the parser,
the expansion (usage) of these variables, when not enclosed in { .. }, is not supported.

The problem is located in eval_env.cc, which uses the statement "if (!(('a' <= c && c <= 'z') || ('0' <= c && c <= '9') || c == '_'))" to detect errors with variable names.

By adding ('A' <= c && c <= 'Z') in the above condition solves this.

[EDIT]

Here is the patch.

diff -Naur ninja/src/eval_env.cc ninja-new/src/eval_env.cc
--- ninja/src/eval_env.cc   2011-02-21 00:14:12.000000000 +0200
+++ ninja-new/src/eval_env.cc   2011-02-21 09:08:56.000000000 +0200
@@ -56,7 +56,7 @@
     } else {
       for (end = start; end < input.size(); ++end) {
         char c = input[end];
-        if (!(('a' <= c && c <= 'z') || ('0' <= c && c <= '9') || c == '_'))
+        if (!(('a' <= c && c <= 'z') || ('A' <= c && c <= 'Z') || ('0' <= c && c <= '9') || c == '_'))
           break;
       }
       if (end == start) {

I thing that in bootstrap.sh #!/bin/bash should be replaced by #!/bin/sh, because on FreeBSD and some other system /bin/bash is not available by default. And this script don't use any bashism that don't work on standard /bin/sh.

Of course I can install bash from FreeBSD ports, but this goes to /usr/local/bin/bash,
so I still need change this line. Another option is using #!/usr/bin/env bash for shearching in $PATH dirs, but #!/bin/sh still looks better for me.

Behold:

fischman@trap ~/src/ninja {master} $ ninja ninja
build stopped: no work to do.
fischman@trap ~/src/ninja {master} $ ninja -k ninja
[1/10/1] DOXYGEN doc/doxygen.config
FAILED: doxygen doc/doxygen.config
/bin/sh: doxygen: command not found
build stopped: subcommand failed.
fischman@trap ~/src/ninja {master} exit:1 $ ninja
[1/9/1] DOXYGEN doc/doxygen.config
FAILED: doxygen doc/doxygen.config
/bin/sh: doxygen: command not found
build stopped: subcommand failed.
fischman@trap ~/src/ninja {master} exit:1 $ 

Looks a lot like omitting the numeric value to the -k flag causes it to silently eat the next param as the -k value.

It would be better to set the fd for .build_log to close-on-exec. May need something different on the Windows platform.

Ninja does not seem to support multiple outputs properly. Even though it claims to do so in the manual page: "
Comparison to GNU make ... A build edge may have multiple outputs. " The following is what I've tried:

$ ls
build.ninja
$ cat build.ninja
rule fork
description = FORK ($in) to ($out)
command = touch $out
build out1 out2: fork in.txt
$ ninja
ninja: error: 'in.txt', needed by 'out1', missing and no known rule to make it
$ touch in.txt
$ ninja
[1/1] FORK (in.txt) to (out1)
$ ninja -v
[1/1] touch out1
$ ls
build.ninja in.txt out1
$ ninja -n -v
[1/1] touch out1
$ touch out2
$ ninja -n -v
build stopped: no work to do.
$ ls
build.ninja in.txt out1 out2
$ ninja -t clean
Cleaning... 2 files.
$ ls
build.ninja in.txt

I expect this build.ninja file to create two files, but only out1 is created. Oddly enough, ninja somehow knows it should build 2 files, so it always shows command need to run until out2 is manually created. And -t clean knows to delete 2 files as well.

Looking into the source code, it appears that the parsing code (parser.cc) does handle multiple outputs:

421   for (;;) {
422     if (tokenizer_.PeekToken() == Token::COLON) {
423       tokenizer_.ConsumeToken();
424       break;
425     }
426
427     string out;
428     if (!tokenizer_.ReadIdent(&out))
429       return tokenizer_.ErrorExpected("output file list", err);
430     outs.push_back(out);
431   }

But in EdgeEnv (graph.cc), only the first out file is returned:

107     } else if (var == "out") {
108       result = edge_->outputs_[0]->file_->path_;
109     }  ...

I wonder whether this is the only place that needs fix though.

When we've been waiting for commands for more than a second with no output, print some "still waiting for xyz" text.

Ninja (bb52198) just crashed on me for the first time in a while with the below backtrace according to gdb. Re-run worked fine.

Core was generated by `/home/fischman/src/ninja/ninja -C ninja/Debug -j32 -j320 chrome'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007f4a1c2c7e6f in __libc_free (mem=0x7da48d0) at malloc.c:3736
3736    malloc.c: No such file or directory.
        in malloc.c
(gdb) bt
#0  0x00007f4a1c2c7e6f in __libc_free (mem=0x7da48d0) at malloc.c:3736
#1  0x0000000000406103 in deallocate (this=0x7da48a0, edge=0x19363d0, success=true, output="", start_time=0x7fff9a4092dc, end_time=<value optimized out>) at /usr/include/c++/4.4/ext/new_allocator.h:95
#2  _M_put_node (this=0x7da48a0, edge=0x19363d0, success=true, output="", start_time=0x7fff9a4092dc, end_time=<value optimized out>) at /usr/include/c++/4.4/bits/stl_tree.h:363
#3  _M_destroy_node (this=0x7da48a0, edge=0x19363d0, success=true, output="", start_time=0x7fff9a4092dc, end_time=<value optimized out>) at /usr/include/c++/4.4/bits/stl_tree.h:384
#4  erase (this=0x7da48a0, edge=0x19363d0, success=true, output="", start_time=0x7fff9a4092dc, end_time=<value optimized out>) at /usr/include/c++/4.4/bits/stl_tree.h:1348
#5  erase (this=0x7da48a0, edge=0x19363d0, success=true, output="", start_time=0x7fff9a4092dc, end_time=<value optimized out>) at /usr/include/c++/4.4/bits/stl_map.h:567
#6  BuildStatus::BuildEdgeFinished (this=0x7da48a0, edge=0x19363d0, success=true, output="", start_time=0x7fff9a4092dc, end_time=<value optimized out>) at src/build.cc:104
#7  0x0000000000406361 in Builder::FinishEdge (this=0x7fff9a409520, edge=0x19363d0, success=false, output="") at src/build.cc:516
#8  0x0000000000406631 in Builder::Build (this=0x7fff9a409520, err=0x7fff9a409820) at src/build.cc:444
#9  0x0000000000403dd7 in main (argc=<value optimized out>, argv=0x1) at src/ninja.cc:562

It's not possible using the web browser (python plugin) to browse a project that uses a different build file name to build.ninja

I'd like to make it into patch, but I don't know various ifdefs for Linux/BSD/Mac platforms, so here's a version of GuessParallelism() that works on Mac OS X, and should work on BSD:

#include <sys/sysctl.h>

int GuessParallelism() {
  int processors = 0;

  size_t procSize = sizeof(processors);
  int name[] = {CTL_HW, HW_NCPU};
  if (sysctl(name, 2, &processors, &procSize, NULL, 0))
    return 2;

  switch (processors) {
  case 0:
  case 1:
    return 2;
  case 2:
    return 3;
  default:
    return processors + 2;
  }
}

E.g. that it runs via sh -c. Similar for Windows.

When adding a rule which parses gcc -MF files, the dependencies seems to be recorded, but not shown on -t graph or -t browse commands. See my repository for a simple example https://git.bettercodes.org/shinobi

The file build.ninja, won't show dependency between main.cc and libt1.h

cxx = g++
cflags = -Wall -Wextra
rule cxx
command = $cxx $includes $incdep -MMD -MF $out.d -fPIC $cflags -c $in $
-o $out
description = CXX $out
depfile = $out.d
rule ar
command = ar crs $out $in
description = AR $out
rule exe
command = g++ -o $out $in $libs
description = LNK $out
build main/debug/main.o: cxx main/main.cc
incdep = -I'test' -I'/usr/include/'
build stpub/debug/main: exe main/debug/main.o
libs = -Lstpub/debug -ltest -L/usr/lib -lssl
build test/debug/get12.o: cxx test/get12.cc
incdep =
build test/debug/get11.o: cxx test/get11.cc
incdep =
build stpub/debug/libtest.a: ar test/debug/get12.o test/debug/get11.o
libs =


Hello,
I found a problem when run bootstrap.sh on Mac OS X and FreeBSD.

% ./bootstrap.sh 
Building ninja manually...
In file included from src/browse.cc:20:
src/../build/browse_py.h:249: error: \x used with no following hex digits
src/../build/browse_py.h:249: error: \x used with no following hex digits
(many same lines...)

Because od command prints many blank characters.

% cat src/browse.py | od -t x1 -A n -v | head -3
           23  21  2f  75  73  72  2f  62  69  6e  2f  70  79  74  68  6f
           6e  0a  23  0a  23  20  43  6f  70  79  72  69  67  68  74  20
           32  30  30  31  20  47  6f  6f  67  6c  65  20  49  6e  63  2e

I hope my gist help you.
https://gist.github.com/860677

On Windows (XP or greater), Command lines are limited in length to 8,191 bytes.

Windows compiler toolsets (and gcc) get around this by allowing the @filepath syntax on the command line.

e.g. link.exe @temp\MyLongLinkLinesInAFile.rsp

will run link.exe, which will then use the contents of the file Temp\MyLongLinkLinesInAFile.rsp as if they were on the command line.

This issue is to suggest that it is only ninja that could know the command line is being exceeded. A generator of .ninja files could know they are getting close, but couldn't know that 8.191 bytes exactly has been reached.

This issue is raised to ask if a patch to ninja that wrote >8191 character command lines into a file then called the build rule with the file (ninja to name the file) could be acceptable. If so, what exactly should the patch do? I propose:

ninja detects it's on Windows and total command line to run is >8,191 characters
ninja creates a .rsp file in Temp with a unique name
ninja writes command line argument contents into that file
ninja invokes the original command line first argument but with the @filename syntax
ninja cleans up the .rsp file

Do others have any thoughts on this?

Repro steps:

1. In a Chrome checkout, edit chrome/browser/resources/sync_internals/sync_index.html (add a comment or something)
2. Do ninja -j100 chrome

Expected results:
stuff in chrome/chrome_extra_resources.ninja gets built, and chrome is rebuilt with the updated resources.

Actual results:
ninja does nothing

If we have a project consisting of two files:

foo.cpp
bar.cpp

and each of them has an independent #include chain, then we deal with this using .dep files:

rule cxx
depfile = $out.dep
command = gcc -c -o $out -MF $out.dep

this is great and works.

I also have a prebuild step which I want to run. This does not produce any output (or it might). I want it to run before the files are compiled, but I only want it to run if actual work needs doing. In otherwords, I want it to depend on everything that my .cpp files depend on, since if any of them need redoing then I want to execute.

I would hope to be able to do this something like:

rule execute
depfile = foo.obj.dep bar.obj.dep
command = ./prebuild.sh

but this would need me to depend on multiple .dep files and this is not supported.

It'snot just prebuild either, we also have a couple of steps in our build chain which we want to execute (prior to compilation) if, and only if, any compilation would actually need done.

In my usage, depfiles didn't handle things like c:\foo well. I think ninja is expecting the format to be ident: ident+ newline. ":" is not an identifier char. I worked around this by treating quoted entities as identifiers for makefile_flavor only.

I'm not sure if depfiles was intended solely for gcc, which I'm not using. If it is I'd recommend making the depfiles rule key mention gcc specifically (i.e. gccdepfiles), and have a generic ninja-controlled format for depfiles [if depfiles are smiled upon].

Using ninja on a Win32 environment, we occasionally see some weird problems. Sometimes, it hangs forever. And sometimes, we get fatal messages, "CreateNamedPipe: There is a process on other end of the pipe."

I spent a little time to look into subprocess-win32.cc, and in particular the SetupPipe function. It seems to me that ninja is using a strange way to decide named pipe name:

snprintf(pipe_name, sizeof(pipe_name), "\.\pipe\ninja_%p_out", ::GetModuleHandle(NULL));

I don't quite understand what is the intention here. If it is to ensure each ninja.exe is using a different pipe name, then the name of the pipe should at least contain a process ID. As it is, it is likely that different ninja.exe on the same machine may use the same pipe name, and I wonder whether that work well.

The other factor is that it might be better to use different names for the pipes in the same process. That can be achieved easily by appending the "this" pointer of the subprocess object to the pipe name. I don't quite know whether it is by design that all pipes in the same process should use the same named pipe.

Touching third_party/ffmpeg/patched-ffmpeg-mt/libavformat/matroskadec.c correctly makes ninja recompile & relink ffmpegsumo, but the same touch doesn't make ninja rebuild test_shell
$ ninja -v test_shell
WARNING: multiple rules generate ninja/gen/protoc_out/chrome/browser/history/in_memory_url_index_cache.pb.o. build will not be correct; continuing anyway
$ touch third_party/ffmpeg/patched-ffmpeg-mt/libavformat/matroskadec.c
$ ninja -v test_shell
WARNING: multiple rules generate ninja/gen/protoc_out/chrome/browser/history/in_memory_url_index_cache.pb.o. build will not be correct; continuing anyway
$

As a result, introducing an #error into ffmpeg sources fails to cause ninja test_shell rebuild to fail if a previous build already created ffmpegsumo.

(I reported this via direct email a while ago but figured I'd open an issue to avoid losing track of it; feel free to resolve if it isn't helpful)

EDIT

Hi martine, thanks for the response.

I have included a test case for variables with uppercase characters, named OneVariableUpperCase. It is basically a copy-paste of the OneVariable test with the variable name changed.

Here is the new patch which includes both the changes to the parser and the test case.

diff -Naur ninja/src/eval_env.cc ninja-new/src/eval_env.cc
--- ninja/src/eval_env.cc   2011-02-21 00:14:12.000000000 +0200
+++ ninja-new/src/eval_env.cc   2011-02-26 03:29:29.000000000 +0200
@@ -56,7 +56,7 @@
     } else {
       for (end = start; end < input.size(); ++end) {
         char c = input[end];
-        if (!(('a' <= c && c <= 'z') || ('0' <= c && c <= '9') || c == '_'))
+        if (!(('a' <= c && c <= 'z') || ('A' <= c && c <= 'Z') || ('0' <= c && c <= '9') || c == '_'))
           break;
       }
       if (end == start) {
diff -Naur ninja/src/ninja_test.cc ninja-new/src/ninja_test.cc
--- ninja/src/ninja_test.cc 2011-02-21 00:14:12.000000000 +0200
+++ ninja-new/src/ninja_test.cc 2011-02-26 04:35:08.000000000 +0200
@@ -80,6 +80,17 @@
   env.vars["var"] = "there";
   EXPECT_EQ("hi there", str.Evaluate(&env));
 }
+TEST(EvalString, OneVariableUpperCase) {
+  EvalString str;
+  string err;
+  EXPECT_TRUE(str.Parse("hi $VaR", &err));
+  EXPECT_EQ("", err);
+  EXPECT_EQ("hi $VaR", str.unparsed());
+  TestEnv env;
+  EXPECT_EQ("hi ", str.Evaluate(&env));
+  env.vars["VaR"] = "there";
+  EXPECT_EQ("hi there", str.Evaluate(&env));
+}
 TEST(EvalString, Error) {
   EvalString str;
   string err;

ORIGINAL

Although the definition of variables with upper-case characters is recognized by the parser,
the expansion (usage) of these variables, when not enclosed in { .. }, is not supported.

The problem is located in eval_env.cc, which uses the statement "if (!(('a' <= c && c <= 'z') || ('0' <= c && c <= '9') || c == '_'))" to detect errors with variable names.

By adding ('A' <= c && c <= 'Z') in the above condition solves this.

Here is the patch.

diff -Naur ninja/src/eval_env.cc ninja-new/src/eval_env.cc
--- ninja/src/eval_env.cc   2011-02-21 00:14:12.000000000 +0200
+++ ninja-new/src/eval_env.cc   2011-02-21 09:08:56.000000000 +0200
@@ -56,7 +56,7 @@
     } else {
       for (end = start; end < input.size(); ++end) {
         char c = input[end];
-        if (!(('a' <= c && c <= 'z') || ('0' <= c && c <= '9') || c == '_'))
+        if (!(('a' <= c && c <= 'z') || ('A' <= c && c <= 'Z') || ('0' <= c && c <= '9') || c == '_'))
           break;
       }
       if (end == start) {

In a virgin ninja checkout:

$ ./bootstrap.sh 
Building ninja manually...
Building ninja using itself...
[13/14] LINK ninja
FAILED: ccache gcc '-B/usr/local/google/home/mihaip/chrome/bin'  -o ninja build/ninja.o -Lbuild -lninja
/usr/bin/ld: build/ninja.o: in function (anonymous namespace)::CollectTargetsFromArgs(State*, int, char**, std::vector<Node*, std::allocator<Node*> >*, std::basic_string<char, std::char_traits<char>, std::allocator<char> >*):/usr/include/c++/4.4/bits/basic_string.h:227: error: undefined reference to 'std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Rep::_S_empty_rep_storage'
/usr/bin/ld: build/ninja.o: in function (anonymous namespace)::CollectTargetsFromArgs(State*, int, char**, std::vector<Node*, std::allocator<Node*> >*, std::basic_string<char, std::char_traits<char>, std::allocator<char> >*):src/ninja.cc:120: error: undefined reference to 'std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&)'
...

If I change bootstrap.sh to set LDFLAGS="-lstdc++", then it links. This used to work in April, but doesn't anymore.

Rather than counting CPUs, just keep spawning subprocesses until load goes up.
Like make's -l flag.

It would be cool if ninja would do simple 1-edit-distance spell checking of target names, like so:
$ ninja fo.o
ninja: error: unknown target 'fo.o', did you mean 'foo.o'?

Make does this, we should too.

While I'm on the subject, it'd be cool to support writing outputs directly to stdout as well, like how redo does it.

In my case, this was a problem in build.ninja where I forgot to set a variable which was used as a dependency.

I made a simple patch: https://gist.github.com/1252031

Since ninja chdir's into the output directory & refers to source files using ../../ prefixes, emacs' compilation-mode (and in particular, next-error) stopped working. I originally solved the problem by having my ninja-wrapping shell-script emit an appropriate line, but this is more principled:

diff --git a/src/ninja.cc b/src/ninja.cc
index c70aa43..cb7d2fa 100644
--- a/src/ninja.cc
+++ b/src/ninja.cc
@@ -449,6 +449,7 @@ int main(int argc, char** argv) {
 #endif
       Fatal("chdir to '%s' - %s", working_dir, strerror(errno));
     }
+    printf("ninja: Entering directory `%s'\n", working_dir);
   }

   bool rebuilt_manifest = false;

(note that wrapping the %s with `' is what make does and what emacs looks for, and that the first character there is a back-quote, not an apostrophe)

(Disclaimer: Just making you aware of this issue. It's not an issue for 99% of people and I don't actually think it's worth fixing other than as a fork if a team needs it)

If you have a file with a '$' in it, it will not be passed to into sh correctly. For example, consider the following build rule (assume a cxx rule has already been set up that works properly).

build $builddir/test$$file.cc.o: cxx test$$file.cc

The double '$' will cause ninja to handle this correctly and will attempt to build the file test$file.cc. On the other hand, sh expands this to "test.cc" and the output file is wrong. This causes ninja to rebuild it.

This gets fixed by escaping the '$' symbols right before sending it to subprocess (but not before that, as that screws up it finding the files).

We have a very large project, so it is not very surprising that it takes about 1 hour to build. But I just realised that to clean it takes more time than building it.

Say TestSuite is the highest level target that depends on all other targets. "ninja -t clean TestSuite" takes hours to finish, although "ninja -t clean" takes less than 1 minute.

I had a look of the source code (clean.cc). It appears that the function Cleaner::DoCleanTarget() is repetitively visiting the same Node. So if one target output file is depended by hundreds of others, DoCleanTarget will be called for it hundreds times. I have a naive fix that uses a set to record visited Node, and does not call it again if a Node is in the set. This has bring the time of "ninja -t clean TestSuite" to within a minute.

I am sure there are better ways to fix it. But if you do not mind, I will send a pull request with the naive fix.

ninja.cc uses the .incbin pseudo op which was included in gas since version 2.12. Apple's last included gas version is 1.38 so the file won't assemble.

If you want to use ninja, delete references for the browse command.

I found that ninja does not tokenize multiple file names defined by a variable. The following is an example to demostrate this.

$ ls
build.ninja novar.ninja

$ cat build.ninja
rule cat
command = cat $in > $out
sourcefiles = in1.txt in2.txt
build out: cat $sourcefiles

$ cat novar.ninja
rule cat
command = cat $in > $out
build out: cat in1.txt in2.txt

$ touch in1.txt

$ touch in2.txt

$ ninja
ninja: error: 'in1.txt in2.txt', needed by 'out', missing and no known rule to make it

$ ninja -f novar.ninja
[1/1] cat in1.txt in2.txt > out

In the build.ninja, ninja parses $sourcefiles as a single input file. I had a look in the source code, in ManifestParser::ParseEdge in parsers.cc, the variable evaluation is done after splitting the input, dependencies, etc.

I wonder whether this is a design decision. If it is, it would be better to clarify this in the manual page, because this is a different behaviour from make.

Hello,
I want to build ./hello.o from ./hello.c.
However, I got a error.

ninja: error: expected makefile to mention './hello.o', got 'hello.o'

You know how to get it to work?
https://gist.github.com/922998

Just saw this once, hasn't recurred on re-runs. No corefile dropped (or else it's been overwritten).

$ time ../goma-ninja -k ppapi_example_c_stub                                       
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/predepends.stamp. build will not be correct; continuing anyway           
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/third_party/mesa/MesaLib/src/egl/main/eglapi.o. build will not be correct; continuing anyway
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/third_party/mesa/MesaLib/src/egl/main/eglarray.o. build will not be correct; continuing anyway
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/third_party/mesa/MesaLib/src/egl/main/eglconfig.o. build will not be correct; continuing anyway
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/third_party/mesa/MesaLib/src/egl/main/eglcontext.o. build will not be correct; continuing anyway
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/third_party/mesa/MesaLib/src/egl/main/eglcurrent.o. build will not be correct; continuing anyway
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/third_party/mesa/MesaLib/src/egl/main/egldisplay.o. build will not be correct; continuing anyway
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/third_party/mesa/MesaLib/src/egl/main/eglglobals.o. build will not be correct; continuing anyway
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/third_party/mesa/MesaLib/src/egl/main/eglimage.o. build will not be correct; continuing anyway
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/third_party/mesa/MesaLib/src/egl/main/egllog.o. build will not be correct; continuing anyway
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/third_party/mesa/MesaLib/src/egl/main/eglmisc.o. build will not be correct; continuing anyway
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/third_party/mesa/MesaLib/src/egl/main/eglmode.o. build will not be correct; continuing anyway
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/third_party/mesa/MesaLib/src/egl/main/eglscreen.o. build will not be correct; continuing anyway
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/third_party/mesa/MesaLib/src/egl/main/eglstring.o. build will not be correct; continuing anyway
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/third_party/mesa/MesaLib/src/egl/main/eglsurface.o. build will not be correct; continuing anyway
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/third_party/mesa/MesaLib/src/egl/main/eglsync.o. build will not be correct; continuing anyway
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/ppapi/lib/gl/egl/egldriver.o. build will not be correct; continuing anyway
ninja: WARNING: multiple rules generate ninja/obj/ppapi_egl/ppapi/lib/gl/egl/egldriver_ppapi.o. build will not be correct; continuing anyway
ninja: WARNING: multiple rules generate ppapi_egl. build will not be correct; continuing anyway                                      
[567/4708] STAMP ninja/obj/lastchange/actions.stamp*** glibc detected *** /home/fischman/src/ninja/ninja: double free or corruption (!prev): 0x000000000179fea0 ***
======= Backtrace: =========
/lib/libc.so.6(+0x775b6)[0x7f04bae9f5b6]
/lib/libc.so.6(cfree+0x73)[0x7f04baea5e83]
/home/fischman/src/ninja/ninja[0x404645]
/home/fischman/src/ninja/ninja[0x405c05]
/home/fischman/src/ninja/ninja[0x405877]
/home/fischman/src/ninja/ninja[0x4031e5]
/lib/libc.so.6(__libc_start_main+0xfd)[0x7f04bae46c4d]
/home/fischman/src/ninja/ninja[0x402059]
======= Memory map: ========
00400000-00418000 r-xp 00000000 fc:01 5378466                            /usr/local/google/fischman/src/ninja/ninja
00418000-00419000 r--p 00018000 fc:01 5378466                            /usr/local/google/fischman/src/ninja/ninja
00419000-0041a000 rw-p 00019000 fc:01 5378466                            /usr/local/google/fischman/src/ninja/ninja
00ac6000-04acf000 rw-p 00000000 00:00 0                                  [heap]
7f04b4000000-7f04b4021000 rw-p 00000000 00:00 0
7f04b4021000-7f04b8000000 ---p 00000000 00:00 0
7f04bae28000-7f04bafa2000 r-xp 00000000 fc:00 788006                     /lib/libc-2.11.1.so
7f04bafa2000-7f04bb1a1000 ---p 0017a000 fc:00 788006                     /lib/libc-2.11.1.so
7f04bb1a1000-7f04bb1a5000 r--p 00179000 fc:00 788006                     /lib/libc-2.11.1.so
7f04bb1a5000-7f04bb1a6000 rw-p 0017d000 fc:00 788006                     /lib/libc-2.11.1.so
7f04bb1a6000-7f04bb1ab000 rw-p 00000000 00:00 0
7f04bb1ab000-7f04bb1c1000 r-xp 00000000 fc:00 789420                     /lib/libgcc_s.so.1
7f04bb1c1000-7f04bb3c0000 ---p 00016000 fc:00 789420                     /lib/libgcc_s.so.1
7f04bb3c0000-7f04bb3c1000 r--p 00015000 fc:00 789420                     /lib/libgcc_s.so.1
7f04bb3c1000-7f04bb3c2000 rw-p 00016000 fc:00 789420                     /lib/libgcc_s.so.1
7f04bb3c2000-7f04bb444000 r-xp 00000000 fc:00 790471                     /lib/libm-2.11.1.so
7f04bb444000-7f04bb643000 ---p 00082000 fc:00 790471                     /lib/libm-2.11.1.so
7f04bb643000-7f04bb644000 r--p 00081000 fc:00 790471                     /lib/libm-2.11.1.so
7f04bb644000-7f04bb645000 rw-p 00082000 fc:00 790471                     /lib/libm-2.11.1.so
7f04bb645000-7f04bb73b000 r-xp 00000000 fc:00 1311197                    /usr/lib/libstdc++.so.6.0.13
7f04bb73b000-7f04bb93b000 ---p 000f6000 fc:00 1311197                    /usr/lib/libstdc++.so.6.0.13
7f04bb93b000-7f04bb942000 r--p 000f6000 fc:00 1311197                    /usr/lib/libstdc++.so.6.0.13
7f04bb942000-7f04bb944000 rw-p 000fd000 fc:00 1311197                    /usr/lib/libstdc++.so.6.0.13
7f04bb944000-7f04bb959000 rw-p 00000000 00:00 0
7f04bb959000-7f04bb979000 r-xp 00000000 fc:00 787461                     /lib/ld-2.11.1.so
7f04bbb4d000-7f04bbb51000 rw-p 00000000 00:00 0
7f04bbb74000-7f04bbb78000 rw-p 00000000 00:00 0
7f04bbb78000-7f04bbb79000 r--p 0001f000 fc:00 787461                     /lib/ld-2.11.1.so
7f04bbb79000-7f04bbb7a000 rw-p 00020000 fc:00 787461                     /lib/ld-2.11.1.so
7f04bbb7a000-7f04bbb7b000 rw-p 00000000 00:00 0
7fff364d2000-7fff36516000 rw-p 00000000 00:00 0                          [stack]
7fff365e6000-7fff365e7000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aborted (core dumped)

real    0m2.571s
user    0m7.120s
sys     0m4.000s

On linux ninja ensures paths exist before running a command that mentions an output in that dir. This fails on Windows.

The canonical example is on Windows a build.ninja file thus:

builddir = build

rule copy
command = copy $in $out

build $builddir\fred\test.out: copy test.in

This fails with:

C:\Temp\ninjatest>ninja -f build.ninja
[1/1] copy test.in build\fred\test.out
FAILED: copy test.in build\fred\test.out
The system cannot find the path specified.
0 file(s) copied.
build stopped: subcommand failed.

The equivalent test on linux (changing \ to /) works fine.

A flaw in the tokenizer. Need to think about how to fix it right.

If ninja reruns an action that produce multiple outputs, I'd expect it to rebuild all targets that depend on any of the outputs. However, it seems that ninja does not bahalf as I expect. Given the following ninja file:

$ ls
build.ninja
$ cat build.ninja
rule fork
description = forking $in to $out...
command = touch $out

rule cxx
description = compiling $in to $out...
command = cp $in $out

build x.s.cpp x.r.cpp: fork x.cpp
build x.o: cxx x.cpp | x.s.cpp

I do a normal build.

$ touch x.cpp
$ ninja -v
[1/2] touch x.s.cpp x.r.cpp
[2/2] cp x.cpp x.o

Now, if I remove x.r.cpp, and I get the following.

$ rm x.r.cpp
$ ninja -v
[1/1] touch x.s.cpp x.r.cpp

I think this is an incorrect behaviour. I'd expect it to rebuild the x.cpp to produce x.o. Btw, the next no-op build is not no-op.

$ ninja -v
[1/1] cp x.cpp x.o

Can you please confirm whether this is a bug?

Thanks

Using the previously-logged timing info, estimate the remaining time on a build.

Parallelization may make this difficult to impossible.

Currently, if ninja gets an error return from CreateProcess (eg, unable to find the exe program), it calls Fatal and therefore aborts the whole ninja process immediately. The user only gets a message like this:

ninja: FATAL: CreateProcess: The specified path is invalid.

This message does not even include the failed command line.

I think, If CreateProcess fails to find the executable file, it should be treated as a normal build action failure.

The canonical example is a ninja.build file thus:

builddir = build

rule copy
command = copy $in $out

build $builddir/test.out: copy test.in

When run, this will fail on windows with:

C:\temp>ninja.exe
[1/1] copy test.in build/test.out
FAILED: copy test.in build/test.out
The syntax of the command is incorrect.
build stopped: subcommand failed.

This is because the windows copy command cannot cope with "/" as the path separator. copy is only one example of a Windows command that insists on \ as the path separator. There are many others.

What should be the fix here? I had a fork that rewrote the paths after parsing on Windows into "" separators, but it's not been taken to main. Would that be a desired fix? If so I can rewrite the fork to just contain that fix if desired? Alternately, what fix should be done to get path separators working the best on Windows?

Kind of related: should/could build.ninja files change so that:

"" is not the line continuation character?
"" is a valid directory separator character (and so is /, still)?

The following will fail:

#---------
rule foo
  depfile = $out.dep
  command = touch $out

build this will break: foo because of the depfile

ninja: error: loading 'build.ninja': line 6, col 24: dependency files only work with single-output rules

whereas the following is valid in gcc:

bash-3.2$ gcc -c foo.cpp bar.cpp -M
foo.o: foo.cpp foo.h
bar.o: bar.cpp bar.h

i.e. multiple targets in the .dep files are valid.

We have a need to support the use of a shared .dep file by multiple targets.

Other than a design consideration, is there any hard and fast "laws of physics" type reason why this is not possible?

Here's a more specific canonical case:

#snip ---- build.ninja:
#
# This will create 3 files:
#
#   foo
#   foo.2
#
# and will create a foo.dep file with the following:
#
# foo: foo.h
# foo.2: foo.h
#
# so if `foo.h' changes, then both of these targets are dirty and need rebuilt.
rule createTwoFiles
  depfile = $out.dep
  command = cmd.exe /c touch $out $out.2 && echo $out: foo.h > $out.dep && echo $out.2: foo.h >> $out.dep

build foo foo.2: createTwoFiles someinputfile

What is going on here is that:

• I have a rule which produces two outputs from a single input
• This tool writes a depfile, indicating that 'foo.h' is depended on by both files. If 'foo.h' is modified, both the foo' edge and thefoo.2' edge is dirty
• the dependencies for foo' andfoo2' are not known in advance, and are determined as a result of executing the rule (similar to `gcc -M')

What do we need to do to get this behaviour?

Please find an example below, which illustrates a situation of incorrect ninja behaviour.

In short, let's assume we have a rule with two outputs. If one of the output files is missing, ninja will regenerate it, along with the second file. However, that second file will not be marked as dirty, and nodes depending on it will not wait for it being re-built.

This confuses ninja (it may crash, or end up with mis-numbered task, eg [4/3]) and creates a race condition (if two tasks accesing the second output file run in parallel).

For the following example, please create 4 files: linux.ninja, test.sh a.in and b.in.

1. linux.ninja:
    rule cxx
      command = cat $in $byproduct > $out
    
    rule cxx_with_byproduct
      command = echo Test passed | tee $byproduct > $main_out
    
    rule link
      command = cat $in > $out
    
    build a.out: cxx a.in | b.byproduct 
      byproduct=b.byproduct
    
    build b.out b.byproduct: cxx_with_byproduct b.in
      main_out=b.out
      byproduct=b.byproduct
    
    build result: link a.out b.out

    build all: phony result

2. test.sh:
    #!/bin/sh
    
    # Original contens of the byproduct file, which should not be used
    echo Test failed > b.byproduct
    
    # Modify a.in to trigger rebuild of a.out
    sleep 1
    touch a.in
    
    # Delete b.out to trigger rebuild of b.out AND b.byproduct
    sleep 1
    rm -f b.out
    # touch b.in (if input is modified instead of the output, test will pass)
    
    # TEST 1
    # run ninja in silent mode - show incorrect order of steps
    # sleep 1
    # ninja -f linux.ninja -n -v all
    
    # TEST 2
    # run ninja (with multiple threads)
    sleep 1
    ninja -f linux.ninja -j 10 all
    
    # a.out should contain "Test passed", 
    # which was added to b.byproduct during the build
    cat a.out

3. a.in
    Result of the test:

4.b.in
    Not important for the test

Then make sure ninja binary exists on your PATH and run test.sh.
It should print out

  [1/3] cat a.in b.byproduct > a.out
  [2/3] echo Test passed | tee b.byproduct > b.out
  [3/3] cat a.out b.out > result
  Result of the test:
  Test failed

This means that b.byproduct was picked up in task [1/3] before it was modified in [2/3]. Nothing would prevent these two running in parallel.

You can also uncommenti TEST1 in test.sh to see a dry run. This may lead to a crash or misnumbered output (last task is [4/3], cat a.in b.byproduct gets executed twice).

Apparently, this issue seems linked to #118 (Dependent targets on actions that produce multiple outputs).

Short analysis shows, that if an output of a rule is marked as dirty (e.g. because the file is missing), its sibling outputs are left "clean" and thus not expected to be rebuilt.

Can you please confirm whether the above observation is correct and this is indeed a bug?

Thanks
Petr

Like ctl-f7 in MSVC.

Whenever I do 'ninja chrome' I get:

[3503/6643] CXX ninja/obj/net/net/socket/ssl_host_info.o
FAILED: g++ -pthread -Wl,-z,noexecstack -Wl,-Bsymbolic -Lninja/gen/third_party/ffmpeg -o ninja/ffmpegsumo_nolink -Wl,-rpath=ninja/lib -Wl,--start-group ninja/obj/ffmpegsumo_nolink/third_party/ffmpeg/dummy_nolink.o ninja/lib/libffmpegsumo.so -Wl,--end-group -lz
/usr/bin/ld: ninja/lib/libffmpegsumo.so: error: undefined reference to 'vpx_codec_decode'
/usr/bin/ld: ninja/lib/libffmpegsumo.so: error: undefined reference to 'vpx_codec_get_frame'
/usr/bin/ld: ninja/lib/libffmpegsumo.so: error: undefined reference to 'vpx_codec_error'
/usr/bin/ld: ninja/lib/libffmpegsumo.so: error: undefined reference to 'vpx_codec_error_detail'
/usr/bin/ld: ninja/lib/libffmpegsumo.so: error: undefined reference to 'vpx_codec_destroy'
/usr/bin/ld: ninja/lib/libffmpegsumo.so: error: undefined reference to 'vpx_codec_version_str'
/usr/bin/ld: ninja/lib/libffmpegsumo.so: error: undefined reference to 'vpx_codec_build_config'
/usr/bin/ld: ninja/lib/libffmpegsumo.so: error: undefined reference to 'vpx_codec_vp8_dx_algo'
/usr/bin/ld: ninja/lib/libffmpegsumo.so: error: undefined reference to 'vpx_codec_dec_init_ver'
/usr/bin/ld: ninja/lib/libffmpegsumo.so: error: undefined reference to 'vpx_codec_encode'
/usr/bin/ld: ninja/lib/libffmpegsumo.so: error: undefined reference to 'vpx_codec_get_cx_data'
/usr/bin/ld: ninja/lib/libffmpegsumo.so: error: undefined reference to 'vpx_codec_control_'
/usr/bin/ld: ninja/lib/libffmpegsumo.so: error: undefined reference to 'vpx_codec_vp8_cx_algo'
/usr/bin/ld: ninja/lib/libffmpegsumo.so: error: undefined reference to 'vpx_codec_enc_config_default'
/usr/bin/ld: ninja/lib/libffmpegsumo.so: error: undefined reference to 'vpx_codec_err_to_string'
/usr/bin/ld: ninja/lib/libffmpegsumo.so: error: undefined reference to 'vpx_codec_enc_init_ver'
/usr/bin/ld: ninja/lib/libffmpegsumo.so: error: undefined reference to 'vpx_img_wrap'
collect2: ld returned 1 exit status

It's been failing for a while, although i'm not sure when it started. My gyp-related env vars are:

$ set | grep GYP
GYP_DEFINES='disable_nacl=1 remove_webcore_debug_symbols=1 enable_svg=0'
GYP_GENERATORS=make,ninja

1. build
2. ninja -t clean
3. note the .d files remain

I'm building chromium to asan/Release, and interrupted the previous build. Restarting the build fails with:

ninja: error: expected makefile to mention 'obj/third_party/angle/src/compiler/translator_common.debug.o', got 'debug.o'

so I ninja -t clean; no joy. So I ./build/gyp_chromium; no joy. Finally I rm -rf asan/Release, re-run gyp_chromium, and things work great. After this happened a few times I chased it down to (I think) errant .o.d files not being cleaned up by the clean tool and somehow confusing ninja. Here's a console log illustrating the problem, the fix, and the subsequent workingness (without a gyp_chromium re-run, or a blowing away of the output directory):

$ ../ninja DumpRenderTree
Using FLAVOR=asan/Release
ninja: error: expected makefile to mention 'obj/third_party/angle/src/compiler/translator_common.debug.o', got 'debug.o'
$ find asan/Release/ -type f -name '*.o.d'|xargs rm
$ ../ninja DumpRenderTree
Using FLAVOR=asan/Release
[4342/319/448] LINK ssl_false_start_blacklist_process

(../ninja is a wrapper that adds the -C flag etc).

Perhaps via a -d (debug) flag.

When I tried to run ninja_test from a directory where I don't have write access, I got a segmentation fault as showed lower, instead of the standard "stat(xx[truncated]xx): File name too long" error message then Success

Actual Error Log :
[----------] 3 tests from DiskInterfaceTest
[ RUN ] DiskInterfaceTest.Stat
Segmentation fault

To make a long story short - I build a lot of internal libraries in a project and to make the include paths more manageable I copy the include files to a central destination. changing the include files makes ninja confused when there are implicit and explicit dependencies to the same include files in a build.

The simplest way to reproduce the problem is this example (in my real case, the dependency on foo.h is throgh a depfile, not an implicit dependency):
$ ls
build.ninja
$ cat build.ninja
rule touch
command = touch $out
description = touching $out

build foo.h: touch foo.h.in
build l.o: touch l.c | foo.h
build bar: touch | foo.h
build foo: touch foo.h
build all: phony foo bar
$ touch foo.h.in
$ ~/src/martine-ninja-5f01572/ninja -v all
touch foo.h
touch bar
touch foo
$ touch foo.h.in
$ valgrind ~/src/martine-ninja-5f01572/ninja -v all
[... snip ...]
touch foo.h
touch bar
touch bar
touch foo
==10933== Invalid read of size 8
==10933== at 0x405BB8: BuildStatus::BuildEdgeFinished(Edge_) (build.cc:93)
==10933== by 0x405CD7: Builder::FinishEdge(Edge_) (build.cc:462)
==10933== by 0x405E4D: Builder::Build(std::string_) (build.cc:411)
==10933== by 0x402F09: main (ninja.cc:239)
==10933== Address 0x4c22598 is 0 bytes after a block of size 72 alloc'd
==10933== at 0x4A0666E: operator new(unsigned long) (vg_replace_malloc.c:220)
==10933== by 0x40603A: Builder::Builder(State_, BuildConfig const&) (build.cc:358)
==10933== by 0x402E34: main (ninja.cc:226)
==10933==
==10933== Invalid free() / delete / delete[]
==10933== at 0x4A05743: operator delete(void_) (vg_replace_malloc.c:346)
==10933== by 0x40A478: __gnu_cxx::new_allocator<std::Rb_tree_node<std::pair<Edge const, timeval> > >::deallocate(std::Rb_tree_node<std::pair<Edge* const, timeval> >, unsigned long) (new_allocator.h:94)
==10933== by 0x40A4A0: std::Rb_tree<Edge, std::pair<Edge* const, timeval>, std::_Select1st<std::pair<Edge* const, timeval> >, std::less<Edge*>, std::allocator<std::pair<Edge* const, timeval> > >::_M_put_node(std::Rb_tree_node<std::pair<Edge* const, timeval> >) (stl_tree.h:362)
==10933== by 0x40A4EB: std::Rb_tree<Edge, std::pair<Edge* const, timeval>, std::Select1st<std::pair<Edge* const, timeval> >, std::less<Edge*>, std::allocator<std::pair<Edge* const, timeval> > >::destroy_node(std::Rb_tree_node<std::pair<Edge* const, timeval> >) (stl_tree.h:392)
==10933== by 0x40BB71: std::Rb_tree<Edge, std::pair<Edge* const, timeval>, std::Select1st<std::pair<Edge* const, timeval> >, std::less<Edge*>, std::allocator<std::pair<Edge* const, timeval> > >::erase(std::Rb_tree_iterator<std::pair<Edge* const, timeval> >) (stl_tree.h:1189)
==10933== by 0x40BBA4: std::map<Edge*, timeval, std::less<Edge*>, std::allocator<std::pair<Edge* const, timeval> > >::erase(std::Rb_tree_iterator<std::pair<Edge* const, timeval> >) (stl_map.h:446)
==10933== by 0x405C3C: BuildStatus::BuildEdgeFinished(Edge) (build.cc:95)
==10933== by 0x405CD7: Builder::FinishEdge(Edge) (build.cc:462)
==10933== by 0x405E4D: Builder::Build(std::string) (build.cc:411)
==10933== by 0x402F09: main (ninja.cc:239)
==10933== Address 0x4c22568 is 24 bytes inside a block of size 72 alloc'd
==10933== at 0x4A0666E: operator new(unsigned long) (vg_replace_malloc.c:220)
==10933== by 0x40603A: Builder::Builder(State, BuildConfig const&) (build.cc:358)
==10933== by 0x402E34: main (ninja.cc:226)

Adding a check for "i == running_edges_.end()" in BuildEdgeFinished and returning early hides the problem, but it's not the correct thing to do since in the real case I see files recompiled before their include files are rebuilt.

Speaking of dependencies. Would it be possible to make something equivalent to depfile, but make those dependencies explicit? I'm pretty sure this would make this problem go away (and make my life easier).

# build.ninja
include unknown.ninja

If unknown.ninja is missing, this filename don't appear in the error message.

ninja: error: loading 'build.ninja': No such file or directory

subninja has same problem.