With the attached Makefile, make 3.80+3.81.b4-1 is much slower than 3.80-9 at running "make -wr stage1/ghc-6.4.1" (only a few seconds in the cutdown case, but more in the real thing - I never waited for it to terminate to know how much more). Thanks Ian $ dpkg -s make | grep Version Version: 3.80+3.81.b4-1 $ time make -wr stage1/ghc-6.4.1 make: Entering directory `/tmp/wibble' Makefile:20: stage1/profiling/CostCentre.o stage1/profiling/SCCfinal.o stage1/main/Config.o make: *** No rule to make target `profiling/CostCentre.lhs', needed by `stage1/profiling/CostCentre.o'. Stop. make: Leaving directory `/tmp/wibble' real 0m4.318s user 0m4.315s sys 0m0.002s $ $ dpkg -s make | grep Version Version: 3.80-9 $ time make -wr stage1/ghc-6.4.1 make: Entering directory `/tmp/wibble' Makefile:20: stage1/profiling/CostCentre.o stage1/profiling/SCCfinal.o stage1/main/Config.o make: *** No rule to make target `profiling/CostCentre.lhs', needed by `stage1/profiling/CostCentre.o'. Stop. make: Leaving directory `/tmp/wibble' real 0m0.008s user 0m0.006s sys 0m0.003s $
I've just tried the real thing, and it's still thinking after >24 hours of CPU time, so this is effectively going to cause FTBFSs for me (although technically I think it will eventually terminate). Thanks Ian
severity 346248 important reassign 348633 ghc6, make merge 346248 348633 thanks I think it ended after <8 hours for me, running at 2.4GHz, but I can't be sure because the ssh connection died before this.. /usr/bin/screen next time.
reassign 348633 ghc6 severity 348633 serious retitle 348633 ghc6: effective FTBFS because of a bug in /usr/bin/make block 348633 with 346248 thanks Ian is right, make has now been processing ghc6 for 11 hours (/tmp/buildd/ghc6-6.4.1/ghc/compiler). Why did you say, that you think it eventually terminates?
Because the testcase I gave when reporting the bug terminates, and I believe the problem is just a larger instance of that case. Thanks Ian
Follow-up Comment #1, bug #15584 (project make): The makefile itself does not give me any idea what might be wrong. Can you provide a minimal but complete test case that reproduces the problem (I don't know what stage1/ghc-6.4.1 is).
Follow-up Comment #2, bug #15584 (project make):
Actually, Boris, this makefile IS sufficient to reproduce the problem. Stick
it in an empty directory. Run it as shown in the example. You'll see that
older versions of make fail as shown almost immediately, while the latest CVS
version will do a LOT of work, then fail after a few seconds.
Using -d you can see what's going on: the old version of make will see that
the first prereq cannot be created, then stop:
Considering target file `stage1/profiling/CostCentre.o'.
File `stage1/profiling/CostCentre.o' does not exist.
Looking for an implicit rule for `stage1/profiling/CostCentre.o'.
Trying pattern rule with stem `profiling/CostCentre'.
Trying implicit prerequisite `profiling/CostCentre.hs'.
Trying pattern rule with stem `profiling/CostCentre'.
Trying implicit prerequisite `profiling/CostCentre.lhs'.
Found an implicit rule for `stage1/profiling/CostCentre.o'.
Considering target file `profiling/CostCentre.lhs'.
File `profiling/CostCentre.lhs' does not exist.
Looking for an implicit rule for `profiling/CostCentre.lhs'.
No implicit rule found for `profiling/CostCentre.lhs'.
Finished prerequisites of target file `profiling/CostCentre.lhs'.
Must remake target `profiling/CostCentre.lhs'.
make: *** No rule to make target `profiling/CostCentre.lhs', needed by
`stage1/profiling/CostCentre.o'. Stop.
However, the new version of make does NOT stop and continues to look for
other prerequisites, and gets into some bizarre sort of full-tree walk. The
full -d output I got was 2297723 _LINES_ long! The first part, matching the
above, was:
Considering target file `stage1/ghc-6.4.1'.
File `stage1/ghc-6.4.1' does not exist.
Looking for an implicit rule for `stage1/profiling/CostCentre.o'.
Trying pattern rule with stem `profiling/CostCentre'.
Trying implicit prerequisite `profiling/CostCentre.hs'.
Trying pattern rule with stem `profiling/CostCentre'.
Trying implicit prerequisite `profiling/CostCentre.lhs'.
Found an implicit rule for `stage1/profiling/CostCentre.o'.
Looking for an implicit rule for `profiling/CostCentre.lhs'.
No implicit rule found for `profiling/CostCentre.lhs'.
Looking for an implicit rule for `stage1/utils/Util.hi'.
Trying pattern rule with stem `Util'.
After make sees that it can't find an implicit rule for one of the prereqs,
instead of stopping it continues with the next prereq. Then, later on it
gets into some kind of pathological recursion, "Pruning file `Config.hs'."
over and over at different depths.
Unfortunately I didn't have time to get any further into this last night.
Follow-up Comment #3, bug #15584 (project make): Seem like my improvement in implicit.c around line 668 uncovered this. Before make would return from pattern_search with either failure or an implicit rule for which all implicit prerequisites either exist or can be built. Now, if an implicit prerequisite is also an explicit prerequisite for this target, pattern_search skips its normal checks because this prerequisite has to be built no matter which implicit rule we choose. Sometimes, like in our case, there may actually be no rule to built such a prerequisite (profiling/CostCentre.lhs here). Some code in remake.c (either around line 451, or 957, or both) "assumes" that there is a rule for each implicit prerequisite if pattern_search did not fail. Just need to find that piece of code and fix it.
Update of bug #15584 (project make):
Severity: 3 - Normal => 5 - Blocker
Update of bug #15584 (project make):
Assigned to: None => bosk
Follow-up Comment #5, bug #15584 (project make): It's a bit tricky when it comes to intermediates. Old algo would sometimes choose subsequent rules because one of the pattern prereqs can only satisfy as an intermediate (even though it is explicitly mentioned in the non-pattern rule). Also the new algorithm improves diagnostics: make will print that it was unable to build foo.h instead of saying that it could not find a rule to build foo.c. I will take care of this bug.
Follow-up Comment #6, bug #15584 (project make): After all, this problem does not appear to be related to my implicit.c change. It's actually quite obvious from the -d output: both versions conclude that there is no implicit rule for profiling/CostCentre.lhs and while 3.80 stops, 3.81 goes on. Next thing that I found is that if we remove .SECONDARY: from the original makefile, make 3.81 returns immediately. So it is related to the fact that all targets are treated as intermediates. I was trying to figure out what are all the differences between normal and intermediate targets. GNU make manuall has this strange text: "The first difference is what happens if the intermediate file does not exist. If an ordinary file B does not exist, and `make' considers a target that depends on B, it invariably creates B and then updates the target from B. But if B is an intermediate file, then `make' can leave well enough alone. It won't bother updating B, or the ultimate target, unless some prerequisite of B is newer than that target or there is some other reason to update that target." I wonder what those "other reasons" are. Like, does mentioning an intermediate target as a goal quilify? What about depending on the target that was mentioned as a goal? Anyway, the problem appears to be in check_dep() in remake.c. Apparently if check_dep is called for an intermediate, non-existent file for which there is no implicit rule and which has intermediate non-existent prereqs (for which there is no implicit rule, etc.), it will recursively check every prereq and at the end return 0, without updating anything. That's exactly what happens in this case. Whether it is correct behaviour or not - I don't know. It depends on what that piece of the manual above actualy means.
Follow-up Comment #7, bug #15584 (project make): not exist, then target will only be updated if B has to be rebuilt because one of B's prereqs is newer: B will not be rebuilt just because it does not exist. "Some other reason" means if you have a rule "target: B A" in the same situation above, then target could be rebuilt because of A, even if it doesn't need to be rebuilt because of B. Your description of how check_dep() behaves sounds correct on the surface: you can't know whether B needs to be rebuilt unless you check all its prerequisites. However, there's SOMETHING pathalogical going on here: if you run the test with -d and capture all the output you can see that as the processing goes along the dep check make performs gets more and more elaborate and the number of times it does "Pruning file `Config.hs'." increases almost exponentially. I can't see any justification for that behavior offhand... is there one? I see you qualify the situation by saying there is no implicit rule for the non-existent file... if there's no implicit or explicit rule for that target then does it make sense to continue looking at that target's prerequisites? Suppose we found one of them and it needed to be rebuilt: what then? Does that mean the same thing as an empty rule: target : B.x ; @echo cp $< $@ %.x : %.y B.y : ; @echo cp $< $@ .INTERMEDIATE: B.x ?? Hm.
Follow-up Comment #8, bug #15584 (project make): Then, since none of the targets in this makefile exist and all of them are intermediate, we shouldn't build anything and should not fail. Or if the target is also a goal then this logic does not apply? Even if that's the case we should fail to build stage1/ghc-6.4.1, not profiling/CostCentre.lhs or stage1/profiling/CostCentre.o? if it doesn't need to be rebuilt because of B. Ok, got it. Thanks for the clarification. That's definitely not how 3.80 works. So we are talking about some new way of doing things in 3.81 that caused this behavior. I see those except my log is full of "Prunning file `Makefile'.". This seems to happen multiple times in a row at different nesting levels. I will try to investigate this. Well, if there is no rule and we don't need to rebuild it then I suppose we don't fail. then? Then we fail. For now I am going to assume that what check_dep does is correct and will try to find out why do we get all those "Prunning..." messages.
Follow-up Comment #9, bug #15584 (project make): Actually, now that I think about it I'm confused. You _can't_ have a situation like this: target : B.x ; @echo cp $< $@ %.x : %.y B.y : ; @echo cp $< $@ because you can't define an empty pattern rule... an empty pattern rule deletes the pattern. So, I don't see how the situation you describe: "an intermediate, non-existent file for which there is no implicit rule" could ever be correct... let me think. I suppose there could be an _explicit_ rule; does that make a difference? That could happen because people can declare targets as intermediate even though they are explicitly mentioned (which normally would disqualify them from being intermediate). So, something like this: target : B.x ; @echo cp $< $@ B.x : B.y B.y : ; @echo cp $< $@ .INTERMEDIATE: B.x Offhand this seems legal to me; would this trigger the issue? If there's no implicit OR explicit rule, then I guess we have one of two cases: either make should check all the prerequisites of the intermediate file (B.x above) and if none of them need to be rebuilt, make says "OK". Or, make should fail immediately because it doesn't know how to build the intermediate file (B.x) regardless of whether or not it actually needs to know. For explicit rules this is perfectly legal: if there's no command to update a target then it's just considered "magically updated" without actually doing anything.
Follow-up Comment #10, bug #15584 (project make): So I did some mode digging and it appears that make just does what it's supposed to do. If you look into this makefile carefully you will see the pattern rule that looks like this %.hi: %.o which pretty much makes every single .o file depend (directly and inderectly) on all others. Now because all of the targets in this makefile are intermediate and non-existed, make dutifully traverses the whole grpath for every .o file and figures out that it cannot build anyhting. Since it doesn't set any flags like updated or this_target_is_intermediate_and_does_not_exist_and_i_already_tried_to_build _it, it does *complete* graph traversal for every .o file it tries to build. I don't see any way (and don't really want) to fix this in make without doing a major surgery on of how things work. To summarize, the long time is the result of three things: 1. Strange desire of the makefile author to have all files intermediate. 2. Pretty much fully-connected dependency graph. 3. Brain-damaged intermediate file logic in GNU make. I therefore suggest that we leave this corner case alone.
Follow-up Comment #11, bug #15584 (project make): Hm. OK. I guess the only question is, why does the older version of GNU make not have this problem? We must have changed the behavior of intermediate files or something that caused this change in behavior. If we can understand what change we made and that it was legitimate (to fix another bug) then I'm happy to leave this alone for 3.81. Thanks for looking into this one Boris!
Follow-up Comment #12, bug #15584 (project make): It appears that 3.80 has a bug in handling of .SECONDARY. It does not actually mark individual targets as intermedite. As a result, check_dep doesn't treat any target in this makefile as intermediate. In CVS the following code was added in file.c around line 692: hash_map (&files, set_intermediate);
Update of bug #15584 (project make):
Severity: 5 - Blocker => 2 - Minor
Item Group: Bug => Enhancement
Component Version: None => CVS
Summary: 3.81.b4 much slower than 3.80 in some situations =>
3.81 does way too much work in some pathalogical situations.
Update of bug #15584 (project make):
Assigned to: bosk => None
severity 346248 wishlist
thanks
Hi,
This is the analysis from upstream:
So I did some mode digging and it appears that make just does what
it's supposed to do. If you look into this makefile carefully you
will see the pattern rule that looks like this %.hi: %.o which pretty
much makes every single .o file depend (directly and inderectly) on
all others. Now because all of the targets in this makefile are
intermediate and non-existed, make dutifully traverses the whole
grpath for every .o file and figures out that it cannot build
anyhting. Since it doesn't set any flags like updated or
this_target_is_intermediate_and_does_not_exist_and_i_already_tried_to_build_it,
it does *complete* graph traversal for every .o file it tries to
build. I don't see any way (and don't really want) to fix this in
make without doing a major surgery on of how things work. To
summarize, the long time is the result of three things:
1. Strange desire of the makefile author to have all files intermediate.
2. Pretty much fully-connected dependency graph.
3. Brain-damaged intermediate file logic in GNU make.
It appears that 3.80 has a bug in handling of .SECONDARY. It does not
actually mark individual targets as intermedite. As a result,
check_dep doesn't treat any target in this makefile as intermediate
So, this is being marked as a feature request.
manoj
retitle 346248 [regression] make: slow to realize it's unable to build a .SECONDARY target quit Ian Lynagh wrote:
Yes: Mon Feb 27 10:59:39 GMT 2006 Simon Marlow <simonmar@microsoft.com> * remove empty .SECONDARY target This works around a problem with recent versions of GNU make that take a long time when all targets are declared intermediate with .SECONDARY. See https://savannah.gnu.org/bugs/?func=detailitem&item_id=15584 for discussion of the GNU make issue. - -# This line prevents GNU make from deleting any intermediate targets: - -.SECONDARY: Thanks Ian
Additional Item Attachment, bug #15584 (project make): File name: BadMakefile Size:2 KB
Follow-up Comment #14, bug #15584 (project make): I ran into this problem with some real work. I realize it's probably not common to have such a large tree of secondary/intermediate files, but due to the nature of our build process it would be nice if that worked without hitting this performance limitation. In case it clarifies anything, I've attached a makefile (named BadMakefile) that replicates the problem. You can see the pattern; adding another level of hierarchy approximately doubles the time it takes to "make all". Also I predict that if you run this with "make -d" it will produce ~10GB of output.
Update of bug #15584 (project make):
Component Version: SCM => 3.81