aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
| | | | * | Merge pull request #98 from danw/regen_side_effectsDan Willemsen2016-10-041-4/+4
| | | | |\ \ | | | | | | | | | | | | | | Run regen commands sequentially
| | | | | * | Run regen commands sequentiallyDan Willemsen2016-10-031-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When I remove --ignore-dirty=$(OUT_DIR)/% from an Android build, it turns out that we have $(shell) commands that have side effects and cannot be reordered in relationship to one another. The trivial case is a sequence of 'rm', 'write', 'write'..., 'cmp'. The 'rm' and 'write' commands can be collapsed into a single $(file) function call, but it still needs to be ordered for the 'cmp' function to work properly. I've been making changes to prevent this from slowing down regen too much, but it's still a 2-3x slowdown overall (0.3s -> 0.8s for aosp_arm64-eng on aosp master).
| | | | * | | Merge pull request #97 from danw/file_funcDan Willemsen2016-10-047-34/+379
| | | | |\ \ \ | | | | | | | | | | | | | | | | Implement the `file` function to read and write files
| | | | | * | | Implement the `file` function to read and write filesDan Willemsen2016-10-037-34/+379
| | | | | |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows us to do file reading and writing without $(shell). Besides being simpler, this also allows faster regen times, since we can just stat the files to be read, or directly write to the files that need to be written.
| | | | * | | Merge pull request #96 from danw/opt_runcommandShinichiro Hamaji2016-10-0310-19/+92
| | | | |\ \ \ | | | | | | | | | | | | | | | | Optimize RunCommand by removing /bin/sh wrapper when possible
| | | | | * | | Optimize RunCommand by removing /bin/sh wrapper when possibleDan Willemsen2016-10-018-22/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For every $(shell echo "test: $PATH") command, when SHELL is /bin/bash, we essentially run: (each arg wrapped in []) [/bin/sh] [-c] [/bin/bash -c "echo \"test: \$PATH\""] This is redundant, since we can just use SHELL, and then we don't need to do an extra level of shell escaping either. This change makes us run this instead: [/bin/bash] [-c] [echo "test: $PATH"] If SHELL is more complicated than an absolute path to a binary, then we'll fall back to /bin/sh. Using the benchmark introduced in the last change, this reduces a minimal RunCommand execution with a simple SHELL from 3.7ms to 1.3ms. For a more complex benchmark (though less normalized), for an AOSP Android build, this change shrinks the average time spent in $(shell) functions from 4.5ms to 3ms. Change-Id: I622116e33565e58bb123ee9e9bdd302616a6609c
| | | | | * | | Add simple benchmark for RunCommandDan Willemsen2016-10-013-1/+54
| | | | | |/ / | | | | | | | | | | | | | | | | | | | | | Change-Id: I7f3aabdd8fc01f9ddbcf23e586b6f5e81ab8cbce
| | | | * | | Merge pull request #95 from danw/readonlyShinichiro Hamaji2016-10-0310-10/+234
| | | | |\ \ \ | | | | | |/ / | | | | |/| | Support marking variables as readonly
| | | | | * | Support marking variables as readonlyDan Willemsen2016-09-2910-10/+234
| | | | |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When the magic variable .KATI_READONLY is set to a variable name, any further attempts to modify the named variable will result in an error. FOO := bar .KATI_READONLY := FOO FOO := baz # Error! This is useful to make some global configuration readonly so that another makefile cannot change it. In Android, we emulated this by backing up some global configuration before including the Android.mk files, then comparing the current values to the backed up values after they've been included. But this means we don't know the location that modified the variable, just that something did. And it's not perfect, since the backup can also be changed. Something similar to this could be implemented with `override`, but then setting the variable silently fails, and it still could be overriden with another override.
* | | | | | Merge remote-tracking branch 'aosp/upstream' into master am: 38315d2f21 am: ↵Dan Willemsen2016-09-219-19/+25
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 56fa7f0d2c am: 09ad1f21ae Change-Id: Ibf044ca94fa6b5a1a4ad1c3e962f0e4f53e908ce
| * | | | | Merge remote-tracking branch 'aosp/upstream' into master am: 38315d2f21Dan Willemsen2016-09-219-19/+25
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | am: 56fa7f0d2c Change-Id: Id2a4ccee547ca282bf0cfaab525e542207ead0e1
| | * | | | Merge remote-tracking branch 'aosp/upstream' into masterDan Willemsen2016-09-219-19/+25
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | am: 38315d2f21 Change-Id: I18034bbf12e5e2730e428ce8e6adf45c9ed80456
| | | * | | Merge remote-tracking branch 'aosp/upstream' into masterDan Willemsen2016-09-209-19/+25
| | |/| | | | | | |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * aosp/upstream: Workaround lifetime problem identified by ASAN Stop using .PHONY as a build target in phony.mk Don't ignore all date/echo commands ninja: Don't emit special targets (.*) Add --regen_debug Fix unused parameter warning for !SSE4.2 Bug: 30947985 Test: Passes upstream kati test suite, manual android testing Change-Id: I5cd822b7b56085f8784c40fa6d03e6ac0db67f8b
| | | * | Merge pull request #92 from danw/asan_workaroundDan Willemsen2016-09-202-9/+9
| | | |\ \ | | | | | | | | | | | | Workaround lifetime problem identified by ASAN
| | | | * | Workaround lifetime problem identified by ASANDan Willemsen2016-09-202-9/+9
| | | |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we run ConcatDir in NinjaGenerator::GenerateStamp, we were hitting a container overflow from ASAN. When dumping the memory contents, it looked like the StringPiece's data was pointed to a corrupted piece of memory. Instead of tracking it down, just copy the strings upon creation. This is causing kati to take a few extra seconds on master (5% increase for a build that otherwise does nothing), so this should probably be investigated. Change-Id: I197e4a522dd32e5c5ed5e7a0c6659aa486dfdc94
| | | * | Merge pull request #94 from danw/date_echo_no_ignoreShinichiro Hamaji2016-09-201-1/+3
| | | |\ \ | | | | | | | | | | | | Don't ignore all date/echo commands
| | | | * | Don't ignore all date/echo commandsDan Willemsen2016-09-191-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | echo commands may have side-effects (writing a file), or be testing a condition and echoing a different value. date commands may be part of a larger command as well. In any case, if the command changes, and would cause changes to the ninja file, then we want to regenerate the ninja file. Just whitelist `date +%s` for now, since Android uses that as its only build-time date source, and we don't want to rebuild the ninja file every time for that. Eventually, it should be removed, but that means removing all makefile choices on BUILD_NUMBER, since that depends on BUILD_DATETIME as well. In either case, the real value should never be inserted into the ninja file. Change-Id: I3b36c8b46f747c1b22a3faacf1fc34683faa9b70
| | | * | | Stop using .PHONY as a build target in phony.mkhamaji2016-09-201-3/+0
| | | | | | | | | | | | | | | | | | | | | | | | This testcase makes little sense.
| | | * | | Merge pull request #93 from danw/ninja_special_targetsShinichiro Hamaji2016-09-201-0/+3
| | | |\ \ \ | | | | | | | | | | | | | | ninja: Don't emit special targets (.*)
| | | | * | | ninja: Don't emit special targets (.*)Dan Willemsen2016-09-191-0/+3
| | | | |/ / | | | | | | | | | | | | | | | | | | | | | | | | These aren't meant to be called directly, so there's no point putting them in the ninja file.
| | | * | | Merge pull request #91 from danw/regen_debugShinichiro Hamaji2016-09-203-5/+9
| | | |\ \ \ | | | | | | | | | | | | | | Add --regen_debug
| | | | * | | Add --regen_debugDan Willemsen2016-09-193-5/+9
| | | | |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This prints information about the regen process during a normal build, including which shell commands where rerun or ignored. My main goal is to understand if it's safe to remove our --ignore_dirty argument. So I need to understand which files were dirty, but ignored. And I'd also like to know which shell commands were run (or would be run if not ignored), to identify any that may cause side-effects (whether desired or not). It's different from --dump-kati-stamp, since --dump-kati-stamp stops kati before actually building anything, and continues after the first dirty indication in an attempt to find every difference.
| | | * | | Merge pull request #90 from danw/unused_parameter_warningShinichiro Hamaji2016-09-201-1/+1
| | | |\ \ \ | | | | |/ / | | | |/| | Fix unused parameter warning for !SSE4.2
| | | | * | Fix unused parameter warning for !SSE4.2Dan Willemsen2016-09-191-1/+1
| | | |/ / | | | | | | | | | | | | | | | Change-Id: I7099c92120dde0d985bc9d19fe0b2573027e5d79
* | | | | Merge remote-tracking branch \\\'aosp/upstream\\\' am: fa3234caba am: e1de29e542Shinichiro Hamaji2016-07-1223-81/+297
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | am: cdb799601c Change-Id: I5fe28794ce5e8fd844f9a68d077f7e2c33acb66a
| * | | | Merge remote-tracking branch \\'aosp/upstream\\' am: fa3234cabaDan Willemsen2016-07-1223-81/+297
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | am: e1de29e542 Change-Id: Idcc752a99c134f2bf0c403c84ba1fed621e0961b
| | * | | Merge remote-tracking branch \'aosp/upstream\'Shinichiro Hamaji2016-07-1223-81/+297
| |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | am: fa3234caba Change-Id: I1eb4edbfe2f5d4fbfe07e1eb15884f0df2e3b0b9
| | * | | Merge remote-tracking branch 'aosp/upstream'Dan Willemsen2016-07-1123-81/+297
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * aosp/upstream: [C++] Remove code only for non-SSE environment [C++] Do not use STTNI when string length is <16 [C++] Fix strutil_test for non-SSE build [C++] Fix a typo in PR#48 Add unit tests for invalid string accesses [C++] Allow targets which start with `='. Add Po Hu to AUTHORS/CONTRIBUTORS Add testcases for commandline variables [C++] A follow-up for the previous change [C++] Allow pass command line variables from make to ckati [C++] Allow command line variables to override file variables [C++] Do not evaluate recursive variable in ifdef [C++] Fix handling CRLF after a backslash [C++] Fix $(join) for #76 [C++] $(eval) stops when first character is '#' Change-Id: I32f5359442b66e1627e3cfdba1ad1e7fd9c32abd
| | | * | [C++] Remove code only for non-SSE environmentShinichiro Hamaji2016-06-301-51/+0
| | | | | | | | | | | | | | | | | | | | Now SkipUntil works both with and without SSE.
| | | * | [C++] Do not use STTNI when string length is <16Shinichiro Hamaji2016-06-301-15/+31
| | | | |
| | | * | [C++] Fix strutil_test for non-SSE buildShinichiro Hamaji2016-06-301-1/+3
| | | | |
| | | * | [C++] Fix a typo in PR#48Shinichiro Hamaji2016-06-301-1/+1
| | | | |
| | | * | Merge pull request #84 from danw/invalid_string_accessShinichiro Hamaji2016-06-303-1/+54
| | | |\ \ | | | | | | | | | | | | Add unit tests for invalid string accesses
| | | | * | Add unit tests for invalid string accessesDan Willemsen2016-06-293-1/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A string may be allocated at the end of a page, and the next page may not be readable, so reading beyond the null character may not be safe. We've hit this with real makefiles, but here's a directed test that can reproduce it in other environments. This also updates the travis config to run the unit tests.
| | | * | | [C++] Allow targets which start with `='.Shinichiro Hamaji2016-06-305-7/+51
| | | |/ / | | | | | | | | | | | | | | | but with a kati-specific warning message. This fixes #83.
| | | * | Add Po Hu to AUTHORS/CONTRIBUTORSShinichiro Hamaji2016-06-252-1/+3
| | | | |
| | | * | Merge branch 'jzhupo-master'Shinichiro Hamaji2016-06-256-0/+129
| | | |\ \ | | | | |/ | | | |/|
| | | | * Add testcases for commandline variablesShinichiro Hamaji2016-06-254-0/+117
| | | | |
| | | | * [C++] A follow-up for the previous changeShinichiro Hamaji2016-06-251-13/+5
| | | | |
| | | | * [C++] Allow pass command line variables from make to ckatiPo Hu2016-06-251-0/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | They will be preserved in MAKEFLAGS as environment variable. TODO: Set MAKEFLAGS as file variable instead of environment. $ cat Makefile VAR := FAIL $(info VAR=$(VAR),$(origin VAR)) $(info MAKEFLAGS=$(MAKEFLAGS),$(origin MAKEFLAGS)) all: @ckati last last: @echo $@ $ make VAR=PASS VAR=PASS,command line MAKEFLAGS=,file VAR=PASS,command line MAKEFLAGS=VAR=PASS,environment last
| | | | * [C++] Allow command line variables to override file variablesPo Hu2016-06-251-0/+4
| | | |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Test case: $ cat Makefile VAR := FAIL $(info $(VAR)) all: @echo $(VAR) $ ckati -f Makefile VAR=PASS PASS PASS
| | | * [C++] Do not evaluate recursive variable in ifdefShinichiro Hamaji2016-06-092-2/+9
| | | | | | | | | | | | | | | | This fixes #79
| | | * [C++] Fix handling CRLF after a backslashShinichiro Hamaji2016-06-072-0/+3
| | | | | | | | | | | | | | | | This fixes #77
| | | * [C++] Fix $(join) for #76Shinichiro Hamaji2016-06-072-2/+8
| | | |
| | | * Merge pull request #75 from stefanb2/topic-issue-74Shinichiro Hamaji2016-06-032-4/+9
| | | |\ | | | | | | | | | | [C++] $(eval) stops when first character is '#'
| | | | * [C++] $(eval) stops when first character is '#'Stefan Becker2016-06-022-4/+9
| | | |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Regression when compared to GNU make behaviour. Test case: $ cat ../Makefile.comment-in-macro .PHONY: all define _rule # comment all: : endef $(eval $(_rule)) $ make -f Makefile.comment-in-macro : $ ckati --ninja --ninja_dir . --gen_all_targets -f Makefile.comment-in-macro *** No targets. Fixes https://github.com/google/kati/issues/74
* | | | Merge remote-tracking branch \'aosp/upstream\' into master am: ff2d59e2e0 ↵Shinichiro Hamaji2016-05-3038-59/+361
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | am: 2ca84c70fc am: 32b10dc028 * commit '32b10dc02806b70ffdad7a6e3c5b767c5a6de507': (35 commits) Handle nested define/endef Update findleaves.py and add a few testcases [C++] Add support for multiple filenames to findleaves emulation Support all kinds of command line variables Allow NULL filename for -d flag [C++] Add -d flag to make debugging slightly easier Normalize log for recent ninja Always sort glob results Skip 3 tests which fail with make 4 Skip shell_var_with_args.mk with GNU make 4 Use override in posix_var.mk to fix test with make 4 Stop overwriting /bin/sh by bash on travis Normalize Unicode quotes for recent find Explicitly use SHELL=/bin/bash [C++] Stop using an uninitialized value Fix multi_implicit_output_patterns.mk for GNU make 4 Clean up normalization in runtest.rb a bit Remove test output in ckati_clean Normalize GNU make 4.00 in runtest.rb Suppress GNU make jobserver magic in runtest.rb ... Change-Id: Ia9b00b3f24e69ead1aab269053afef6d892a30b2
| * | | Merge remote-tracking branch \'aosp/upstream\' into master am: ff2d59e2e0Colin Cross2016-05-3038-59/+361
|/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | am: 2ca84c70fc * commit '2ca84c70fcae58ea9186310a68f0b4b24d511178': (35 commits) Handle nested define/endef Update findleaves.py and add a few testcases [C++] Add support for multiple filenames to findleaves emulation Support all kinds of command line variables Allow NULL filename for -d flag [C++] Add -d flag to make debugging slightly easier Normalize log for recent ninja Always sort glob results Skip 3 tests which fail with make 4 Skip shell_var_with_args.mk with GNU make 4 Use override in posix_var.mk to fix test with make 4 Stop overwriting /bin/sh by bash on travis Normalize Unicode quotes for recent find Explicitly use SHELL=/bin/bash [C++] Stop using an uninitialized value Fix multi_implicit_output_patterns.mk for GNU make 4 Clean up normalization in runtest.rb a bit Remove test output in ckati_clean Normalize GNU make 4.00 in runtest.rb Suppress GNU make jobserver magic in runtest.rb ... Change-Id: I95545a47bab08be7d3d980ccee23d29e29d635ab
| * | | Merge remote-tracking branch \'aosp/upstream\' into masterShinichiro Hamaji2016-05-3038-59/+361
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | am: ff2d59e2e0 * commit 'ff2d59e2e082d17ae04f43d409244440a1687856': (35 commits) Handle nested define/endef Update findleaves.py and add a few testcases [C++] Add support for multiple filenames to findleaves emulation Support all kinds of command line variables Allow NULL filename for -d flag [C++] Add -d flag to make debugging slightly easier Normalize log for recent ninja Always sort glob results Skip 3 tests which fail with make 4 Skip shell_var_with_args.mk with GNU make 4 Use override in posix_var.mk to fix test with make 4 Stop overwriting /bin/sh by bash on travis Normalize Unicode quotes for recent find Explicitly use SHELL=/bin/bash [C++] Stop using an uninitialized value Fix multi_implicit_output_patterns.mk for GNU make 4 Clean up normalization in runtest.rb a bit Remove test output in ckati_clean Normalize GNU make 4.00 in runtest.rb Suppress GNU make jobserver magic in runtest.rb ... Change-Id: Iaeea51db1eecc40c3b755504ff8eb67b8d40c0aa
| | * | Merge remote-tracking branch 'aosp/upstream' into masterColin Cross2016-05-3038-59/+361
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 390115c Handle nested define/endef c15a824 Update findleaves.py and add a few testcases af13468 Merge pull request #73 from colincross/findleaves f0a6fdf [C++] Add support for multiple filenames to findleaves emulation 1c3a695 Support all kinds of command line variables ac6f169 Allow NULL filename for -d flag c58db9a [C++] Add -d flag to make debugging slightly easier 38892d8 Normalize log for recent ninja 9f6343c Always sort glob results 8082dcb Skip 3 tests which fail with make 4 1561f68 Skip shell_var_with_args.mk with GNU make 4 a1ded1c Use override in posix_var.mk to fix test with make 4 fc24bd2 Stop overwriting /bin/sh by bash on travis 913eb74 Normalize Unicode quotes for recent find 2d2ed95 Explicitly use SHELL=/bin/bash d07e297 [C++] Stop using an uninitialized value 855fea4 Fix multi_implicit_output_patterns.mk for GNU make 4 dc258cb Clean up normalization in runtest.rb a bit 3009771 Merge pull request #66 from stefanb2/topic-issue-65 bfae810 Remove test output in ckati_clean a24a7a0 Normalize GNU make 4.00 in runtest.rb e4e56f3 Suppress GNU make jobserver magic in runtest.rb 2941ea0 [C++] Handle .POSIX at eval time 03fa345 Add testcase/posix_var.mk a8fafa4 Remove Go related targets from test and clean d8d43ea [C++] Reduce unnecessary string allocation c6926b0 Add testcase/call_with_whitespace.mk 5dfb361 Add testcase/recursive_marker.mk 02e5ee3 [C++] Remove unnecessary #include 6823600 Add a test case to override_override.mk 4e60e5f Merge pull request #58 from stefanb2/topic-issue-57 952d445 [C++] Ignore recursive marker in recipes 786881c [C++] Replace erroneous return in EvalInclude() d4f2871 [C++] Store SHELL value in command result 29b9b74 [C++] Honor "override" when setting global variable 167e1f7 [C++] Ignore white space around $(call) function name 187bf08 [C++] Add support for .POSIX: cd060c5 add Stefan to CONTRIBUTORS Change-Id: I7697703aa2a0eb37202645b1895899807e6426b2
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-20 00:59:57 +0000