aboutsummaryrefslogtreecommitdiff
path: root/python/scripts
Commit message (Collapse)AuthorAgeFilesLines
* Fix non-deterministic python compilationAnas Sulaiman2024-02-071-9/+12
| | | | | | | | | | | | This CL fixes timestamps for zip entries to a hardcoded value to ensure deterministic zip archives across checkouts, not just for incremental builds. The value chosen for the fixed date is the same as the one used by soong_zip. Bug: b/322788229 Test: Ran a few builds and verified that mdate for zipped files are fixed as well as the generated zip files have identical hashes. Change-Id: I3cd6fcf559d4d83d8813c93319b46e267ae64a2b
* Fix non-deterministic python compilationAnas Sulaiman2024-02-011-9/+19
| | | | | | | | Timestamps in pyc files and zip archives were causing cache misses in RBE. Bug: b/322788229 Test: Ran a few local builds and verified improved cache hit rate for metalava actions. Change-Id: I9375694d26a107f5c0c43bbd685bd8e86349d187
* stub_template_host redirect SIGINT and SIGTERM to subprocessQiao Yang2023-03-221-1/+13
| | | | | Test: manual Change-Id: I561f581f7413dc18b2d449f4ef338db3b6b9ca47
* Precompile python sourcesCole Faust2023-01-271-0/+61
| | | | | | | | | | | This signifigantly improves the startup time of soong-built python binaries. For example, running `m apexer && time out/host/linux-x86/bin/apexer` gives 0.734s before this cl, and 0.094s after. Fixes: 259718110 Test: Presubmits Change-Id: Ib19e83e2c60c39a849525be117279c318de3afa7
* Remove flags that enable the new python path behaviorCole Faust2022-10-211-10/+1
| | | | | | | | | The new behavior has been enabled by default, and these flags aren't necessary anymore. Fixes: 245583294 Test: m py_dont_import_folder_of_entrypoint_test && /ssd/aosp-master/out/host/linux-x86/testcases/py_dont_import_folder_of_entrypoint_test/x86_64/py_dont_import_folder_of_entrypoint_test Change-Id: I5b6f98da51791bc5d28662ef799a10c1bb6a35a0
* Add flag to not add directory of entrypoint to sys.pathCole Faust2022-09-261-0/+6
| | | | | | | | | | | | | | | | | | | | The python interpreter will by default add the directory of the entrypoint script to the beginning of sys.path. This can be disabled in python 3.11+ (which is not released yet) using the PYTHON_SAFE_PATH environment variable or the -P flag. As a workaround to have this behavior in older python versions, we can make an __soong_entrypoint_redirector__.py file at the root of the zip file that is the entrypoint, and then that file will redirect to the real entrypoint. This brings non-embedded-launcher python modules closer to the embedded launcher version. The embedded launcher binaries already act like this because they start at an __main__.py file at the root of the zip file. Bug: 245583294 Test: m py_dont_import_folder_of_entrypoint_test && out/host/linux-x86/nativetest64/py_dont_import_folder_of_entrypoint_test/py_dont_import_folder_of_entrypoint_test Change-Id: I39aaf04fb19c3ba7f5c9d98220872d6d08abf736
* Add flag to not add top-level modules to PYTHONPATHCole Faust2022-09-231-51/+16
| | | | | | | | | | | | | | stub_template_host.txt added all the top-level modules to the PYTHONPATH, which isn't correct, and caused absl.logging to override the built-in logging module. Removing this also makes it more consistent with python binaries built with embedded_launcher: true. embedded_launcher: true binaries don't add the top-level modules. Fixes: 245583294 Test: m py_dont_add_top_level_dirs_test && out/host/linux-x86/testcases/py_dont_add_top_level_dirs_test/x86_64/py_dont_add_top_level_dirs_test Change-Id: Id3069565d2b2c4b2bda0ff5301e757a7b4201751
* Don't close file descriptors when starting python programCole Faust2022-09-091-1/+3
| | | | | | | | | | | Python programs run via a wrapper script. That wrapper script can be started with open file descriptors, for example by using the <(echo foo) syntax in bash. The shouldn't be closed when starting the real script. Bug: 203436762 Test: Presubmits Change-Id: Ib20450f713b15c8a5527d1e433bfd9635cead85f
* Change par file generate rules.yangbill2021-04-191-2/+2
| | | | | | | | | | | Make sure the shebang of the par file is consistent with the main python script rather than use /usr/bin/python arbitrarily. Bug: 185399398 Bug: 185094069 Test: m atest ; atest-dev --help m atest-py2 ; atest-py2-dev --help Change-Id: Ice4590b3d173e6d11e608c2ca29b297df0dc4089
* Simplify python launcher, use __main__.pyDan Willemsen2019-02-151-0/+12
| | | | | | | Uses more python rather than C++, and skips less of Py_Main. Test: build/soong/python/tests/runtests.sh Change-Id: I03997d88e2e16047c96bb4e00e530229c42b3325
* Pass -S option to wrapper for python binary.yangbill2019-01-041-1/+4
| | | | | | | | | | | | | | Prevent problem for some users using PAR file built by android's build system which cause by the different site package of python installed by user themselves. Bug: 120469915 Test: 1. m atest atest --help 2. apply aosp/861470 full build Change-Id: I72c83f9397f0e88eecfcd32806f1746520299447
* Define runfiles_path before try so that finally can use it.Nicolas Geoffray2018-10-111-1/+3
| | | | | | | | Otherwise, python will complain the variable has not been initialized, in case ExtractRunFiles throws an exception. Test: m Change-Id: I3d9e3cda5e558436ee4182c44e4345151c031e41
* Fix embedded_launcher can't find filesNan Zhang2018-06-041-3/+2
| | | | | | | | | | | The problem came from Python libraries doesn't know the information that we enabled embedded_launcher (only Python binary knows about that). And we can't simply remove runfiles dir for Python libraries since host Python mode need this. Bug: b/80441699 Test: m perf_profo_flames Change-Id: I73ffc4d7504f95a708ae7cca47bc6c15a673aa31
* Revert "Revert "Revert "Revert "Add path interposer""""Dan Willemsen2018-05-311-2/+0
| | | | | | | | | | | | | | | | | | This reverts commit 09f4540d667a98e548bf5dfc94784382ebedc963. Fixes the raw call to net.Listen in the tests to go through the listen() helper and use the long socket path fallbacks. Removes the use of timeouts from the tests -- the behaviors being tested did not rely on timeouts, so removing them will reduce the flakiness if the build is heavily loading the machine at the same time the test is running. Also fixes some potential nil pointer dereferences. Test: OUT_DIR=<really long> m blueprint_tools Test: `while .../soong-ui-build-paths/test/test; do sleep 0.01; done` with a build running Change-Id: I16d44be7517bc415f1c808284088f4ba40df3bfa
* Revert "Revert "Revert "Add path interposer"""Dan Willemsen2018-05-251-0/+2
| | | | | | | | | | This reverts commit c59a92cb1a73468a0e2af7102632dda48d811f5b. Reason for revert: tests are broken with long OUT_DIRs They're directly calling net.Listen, and not using the fallback for long socket names. Change-Id: Id14cbd499fd9b36c6926b7552d3554340cb0916c
* Revert "Revert "Add path interposer""Dan Willemsen2018-05-251-2/+0
| | | | | | | | | | | | | | | | | | | This reverts commit 96c957ae20d8418f85cf10b4ee06ce9781d86c47. Fixes issues on some machines where the socket in TMPDIR ended up with a unix domain socket pathname over 107 characters long, which Go will reject due to underlying limitations in the system calls. If this happens, we'll fall back to opening the directory, then using /proc/self/fd/#/<file>, or manually creating a similar symlink in /tmp. Also fixes some issues on Mac where os.Executable returns the symlink instead of the underlying file, sending a message over a unix domain socket will block if the reader isn't reading, and sandboxing was preventing us from running `ps`. Test: m blueprint_tools Test: m blueprint_tools on mac Change-Id: Ib19ccfe10cb0a79f1476fb1d5cd20ed0495be367
* Revert "Add path interposer"Dan Willemsen2018-05-191-0/+2
| | | | | | | | This reverts commit a14704c12bbcc1eb207d1c0ff3df0140a4582fc5. Reason for revert: breaking builds Change-Id: I920b0ff41823428a0baf59b3fd82cbcc766babca
* Add path interposerDan Willemsen2018-05-181-2/+0
| | | | | | | | | | | | | | | | | | | This will allow us to track (and eventually limit) the commands that the build references via $PATH. These are mostly implicit dependencies on the host system -- for Linux, we assume something similar to Ubuntu 14.04 with a few extra packages, but this will let us better define that. This will not catch uses of tools with absolute paths (/bin/bash, etc), but most uses shouldn't be relying on absolute path names anyways. Adds ~400ms on the first startup, ~140ms on subsequent runs, and overhead of a few ms for every forwarded execution. Test: m Test: build/soong/build_test.bash Test: Add `gcc --version`, TEMPORARY_DISABLE_PATH_RESTRICTIONS=true m Change-Id: Id68cbb1c8ceef65bbbb10751e83722c7662d2351
* Supported python build in host side.Nan Zhang2017-05-051-0/+91
The base module handles all the common functionalites, such as version compatibilty check, version variations split, source file format check, source/data file duplicate check. The library/binary module focuses on how to generate binary build actions, such as setting up stub script, zipping, filling in __init__.py in runfiles dir tree. Bug: b/31676493 Test: go test under python package Change-Id: I06608369f350f7195873d459e1c8d1bdb811e77e