aboutsummaryrefslogtreecommitdiff
path: root/zip/zip.go
Commit message (Collapse)AuthorAgeFilesLines
* Add `-quiet` option in soong_zipJihoon Kang2023-09-131-3/+10
| | | | | | | | The quiet option prevents warnings from being printed to the console. Test: m soong_zip Bug: 300166930 Change-Id: I4c2c5f16c45c2874a2a2cbb1e3f397124043e472
* Add flag to skip writing output from soong_zip.Dan Albert2023-07-131-2/+5
| | | | | | | | | | | | | | | | | | | | We want to measure how much of build time is spent on writing large zips to disk. We can do this by doing the rest of the work but skipping the write itself. Some local and trivial testing shows this is probably a lot of the cost of these rules: $ python -m timeit -s "import os" -r 10 "os.system('soong_zip -write_if_changed -o foo.zip -D prebuilts/sdk')" 1 loop, best of 10: 8.37 sec per loop $ python -m timeit -s "import os" -r 10 "os.system('soong_zip -o foo.zip -D prebuilts/sdk')" 1 loop, best of 10: 5.39 sec per loop $ python -m timeit -s "import os" -r 10 "os.system('soong_zip -n -o foo.zip -D prebuilts/sdk')" 1 loop, best of 10: 3.66 sec per loop Bug: None Test: treehugger Change-Id: I43acdb08d0e00efaba9e5d7add972a7ec41646d2
* Add -e argument to soong_zip to allow setting an explicit filenameColin Cross2023-05-091-1/+14
| | | | | | | | | | | | soong_zip normally takes the name of the input source file as the name of the file in the zip, which is ideal for zipping up directories but not for constructing arbitrary zip files. Add a -e argument that explicitly sets the path in the zip file for the next -f argument. Bug: 254867347 Test: zip_test.go Change-Id: If9d62c1a0064a485aebddf6d2a661f63f3e60b0f
* Support storing SHA256 checksum for files in soong_zipZhenhuang Wang2023-01-251-17/+59
| | | | | | | | | | | Add a -sha256 argument to soong_zip that computes SHA256 checksum for each file and stores the value in file header. The SHA information can then be used by downstream systems that use content addressing. Bug: 259513199 Test: zip_test.go Test: soong_zip -o test.zip -D test_dir -sha256 Change-Id: I20e9f424bd0a4604f0dc7cc77bd65f10eb49a163
* Allow duplicate files inputs in soong_zipColin Cross2022-08-151-4/+48
| | | | | | | | | | | Accept duplicate file inputs in soong_zip when they are the same source file. This came up when trying to zip lint srcs, as some java modules have duplicate source files that seem to be ignored by javac. Test: TestZip Bug: 216456886 Change-Id: I8c43df9aded8cf094afaed79cca2b9eb091cc861
* Use broader permissions when archiving files.Chris Gross2021-06-021-3/+3
| | | | | | | | | | | Using broader premissions for archived files allows them to be more easily used when extracted. For example, defaulting regular files to 0644 will allow other tooling to use a file without the need to change permissions manually. Bug: 189919409 Test: m dist and inspected file permissions of archived files Change-Id: I4a0f8075206391254c639ecf865639bb9e8df0bf
* Always call SetMode in soong_zipColin Cross2021-05-111-1/+3
| | | | | | | | | | In addition to setting the mode, SetMode also marks the file as created by Unix, which seems to cause Debian's unzip tool to handle UTF-8 filenames correctly. Fixes: 184575223 Test: TestZip Change-Id: I0e839dc3d27aaa9abced0eb1d3c4c0f8eed3e3c4
* Follow blueprint change to return GlobResult from GlobColin Cross2021-04-091-5/+5
| | | | | | | | | Follow I2159cc9d85f388073198eac7456e5bf43e813096 that makes Glob return a GlobResult. Bug: 159845846 Test: glob_test.go Change-Id: Ia771bdbdf1eb668623c4b3f00bf65e0e1e3a55c0
* Move response file handling to a separate packageColin Cross2021-03-251-48/+4
| | | | | | | | sbox is going to need to read and write response files, move ReadRspFile to its own package. Test: response_test.go Change-Id: Iecb5486b4aaeb2531828743ad8ef784df675e18e
* Fix removing partial zip file on errorColin Cross2021-03-161-4/+6
| | | | | | | | | Fix the shadowed err variable so that it is visible to the defer function that removes the partial zip file on error. Bug: 182761771 Test: manual Change-Id: Ia67ac4bca5ef0107466edf25a76c74d70c243d3f
* soong_zip: don't glob RSP file entriesColin Cross2021-02-031-0/+3
| | | | | | | | | | File arguments are considered globs, but RSP file entries should be considered filenames without globbing. Escape RSP file entries so they don't get treated as globs later. Bug: 179297188 Test: zip_test.go Change-Id: I1de7829c668538204fc4a2ac793a4f88dbc886c9
* Document usage, perform minor cleanups.Sasha Smundak2020-11-161-8/+9
| | | | | | Test: treehugger Bug: 173466220 Change-Id: I5aa09c2941c0eceb8ae635b9e613ad932d38aebb
* Support ninja rsp files in soong_zipColin Cross2020-08-191-0/+24
| | | | | | | | | | Add a -r argument to soong_zip that reads a list of files from a file like the -l argument but treats it as a Ninja rsp file with escaping. Replace the -l arguments in Soong that are using rsp files with -r. Fixes: 162435077 Test: TestReadRespFile, TestZip Change-Id: I4605312e99406ab1bd0c37af9c5ad212393f0403
* -l option soong_zip can accept a file having space separated listJiyong Park2019-11-041-1/+1
| | | | | | | | | | | | -l option of soong_zip is used to specify the list of input files that should be zipped. However, it only accepted newline-separated list in the file. This prevented us from using a rspfile as paths in rspfile are space-separated in a single line. Fixing the problem by splitting the file content by any unicode whitespace character including newline and space. Test: zip_test Change-Id: Iba572109e30c01540eacf20bd2794ba60c870fa3
* Support moving sources in srcjars in soong_zipColin Cross2019-06-181-28/+60
| | | | | | | | | Add a -srcjar argument to soong_zip that causes it to read the package statement of each .java file and use that to place the source file at a path that matches the package. Test: jar_test.go, zip_test.go Change-Id: I36017e42445ba3b0a82a10a8d81e8ac0cca096f2
* Fix soong_zip printing warnings with --ignore_missing_filesDan Willemsen2018-12-271-3/+3
| | | | | | | | We weren't defaulting args.Stderr to os.Stderr, so we had been calling fmt.Fprintln with `nil`. Test: soong_zip --ignore_missing_files -C test/ -f abc -o test.zip Change-Id: I0421f4bfb07fa36edf86c68ffc847995777c8221
* soong_zip: add --ignore_missing_files flagColin Cross2018-09-281-17/+46
| | | | | | | | | | | | | | | soong_zip builds a list of files to zip early and then starts zipping them all. If a directory being zipped is concurrently modified, a file that existed when soong_zip started may not still exist. Add a flag that continues when an expected file does not exist. Print a warning, since this should be rare in normal usages but is a sign of a problem if it happens regularly. Test: zip_test.go Test: m checkbuild Test: m platform Change-Id: I78426fe66fded8528ddd436c0f71a7442183cfeb
* Add a --symlinks argument to soong_zipColin Cross2018-09-281-10/+26
| | | | | | | | | | | | | | Add a --symlinks argument that defaults to true to soong_zip. Passing --symlinks=false will cause it to follow symlinks instead of storing them in the zip file. Relands I4deb98daa9d4ba9f94e3d7670c117fe00381d2ba with tests. Bug: 112843624 Test: glob_test.go Test: zip_test.go Test: m checkbuild Change-Id: I0eff9c1f2dba79e873fda381ff585df55d5aaaad
* soong_zip: support globs in -f and -D argumentsColin Cross2018-09-281-19/+57
| | | | | | | | | | | | | | | | | | | | -f and -D arguments can now take globs in the Soong format. Also update the use of soong_zip that jars resources to escape the globs in the arguments, and then shell-escape them when writing to the rsp file so the glob escape are not intepreted by ReadRespFile. Also remove an unused argument to the buildAAR rule that could have contained values that needed escaping. Relands I7f20bb169dc01f952d2a7681ec6ee9c05737ed37 with a fix for trailing "\n" in list files, which causes a call to pathtools.Glob("") that returns "./", which could then get incorrectly translated to "../../../" in the zip file. Also adds tests. Test: m checkbuild Test: zip_test.go Change-Id: I54b8eef9231875e6042a32c9f8bcc5c2f779922a
* soong_zip: Add testsColin Cross2018-09-281-21/+40
| | | | | | | | | | Add test that cover basic command line usage of soong_zip. -D is not covered yet as the implementation will be replaced with one that is also more easily testable in the next patch. Bug: 116751500 Test: zip_test.go Change-Id: I5a1bcee74ebc9cb3cf332c36f89bc12c0e807ad2
* soong_zip: move profiling from zip library to soong_zipColin Cross2018-09-281-30/+0
| | | | | | | | | Profiling is only used by the standalone soong_zip command, move it out of the shared zip library. Bug: 116751500 Test: m checkbuild Change-Id: I443c34fb39cf8955e163a7720d6f7ed585e4172a
* soong_zip: move args parsing into zip.FileArgsBuilderColin Cross2018-09-281-16/+89
| | | | | | | | | Add a builder that can construct []FileArg for zip.Run to use. This will help writing tests that cover the command line parsing. Test: later cl Bug: 116751500 Change-Id: I15b5144f5094ab154f13de9c8a84b82db2da4e67
* Revert "soong_zip: support globs in -f and -D arguments"Colin Cross2018-09-261-13/+2
| | | | | | | | | This reverts commit 08e28abc4ecd10a0e0ab2dcb683560f9c6331e1b. Reason for revert: inserts extra ../../ entries in the zip file. Bug: 116737386 Change-Id: I048f94889a66fe618058e37a5827d0cf6b1a6b68
* Revert "Add a --symlinks argument to soong_zip"Colin Cross2018-09-261-22/+8
| | | | | | | This reverts commit d59dab94c4897f504ca7a3a2b227ca6b000bfaa4. Bug: 112843624 Change-Id: I3d0f1b61e899b162e65c18662f9d27dd794a9a30
* Add a --symlinks argument to soong_zipColin Cross2018-09-211-8/+22
| | | | | | | | | | | Add a --symlinks argument that defaults to true to soong_zip. Passing --symlinks=false will cause it to follow symlinks instead of storing them in the zip file. Bug: 112843624 Test: glob_test.go Test: m checkbuild Change-Id: I4deb98daa9d4ba9f94e3d7670c117fe00381d2ba
* soong_zip: support globs in -f and -D argumentsColin Cross2018-09-211-2/+13
| | | | | | | | | | | | | | -f and -D arguments can now take globs in the Soong format. Also update the use of soong_zip that jars resources to escape the globs in the arguments, and then shell-escape them when writing to the rsp file so the glob escape are not intepreted by ReadRespFile. Also remove an unused argument to the buildAAR rule that could have contained values that needed escaping. Test: m checkbuild Change-Id: I7f20bb169dc01f952d2a7681ec6ee9c05737ed37
* soong_zip: add support for -j to junk pathsColin Cross2018-09-191-7/+14
| | | | | | | | | | | | | | When -j is specified ignore the path to the source file and just zip it with its filename. -j overrides -C, and -C overrides -j, so -j -f path1 -C dir -f path2 will junk the path for path1, but treat path2 as relative to dir. Remove the filepath.Clean for the FileArgs, it would convert "" to "." sometimes, and everything gets cleaned in zip already for the non-command-line use cases. Test: m checkbuild Change-Id: I7d90572c622ee0c6f35967ff31d067b5312c72eb
* Don't attempt to deflate when compression level is 0Colin Cross2018-09-141-4/+9
| | | | | | | | | | soong_zip -L 0 would attempt to compress with deflate at compression level 0, then pick store instead of deflate because the overhead of deflate would always make it larger than the uncompressed size. Just use store instead. Test: m checkbuild Change-Id: I69610bd755b55d97a47b8c6cf4fbbee3b5632db6
* soong_zip: set local header crc and size for symlinksColin Cross2018-06-221-1/+4
| | | | | | | | | | | | | | | | | | | | | | | Getting a crc and size into the local header requires setting it before writing the payload, or using a streaming data header after the payload with the crc and size. Stored (uncompressed) entries are not allowed to use a streaming data header. Symlinks are always stored uncompressed, so set the crc and size in the file header before writing the payload. Also set the mode to 0777 to match the behavior of zip. This relands I66c5d41f0a5b23b828d6a03a3790afedc5a97625 with fixes for the size and mode. Test: m checkbuild Test: zipdetails on zip with symlink created with soong_zip has same crc in local header and central header. Test: Compare zipdetails output of zip containing symlink created by soong_zip and by zip --symlinks -X. Bug: 110716403 Change-Id: Iec0bc9056a0d9cdab76f015844aca9c711e72e5b
* Revert "soong_zip: set local header crc for symlinks"Colin Cross2018-06-231-2/+0
| | | | | | | | | | | | This reverts commit b88c4dfb8794c0618067c127794445e8fd821c33. This made it worse, now that the local header crc matches the global header, unzip will attempt to unzip the symlink. However, the local header size is still incorrect, so it produces an empty file instead of a symlink. Bug: 110716403 Change-Id: Ifd9aabc04d180b7ead4aec8566c151206b6b7c57
* soong_zip: set local header crc for symlinksColin Cross2018-06-231-0/+2
| | | | | | | | | | | | | | | | Getting a crc into the local header requires setting it before writing the payload, or using a streaming data header after the payload with the crc. Stored (uncompressed) entries are not allowed to use a streaming data header. Symlinks are always stored uncompressed, so set the crc in the file header before writing the payload. Test: m checkbuild Test: zipdetails on zip with symlink created with soong_zip has same crc in local header and central header. Bug: 110716403 Change-Id: I66c5d41f0a5b23b828d6a03a3790afedc5a97625
* Add Respfile support for soong_zip.Nan Zhang2018-02-131-0/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Sometime the size of our command line passed to soong_zip go program exceeds the cmdline size limit. So add an RespFile support with "@" special character prefix. The args in the cmdline will be considered together with the args in RespFile during soong_zip running. Test: real tests in my local machine, and compare the res/libphonenumber.jar before and after changes. ./cmd -o test.zip '""'-C -> [./cmd,-o,test.zip,""-C] ./cmd -o test.zip '-C -f -> [./cmd,-o,test.zip,-C -f] ./cmd -o test.zip '\"'-C -f -> [./cmd,-o,test.zip,\"-C -f] ./cmd -o test.zip '\\'-C -f -> [./cmd,-o,test.zip,\\-C -f] ./cmd -o test.zip '\a'-C -f -> [./cmd,-o,test.zip,\a-C -f] ./cmd -o test.zip \'-C -> [./cmd,-o,test.zip,'-C] ./cmd -o test.zip \\-C -> [./cmd,-o,test.zip,\-C] ./cmd -o test.zip \"-C -> [./cmd,-o,test.zip,"-C] ./cmd -o test.zip "'"-C -> [./cmd,-o,test.zip,'-C] ./cmd -o test.zip "\\"-C -f -> [./cmd,-o,test.zip,\a-C -f] ./cmd -o test.zip "\""-C -f -> [./cmd,-o,test.zip,"a-C -f] Bug: b/72484223 Change-Id: I83c3630b70c8396c8e8a3f266244d868d754c4e8
* Add --write_if_changed argument to soong_zipColin Cross2017-11-131-17/+39
| | | | | | | | | Simplify making restat rules by adding a --write_if_changed argument to soong_zip that generates the output file into memory and then compares it to the version on disk before writing it out. Test: builds Change-Id: I5059a1e3a11e79b0ca538b3b829bc7479c126ce6
* split soong_zip into a library and a binaryJeff Gaston2017-10-191-0/+760
to make it faster/easier to invoke from other Go programs (such as multiproduct_kati) Bug: 67478260 Test: m -j Change-Id: Idd2671a44290550197c88f53dd11a6dd39c85cc5
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-20 08:11:28 +0000