summaryrefslogtreecommitdiff
path: root/payload_generator
Commit message (Collapse)AuthorAgeFilesLines
* Allow parsing boot image with header version 3Tianjie2020-03-264-37/+157
| | | | | | | | | | | | | | | | | The boot image header version no longer contains a page_size field; and it assumes the block size is always 4096. Also, the order of the header fields are different in version 3 from the previous versions. While the position of "header_version" is fixed at position #9 across all image headers. This cl handles header version 3 separately from the previous versions. Also create a stub implementation since the parsing of boot image is only used in android. Bug: 152371989 Test: unit tests, generate incremental OTA for the affected package, build with stub file Change-Id: Iea2145e0c234216f9ee42c571bd53dc93b4b9d2e (cherry picked from commit 37ae85653198123fc28a4ce1de25d2e56ee6a3a2)
* Update keys in dynamic_partitions_info.txt.Yifan Hong2019-11-071-4/+8
| | | | | | | | | These key names have been changed to match the same keys in misc_info.txt. Read new key names as well. Test: m otapackage -j Change-Id: I70e009f2e5668863fd54db45b2b230c7f4a8e896
* Add a command in delta_generator to calculate the maximum signature sizeTianjie Xu2019-10-141-0/+28
| | | | | | | | | | | The ota generation script used to call openssl functions to get the signature size. This is no longer viable for EC keys. So we add this new functionality here in the delta_generator. Because the signature size will be later used by the delta_generator to sign the payload. Bug: 141244025 Test: call the binary with new option Change-Id: Id743325242faf7a2b2dcec5e218219dba12a8e88
* Add EC key signing supportTianjie Xu2019-10-104-30/+102
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The DER encoded signature size of ECDSA with P-256 NIST CURVE is nondeterministic for different input of sha256 hash. For example, the signature size can be 70, 71, 72 bytes with the maximum possible size of 72 bytes. However, we need the size of the serialized signatures protobuf string to be fixed before signing; because the size is part of the content to be signed. To achieve that, we can add padding to the signature; and update the definition of the signature proto to include the unpadded signature size. message Signatures { message Signature { optional uint32 version = 1; optional bytes data = 2; optional fixed32 unpadded_signature_size = 3; } repeated Signature signatures = 1; } Therefore the payload verifier will read the unpadded signature and use it to verify against the public keys. For RSA signatures, the signature data already has the correct size. So the legacy update_engine on the old devices will still be able to verify these signatures in new proto format. We also need to update the version in signature proto, and the minor version of update_engine. The EC key in the unittest is generated with the command: openssl ecparam -name prime256v1 -genkey -noout -out prime256v1-key.pem openssl pkey -in prime256v1-key.pem -out unittest_key_EC.pem Bug: 141244025 Test: unit tests pass, sign a package with EC key and and install on sailfish Change-Id: I0a16c9f2f2c7fe9ccc1070c87fbbd6b94bc1f542
* Factor out the RSA verification in payload verifierTianjie Xu2019-10-042-37/+54
| | | | | | | | | | | | | Right now, the payload is always signed with a RSA key, and the payload verifier takes the public key as a PEM encoded string. As we want to support payload signing with EC keys, we need to figure out the key types first in the verifier. So, add an overload function in payload verifier to accept EVP_PKEY as the public key. Bug: 141244025 Test: unittests pass Change-Id: Ibbdac5a7a3de48347100861aeac0013bff43da6f
* Add snapshot_enabled field to DeltaArchiveManifest.Yifan Hong2019-10-021-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When this field is set, OTA client should try to use snapshots during updates. If it is not set, OTA client MUST NOT use snapshots. This field is set iff the target build has Virtual A/B feature enabled / retrofitted (except for secondary OTAs, where system_other should be written directly). In follow-up CLs, DynamicPartitionControlAndroid only maps partitions as snapshots if this flag is set. DeltaPerformer may skip in-place SOURCE_COPY operations if snapshots are created. Note that: - On retrofit Virtual A/B devices, this field is always set. If updating from a non Virtual A/B build to a Virtual A/B build, the OTA client on the device would simply ignore this field and perform a regular A/B OTA. - When downgrading a retrofit Virtual A/B device back to a regular A/B build, this field is NOT set, and the OTA client will create the target slot partitions as usual. - When sideloading a full OTA in recovery, OTA client cannot create any snapshots. This field is effectively ignored (as if it were set to false). Fixes: 141720569 Test: build OTA Test: update_engine_unittests Test: apply Virtual A/B OTA Change-Id: I1939b24f6687f66a682cd6b7ae826f27acf98e2f
* Signature size now uses size_tTianjie Xu2019-10-014-9/+11
| | | | | | | | Signature size shouldn't have negative values. Make its usage more consistent. Test: mma Change-Id: Ie6cbf49694fcbe0348d6338373423d77d3edfdd4
* Add a flag to disable the fec data computationTianjie Xu2019-08-224-6/+35
| | | | | | | | | | | The write of fec data is currently mandatory in the update engine. This step reduces the update package size but it's also time comsuming. So we add a flag to allow partners make the trade off and disable the on device fec data computation. Bug: 139723500 Test: generate and apply incremental updates for verified boot 1.0 & 2.0 Change-Id: Ic7c63396bb4d4fbbc3c3b2e9ff3804c9ff941f2f
* Only scan for deflates in regular filesHåkan Kvist2019-07-161-2/+13
| | | | | | | | | | | It only makes sense to scan for deflates in regular files. Scanning a symlink to a zip/gzip file would crash the generator. Test: Run ota package generation script with image including a symlink Bug: 137128486 Change-Id: I16f9040f2e483dcbb6a77d6dc56d38d32529521c
* Rename product_services to system_extJustin Yun2019-07-011-6/+6
| | | | | | Bug: 134359158 Test: run tests for update_engine_unittests Change-Id: I5d5b7f33f2b106f23152049f07e4b44c0cdcdc7e
* Support signature size of 512 bytesxunchang2019-04-081-2/+2
| | | | | | | | | The signature size will be 512 bytes for the payload signed with 4096 bits RSA key. Bug: 129163830 Test: generate and verify an OTA Change-Id: I18710218e4a3a030e257c594f416831cbfa8c041
* Support signing payload with 4096 bits RSA keysxunchang2019-04-022-21/+25
| | | | | | | | | | | | | | | | | | | The 32 bytes sha256 hash was padded to 256 bytes before payload signing and verification. During the padding, we appended a hard coded header according to RFC3447 spec. As we want to support signing with 4096 bits keys, the format of the padding doesn't change but the length needs adjustion. Now callers will pass in the RSA size in bytes in the padding function. And the verification function will now take the raw 32 bytes sha256 hash instead of the padded value. The new key for unittest is generated by: openssl genrsa -out unittest_key_RSA4096.pem 4096 Bug: 129163830 Test: unit tests pass, create and install an update signed by 4096 bits key. Change-Id: I8e0d02ddb1472e22976c0f170e8bf2b8b094c7d4
* Use InstallOperation::Type instead of InstallOperation_Type.Sen Jiang2019-02-2612-29/+29
| | | | | Test: mma Change-Id: I4d23856904eab70ece686fe571c2db517b60835b
* Use string for Signatures protobuf.Sen Jiang2019-02-224-92/+79
| | | | | | | | To avoid conversion between brillo::Blob and string. Also use SerializeToString() instead of AppendToString(). Test: update_engine_unittests Change-Id: I0ad0fb5f45223f9f51f940de727660a9e62052bb
* update_engine: Run clang-format on payload_generator/Amin Hassani2019-01-1653-762/+689
| | | | | | | | | | | BUG=none TEST=unittest Change-Id: I00cafb1fc709d8f02119cb0058cfcb0b573ef25d Reviewed-on: https://chromium-review.googlesource.com/1409927 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
* Use base::TimeTicks.Sen Jiang2019-01-101-3/+2
| | | | | | | | base::Time is wall clock time which might be decreased, base::TimeTicks should be used to track the time difference. Test: unit test Change-Id: I1ed2c74137955a8351815ebda8e6c7330d7b1012
* Don't use temp file for public key.Sen Jiang2019-01-092-18/+22
| | | | | | | | | If using public key from Omaha response, we wrote it to a temp file which will be read back, this is unnecessary since we can keep it in memory. Test: update_engine_unittests Change-Id: Ib9f7a9292b71b9d91a0b24c687cc989b79d3812b
* Merge aosp/upstream-master into aosp/master.Sen Jiang2019-01-041-1/+1
|\ | | | | | | | | | | | | Added BinderUpdateEngineClient::AttemptInstall(). Test: update_engine_unittests Change-Id: Id6911f49d763b0d572658acb7d66857016bf6969
| * update_engine: brillo_update_payload: Switch to major version 2Amin Hassani2019-01-021-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Per this CL, we switch the payloads to major version 2 from servers. For more information regarding this switch refer to the attached issue. We also change the default major version to 2. BUG=chromium:794404 TEST=brillo_update_payload Change-Id: I8403805c07210c3d947100efce74dc184a022cdb Reviewed-on: https://chromium-review.googlesource.com/1234590 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
* | update_engine: Remove problematic deflate caches.Amin Hassani2019-01-041-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | This patch calls puffin::RemoveDeflatesWithBadDistanceCaches() to ignore certain deflates that older puffin clients fail to properly process. Unfortunately this adds another round of puffin call which will add a few seconds to paygen, but we can't get out of it at this point yet. In the future we can increase a minor version for it. Bug: crbug.com/915559 Test: generate delta update from payloads with problematic deflates Change-Id: I77416d4005b5ad2fbb47c129541c811b6a380f60
* | Add support for parsing Verified Boot 1.0 metadata.Sen Jiang2019-01-021-57/+147
| | | | | | | | | | | | | | | | | | | | | | | | This is for devices that uses A/B but not AVB. Now libfec is used on host as well. Bug: 28171891 Test: update_engine_unittests Test: brillo_update_payload generate Test: brillo_update_payload verify Change-Id: I71ac5868b8eb53cb688fe939c298965d59b574ea
* | Always filter out all zero blocks from old_file_extents.Sen Jiang2018-12-043-16/+22
| | | | | | | | | | | | | | | | | | | | Even though we are allowed to read the old blocks multiple times for non-inplace update, we should still skip those blocks that are all zero. Bug: 120465293 Test: no zero blocks in src_extents in delta payload Change-Id: I48bdc06680316d2c9b373e9d929fdff1ad797a55
* | Fix partition spelling.Sen Jiang2018-11-293-8/+8
| | | | | | | | | | | | | | Parition -> Partition Test: mma Change-Id: Ibf4898c443fadd70e45789a55566f25dc4af8b68
* | Add apex file to the list of zip extensions.Sen Jiang2018-11-211-2/+2
| | | | | | | | | | | | | | This is an Android specific package format. Test: None Change-Id: I28bb14d4b5cc5c5623851138028636c7c6f5f403
* | Use a similar file if no old files matches new file name.Sen Jiang2018-11-213-1/+98
| | | | | | | | | | | | | | | | | | | | | | Find the old file with the shortest levenshtein distance to the new file name, and use that for diff operations. This works great if the file name has version number in it, but even for a completely new file, using a similar file could still help. Test: generated a diff Change-Id: I45062f81772220a5d5872a98a7af01ab69837dcc
* | Support payload larger than 4GiB.Sen Jiang2018-11-142-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | The data_offset and data_length field in protobuf are updated from uint32 to uint64, thanks to the variable length encoding in protobuf, this change is backward compatible, this does not change the format of the payload manifest in any way, new client with this change will be able to use 64 bits data_offset, while old client will get the field as if it was casted to uint32_t. Bug: 119222724 Test: generate and apply a 5GiB payload Change-Id: Ie5cc8a9b75ef3a9fa4cb1cda4d403ef9fc174cb3
* | Parse deflates in gzip files.Sen Jiang2018-11-133-38/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | The diff for NOTICE.xml.gz is reduced from 200+K to 17K. Also we are now honoring puffin::LocateDeflatesInGzip() failures, since if won't complain about padding data anymore, and puffin::LocateDeflateSubBlocksInZipArchive() is updated to puffin::LocateDeflatesInZipArchive(). Test: generate a delta payload extracting deflates in gzip Change-Id: I44b816cff8ac99ce78f758eef5875fa99e36e369
* | Remove ZeroPadExtentWriter.Sen Jiang2018-11-021-3/+0
| | | | | | | | | | | | | | | | | | | | We don't generate replace operations with trailing zero removed, so we can remove ZeroPadExtentWriter and ExtentWriter::End(). Bug: 78792859 Test: update_engine_unittests Test: brillo_update_payload verify Change-Id: I3c36d80a9c4475fda0b32c86c6503ab8b179b88f
* | Print total number of operations.Sen Jiang2018-11-021-1/+4
| | | | | | | | | | | | | | | | Also added a flush after all the printf so that it won't be mixed with other logs. Test: brillo_update_payload generate Change-Id: Ic5235503a81703495178684e6dbb10b8b6e624dd
* | Consider diff operation overhead.Sen Jiang2018-11-022-3/+71
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Storing a diff operation has more overhead over replace operation in the manifest, we need to store an additional src_sha256_hash which is 32 bytes and not compressible, and also src_extents which could use anywhere from a few bytes to hundreds of bytes depending on the number of extents. We should consider this overhead when deciding whether to use a diff operation over a replace operation, replace operation should be prefered if payload size is similar, because they don't rely on existing data on disk and they could be merged with other replace operations. Test: brillo_update_payload generate Test: brillo_update_payload verify Change-Id: I850df7baf72fdd1f5b70b22506ebacadb60db58b
* | Add dynamic_partition_metadata to update manifestYifan Hong2018-10-265-0/+197
| | | | | | | | | | | | | | | | | | | | | | | | At build time, it will be filled with BOARD_SUPER_PARTITION_GROUPS, BOARD_*_SIZE, and BOARD_*_PARTITION_LIST. Only the information from the new target_files package is used. The META/dynamic_partition_info.txt from old target_files is ignored. Test: builds and manual OTA Bug: 117182932 Change-Id: I02ce99caaf7d01cec1470f7262c45490c15dfcb7
* | Prefix operation name with partition name.Sen Jiang2018-10-261-10/+9
| | | | | | | | | | | | | | | | | | | | | | | | In the final breakdown we can now easily spot which partition the largest operations come from. Also changed LOG(INFO) to printf to get rid of the timestamp in the report making it difficult to compare. Bug: None Test: check log of generating payload Change-Id: Iec26e864ffc1cd507c9e78733f5cda1f53886db1
* | Only filter old_visited_blocks for inplace update.Sen Jiang2018-10-251-2/+6
| | | | | | | | | | | | | | | | For inplace update, old blocks can only be visited once, but there's no such restrictions for minor version 2 and up. Test: generate a payload Change-Id: I28ddde27e9de8070e5185ad9837bfe2a79d49d27
* | Replace LoadPayloadMetadata() with PayloadMetadata class.Sen Jiang2018-10-223-155/+31
| | | | | | | | | | | | | | Removed duplicated payload header parsing logic. Test: update_engine_unittests Change-Id: I457c5cb86fa16e97b7a0c34d4039c46b86cd2957
* | Support writing FEC.Sen Jiang2018-10-034-3/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After hash tree is written, we re-read the partition to encode FEC, this cannot be done incrementally for each Update() like the hash tree, because the data needed to encode each rs block are spreaded across entire partition, and we can not afford to use that much memory. For each round, we encode block_size number of rs blocks, which will produce block_size * fec_roots bytes of FEC data, this will allow us to read one block at a time instead of one byte. Bug: 28171891 Test: update_engine_unittests Test: brillo_update_payload generate Test: brillo_update_payload verify Change-Id: I35ba7e0647b9ee5a97b972dc480deef60d813676
* | payload_generator: Use actions to apply payload.Sen Jiang2018-09-261-32/+54
| | | | | | | | | | | | | | | | | | Verity data is written by FilesystemVerifierAction(), so we need to use the actions to apply payload correctly. Bug: 28171891 Test: brillo_update_payload verify Change-Id: I106053a6291282538008f7fbbaba0e6d14346770
* | Add unittest for payload_generation_config_android.Sen Jiang2018-09-261-0/+188
| | | | | | | | | | | | Bug: 28171891 Test: update_engine_unittests Change-Id: Id2566cfe4167da90fbb00e0e29700841db206edc
* | payload_generator: Verify hash tree.Sen Jiang2018-09-261-0/+30
| | | | | | | | | | | | | | | | | | | | Generate the hash tree based on the descriptor and verify that it matches with the hash tree stored in the image before enabling verity hash tree configs. Bug: 28171891 Test: generate a payload with verity hash tree Change-Id: I1f2a220d631093607e648afc105ca9b5379cc74a
* | Generate verity config.Sen Jiang2018-09-269-2/+222
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A new minor version 6 kVerityMinorPayloadVersion is added, client having a minor version >= 6 means it support writing verity hash tree and fec given verity config from payload. When generating payload in Android, if the source minor version is >= 6, then we try to find the AVB footer for each partition image and parse the AVB hashtree descriptor, if found then write verity config to the payload, the verification of verity config will be added in a follow up CL. AVB is an Android specific format, so we will skip this for Chrome OS for now, if later we decided to enable verity feature in Chrome OS, we can implement the parsing logic for Chrome OS verity metadata format in payload_generation_config_chromeos.cc Bug: 28171891 Test: brillo_update_payload verify Change-Id: Ic319ea05e9f0095e3a4721df79a6ab516fa8a915
* | Merge remote-tracking branch 'aosp/upstream-master' into aosp/master.Sen Jiang2018-08-135-23/+34
|\| | | | | | | | | | | | | | | | | | | | | | | The following change is reverted because aosp has newer libchrome. 71818c84 Partially Revert 2b9d241 Added stub override for ReportInternalErrorCode(). Fixed RunPosinstallAction typo. Bug: 112326236 Test: update_engine_unittests Change-Id: Ieaae0eef425cbb1278067a48aa19b14ed056317a
| * delta_generator: Use "kernel" and "root" partition names internallyTudor Brindus2018-07-094-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit abandons using "boot" and "system" names internally for major version 1 payloads in favor of "kernel" and "root", which is symmetrical with what's used in major version 2 payloads. BUG=chromium:794404 TEST=no errors running brillo_update_payload with FORCE_MAJOR_VERSION set to either 1 or 2; paycheck.py <generated payload> --check and printing partition names in protobuf confirms correct names are generated Change-Id: I396274812d5586e19a021c05b8b7a415aac976bf Reviewed-on: https://chromium-review.googlesource.com/1119573 Commit-Ready: Tudor Brindus <tbrindus@chromium.org> Tested-by: Tudor Brindus <tbrindus@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org>
| * Merge commit '3538a883700a86c8224bff075e1cc4bab552d2ed' into patch_branchchrome-bot2018-06-058-93/+125
| |\ | | | | | | | | | Change-Id: I48fa7c6e8d3977813623d05353bb3c005d19ba83
| | * Merge remote-tracking branch 'cros/upstream' into cros/masterAmin Hassani2018-06-048-93/+125
| | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | BUG=chromium:815356 TEST=unittest TEST=precq TEST=cros flash Change-Id: I8d9c37411708d0cae61613b285123a090ce6deb1
| * | | update_engine: Rename signing variables.Amin Hassani2018-06-041-9/+16
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rename 'signatures' to 'payload_signatures' so there is a clear distinction between metadata and payload signatures and their input files. BUG=chromium:845614 TEST=payload tryjobs CQ-DEPEND=CL:1069530 Change-Id: Ic2e924871611af1b1f0ee2f1dfd7c6b9f52decc8 Reviewed-on: https://chromium-review.googlesource.com/1069527 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Nicolas Norvez <norvez@chromium.org> Reviewed-by: Xiaochu Liu <xiaochu@chromium.org>
| * | update_engine: puffdiff .zvoice filesAmin Hassani2018-04-021-5/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | .zvoice files in speech_synthesis are large zip files. Start using puffin on them to reduce the payload size whenever they change. BUG=none TEST=unittests TEST=brillo_update_payload Change-Id: Ib8ec31af831e44aa6bec3c024235728bd2d6aec5 Reviewed-on: https://chromium-review.googlesource.com/986736 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Nicolas Norvez <norvez@chromium.org>
* | | Use ScopedTempFile in unit test.Sen Jiang2018-07-095-98/+53
|\ \ \ | | | | | | | | | | | | | | | | | | | | am: 0779a151e5 Change-Id: I883cc93e672208f14bbb9478f188140d47610c83
| * | | Use ScopedTempFile in unit test.Sen Jiang2018-07-035-98/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Replace these 3 lines of code: string path; ASSERT_TRUE(utils::MakeTempFile("name-XXXXXX", &path, nullptr)); ScopedPathUnlinker path_unlinker(path); with one liner: test_utils::ScopedTempFile file("name-XXXXXX"); Bug: None Test: unit test Change-Id: Ic5be7dc8339842270023055bcc3a97e526953f04
* | | | Parse Android boot.img.Sen Jiang2018-07-029-13/+327
|\| | | | | | | | | | | | | | | | | | | | | | | am: 0a582fbf50 Change-Id: I9de124d3a988fbc1bec03079fbf7350cdb0dcba9
| * | | Parse Android boot.img.Sen Jiang2018-06-299-13/+327
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Android boot.img contains kernel, ramdisk and optionally second stage bootloader. Both kernel and ramdisk could be compressed by gzip. This patch add a new boot img filesystem to parse boot.img to split it into |File|s, and find the deflates to utilize puffin, which could save us several MBs in delta payload size. As a side effect, generating delta payload for boot partition is much faster because it is now splitted into smaller files, and we can make use of multithreading. Bug: 110494725 Test: generate a delta payload Change-Id: If41468b37a407fe1b7a70c2b61f632f7df8176d5
* | | | Enable BCJ filters for X86 and ARM ELF binary. am: 5288bd128eSen Jiang2018-06-072-4/+53
|\| | | | | | | | | | | | | | | | | | | | | | | am: 4a3bfdfb9b Change-Id: If1e5c76c941609145c1289b70624f4070d7fc9ed
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-20 11:24:40 +0000