-
Notifications
You must be signed in to change notification settings - Fork 571
[FastDev] Add faster FastDeploy2 #11795
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
simonrozsival
wants to merge
22
commits into
main
Choose a base branch
from
android-fastdeploy2-clean
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+2,216
−1
Open
Changes from all commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
1bb8120
Add manifest-driven FastDeploy2
simonrozsival 16b491d
Fix FastDeploy2 copy fallback from symlinks
simonrozsival 807edd7
Address FastDeploy2 review feedback
simonrozsival 00522fe
Shorten FastDeploy2 symlink shell commands
simonrozsival 149eb9d
Remove unused FastDeploy2 helpers
simonrozsival 6edf60e
Fix FastDeploy2 copy recovery edge cases
simonrozsival 5750867
Address FastDeploy2 review cleanup
simonrozsival ace7857
Simplify FastDeploy2 implementation
simonrozsival d20c46d
Reduce FastDeploy2 binlog noise
simonrozsival 03b083f
Use Android temp directory for FastDeploy2 staging
simonrozsival f0d4842
Track FastDeploy2 device state with manifest hashes
simonrozsival 9a6c077
Merge branch 'main' into android-fastdeploy2-clean
simonrozsival ba74454
Fix FastDeploy2 device-test regressions
simonrozsival adbd2dd
FastDeploy2: preserve subdirectories in symlink full rewrite
simonrozsival 9f06e2f
FastDeploy2: use device-relative path in abi-skip sync marker
simonrozsival 76f85a9
FastDeploy2: fix device manifest-hash read so incremental deploys work
simonrozsival 034bad6
FastDeploy2: drop unused machine-specific LocalPath from manifest
simonrozsival d1aa3bb
FastDeploy2: address review feedback
simonrozsival 9b942c4
Revert accidental submodule bumps
simonrozsival dfb6a86
Potential fix for pull request finding
simonrozsival ccc45ff
[FastDeploy2] Address PR feedback
simonrozsival 809de7c
[FastDev] Batch run-as commands by shell command length; misc cleanup
simonrozsival File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,166 @@ | ||
| # FastDeploy2 | ||
|
|
||
| `FastDeploy2` is the fast-deployment strategy used by `Install` builds (it is the | ||
| default; the legacy strategy is still available as `FastDeploy`). Fast deployment | ||
| keeps the installed `.apk` small and avoids a full re-install on every `F5`: the | ||
| application assemblies (and, optionally, environment files) are pushed to the | ||
| device separately and surfaced to the app through an *override directory*, so an | ||
| inner-loop change only re-transfers the files that actually changed. | ||
|
|
||
| This document describes how the [`FastDeploy2`][task] MSBuild task works: the | ||
| stages it runs, the `adb` commands it issues, and the properties that control it. | ||
|
|
||
| [task]: ../../src/Xamarin.Android.Build.Debugging.Tasks/Tasks/FastDeploy2.cs | ||
|
|
||
| ## MSBuild properties | ||
|
|
||
| The task is invoked from `Xamarin.Android.Common.Debugging.targets`. The | ||
| properties intended for end users are: | ||
|
|
||
| | Property | Default | Description | | ||
| | --- | --- | --- | | ||
| | `$(_AndroidFastDevStrategy)` | `FastDeploy2` | `FastDeploy` or `FastDeploy2`. Set to `FastDeploy` to fall back to the legacy strategy. | | ||
| | `$(_AndroidFastDeployAppFileTransferMode)` | `Symlink` (for `FastDeploy2`) | How staged files are surfaced in the override directory: `Symlink` or `Copy`. | | ||
| | `$(AndroidFastDeploymentAdbCompressionAlgorithm)` | `any` | The `adb push -z` compression algorithm. `FastDeploy2` relies on a modern Android SDK Platform-Tools `adb` for multi-file `push -z` support. | | ||
|
|
||
| The following internal/unsupported properties tune batching. They exist mainly so | ||
| the batching paths can be exercised with smaller batches while testing; their | ||
| defaults match the matching task properties: | ||
|
|
||
| | Property | Default | Description | | ||
| | --- | --- | --- | | ||
| | `$(_AndroidFastDeployStaleFileRemovalBatchSize)` | `100` | Number of stale override files deleted per `rm` invocation. | | ||
| | `$(_AndroidFastDeployCopyBatchSize)` | `25` | Number of files copied per batch when staging fast-deployment files. | | ||
| | `$(_AndroidFastDeployMaxShellCommandLength)` | `900` | Maximum length of a single `adb shell` command line before it is split. | | ||
| | `$(_AndroidFastDeployMaxAdbCommandLength)` | `4096` | Maximum length of a single `adb` command line before it is split. | | ||
|
|
||
| ## On-device layout | ||
|
|
||
| * **Staging directory:** `/data/local/tmp/fastdeploy2/<package-name>/<user-id>`. | ||
| Files are pushed here first (this location is writable by `adb` without | ||
| `run-as`). | ||
| * **Override directory:** `files/.__override__` inside the application's private | ||
| data directory (resolved with `run-as`). The runtime loads assemblies from here | ||
| in preference to the ones embedded in the `.apk`. | ||
| * **Manifest markers:** a `.fastdeploy2-manifest-hash` file is written to both the | ||
| staging and override directories. It records the hash of the last successfully | ||
| deployed manifest so the next build can detect whether the device is already up | ||
| to date and skip redundant work. | ||
|
|
||
| ## Stages | ||
|
|
||
| ### 1. Resolve the device | ||
|
|
||
| The target device is resolved from `$(AdbTarget)` via `AndroidHelper.ParseTarget` | ||
| (which lists devices with `adb devices`). Only the resolved device id is kept; it | ||
| is passed to every subsequent command as `adb -s <id> …`. | ||
|
|
||
| ### 2. Validate device state | ||
|
|
||
| Two system properties are read and the deployment is aborted with a coded error | ||
| if either makes fast deployment unsafe: | ||
|
|
||
| ``` | ||
| adb shell getprop log.redirect-stdio # XA0128 if "true" | ||
| adb shell getprop ro.boot.disable_runas # XA0131 if "true" | ||
| ``` | ||
|
|
||
| ### 3. Inspect the installed app | ||
|
|
||
| `CheckAppInstalledAndDebuggable` discovers the application's private data | ||
| directory and current process id, and detects whether the package is installed, | ||
| debuggable, or a system application. It runs (via `run-as`, falling back to `su` | ||
| for system apps): | ||
|
|
||
| ``` | ||
| adb shell run-as <package> sh -c 'pwd; pidof <package> 2>/dev/null || true' | ||
| ``` | ||
|
|
||
| Depending on the output it may force a re-install (package not debuggable) or | ||
| treat the package as not installed. | ||
|
|
||
| ### 4. (Re)install the `.apk` when needed | ||
|
|
||
| The `.apk` is (re)installed when it is out of date, when `ReInstall` is set, or | ||
| when the app is not yet installed. Installation uses `adb install`: | ||
|
|
||
| ``` | ||
| adb install -r -d [-t] [--user <id>] <path-to-apk> | ||
| ``` | ||
|
|
||
| * `-r` is added when reinstalling, `-d` always allows a version downgrade | ||
| (matching the legacy behavior on API 19+), and `-t` allows test packages. | ||
| * On an `INSTALL_FAILED_ALREADY_EXISTS` failure the package is uninstalled | ||
| preserving data (`pm uninstall -k`) and the install is retried. | ||
| * On an "incompatible/requires uninstall" failure | ||
| (`INSTALL_FAILED_UPDATE_INCOMPATIBLE`, `INSTALL_PARSE_FAILED_INCONSISTENT_CERTIFICATES`, | ||
| `INSTALL_FAILED_VERSION_DOWNGRADE`, …) the package is fully uninstalled and the | ||
| install is retried. | ||
| * Other failures are reported with an `ADB####` error code (for example, | ||
| `ADB0020` for an incompatible ABI or `ADB0060` for insufficient storage). | ||
|
|
||
| If `$(EmbedAssembliesIntoApk)` is `true`, the override directory is removed and | ||
| deployment stops here — there are no separate files to push. | ||
|
|
||
| ### 5. Terminate the running app | ||
|
|
||
| Before swapping files, the app is stopped so it reloads them on next launch: | ||
|
|
||
| ``` | ||
| adb shell pidof <package> # only for system apps; otherwise the pid from stage 3 is used | ||
| adb shell am force-stop <package> | ||
| ``` | ||
|
|
||
| ### 6. Deploy the fast-deployment files | ||
|
|
||
| This is the incremental core (`DeployFastDevFilesWithAdbPush`): | ||
|
|
||
| 1. **Build the current manifest.** Each file to deploy is recorded with its | ||
| size and last-write time; the set of | ||
| `{ relative-path → (size, mtime) }` forms the manifest. A single SHA256 hash | ||
| over the whole manifest is used as the device readiness marker (see below). | ||
| 2. **Compare against the device.** The previous manifest is read from `obj`, and | ||
| the on-device `.fastdeploy2-manifest-hash` markers are read to confirm the | ||
| device still matches it. If the staging directory is not in the expected | ||
| state it is reset: | ||
| ``` | ||
| adb shell rm -rf <staging-dir> | ||
| ``` | ||
| 3. **Create staging directories** for the files being deployed: | ||
| ``` | ||
| adb shell mkdir -p <dir> [<dir> …] # batched up to MaxShellCommandLength | ||
| ``` | ||
| 4. **Remove stale files** that are no longer part of the app: | ||
| ``` | ||
| adb shell rm -f <file> [<file> …] # batched up to StaleFileRemovalBatchSize / MaxAdbCommandLength | ||
| ``` | ||
| 5. **Upload changed files** (only files whose size or last-write time changed), | ||
| grouped by | ||
| directory and batched up to `MaxAdbCommandLength`: | ||
| ``` | ||
| adb push -z <algorithm> <local-file> [<local-file> …] <remote-dir> | ||
| ``` | ||
| 6. **Update the override directory** so it points at the freshly staged files, | ||
| using one of two modes: | ||
| * **`Symlink` (default):** for each directory, symlink the staged files into | ||
| the override directory, leaving subdirectories untouched. Roughly: | ||
| ``` | ||
| adb shell run-as <package> sh -c \ | ||
| 'd=<override-dir>;s=<staging-dir>;mkdir -p "$d"&&cd "$d"&& \ | ||
| for e in ./*;do [ -d "$e" ]||rm -f "$e";done&& \ | ||
| for f in "$s"/*;do [ -d "$f" ]||ln -sf "$f" .;done' | ||
| ``` | ||
| If the device does not support symlinking into the override directory, the | ||
| task automatically falls back to `Copy`. | ||
| * **`Copy`:** the staged files are copied into the override directory instead | ||
| of symlinked. | ||
| 7. **Mark success.** When the override directory is up to date, the current | ||
| manifest hash is written to the staging and override markers, and the manifest | ||
| is saved to `obj` for the next incremental build. | ||
|
|
||
| ## Error codes | ||
|
|
||
| Install failures are reported with `ADB####` codes; fast-deployment shell | ||
| failures (`mkdir`/`rm`/`push`/`ln`) are reported with `XA0129`. `run-as` | ||
| diagnostics map to `XA0131`–`XA0137`. See the | ||
| [build/deploy message docs](../docs-mobile/messages/index.md) for details. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| #### Application build and deployment | ||
|
|
||
| - [GitHub PR #11795](https://github.com/dotnet/android/pull/11795): | ||
| Added a faster `FastDeploy2` fast-deployment strategy, now the default for | ||
| app install fast deployment. Three new MSBuild properties control it: | ||
| - `$(_AndroidFastDevStrategy)`: `FastDeploy` or `FastDeploy2` (default). | ||
| Set to `FastDeploy` to fall back to the legacy strategy. | ||
| - `$(_AndroidFastDeployAppFileTransferMode)`: `Symlink` (default for | ||
| `FastDeploy2`) or `Copy`. | ||
| - `$(AndroidFastDeploymentAdbCompressionAlgorithm)`: the `adb push -z` | ||
| compression algorithm, `any` by default. `FastDeploy2` relies on a modern | ||
| Android SDK Platform-Tools `adb` for multi-file `push -z` support. | ||
|
|
||
| See the [FastDeploy2 guide](../guides/FastDeploy2.md) for a detailed | ||
| description of the deployment stages and the `adb` commands they run. | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.