diff --git a/Documentation/guides/FastDeploy2.md b/Documentation/guides/FastDeploy2.md new file mode 100644 index 00000000000..ba80942e125 --- /dev/null +++ b/Documentation/guides/FastDeploy2.md @@ -0,0 +1,166 @@ +# FastDeploy2 + +`FastDeploy2` is the fast-deployment strategy used by `Install` builds (it is the +default; the legacy strategy is still available as `FastDeploy`). Fast deployment +keeps the installed `.apk` small and avoids a full re-install on every `F5`: the +application assemblies (and, optionally, environment files) are pushed to the +device separately and surfaced to the app through an *override directory*, so an +inner-loop change only re-transfers the files that actually changed. + +This document describes how the [`FastDeploy2`][task] MSBuild task works: the +stages it runs, the `adb` commands it issues, and the properties that control it. + +[task]: ../../src/Xamarin.Android.Build.Debugging.Tasks/Tasks/FastDeploy2.cs + +## MSBuild properties + +The task is invoked from `Xamarin.Android.Common.Debugging.targets`. The +properties intended for end users are: + +| Property | Default | Description | +| --- | --- | --- | +| `$(_AndroidFastDevStrategy)` | `FastDeploy2` | `FastDeploy` or `FastDeploy2`. Set to `FastDeploy` to fall back to the legacy strategy. | +| `$(_AndroidFastDeployAppFileTransferMode)` | `Symlink` (for `FastDeploy2`) | How staged files are surfaced in the override directory: `Symlink` or `Copy`. | +| `$(AndroidFastDeploymentAdbCompressionAlgorithm)` | `any` | The `adb push -z` compression algorithm. `FastDeploy2` relies on a modern Android SDK Platform-Tools `adb` for multi-file `push -z` support. | + +The following internal/unsupported properties tune batching. They exist mainly so +the batching paths can be exercised with smaller batches while testing; their +defaults match the matching task properties: + +| Property | Default | Description | +| --- | --- | --- | +| `$(_AndroidFastDeployStaleFileRemovalBatchSize)` | `100` | Number of stale override files deleted per `rm` invocation. | +| `$(_AndroidFastDeployCopyBatchSize)` | `25` | Number of files copied per batch when staging fast-deployment files. | +| `$(_AndroidFastDeployMaxShellCommandLength)` | `900` | Maximum length of a single `adb shell` command line before it is split. | +| `$(_AndroidFastDeployMaxAdbCommandLength)` | `4096` | Maximum length of a single `adb` command line before it is split. | + +## On-device layout + +* **Staging directory:** `/data/local/tmp/fastdeploy2//`. + Files are pushed here first (this location is writable by `adb` without + `run-as`). +* **Override directory:** `files/.__override__` inside the application's private + data directory (resolved with `run-as`). The runtime loads assemblies from here + in preference to the ones embedded in the `.apk`. +* **Manifest markers:** a `.fastdeploy2-manifest-hash` file is written to both the + staging and override directories. It records the hash of the last successfully + deployed manifest so the next build can detect whether the device is already up + to date and skip redundant work. + +## Stages + +### 1. Resolve the device + +The target device is resolved from `$(AdbTarget)` via `AndroidHelper.ParseTarget` +(which lists devices with `adb devices`). Only the resolved device id is kept; it +is passed to every subsequent command as `adb -s …`. + +### 2. Validate device state + +Two system properties are read and the deployment is aborted with a coded error +if either makes fast deployment unsafe: + +``` +adb shell getprop log.redirect-stdio # XA0128 if "true" +adb shell getprop ro.boot.disable_runas # XA0131 if "true" +``` + +### 3. Inspect the installed app + +`CheckAppInstalledAndDebuggable` discovers the application's private data +directory and current process id, and detects whether the package is installed, +debuggable, or a system application. It runs (via `run-as`, falling back to `su` +for system apps): + +``` +adb shell run-as sh -c 'pwd; pidof 2>/dev/null || true' +``` + +Depending on the output it may force a re-install (package not debuggable) or +treat the package as not installed. + +### 4. (Re)install the `.apk` when needed + +The `.apk` is (re)installed when it is out of date, when `ReInstall` is set, or +when the app is not yet installed. Installation uses `adb install`: + +``` +adb install -r -d [-t] [--user ] +``` + +* `-r` is added when reinstalling, `-d` always allows a version downgrade + (matching the legacy behavior on API 19+), and `-t` allows test packages. +* On an `INSTALL_FAILED_ALREADY_EXISTS` failure the package is uninstalled + preserving data (`pm uninstall -k`) and the install is retried. +* On an "incompatible/requires uninstall" failure + (`INSTALL_FAILED_UPDATE_INCOMPATIBLE`, `INSTALL_PARSE_FAILED_INCONSISTENT_CERTIFICATES`, + `INSTALL_FAILED_VERSION_DOWNGRADE`, …) the package is fully uninstalled and the + install is retried. +* Other failures are reported with an `ADB####` error code (for example, + `ADB0020` for an incompatible ABI or `ADB0060` for insufficient storage). + +If `$(EmbedAssembliesIntoApk)` is `true`, the override directory is removed and +deployment stops here — there are no separate files to push. + +### 5. Terminate the running app + +Before swapping files, the app is stopped so it reloads them on next launch: + +``` +adb shell pidof # only for system apps; otherwise the pid from stage 3 is used +adb shell am force-stop +``` + +### 6. Deploy the fast-deployment files + +This is the incremental core (`DeployFastDevFilesWithAdbPush`): + +1. **Build the current manifest.** Each file to deploy is recorded with its + size and last-write time; the set of + `{ relative-path → (size, mtime) }` forms the manifest. A single SHA256 hash + over the whole manifest is used as the device readiness marker (see below). +2. **Compare against the device.** The previous manifest is read from `obj`, and + the on-device `.fastdeploy2-manifest-hash` markers are read to confirm the + device still matches it. If the staging directory is not in the expected + state it is reset: + ``` + adb shell rm -rf + ``` +3. **Create staging directories** for the files being deployed: + ``` + adb shell mkdir -p [ …] # batched up to MaxShellCommandLength + ``` +4. **Remove stale files** that are no longer part of the app: + ``` + adb shell rm -f [ …] # batched up to StaleFileRemovalBatchSize / MaxAdbCommandLength + ``` +5. **Upload changed files** (only files whose size or last-write time changed), + grouped by + directory and batched up to `MaxAdbCommandLength`: + ``` + adb push -z [ …] + ``` +6. **Update the override directory** so it points at the freshly staged files, + using one of two modes: + * **`Symlink` (default):** for each directory, symlink the staged files into + the override directory, leaving subdirectories untouched. Roughly: + ``` + adb shell run-as sh -c \ + 'd=;s=;mkdir -p "$d"&&cd "$d"&& \ + for e in ./*;do [ -d "$e" ]||rm -f "$e";done&& \ + for f in "$s"/*;do [ -d "$f" ]||ln -sf "$f" .;done' + ``` + If the device does not support symlinking into the override directory, the + task automatically falls back to `Copy`. + * **`Copy`:** the staged files are copied into the override directory instead + of symlinked. +7. **Mark success.** When the override directory is up to date, the current + manifest hash is written to the staging and override markers, and the manifest + is saved to `obj` for the next incremental build. + +## Error codes + +Install failures are reported with `ADB####` codes; fast-deployment shell +failures (`mkdir`/`rm`/`push`/`ln`) are reported with `XA0129`. `run-as` +diagnostics map to `XA0131`–`XA0137`. See the +[build/deploy message docs](../docs-mobile/messages/index.md) for details. diff --git a/Documentation/release-notes/11795.md b/Documentation/release-notes/11795.md new file mode 100644 index 00000000000..b61f2e06637 --- /dev/null +++ b/Documentation/release-notes/11795.md @@ -0,0 +1,15 @@ +#### Application build and deployment + +- [GitHub PR #11795](https://github.com/dotnet/android/pull/11795): + Added a faster `FastDeploy2` fast-deployment strategy, now the default for + app install fast deployment. Three new MSBuild properties control it: + - `$(_AndroidFastDevStrategy)`: `FastDeploy` or `FastDeploy2` (default). + Set to `FastDeploy` to fall back to the legacy strategy. + - `$(_AndroidFastDeployAppFileTransferMode)`: `Symlink` (default for + `FastDeploy2`) or `Copy`. + - `$(AndroidFastDeploymentAdbCompressionAlgorithm)`: the `adb push -z` + compression algorithm, `any` by default. `FastDeploy2` relies on a modern + Android SDK Platform-Tools `adb` for multi-file `push -z` support. + + See the [FastDeploy2 guide](../guides/FastDeploy2.md) for a detailed + description of the deployment stages and the `adb` commands they run. diff --git a/src/Xamarin.Android.Build.Debugging.Tasks/Tasks/FastDeploy2.Adb.cs b/src/Xamarin.Android.Build.Debugging.Tasks/Tasks/FastDeploy2.Adb.cs new file mode 100644 index 00000000000..338ab6cf324 --- /dev/null +++ b/src/Xamarin.Android.Build.Debugging.Tasks/Tasks/FastDeploy2.Adb.cs @@ -0,0 +1,218 @@ +using System; +using System.Collections.Generic; +using System.Linq; +using System.Text; +using System.Threading.Tasks; + +namespace Xamarin.Android.Tasks +{ + public partial class FastDeploy2 + { + // The high-level device operations below are implemented directly on top of `adb` + // (via RunAdbCommand / RunAdbShellCommand) so that FastDeploy2 does not depend on the + // legacy Mono.AndroidTools / Xamarin.AndroidTools assemblies. + + /// + /// Reads a single system property via adb shell getprop <name>. + /// Returns a trimmed value, or an empty string when the property is unset. + /// + async Task GetDeviceProperty (string name) + { + var result = await RunAdbShellCommand ("getprop", name); + return result.Output?.Trim () ?? ""; + } + + /// + /// Returns the process id of via adb shell pidof, + /// or 0 when the process is not running. + /// + async Task GetProcessId (string packageName) + { + var result = await RunAdbShellCommand ("pidof", packageName); + string output = result.Output?.Trim () ?? ""; + // `pidof` can return multiple, space-separated pids; take the first one. + int space = output.IndexOf (' '); + if (space >= 0) { + output = output.Substring (0, space); + } + return int.TryParse (output, out int pid) ? pid : 0; + } + + /// + /// Force-stops via adb shell am force-stop. + /// + async Task ForceStopPackage (string packageName) + { + await RunAdbShellCommand ("am", "force-stop", packageName); + } + + /// + /// Uninstalls via adb shell pm uninstall, optionally + /// preserving the application's data and cache directories (-k). + /// + async Task UninstallPackage (string packageName, bool preserveData, string user) + { + var args = new List { "pm", "uninstall" }; + if (preserveData) { + args.Add ("-k"); + } + if (!string.IsNullOrEmpty (user)) { + args.Add ("--user"); + args.Add (user); + } + args.Add (packageName); + await RunLoggedDeviceOperation ($"UninstallPackage {string.Join (" ", args)}", () => RunAdbShellCommand (args.ToArray ())); + } + + /// + /// Installs an APK via adb install. On an "already exists" or "requires uninstall" + /// failure the package is uninstalled (optionally preserving data) and the install is + /// retried once, mirroring the legacy behavior. Any other failure throws a + /// carrying the matching ADB#### error code. + /// + async Task InstallApkWithRetry (string apkFile, bool reinstall, bool testOnly, string user) + { + var result = await RunInstallCommand (apkFile, reinstall, testOnly, user); + var kind = ClassifyInstallResult (result); + if (kind == InstallResultKind.Success) { + return; + } + + if (kind == InstallResultKind.AlreadyExists || kind == InstallResultKind.RequiresUninstall) { + bool preserveData = kind == InstallResultKind.AlreadyExists; + LogDebugMessage ($"Package '{PackageName}' could not be installed directly ({kind}). Uninstalling (preserving data: {preserveData}) and retrying."); + await UninstallPackage (PackageName, preserveData: preserveData, user: user); + result = await RunInstallCommand (apkFile, reinstall: true, testOnly: testOnly, user: user); + kind = ClassifyInstallResult (result); + if (kind == InstallResultKind.Success) { + return; + } + } + + throw new FastDeployInstallException (GetInstallErrorCode (kind), result.Output); + } + + async Task RunInstallCommand (string apkFile, bool reinstall, bool testOnly, string user) + { + var args = new List { "install" }; + if (reinstall) { + args.Add ("-r"); + } + // Allow downgrade: matches the legacy AllowDowngrade flag, which was always set on + // API 19+ (every device supported by .NET for Android). + args.Add ("-d"); + if (testOnly) { + args.Add ("-t"); + } + if (!string.IsNullOrEmpty (user)) { + args.Add ("--user"); + args.Add (user); + } + args.Add (GetFullPath (apkFile)); + var operation = $"Install ApkFile={apkFile}, PackageName={PackageName}, ReInstall={reinstall}, User={user ?? ""}, TestOnly={testOnly}"; + return await RunLoggedDeviceOperation (operation, () => RunAdbCommand (args.ToArray ())); + } + + enum InstallResultKind + { + Success, + AlreadyExists, + RequiresUninstall, + IncompatibleCpuAbi, + SdkNotSupported, + InsufficientSpace, + Failed, + } + + /// + /// Classifies adb install output, mirroring the failure categories that the legacy + /// install path raised as typed exceptions. + /// + static InstallResultKind ClassifyInstallResult (AdbCommandResult result) + { + string output = result.Output ?? ""; + + // `adb install` prints "Success" on success; it can also be empty on success. + if (output.IndexOf ("Success", StringComparison.Ordinal) >= 0) { + return InstallResultKind.Success; + } + + // NOTE: match without the trailing ']', since adb may print either + // [INSTALL_FAILED_NO_MATCHING_ABIS] or [INSTALL_FAILED_NO_MATCHING_ABIS: ...]. + if (output.Contains ("[INSTALL_FAILED_INSUFFICIENT_STORAGE") || output.Contains ("[INSTALL_FAILED_MEDIA_UNAVAILABLE")) { + return InstallResultKind.InsufficientSpace; + } + if (output.Contains ("[INSTALL_FAILED_ALREADY_EXISTS")) { + return InstallResultKind.AlreadyExists; + } + if (output.Contains ("[INSTALL_FAILED_OLDER_SDK")) { + return InstallResultKind.SdkNotSupported; + } + if (output.Contains ("[INSTALL_PARSE_FAILED_INCONSISTENT_CERTIFICATES") || + output.Contains ("doesn't support runtime permissions") || + output.Contains ("[INSTALL_FAILED_UPDATE_INCOMPATIBLE") || + output.Contains ("[INSTALL_FAILED_VERSION_DOWNGRADE")) { + return InstallResultKind.RequiresUninstall; + } + if (output.Contains ("[INSTALL_FAILED_CPU_ABI_INCOMPATIBLE") || output.Contains ("[INSTALL_FAILED_NO_MATCHING_ABIS")) { + return InstallResultKind.IncompatibleCpuAbi; + } + + if (result.ExitCode != 0 || + output.IndexOf ("Failure", StringComparison.OrdinalIgnoreCase) >= 0 || + output.IndexOf ("INSTALL_FAILED", StringComparison.OrdinalIgnoreCase) >= 0 || + output.IndexOf ("adb: failed", StringComparison.OrdinalIgnoreCase) >= 0) { + return InstallResultKind.Failed; + } + + // Exit code 0 with no recognizable failure marker: treat as success. + return InstallResultKind.Success; + } + + static string GetInstallErrorCode (InstallResultKind kind) + { + return kind switch { + InstallResultKind.IncompatibleCpuAbi => "ADB0020", + InstallResultKind.RequiresUninstall => "ADB0030", + InstallResultKind.SdkNotSupported => "ADB0040", + InstallResultKind.AlreadyExists => "ADB0050", + InstallResultKind.InsufficientSpace => "ADB0060", + _ => "ADB0010", + }; + } + + /// Quotes an argument for . + static string QuoteProcessArgument (string argument) + { + if (argument == null) { + return "\"\""; + } + var sb = new StringBuilder (); + sb.Append ('"'); + // The .NET process class only supports quoted arguments with escaped quotes/backslashes. + foreach (char c in argument) { + if (c == '"' || c == '\\') { + sb.Append ('\\'); + } + sb.Append (c); + } + sb.Append ('"'); + return sb.ToString (); + } + } + + /// + /// Thrown when adb install fails for a reason FastDeploy2 cannot recover from. The + /// is the ADB#### code reported to MSBuild. + /// + class FastDeployInstallException : Exception + { + public string ErrorCode { get; } + + public FastDeployInstallException (string errorCode, string message) + : base (message) + { + ErrorCode = errorCode; + } + } +} diff --git a/src/Xamarin.Android.Build.Debugging.Tasks/Tasks/FastDeploy2.Manifest.cs b/src/Xamarin.Android.Build.Debugging.Tasks/Tasks/FastDeploy2.Manifest.cs new file mode 100644 index 00000000000..63f46534887 --- /dev/null +++ b/src/Xamarin.Android.Build.Debugging.Tasks/Tasks/FastDeploy2.Manifest.cs @@ -0,0 +1,538 @@ +using System; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Security.Cryptography; +using System.Text; +using System.Text.Json; +using System.Text.Json.Serialization; +using System.Threading.Tasks; + +namespace Xamarin.Android.Tasks +{ + public partial class FastDeploy2 + { + const string RemoteStagingRootPath = "/data/local/tmp/fastdeploy2"; + const string ManifestHashMarker = ".fastdeploy2-manifest-hash"; + + string RemoteStagingRoot => RemoteStagingRootPath; + + async Task DeployFastDevFilesWithAdbPush (string overridePath) + { + var files = PrepareDirectPushFiles (); + var expectedFiles = new HashSet (files.Select (file => file.RelativePath), StringComparer.Ordinal); + var currentManifest = CreateManifest (files); + if (files.Count == 0) { + LogDiagnostic ("No FastDev files were prepared for adb push deployment."); + return true; + } + + string remoteStagingPath = GetRemoteAdbPushStagingPath (); + var previousManifest = LoadPreviousManifest (); + string previousManifestHash = previousManifest == null ? "" : ComputeManifestHash (previousManifest); + var deviceManifestState = previousManifest == null ? new DeviceManifestState () : await GetDeviceManifestState (remoteStagingPath, overridePath); + bool remoteReady = previousManifest != null && string.Equals (deviceManifestState.RemoteHash, previousManifestHash, StringComparison.Ordinal); + bool overrideSymlinksReady = previousManifest != null && string.Equals (deviceManifestState.OverrideHash, previousManifestHash, StringComparison.Ordinal); + if (!remoteReady) { + previousManifest = null; + overrideSymlinksReady = false; + if (!await ResetRemoteStagingDirectory (remoteStagingPath)) { + return false; + } + } + + var changedFiles = GetChangedFiles (currentManifest, previousManifest); + var removedFiles = GetRemovedFiles (currentManifest, previousManifest); + LogDiagnostic ($"FastDeploy2 manifest changed files: {changedFiles.Count}; removed files: {removedFiles.Count}."); + + foreach (var file in files) { + if (changedFiles.Contains (file.RelativePath)) { + LogDebugMessage ($"NotifySync CopyFile {file.RelativePath}."); + } else { + LogDebugMessage ($"NotifySync SkipCopyFile {file.RelativePath} file is up to date."); + } + } + + string output = await CreateRemoteStagingDirectories (remoteStagingPath, expectedFiles); + if (!string.IsNullOrEmpty (output) && IsShellError (output, "mkdir")) { + LogFastDeploy2Error ("XA0129", output, remoteStagingPath); + return false; + } + + if (!await RemoveRemoteStaleFiles (remoteStagingPath, removedFiles)) { + return false; + } + + if (!await UploadChangedFiles (remoteStagingPath, files, changedFiles)) { + return false; + } + + bool result; + if (UseShellSymlinkAppFileTransfer ()) { + result = await UpdateOverrideShellSymlinks (remoteStagingPath, overridePath, currentManifest, previousManifest, overrideSymlinksReady, removedFiles); + } else { + result = await UpdateOverrideCopies (remoteStagingPath, overridePath); + } + + if (result) { + string currentManifestHash = ComputeManifestHash (currentManifest); + if (!await MarkRemoteManifest (remoteStagingPath, currentManifestHash)) { + return false; + } + if (UseShellSymlinkAppFileTransfer () && !await MarkOverrideManifest (overridePath, currentManifestHash)) { + return false; + } + WriteManifest (currentManifest); + } + return result; + } + + bool UseShellSymlinkAppFileTransfer () + { + return string.Equals (AppFileTransferMode, "Symlink", StringComparison.OrdinalIgnoreCase); + } + + async Task UpdateOverrideShellSymlinks (string remoteStagingPath, string overridePath, ManifestData currentManifest, ManifestData previousManifest, bool overrideSymlinksReady, List removedFiles) + { + var previousSymlinkManifest = overrideSymlinksReady ? previousManifest : null; + var newFiles = previousSymlinkManifest == null ? + new HashSet (currentManifest.Files.Keys, StringComparer.Ordinal) : + new HashSet (currentManifest.Files.Keys.Where (file => !previousSymlinkManifest.Files.ContainsKey (file)), StringComparer.Ordinal); + LogDiagnostic ($"FastDeploy2 symlink update new files: {newFiles.Count}; removed files: {removedFiles.Count}."); + + if (!await RunCombinedShellSymlinkUpdate (remoteStagingPath, overridePath, currentManifest, previousSymlinkManifest, newFiles, removedFiles)) { + return await FallbackToCopy (remoteStagingPath, overridePath); + } + + return true; + } + + async Task RunCombinedShellSymlinkUpdate (string remoteStagingPath, string overridePath, ManifestData currentManifest, ManifestData previousManifest, HashSet newFiles, List removedFiles) + { + var currentByDirectory = GroupFilesByDirectory (currentManifest.Files.Keys); + var newByDirectory = GroupFilesByDirectory (newFiles); + var removedByDirectory = GroupFilesByDirectory (removedFiles); + var directories = new HashSet (currentByDirectory.Keys, StringComparer.Ordinal); + directories.UnionWith (removedByDirectory.Keys); + + foreach (string directory in directories) { + var currentInDirectory = GetFilesInDirectory (currentByDirectory, directory); + var newInDirectory = GetFilesInDirectory (newByDirectory, directory); + var removedInDirectory = GetFilesInDirectory (removedByDirectory, directory); + string targetDirectory = CombineRemotePath (overridePath, directory); + string sourceDirectory = CombineRemotePath (remoteStagingPath, directory); + + if (currentInDirectory.Count > 0 && (previousManifest == null || newInDirectory.Count == currentInDirectory.Count)) { + // Clear and symlink only the files that live directly in this directory, never + // the subdirectories. A plain `rm -rf ./*` + `ln -sf "$s"/* .` would (a) delete + // child directories that other iterations populate and (b) create symlinks to + // staging subdirectories; processing those children would then follow the symlink + // back into the shell-owned staging area and fail with "Permission denied" under + // run-as. Each subdirectory is handled by its own iteration instead. + string script = $"d={QuoteShellArgument (targetDirectory)};s={QuoteShellArgument (sourceDirectory)};mkdir -p \"$d\"&&cd \"$d\"&&for e in ./*;do [ -d \"$e\" ]||rm -f \"$e\";done&&for f in \"$s\"/*;do [ -d \"$f\" ]||ln -sf \"$f\" .;done"; + string output = await RunAsShell (script); + if (RaiseRunAsError (output) || IsShellError (output, "rm") || IsShellError (output, "mkdir") || IsShellError (output, "ln")) { + LogDiagnostic ($"Shell symlink glob update failed with '{output}'."); + return false; + } + continue; + } + + foreach (string script in CreateShellSymlinkScripts (remoteStagingPath, overridePath, newInDirectory, removedInDirectory)) { + string output = await RunAsShell (script); + if (RaiseRunAsError (output) || IsShellError (output, "rm") || IsShellError (output, "mkdir") || IsShellError (output, "ln")) { + LogDiagnostic ($"Shell symlink batch update failed with '{output}'."); + return false; + } + } + } + + return true; + } + + static List GetFilesInDirectory (Dictionary> filesByDirectory, string directory) + { + return filesByDirectory.TryGetValue (directory, out List files) ? files : []; + } + + IEnumerable CreateShellSymlinkScripts (string remoteStagingPath, string overridePath, List newFiles, List removedFiles) + { + foreach (var group in removedFiles.Concat (newFiles).GroupBy (GetDirectoryName, StringComparer.Ordinal)) { + string targetDirectory = CombineRemotePath (overridePath, group.Key); + var prefix = $"d={QuoteShellArgument (targetDirectory)};mkdir -p \"$d\"&&cd \"$d\"&&rm -f"; + foreach (var batch in BatchShellWords (prefix, group.Select (file => QuoteShellArgument (Path.GetFileName (file))))) { + yield return batch; + } + } + + foreach (var group in newFiles.GroupBy (GetDirectoryName, StringComparer.Ordinal)) { + string targetDirectory = CombineRemotePath (overridePath, group.Key); + string sourceDirectory = CombineRemotePath (remoteStagingPath, group.Key); + var prefix = $"d={QuoteShellArgument (targetDirectory)};s={QuoteShellArgument (sourceDirectory)};mkdir -p \"$d\"&&cd \"$d\"&&ln -sf"; + var sources = group.Select (file => "\"$s\"/" + QuoteShellArgument (Path.GetFileName (file))); + foreach (var batch in BatchShellWords (prefix, sources, " .")) { + yield return batch; + } + } + } + + IEnumerable BatchShellWords (string prefix, IEnumerable words, string suffix = "") + { + var builder = new StringBuilder (prefix); + int count = 0; + foreach (string word in words) { + string argument = " " + word; + if (count > 0 && builder.Length + argument.Length + suffix.Length >= MaxAdbCommandLength) { + if (!string.IsNullOrEmpty (suffix)) { + builder.Append (suffix); + } + yield return builder.ToString (); + builder.Clear (); + builder.Append (prefix); + count = 0; + } + builder.Append (argument); + count++; + } + if (count > 0) { + if (!string.IsNullOrEmpty (suffix)) { + builder.Append (suffix); + } + yield return builder.ToString (); + } + } + + async Task FallbackToCopy (string remoteStagingPath, string overridePath) + { + LogDiagnostic ("FastDeploy2 symlink update failed; falling back to copy mode."); + return await UpdateOverrideCopies (remoteStagingPath, overridePath, clearOverrideDirectory: true); + } + + async Task UpdateOverrideCopies (string remoteStagingPath, string overridePath, bool clearOverrideDirectory = false) + { + if (clearOverrideDirectory) { + if (!await ClearOverrideDirectory (overridePath)) { + return false; + } + } else if (!await ClearOverrideSymlinkState (overridePath)) { + return false; + } + + var stagedFileData = await GetRemoteFileData (remoteStagingPath, runAs: false); + if (stagedFileData == null) { + return false; + } + stagedFileData.Remove (ManifestHashMarker); + + var overrideFileData = await GetRemoteFileData (overridePath, runAs: true); + if (overrideFileData == null) { + return false; + } + + if (!await RemoveStaleOverrideFiles (overridePath, stagedFileData, overrideFileData)) { + return false; + } + + return await CopyChangedFiles (remoteStagingPath, overridePath, stagedFileData, overrideFileData); + } + + ManifestData CreateManifest (List files) + { + var manifest = new ManifestData { + DeviceId = GetDeviceId (), + PackageName = PackageName, + UserId = GetUserId (), + PrimaryCpuAbi = PrimaryCpuAbi, + Files = new Dictionary (StringComparer.Ordinal), + }; + foreach (var file in files) { + var info = new FileInfo (file.LocalPath); + manifest.Files [file.RelativePath] = new ManifestEntry { + RelativePath = file.RelativePath, + Size = info.Length, + LastWriteTimeUtcTicks = info.LastWriteTimeUtc.Ticks, + }; + } + return manifest; + } + + HashSet GetChangedFiles (ManifestData currentManifest, ManifestData previousManifest) + { + if (previousManifest == null) { + return new HashSet (currentManifest.Files.Keys, StringComparer.Ordinal); + } + + var changedFiles = new HashSet (StringComparer.Ordinal); + foreach (var entry in currentManifest.Files) { + if (!previousManifest.Files.TryGetValue (entry.Key, out ManifestEntry previous) || + previous.Size != entry.Value.Size || + previous.LastWriteTimeUtcTicks != entry.Value.LastWriteTimeUtcTicks) { + changedFiles.Add (entry.Key); + } + } + return changedFiles; + } + + List GetRemovedFiles (ManifestData currentManifest, ManifestData previousManifest) + { + var removedFiles = new List (); + if (previousManifest == null) { + return removedFiles; + } + + foreach (var entry in previousManifest.Files.Keys) { + if (!currentManifest.Files.ContainsKey (entry)) { + removedFiles.Add (entry); + } + } + return removedFiles; + } + + async Task UploadChangedFiles (string remoteStagingPath, List files, HashSet changedFiles) + { + var changedFileList = files.Where (file => changedFiles.Contains (file.RelativePath)).ToList (); + foreach (var group in changedFileList.GroupBy (file => GetDirectoryName (file.RelativePath), StringComparer.Ordinal)) { + string remoteDirectory = CombineRemotePath (remoteStagingPath, group.Key); + foreach (var batch in BatchPushFilesWithoutSync (group.ToList (), remoteDirectory)) { + var result = await RunAdbCommand (batch.ToArray ()); + if (result.ExitCode != 0) { + LogFastDeploy2Error ("XA0129", result.Output, remoteDirectory); + return false; + } + } + } + return true; + } + + async Task RemoveRemoteStaleFiles (string remoteStagingPath, List removedFiles) + { + foreach (var batch in BatchArguments ("rm", "-f", removedFiles.Select (file => CombineRemotePath (remoteStagingPath, file)))) { + var args = new [] { "shell" }.Concat (batch).ToArray (); + var result = await RunAdbCommand (args); + if (result.ExitCode != 0 || IsShellError (result.Output, "rm")) { + LogFastDeploy2Error ("XA0129", result.Output, remoteStagingPath); + return false; + } + } + return true; + } + + async Task ResetRemoteStagingDirectory (string remoteStagingPath) + { + var result = await RunAdbCommand ("shell", "rm", "-rf", remoteStagingPath); + if (result.ExitCode != 0 || IsShellError (result.Output, "rm")) { + LogFastDeploy2Error ("XA0129", result.Output, remoteStagingPath); + return false; + } + return true; + } + + IEnumerable> BatchPushFilesWithoutSync (List files, string remoteDirectory) + { + var batch = CreatePushArgsPrefix (); + int prefixCount = batch.Count; + int length = EstimateCommandLength (batch) + remoteDirectory.Length + 4; + foreach (var file in files) { + if (Path.GetFileName (file.LocalPath) != Path.GetFileName (file.RelativePath)) { + yield return CreatePushArgs (file.LocalPath, CombineRemotePath (remoteDirectory, Path.GetFileName (file.RelativePath))); + continue; + } + + int itemLength = file.LocalPath.Length + 3; + if (batch.Count > prefixCount && length + itemLength >= MaxAdbCommandLength) { + batch.Add (remoteDirectory); + yield return batch; + batch = CreatePushArgsPrefix (); + length = EstimateCommandLength (batch) + remoteDirectory.Length + 4; + } + batch.Add (file.LocalPath); + length += itemLength; + } + if (batch.Count > prefixCount) { + batch.Add (remoteDirectory); + yield return batch; + } + } + + async Task GetDeviceManifestState (string remoteStagingPath, string overridePath) + { + string remoteMarkerPath = CombineRemotePath (remoteStagingPath, ManifestHashMarker); + string overrideMarkerPath = CombineRemotePath (overridePath, ManifestHashMarker); + string runAsCommand = string.Join (" ", BuildRunAsArgs ().Concat (new [] { + "sh", + "-c", + $"cat {QuoteShellArgument (overrideMarkerPath)} 2>/dev/null || true" + }).Select (QuoteShellArgument)); + // Use `echo` (which emits a real newline) with command substitution rather than + // `printf '\n'`: the backslash escape does not reliably survive the adb/shell quoting + // layers and can arrive at the device as a literal "\n", which merges both values onto + // a single line so ParseDeviceManifestState can never read the override hash and the + // readiness check always fails (forcing a full redeploy on every incremental install). + string script = $"echo \"remote=$(cat {QuoteShellArgument (remoteMarkerPath)} 2>/dev/null)\"; echo \"override=$({runAsCommand} 2>/dev/null)\""; + var result = await RunAdbShellCommand (script); + return ParseDeviceManifestState (result.Output); + } + + async Task ClearOverrideSymlinkState (string overridePath) + { + string markerPath = CombineRemotePath (overridePath, ManifestHashMarker); + string output = await RunAsShell ($"if test -f {QuoteShellArgument (markerPath)}; then rm -rf {QuoteShellArgument (overridePath)}; else rm -f {QuoteShellArgument (markerPath)}; fi"); + if (RaiseRunAsError (output) || IsShellError (output, "rm")) { + LogFastDeploy2Error ("XA0129", output, overridePath); + return false; + } + return true; + } + + async Task ClearOverrideDirectory (string overridePath) + { + string output = await RunAs ("rm", "-rf", overridePath); + if (RaiseRunAsError (output) || IsShellError (output, "rm")) { + LogFastDeploy2Error ("XA0129", output, overridePath); + return false; + } + return true; + } + + async Task MarkRemoteManifest (string remoteStagingPath, string manifestHash) + { + string markerPath = CombineRemotePath (remoteStagingPath, ManifestHashMarker); + var result = await RunAdbShellCommand ($"printf %s {QuoteShellArgument (manifestHash)} > {QuoteShellArgument (markerPath)}"); + if (result.ExitCode != 0 || IsShellError (result.Output, "printf")) { + LogFastDeploy2Error ("XA0129", result.Output, markerPath); + return false; + } + return true; + } + + async Task MarkOverrideManifest (string overridePath, string manifestHash) + { + string output = await RunAsShell ($"mkdir -p {QuoteShellArgument (overridePath)}; printf %s {QuoteShellArgument (manifestHash)} > {QuoteShellArgument (CombineRemotePath (overridePath, ManifestHashMarker))}"); + if (RaiseRunAsError (output) || IsShellError (output, "mkdir") || IsShellError (output, "printf")) { + LogFastDeploy2Error ("XA0129", output, overridePath); + return false; + } + return true; + } + + static DeviceManifestState ParseDeviceManifestState (string output) + { + var state = new DeviceManifestState (); + foreach (string line in output.Split (new char [] { '\r', '\n' }, StringSplitOptions.RemoveEmptyEntries)) { + if (line.StartsWith ("remote=", StringComparison.Ordinal)) { + state.RemoteHash = line.Substring ("remote=".Length).Trim (); + } else if (line.StartsWith ("override=", StringComparison.Ordinal)) { + state.OverrideHash = line.Substring ("override=".Length).Trim (); + } + } + return state; + } + + ManifestData LoadPreviousManifest () + { + string manifestFile = GetManifestFilePath (); + if (!File.Exists (manifestFile)) { + return null; + } + + try { + var manifest = JsonSerializer.Deserialize (File.ReadAllText (manifestFile), typeof (ManifestData), FastDeploy2JsonSerializerContext.Default) as ManifestData; + return IsManifestForCurrentTarget (manifest) ? manifest : null; + } catch (Exception ex) { + LogDiagnostic ($"Ignoring FastDeploy2 manifest '{manifestFile}'. {ex}"); + return null; + } + } + + void WriteManifest (ManifestData manifest) + { + string manifestFile = GetManifestFilePath (); + Directory.CreateDirectory (Path.GetDirectoryName (manifestFile)); + File.WriteAllText (manifestFile, JsonSerializer.Serialize (manifest, typeof (ManifestData), FastDeploy2JsonSerializerContext.Default)); + } + + bool IsManifestForCurrentTarget (ManifestData manifest) + { + return manifest != null && + string.Equals (manifest.DeviceId, GetDeviceId (), StringComparison.Ordinal) && + string.Equals (manifest.PackageName, PackageName, StringComparison.Ordinal) && + string.Equals (manifest.UserId, GetUserId (), StringComparison.Ordinal) && + string.Equals (manifest.PrimaryCpuAbi, PrimaryCpuAbi, StringComparison.Ordinal) && + manifest.Files != null; + } + + static string ComputeManifestHash (ManifestData manifest) + { + using (var hash = SHA256.Create ()) { + byte [] bytes = Encoding.UTF8.GetBytes (GetCanonicalManifestText (manifest)); + return BitConverter.ToString (hash.ComputeHash (bytes)).Replace ("-", "").ToLowerInvariant (); + } + } + + static string GetCanonicalManifestText (ManifestData manifest) + { + var builder = new StringBuilder (); + builder.AppendLine (manifest.DeviceId ?? ""); + builder.AppendLine (manifest.PackageName ?? ""); + builder.AppendLine (manifest.UserId ?? ""); + builder.AppendLine (manifest.PrimaryCpuAbi ?? ""); + foreach (var entry in manifest.Files.OrderBy (entry => entry.Key, StringComparer.Ordinal)) { + builder.Append (entry.Key).Append ('\t') + .Append (entry.Value.RelativePath).Append ('\t') + .Append (entry.Value.Size).Append ('\t') + .AppendLine (entry.Value.LastWriteTimeUtcTicks.ToString ()); + } + return builder.ToString (); + } + + string GetManifestFilePath () + { + return Path.Combine ( + GetFullPath (IntermediateOutputPath), + "fastdeploy2", + GetSafeFileName (GetDeviceId ()), + GetSafeFileName (PackageName), + GetSafeFileName (GetUserId ()), + GetSafeFileName (PrimaryCpuAbi), + "manifest.json"); + } + + static string GetSafeFileName (string value) + { + return string.IsNullOrEmpty (value) ? "_" : Uri.EscapeDataString (value); + } + + class DeviceManifestState { + public string RemoteHash { get; set; } = ""; + public string OverrideHash { get; set; } = ""; + } + + internal class ManifestData { + [JsonPropertyName ("deviceId")] + public string DeviceId { get; set; } + + [JsonPropertyName ("packageName")] + public string PackageName { get; set; } + + [JsonPropertyName ("userId")] + public string UserId { get; set; } + + [JsonPropertyName ("primaryCpuAbi")] + public string PrimaryCpuAbi { get; set; } + + [JsonPropertyName ("files")] + public Dictionary Files { get; set; } + } + + internal class ManifestEntry { + [JsonPropertyName ("relativePath")] + public string RelativePath { get; set; } + + [JsonPropertyName ("size")] + public long Size { get; set; } + + [JsonPropertyName ("lastWriteTimeUtcTicks")] + public long LastWriteTimeUtcTicks { get; set; } + } + } +} diff --git a/src/Xamarin.Android.Build.Debugging.Tasks/Tasks/FastDeploy2.cs b/src/Xamarin.Android.Build.Debugging.Tasks/Tasks/FastDeploy2.cs new file mode 100644 index 00000000000..aa36652d04f --- /dev/null +++ b/src/Xamarin.Android.Build.Debugging.Tasks/Tasks/FastDeploy2.cs @@ -0,0 +1,1221 @@ +using System; +using System.Collections.Generic; +using System.Diagnostics; +using System.IO; +using System.Linq; +using System.Text; +using System.Threading; +using System.Threading.Tasks; + +using Microsoft.Android.Build.Tasks; +using Microsoft.Build.Framework; +using Xamarin.Android.Build.Debugging.Tasks.Properties; + +namespace Xamarin.Android.Tasks +{ + public partial class FastDeploy2 : AsyncTask + { + const string OverridePath = "files/.__override__"; + + public override string TaskPrefix => "FD2"; + + /// + /// Number of stale override files to delete per rm invocation. Exposed as an + /// internal MSBuild property primarily so it can be lowered when testing batching behavior. + /// + public int StaleFileRemovalBatchSize { get; set; } = 100; + + /// + /// Number of files to copy per batch when staging fast-deployment files on the device. + /// Exposed as an internal MSBuild property primarily so it can be lowered when testing. + /// + public int CopyBatchSize { get; set; } = 25; + + /// + /// Maximum length (in characters) of a single adb shell command line before it is + /// split into multiple invocations. + /// + public int MaxShellCommandLength { get; set; } = 900; + + /// + /// Maximum length (in characters) of a single adb command line before it is split + /// into multiple invocations. + /// + public int MaxAdbCommandLength { get; set; } = 4096; + + public string AdbTarget { get; set; } + public string UploadFlagFile { get; set; } + public bool EmbedAssembliesIntoApk { get; set; } + public bool ReInstall { get; set; } = false; + + [Required] + public string PackageName { get; set; } + + public string PackageFile { get; set; } + + public string PrimaryCpuAbi { get; set; } + + public ITaskItem [] FastDevFiles { get; set; } + + public bool PreserveUserData { get; set; } = true; + + public bool DiagnosticLogging { get; set; } = false; + + public string UserID { get; set; } + + public bool IsTestOnly { get; set; } + + [Required] + public string IntermediateOutputPath { get; set; } + + public ITaskItem [] EnvironmentFiles { get; set; } + + public string AdbToolPath { get; set; } + + public string AdbToolExe { get; set; } + + public string AdbPushCompressionAlgorithm { get; set; } = "any"; + + public string AppFileTransferMode { get; set; } = "Copy"; + + string DeviceId = ""; + PackageInfo packageInfo = new PackageInfo (); + DateTime lastUpload = DateTime.MinValue; + Queue diagnosticLogs = new Queue (); + readonly object diagnosticLogsLock = new object (); + + string OverrideFullPath { + get { return packageInfo.IsSystemApplication ? $"{packageInfo.InternalPath}/{OverridePath}" : OverridePath; } + } + + class PackageInfo { + string internalPath = null; + public string InternalPath { + get { return internalPath; } + set { internalPath = value?.Trim () ?? null; } + } + + public bool IsSystemApplication { get; set; } = false; + public bool AdbIsRoot { get; set; } = false; + public string UserId { get; set; } = null; + public string PackageName { get; set; } = null; + public int ProcessId { get; set; } = 0; + } + + class RemoteFileInfo { + public long Size { get; set; } + public long ModifiedTime { get; set; } + } + + class DirectPushFile { + public string LocalPath { get; set; } + public string RelativePath { get; set; } + } + + void LogDiagnostic (string message) + { + if (DiagnosticLogging) { + LogDebugMessage (message); + return; + } + lock (diagnosticLogsLock) { + diagnosticLogs.Enqueue (message); + } + } + + void PrintDiagnostics () + { + while (true) { + string message; + lock (diagnosticLogsLock) { + if (diagnosticLogs.Count == 0) { + break; + } + message = diagnosticLogs.Dequeue (); + } + LogMessage (message); + } + } + + public override bool Execute () + { + var device = AndroidHelper.ParseTarget (AdbTarget, LogMessage, LogCodedError, logErrors: true, engine4: BuildEngine4); + if (device == null) { + PrintDiagnostics (); + return false; + } + DeviceId = device.ID ?? ""; + LogMessage ($"Found device: {DeviceId}"); + + if (string.IsNullOrEmpty (PrimaryCpuAbi) && !EmbedAssembliesIntoApk) { + PrintDiagnostics (); + LogCodedError ("XA0010", Resources.XA0010_NoAbi, DeviceId); + return false; + } + + var flagFilePath = GetFullPath (UploadFlagFile); + lastUpload = File.GetLastWriteTimeUtc (flagFilePath); + LogDiagnostic ($"LastWriteTime of `{flagFilePath}`: {lastUpload}"); + + var lifetime = RegisteredTaskObjectLifetime.AppDomain; + var key = ProjectSpecificTaskObjectKey ($"{DeviceId}_{PackageName}_{GetType ().Name}"); + if (!File.Exists (UploadFlagFile)) { + packageInfo = new PackageInfo (); + } else { + packageInfo = BuildEngine4.GetRegisteredTaskObjectAssemblyLocal (key, lifetime) ?? new PackageInfo (); + } + + try { + return base.Execute (); + } finally { + BuildEngine4.RegisterTaskObjectAssemblyLocal (key, packageInfo, lifetime, allowEarlyCollection: false); + } + } + + public async override Task RunTaskAsync () + { + try { + await RunInstall (); + } catch { + PrintDiagnostics (); + throw; + } + } + + async Task RunInstall () + { + string redirectStdio = await GetDeviceProperty ("log.redirect-stdio"); + if (string.Equals ("true", redirectStdio, StringComparison.OrdinalIgnoreCase)) { + LogFastDeploy2Error ("XA0128", Resources.XA0128_RedirectStdioIsEnabled); + return; + } + + string runAsDisabled = await GetDeviceProperty ("ro.boot.disable_runas"); + if (string.Equals ("true", runAsDisabled, StringComparison.OrdinalIgnoreCase)) { + LogFastDeploy2Error ("XA0131", Resources.XA0131_DeveloperModeNotEnabled); + return; + } + + await CheckAppInstalledAndDebuggable (PackageName); + + if (EmbedAssembliesIntoApk) { + await RemoveOverrideDirectory (); + } + + if (ReInstall && !string.IsNullOrEmpty (PackageFile)) { + await UninstallPackage (PackageName, preserveData: PreserveUserData, user: UserID); + } + + bool packageFileOutOfDate = !string.IsNullOrEmpty (PackageFile) && + (packageInfo.InternalPath.IndexOf ("unknown", StringComparison.OrdinalIgnoreCase) >= 0 || ReInstall || IsPackageFileOutOfDate ()); + + if (packageFileOutOfDate) { + try { + await InstallPackage (); + } catch (Exception ex) { + LogFastDeploy2Error (GetErrorCode (ex), ex.ToString ()); + return; + } + if (!EmbedAssembliesIntoApk && packageInfo.InternalPath.IndexOf ("unknown", StringComparison.OrdinalIgnoreCase) >= 0) { + packageInfo.InternalPath = null; + await CheckAppInstalledAndDebuggable (PackageName); + if (RaiseRunAsError (packageInfo.InternalPath)) { + return; + } + } + } + + if (EmbedAssembliesIntoApk) + return; + + if ((FastDevFiles?.Length ?? 0) == 0 && (EnvironmentFiles?.Length ?? 0) == 0) { + return; + } + + await TerminateApp (); + if (!await DeployFastDevFilesWithAdbPush (OverrideFullPath)) { + LogDiagnostic ("FastDeploy2 deployment did not complete successfully."); + } + } + + bool IsPackageFileOutOfDate () + { + var packageFile = GetFullPath (PackageFile); + var lastPackage = File.GetLastWriteTimeUtc (packageFile); + LogDiagnostic ($"LastWriteTime of `{packageFile}`: {lastPackage}"); + return lastUpload < lastPackage; + } + + async Task CheckAppInstalledAndDebuggable (string packageName) + { + packageInfo.UserId = UserID; + packageInfo.PackageName = packageName; + packageInfo.ProcessId = 0; + await EnsureUserIsRunning (); + string packageInfoOutput = IsSafePackageNameForShell (packageName) ? + await RunAs ("sh", "-c", $"pwd; pidof {packageName} 2>/dev/null || true") : + await RunAs ("pwd"); + ParsePackageInfoOutput (packageInfoOutput); + if (string.IsNullOrEmpty (packageInfo.InternalPath)) { + packageInfo.InternalPath = packageInfoOutput?.Trim (); + } + if (packageInfo.InternalPath.IndexOf ("Permission denied", StringComparison.OrdinalIgnoreCase) >= 0) { + packageInfo.InternalPath = await RunAs ("readlink", "-f", "."); + } + if (packageInfo.InternalPath.IndexOf ("not an application", StringComparison.OrdinalIgnoreCase) >= 0) { + LogDiagnostic ($"Package {packageInfo.PackageName} is a system application."); + packageInfo.IsSystemApplication = true; + var whoami = await RunAdbShellCommand ("whoami"); + packageInfo.AdbIsRoot = whoami.Output.Trim () == "root"; + LogDiagnostic ($"using {(packageInfo.AdbIsRoot ? "root" : $"su {packageInfo.UserId}")} to install fast deployment files."); + packageInfo.InternalPath = $"/data/user/{(packageInfo.UserId ?? "0")}/{packageInfo.PackageName}"; + return; + } + if (packageInfo.InternalPath.IndexOf ("not debuggable", StringComparison.OrdinalIgnoreCase) >= 0) { + LogDiagnostic ($"Package {packageInfo.PackageName} was not debuggable. Forcing ReInstall"); + ReInstall = true; + return; + } + if (packageInfo.InternalPath.IndexOf ("unknown", StringComparison.OrdinalIgnoreCase) >= 0) { + LogDiagnostic ($"Package {packageInfo.PackageName} was not installed."); + return; + } + if (packageInfo.InternalPath.IndexOf ("Permission denied", StringComparison.OrdinalIgnoreCase) >= 0) { + LogDiagnostic ("run-as not supported on this device."); + } + } + + static bool IsSafePackageNameForShell (string packageName) + { + if (string.IsNullOrEmpty (packageName)) { + return false; + } + foreach (char c in packageName) { + if (!(char.IsLetterOrDigit (c) || c == '.' || c == '_')) { + return false; + } + } + return true; + } + + void ParsePackageInfoOutput (string output) + { + if (string.IsNullOrEmpty (output)) { + return; + } + + string [] lines = output.Replace ("\r", "").Split (new char [] { '\n' }, StringSplitOptions.None); + if (lines.Length > 0 && !string.IsNullOrEmpty (lines [0])) { + packageInfo.InternalPath = lines [0].Trim (); + } + if (lines.Length <= 1) { + return; + } + + string pidLine = lines [1].Trim (); + int space = pidLine.IndexOf (' '); + if (space >= 0) { + pidLine = pidLine.Substring (0, space); + } + if (int.TryParse (pidLine, out int pid)) { + packageInfo.ProcessId = pid; + } + } + + async Task EnsureUserIsRunning () + { + var userId = (UserID ?? "").Trim (); + if (userId.Length == 0 || (int.TryParse (userId, out var id) && id == 0)) { + return; + } + LogDiagnostic ($"Ensuring Android user {userId} is in the 'running' state before run-as queries."); + var result = await RunAdbShellCommand ("am", "start-user", "-w", userId); + string output = result.Output; + LogDiagnostic ($"'am start-user -w {userId}' returned: {(string.IsNullOrWhiteSpace (output) ? "" : output.Trim ())}"); + } + + async Task InstallPackage () + { + LogDebugMessage ($"Installing Package {PackageName}"); + await InstallApkWithRetry (PackageFile, reinstall: ReInstall, testOnly: IsTestOnly, user: UserID); + LogDebugMessage ($"Installed Package {PackageName}."); + } + + async Task RemoveOverrideDirectory () + { + await RunAs ("rm", "-Rf", OverrideFullPath); + } + + async Task TerminateApp () + { + var pid = packageInfo.ProcessId; + if (pid == 0 && packageInfo.IsSystemApplication) { + pid = await RunLoggedDeviceOperation ($"GetProcessId {PackageName}", () => GetProcessId (PackageName)); + } + if (pid == 0) { + LogDebugMessage ($"{PackageName} was not running, skipping kill"); + return; + } + LogDebugMessage ($"Terminating {PackageName}..."); + await RunLoggedDeviceOperation ($"ForceStop {PackageName}", () => ForceStopPackage (PackageName)); + LogDebugMessage ($"{PackageName} Terminated."); + } + + async Task CreateRemoteStagingDirectories (string remoteStagingPath, HashSet stagedFiles) + { + var directories = new HashSet (StringComparer.Ordinal) { remoteStagingPath }; + foreach (var file in stagedFiles) { + string directory = GetDirectoryName (file); + if (!string.IsNullOrEmpty (directory)) { + directories.Add (CombineRemotePath (remoteStagingPath, directory)); + } + } + + var output = new StringBuilder (); + foreach (var batch in BatchArguments ("mkdir", "-p", directories)) { + output.Append ((await RunAdbShellCommand (batch.ToArray ())).Output); + } + return output.ToString (); + } + + List PrepareDirectPushFiles () + { + var files = new List (); + foreach (var file in FastDevFiles ?? []) { + string localPath = GetFullPath (file.ItemSpec); + if (!File.Exists (localPath)) { + LogDiagnostic ($"File '{file.ItemSpec}' does not exist. Skipping."); + continue; + } + if (Path.GetExtension (file.ItemSpec) == ".so") { + string abi = AndroidRidAbiHelper.GetNativeLibraryAbi (file); + if (abi != PrimaryCpuAbi) { + LogDebugMessage ($"NotifySync SkipCopyFile {GetAdbPushTargetPath (file)} abi not suitable for this device."); + continue; + } + } + + files.Add (new DirectPushFile { + LocalPath = localPath, + RelativePath = GetAdbPushTargetPath (file), + }); + LogDiagnostic ($"Prepared {file.ItemSpec} => {files [files.Count - 1].RelativePath}"); + } + + if (EnvironmentFiles?.Length > 0) { + byte [] environmentData = CreateEnvironmentFileData (EnvironmentFiles, out DateTime newestFileDateTime); + if (environmentData.Length > 0) { + string environmentFile = Path.Combine (GetFullPath (IntermediateOutputPath), "fastdeploy2-environment", PrimaryCpuAbi, "environment"); + WriteFileIfChanged (environmentFile, environmentData, newestFileDateTime); + files.Add (new DirectPushFile { + LocalPath = environmentFile, + RelativePath = $"{PrimaryCpuAbi}/environment", + }); + } + } + + return files; + } + + bool WriteFileIfChanged (string path, byte [] contents, DateTime modifiedDateTime) + { + if (!Files.HasBytesChanged (contents, path)) { + return false; + } + + Directory.CreateDirectory (Path.GetDirectoryName (path)); + File.WriteAllBytes (path, contents); + File.SetLastWriteTimeUtc (path, modifiedDateTime); + return true; + } + + string GetAdbPushTargetPath (ITaskItem file) + { + string targetPath = file.GetMetadata ("TargetPath"); + if (string.IsNullOrEmpty (targetPath)) { + LogDiagnostic ($"'TargetPath' metadata not found on '{file.ItemSpec}'. Falling back to 'DestinationSubPath'"); + targetPath = file.GetMetadata ("DestinationSubPath"); + } + if (!string.IsNullOrEmpty (targetPath)) { + return targetPath.Replace ("\\", "/"); + } + return Path.GetFileName (file.ItemSpec); + } + + byte [] CreateEnvironmentFileData (ITaskItem [] environments, out DateTime newestFileDateTime) + { + int maxKeyLength = 0; + int maxValueLength = 0; + newestFileDateTime = DateTime.MinValue; + var data = new Dictionary (); + foreach (ITaskItem env in environments ?? []) { + if (!File.Exists (env.ItemSpec)) + continue; + DateTime modifiedDateTime = File.GetLastWriteTimeUtc (env.ItemSpec); + if (modifiedDateTime > newestFileDateTime) + newestFileDateTime = modifiedDateTime; + foreach (string line in File.ReadLines (env.ItemSpec)) { + if (string.IsNullOrEmpty (line)) + continue; + int index = line.IndexOf ('='); + if (index == -1) { + LogDebugMessage ($"Skipping invalid environment line: {line}"); + continue; + } + var key = line.Substring (0, index); + var value = line.Substring (index + 1); + maxKeyLength = Math.Max (maxKeyLength, key.Length); + maxValueLength = Math.Max (maxValueLength, value.Length); + data [key] = value; + } + } + + if (newestFileDateTime == DateTime.MinValue) { + return []; + } + + maxKeyLength++; + maxValueLength++; + + using (var stream = new MemoryStream ()) + using (var binaryWriter = new BinaryWriter (stream, Encoding.ASCII)) { + binaryWriter.Write (Encoding.ASCII.GetBytes ("0x" + maxKeyLength.ToString ("X8") + '\0')); + binaryWriter.Write (Encoding.ASCII.GetBytes ("0x" + maxValueLength.ToString ("X8") + '\0')); + foreach (var kvp in data) { + binaryWriter.Write (Encoding.ASCII.GetBytes (kvp.Key.PadRight (maxKeyLength, '\0'))); + binaryWriter.Write (Encoding.ASCII.GetBytes (kvp.Value.PadRight (maxValueLength, '\0'))); + } + binaryWriter.Flush (); + return stream.ToArray (); + } + } + + async Task> GetRemoteFileData (string rootPath, bool runAs) + { + // The stat format must be quoted so that the `|` separators survive to the device + // shell. `adb shell` re-parses its arguments, so passing the format as an argv element + // (e.g. via RunAdbShellCommand (params string [])) would let the device shell treat the + // `|` characters as pipes. Building a single, explicitly quoted command string avoids it. + string findCommand = $"find {QuoteShellArgument (rootPath)} -type f -exec stat -c '%n|%s|%Y' {{}} +"; + string output; + if (runAs) { + output = await RunAsShell (findCommand); + if (RaiseRunAsError (output)) { + return null; + } + } else { + var result = await RunAdbShellCommand (findCommand); + output = result.Output; + } + + if (IsMissingDirectoryError (output)) { + return new Dictionary (StringComparer.Ordinal); + } + if (IsShellError (output, "find") || IsShellError (output, "stat")) { + LogFastDeploy2Error ("XA0129", output, rootPath); + return null; + } + + return ParseRemoteFileData (rootPath, output); + } + + Dictionary ParseRemoteFileData (string rootPath, string output) + { + var files = new Dictionary (StringComparer.Ordinal); + string prefix = rootPath.TrimEnd ('/') + "/"; + foreach (string line in output.Split (new char [] { '\r', '\n' }, StringSplitOptions.RemoveEmptyEntries)) { + var entries = line.Split (new char [] { '|' }, 3); + if (entries.Length != 3) { + LogDebugMessage ($"Ignoring remote file entry '{line}'. Line is incorrectly formatted."); + continue; + } + string remoteFile = entries [0].Trim (); + if (!remoteFile.StartsWith (prefix, StringComparison.Ordinal)) { + LogDebugMessage ($"Ignoring remote file entry '{line}'. Path is outside '{rootPath}'."); + continue; + } + if (!long.TryParse (entries [1].Trim (), out long size) || !long.TryParse (entries [2].Trim (), out long mtime)) { + LogDebugMessage ($"Ignoring remote file entry '{line}'. Size or timestamp is invalid."); + continue; + } + files [remoteFile.Substring (prefix.Length)] = new RemoteFileInfo { + Size = size, + ModifiedTime = mtime, + }; + } + return files; + } + + async Task RemoveStaleOverrideFiles (string overridePath, Dictionary stagedFiles, Dictionary overrideFiles) + { + var staleFiles = new List (); + foreach (var file in overrideFiles.Keys) { + if (!stagedFiles.ContainsKey (file)) { + staleFiles.Add (CombineRemotePath (overridePath, file)); + } + } + + LogDiagnostic ($"FastDeploy2 removing {staleFiles.Count} stale override files."); + foreach (var batch in BatchShellArguments (new [] { "rm", "-f" }, staleFiles, StaleFileRemovalBatchSize)) { + string output = await RunAs (batch.ToArray ()); + if (RaiseRunAsError (output) || IsShellError (output, "rm")) { + LogFastDeploy2Error ("XA0129", output, overridePath); + return false; + } + } + return true; + } + + async Task CopyChangedFiles (string remoteStagingPath, string overridePath, Dictionary stagedFiles, Dictionary overrideFiles) + { + var changedFiles = new List (); + foreach (var file in stagedFiles) { + if (!overrideFiles.TryGetValue (file.Key, out RemoteFileInfo existing) || + existing.Size != file.Value.Size || + existing.ModifiedTime != file.Value.ModifiedTime) { + changedFiles.Add (file.Key); + } + } + + LogDiagnostic ($"FastDeploy2 copying {changedFiles.Count} changed override files."); + var filesByDirectory = GroupFilesByDirectory (changedFiles); + + foreach (var group in filesByDirectory) { + string targetDirectory = CombineRemotePath (overridePath, group.Key); + string output = await RunAs ("mkdir", "-p", targetDirectory); + if (RaiseRunAsError (output) || IsShellError (output, "mkdir")) { + LogFastDeploy2Error ("XA0129", output, targetDirectory); + return false; + } + + // Remove the current destination files, then copy the freshly staged files in. + // `cp` overwrites anyway, so removing first (rather than interleaving per batch) + // is equivalent and lets each command batch independently by length. + var destinationFiles = group.Value.Select (file => CombineRemotePath (targetDirectory, Path.GetFileName (file))); + foreach (var batch in BatchShellArguments (new [] { "rm", "-f" }, destinationFiles, CopyBatchSize)) { + output = await RunAs (batch.ToArray ()); + if (RaiseRunAsError (output) || IsShellError (output, "rm")) { + LogFastDeploy2Error ("XA0129", output, targetDirectory); + return false; + } + } + + var sourceFiles = group.Value.Select (file => CombineRemotePath (remoteStagingPath, file)); + foreach (var batch in BatchShellArguments (new [] { "cp", "-p" }, sourceFiles, CopyBatchSize, trailing: targetDirectory)) { + output = await RunAs (batch.ToArray ()); + if (RaiseRunAsError (output) || IsShellError (output, "cp")) { + LogFastDeploy2Error ("XA0129", output, targetDirectory); + return false; + } + } + } + + return true; + } + + IEnumerable> BatchArguments (string command, string option, IEnumerable values) + { + var batch = new List { command, option }; + int length = command.Length + option.Length + 2; + foreach (var value in values) { + int itemLength = value.Length + 3; + if (batch.Count > 2 && length + itemLength >= MaxShellCommandLength) { + yield return batch; + batch = new List { command, option }; + length = command.Length + option.Length + 2; + } + batch.Add (value); + length += itemLength; + } + if (batch.Count > 2) { + yield return batch; + } + } + + /// + /// Splits into run-as shell command batches, capping each + /// batch by both items and + /// characters. Each batch starts with (e.g. rm -f) and, + /// when is set, ends with it (e.g. the destination directory of + /// a cp). Length is estimated the same way as (prefix, + /// trailing, and per-value quoting overhead) so a single command never overflows the device + /// shell's command-line limit even when a count-only cap would keep the batch too large. + /// + IEnumerable> BatchShellArguments (IReadOnlyList prefix, IEnumerable values, int maxCount, string trailing = null) + { + int prefixLength = 0; + foreach (var arg in prefix) { + prefixLength += arg.Length + 3; + } + bool hasTrailing = !string.IsNullOrEmpty (trailing); + int trailingLength = hasTrailing ? trailing.Length + 3 : 0; + + var batch = new List (prefix); + int count = 0; + int length = prefixLength + trailingLength; + foreach (var value in values) { + int itemLength = value.Length + 3; + if (count > 0 && (count >= maxCount || length + itemLength >= MaxShellCommandLength)) { + if (hasTrailing) { + batch.Add (trailing); + } + yield return batch; + batch = new List (prefix); + count = 0; + length = prefixLength + trailingLength; + } + batch.Add (value); + length += itemLength; + count++; + } + if (count > 0) { + if (hasTrailing) { + batch.Add (trailing); + } + yield return batch; + } + } + + List CreatePushArgs (string localPath, string remotePath) + { + var args = CreatePushArgsPrefix (); + args.Add (localPath); + args.Add (remotePath); + return args; + } + + List CreatePushArgsPrefix () + { + var args = new List { "push" }; + if (!string.IsNullOrEmpty (AdbPushCompressionAlgorithm)) { + args.Add ("-z"); + args.Add (AdbPushCompressionAlgorithm); + } + return args; + } + + int EstimateCommandLength (List args) + { + int length = 0; + foreach (var arg in args) { + length += arg.Length + 3; + } + return length; + } + + async Task RunAdbCommand (params string [] arguments) + { + return await RunAdbCommand (arguments, environmentVariables: null); + } + + async Task RunAdbShellCommand (params string [] arguments) + { + return await RunAdbCommand (new [] { "shell" }.Concat (arguments).ToArray ()); + } + + async Task RunAdbCommand (string [] arguments, Dictionary environmentVariables) + { + string adb = ResolveAdbPath (); + var adbArguments = new List (); + if (!string.IsNullOrEmpty (DeviceId) && !string.Equals (DeviceId, "any", StringComparison.OrdinalIgnoreCase)) { + adbArguments.Add ("-s"); + adbArguments.Add (DeviceId); + } + adbArguments.AddRange (arguments); + + var stdout = new StringBuilder (); + var stderr = new StringBuilder (); + using var stdoutCompleted = new ManualResetEvent (false); + using var stderrCompleted = new ManualResetEvent (false); + var psi = new ProcessStartInfo { + FileName = adb, + Arguments = string.Join (" ", adbArguments.Select (QuoteProcessArgument)), + UseShellExecute = false, + RedirectStandardOutput = true, + RedirectStandardError = true, + CreateNoWindow = true, + WindowStyle = ProcessWindowStyle.Hidden, + }; + if (environmentVariables != null) { + foreach (var kvp in environmentVariables) { + psi.EnvironmentVariables [kvp.Key] = kvp.Value; + } + } + + LogDiagnostic ($"adb command: {psi.FileName} {psi.Arguments}"); + using (var process = new Process ()) { + process.StartInfo = psi; + process.OutputDataReceived += (sender, e) => { + if (e.Data != null) { + lock (stdout) { + stdout.AppendLine (e.Data); + } + } else { + stdoutCompleted.Set (); + } + }; + process.ErrorDataReceived += (sender, e) => { + if (e.Data != null) { + lock (stderr) { + stderr.AppendLine (e.Data); + } + } else { + stderrCompleted.Set (); + } + }; + + process.Start (); + process.BeginOutputReadLine (); + process.BeginErrorReadLine (); + using (CancellationToken.Register (() => { + try { + if (!process.HasExited) { + process.Kill (); + } + } catch (InvalidOperationException) { + } + })) { + await Task.Run (() => process.WaitForExit (), CancellationToken); + } + stdoutCompleted.WaitOne (TimeSpan.FromSeconds (30)); + stderrCompleted.WaitOne (TimeSpan.FromSeconds (30)); + var result = new AdbCommandResult { + ExitCode = process.ExitCode, + StandardOutput = stdout.ToString ().Trim (), + StandardError = stderr.ToString ().Trim (), + }; + LogAdbCommandResult (result); + return result; + } + } + + void LogAdbCommandResult (AdbCommandResult result) + { + LogDiagnostic ($"adb exit code: {result.ExitCode}"); + LogAdbStream ("stdout", result.StandardOutput); + LogAdbStream ("stderr", result.StandardError); + } + + void LogAdbStream (string name, string value) + { + if (string.IsNullOrEmpty (value)) { + return; + } + LogDiagnostic ($"adb {name}:{Environment.NewLine}{value}"); + } + + async Task RunLoggedDeviceOperation (string operation, Func action) + { + LogDiagnostic ($"Device operation: {operation}"); + await action (); + LogDiagnostic ($"Device operation completed: {operation}"); + } + + async Task RunLoggedDeviceOperation (string operation, Func> action) + { + LogDiagnostic ($"Device operation: {operation}"); + T result = await action (); + LogDiagnostic ($"Device operation completed: {operation} => {result}"); + return result; + } + + List BuildRunAsArgs () + { + List args = new List (); + if (packageInfo.IsSystemApplication) { + if (!packageInfo.AdbIsRoot) { + args.Add ("su"); + args.Add (packageInfo.UserId ?? "0"); + } + return args; + } + args.Add ("run-as"); + args.Add (packageInfo.PackageName); + if (!string.IsNullOrEmpty (packageInfo.UserId)) { + args.Add ("--user"); + args.Add (packageInfo.UserId); + } + return args; + } + + async Task RunAs (params string [] arguments) + { + List args = BuildRunAsArgs (); + args.AddRange (arguments); + var result = await RunAdbShellCommand (args.ToArray ()); + return result.Output; + } + + async Task RunAsShell (string script) + { + List args = BuildRunAsArgs (); + args.Add ("sh"); + args.Add ("-c"); + args.Add (script); + string command = string.Join (" ", args.Select (QuoteShellArgument)); + var result = await RunAdbShellCommand (command); + return result.Output; + } + + static string QuoteShellArgument (string value) + { + return "'" + value.Replace ("'", "'\"'\"'") + "'"; + } + + string ResolveAdbPath () + { + var exe = string.IsNullOrEmpty (AdbToolExe) ? "adb" : AdbToolExe; + return string.IsNullOrEmpty (AdbToolPath) ? exe : Path.Combine (AdbToolPath, exe); + } + + string GetRemoteAdbPushStagingPath () + { + return $"{RemoteStagingRoot}/{PackageName}/{GetUserId ()}"; + } + + string GetUserId () + { + return string.IsNullOrEmpty (UserID) ? "0" : UserID; + } + + string GetDeviceId () + { + if (!string.IsNullOrEmpty (DeviceId)) { + return DeviceId; + } + return string.IsNullOrEmpty (AdbTarget) ? "any" : AdbTarget; + } + + void LogFastDeploy2Error (string errorCode, string error, string file = "") + { + if (!string.IsNullOrEmpty (file)) { + LogDiagnostic ($"{errorCode} while deploying '{file}': {error}"); + } else { + LogDiagnostic ($"{errorCode}: {error}"); + } + PrintDiagnostics (); + if (errorCode == "XA0129") { + LogCodedError (errorCode, Resources.XA0129_ErrorDeployingFile, file); + } else { + LogCodedError (errorCode, error); + } + } + + string GetFullPath (string dir) => Path.IsPathRooted (dir) ? dir : Path.GetFullPath (Path.Combine (WorkingDirectory, dir)); + + static string GetDirectoryName (string file) + { + return Path.GetDirectoryName (file)?.Replace ("\\", "/") ?? ""; + } + + static string CombineRemotePath (string rootPath, string relativePath) + { + return string.IsNullOrEmpty (relativePath) ? rootPath : $"{rootPath}/{relativePath}"; + } + + static Dictionary> GroupFilesByDirectory (IEnumerable files) + { + var filesByDirectory = new Dictionary> (StringComparer.Ordinal); + foreach (string file in files) { + string directory = GetDirectoryName (file); + if (!filesByDirectory.TryGetValue (directory, out List filesInDirectory)) { + filesInDirectory = new List (); + filesByDirectory.Add (directory, filesInDirectory); + } + filesInDirectory.Add (file); + } + return filesByDirectory; + } + + bool RaiseRunAsError (string error) + { + if (TryGetRunAsErrorCode (error, out var err)) { + LogDiagnostic ($"{err.code}: {err.message}"); + PrintDiagnostics (); + LogCodedError (err.code, err.message, error); + return true; + } + return false; + } + + bool TryGetRunAsErrorCode (string error, out (string error, string code, string message) errTuple) + { + errTuple = (error: "unknown", code: "XA0132", message: error); + foreach (var err in runas_codes) { + if (error.IndexOf (err.error, StringComparison.OrdinalIgnoreCase) >= 0) { + errTuple = err; + return true; + } + } + return false; + } + + string GetErrorCode (Exception ex) + { + if (ex is FastDeployInstallException installException) { + return installException.ErrorCode; + } + return GetErrorCode (ex.Message); + } + + static string GetErrorCode (string message) + { + foreach (var errorCode in error_codes) + if (message.IndexOf (errorCode.message, StringComparison.OrdinalIgnoreCase) >= 0) + return errorCode.code; + + return "ADB1000"; + } + + static bool IsShellError (string output, string command) + { + if (string.IsNullOrEmpty (output)) { + return false; + } + return output.IndexOf ($"{command}:", StringComparison.OrdinalIgnoreCase) >= 0 || + output.IndexOf ("No such file or directory", StringComparison.OrdinalIgnoreCase) >= 0 || + output.IndexOf ("Permission denied", StringComparison.OrdinalIgnoreCase) >= 0 || + output.IndexOf ("Read-only file system", StringComparison.OrdinalIgnoreCase) >= 0 || + output.IndexOf ("not found", StringComparison.OrdinalIgnoreCase) >= 0; + } + + static bool IsMissingDirectoryError (string output) + { + return !string.IsNullOrEmpty (output) && + output.IndexOf ("No such file or directory", StringComparison.OrdinalIgnoreCase) >= 0; + } + + struct AdbCommandResult + { + public int ExitCode; + public string StandardOutput; + public string StandardError; + + public string Output { + get { + if (string.IsNullOrEmpty (StandardOutput)) { + return StandardError ?? ""; + } + if (string.IsNullOrEmpty (StandardError)) { + return StandardOutput; + } + return $"{StandardOutput}{Environment.NewLine}{StandardError}"; + } + } + } + + static readonly List<(string error, string code, string message)> runas_codes = new List<(string error, string code, string message)> () { + { (error: "run-as is disabled", code: "XA0131", message: Resources.XA0131_DeveloperModeNotEnabled ) }, + { (error: "Could not set capabilities", code: "XA0131", message: Resources.XA0131_DeveloperModeNotEnabled ) }, + { (error: "unknown", code: "XA0132", message: Resources.XA0132_PackageNotInstalled ) }, + { (error: "Permission denied", code: "XA0133", message: Resources.XA0133_RunAsPermissionDenied ) }, + { (error: "package not debuggable", code: "XA0134", message: Resources.XA0134_RunAsPackageNotDebuggable ) }, + { (error: "package not an application", code: "XA0135", message: Resources.XA0135_RunAsPackageNotAndApplication ) }, + { (error: "has corrupt installation", code: "XA0136", message: Resources.XA0136_RunAsCorruptInstallation ) }, + { (error: "users can run this program", code: "XA0137", message: Resources.XA0137_RunAsOSCorrupt ) }, + { (error: "set SELinux security context", code: "XA0137", message: Resources.XA0137_RunAsOSCorrupt ) }, + { (error: "to package's data directory", code: "XA0137", message: Resources.XA0137_RunAsOSCorrupt ) }, + { (error: "couldn't stat", code: "XA0137", message: Resources.XA0137_RunAsOSCorrupt ) }, + { (error: "has wrong owner", code: "XA0137", message: Resources.XA0137_RunAsOSCorrupt ) }, + { (error: "readable or writable by others", code: "XA0137", message: Resources.XA0137_RunAsOSCorrupt ) }, + { (error: "not a directory", code: "XA0137", message: Resources.XA0137_RunAsOSCorrupt ) }, + { (error: "run-as:", code: "XA0137", message: Resources.XA0137_RunAsOSCorrupt ) }, + }; + + static readonly List<(string code, string message)> error_codes = new List<(string code, string message)> () { + { (code: "ADB1001", message: "failed to create session") }, + { (code: "ADB1002", message: "failed to finalize session") }, + { (code: "ADB1003", message: "product directory not specified; set $ANDROID_PRODUCT_OUT") }, + { (code: "ADB1004", message: "server didn't ACK") }, + { (code: "ADB1005", message: "server killed by remote request") }, + { (code: "ADB1006", message: "timed out waiting for threads to finish reading from ADB server") }, + { (code: "ADB1007", message: "usage:") }, + { (code: "ADB1008", message: "bulkIn endpoint not assigned") }, + { (code: "ADB1009", message: "bulkOut endpoint not assigned") }, + { (code: "ADB1010", message: "cannot start server on remote host") }, + { (code: "ADB1011", message: "cap_clear_flag(INHERITABLE) failed") }, + { (code: "ADB1012", message: "cap_clear_flag(PEMITTED) failed") }, + { (code: "ADB1013", message: "cap_set_proc() failed") }, + { (code: "ADB1014", message: "Client not connected") }, + { (code: "ADB1015", message: "Could not find device interface") }, + { (code: "ADB1016", message: "Could not set SELinux context") }, + { (code: "ADB1017", message: "Could not start mdnsd") }, + { (code: "ADB1018", message: "could not start server") }, + { (code: "ADB1019", message: "couldn't allocate StdinReadArgs object") }, + { (code: "ADB1020", message: "couldn't create USB matching dictionary") }, + { (code: "ADB1021", message: "daemon started successfully") }, + { (code: "ADB1022", message: "daemon still not running") }, + { (code: "ADB1023", message: "error: no emulator detected") }, + { (code: "ADB1024", message: "error: shell command too long") }, + { (code: "ADB1025", message: "Failed to allocate key") }, + { (code: "ADB1026", message: "failed to allocate memory for ShellProtocol object") }, + { (code: "ADB1027", message: "failed to allocate new subprocess") }, + { (code: "ADB1028", message: "Failed to convert to public key") }, + { (code: "ADB1029", message: "failed to create pipe to report error") }, + { (code: "ADB1030", message: "failed to create run queue notify socketpair") }, + { (code: "ADB1031", message: "failed to empty run queue notify fd") }, + { (code: "ADB1032", message: "failed to encode RSA public key") }, + { (code: "ADB1033", message: "Failed to generate new key") }, + { (code: "ADB1034", message: "failed to get matching services") }, + { (code: "ADB1035", message: "failed to get user home directory") }, + { (code: "ADB1036", message: "Failed to get user key") }, + { (code: "ADB1037", message: "failed to make run queue notify socket nonblocking") }, + { (code: "ADB1038", message: "Failed to read key") }, + { (code: "ADB1039", message: "failed to register libusb hotplug callback") }, + { (code: "ADB1040", message: "failed to start daemon") }, + { (code: "ADB1041", message: "failed to write to run queue notify fd") }, + { (code: "ADB1042", message: "Key must be a null-terminated string") }, + { (code: "ADB1043", message: "Pipe stalled, clearing stall") }, + { (code: "ADB1044", message: "Public key too large to base64 encode") }, + { (code: "ADB1045", message: "reply fd for adb server to client communication not specified") }, + { (code: "ADB1046", message: "run queue notify fd was closed") }, + { (code: "ADB1047", message: "Unable to get interface class, subclass and protocol") }, + { (code: "ADB1048", message: "usb_read interface was null") }, + { (code: "ADB1049", message: "usb_write interface was null") }, + { (code: "ADB1050", message: "cannot fit pipe handle value into 32-bits") }, + { (code: "ADB1051", message: "connect error for create") }, + { (code: "ADB1052", message: "connect error for finalize") }, + { (code: "ADB1053", message: "connect error for write") }, + { (code: "ADB1054", message: "could not open adb service") }, + { (code: "ADB1055", message: "couldn't parse 'wait-for' command") }, + { (code: "ADB1056", message: "CreateFileW 'nul' failed") }, + { (code: "ADB1057", message: "only wrote") }, + { (code: "ADB1058", message: "error response") }, + { (code: "ADB1059", message: "failed to install") }, + { (code: "ADB1060", message: "failed to read block") }, + { (code: "ADB1061", message: "failed to write data") }, + { (code: "ADB1062", message: "invalid reply fd") }, + { (code: "ADB1063", message: "pre-KitKat sideload connection failed") }, + { (code: "ADB1064", message: "doesn't match this client") }, + { (code: "ADB1065", message: "sideload connection failed") }, + { (code: "ADB1066", message: "unable to connect for backup") }, + { (code: "ADB1067", message: "unable to connect for restore") }, + { (code: "ADB1068", message: "unable to connect for") }, + { (code: "ADB1069", message: "unexpected output length for") }, + { (code: "ADB1070", message: "expected 'any', 'local', or 'usb'") }, + { (code: "ADB1071", message: "attempted to close unregistered usb_handle for") }, + { (code: "ADB1072", message: "attempted to reinitialize adb_server_socket_spec") }, + { (code: "ADB1073", message: "cannot connect to daemon at") }, + { (code: "ADB1074", message: "Cannot mkdir") }, + { (code: "ADB1075", message: "Connection banner is too long") }, + { (code: "ADB1076", message: "Could not clear pipe stall both ends") }, + { (code: "ADB1077", message: "Could not install smartsocket listener") }, + { (code: "ADB1078", message: "Could not open interface") }, + { (code: "ADB1079", message: "Could not register mDNS service") }, + { (code: "ADB1080", message: "Couldn't create a device interface") }, + { (code: "ADB1081", message: "Couldn't grab device from interface") }, + { (code: "ADB1082", message: "Couldn't query the interface") }, + { (code: "ADB1083", message: "daemon not running; starting now at") }, + { (code: "ADB1084", message: "destroying fde not created by fdevent_create") }, + { (code: "ADB1085", message: "Encountered mDNS registration error") }, + { (code: "ADB1086", message: "not implemented on Win32") }, + { (code: "ADB1087", message: "could not connect to TCP port") }, + { (code: "ADB1088", message: "no emulator connected") }, + { (code: "ADB1089", message: "only supports allocating a pty") }, + { (code: "ADB1090", message: "failed to connect to socket") }, + { (code: "ADB1091", message: "failed to convert errno") }, + { (code: "ADB1092", message: "failed to initialize libusb") }, + { (code: "ADB1093", message: "Failed to parse key") }, + { (code: "ADB1094", message: "failed to set non-blocking mode for fd") }, + { (code: "ADB1095", message: "failed to start subprocess management thread") }, + { (code: "ADB1096", message: "failed to start subprocess") }, + { (code: "ADB1097", message: "FindDeviceInterface - could not get pipe properties") }, + { (code: "ADB1098", message: "Invalid base64 key") }, + { (code: "ADB1099", message: "Key too long") }, + { (code: "ADB1100", message: "No ':' found in shell service arguments") }, + { (code: "ADB1101", message: "observed inotify event for unmonitored path") }, + { (code: "ADB1102", message: "packet data length doesn't match payload") }, + { (code: "ADB1103", message: "Unable to create a device plug-in") }, + { (code: "ADB1104", message: "Unable to create an interface plug-in") }, + { (code: "ADB1105", message: "Unable to get number of endpoints") }, + { (code: "ADB1106", message: "unexpected type for") }, + { (code: "ADB1107", message: "Unknown socket type") }, + { (code: "ADB1108", message: "Unknown trace flag") }, + { (code: "ADB1109", message: "usb_read failed with status") }, + { (code: "ADB1110", message: "usb_write failed with status") }, + { (code: "ADB1111", message: "adb_socket_accept: failed to allocate accepted socket") }, + { (code: "ADB1112", message: "cannot create service socket pair") }, + { (code: "ADB1113", message: "cannot create socket pair") }, + { (code: "ADB1114", message: "Error generating token") }, + { (code: "ADB1115", message: "Error getting user key filename") }, + { (code: "ADB1116", message: "Failed to accept") }, + { (code: "ADB1117", message: "failed to create inotify fd") }, + { (code: "ADB1118", message: "Failed to get adbd socket") }, + { (code: "ADB1119", message: "failed to shutdown writes to FD") }, + { (code: "ADB1120", message: "Failed to write PK") }, + { (code: "ADB1121", message: "failed to write the exit code packet") }, + { (code: "ADB1122", message: "read of inotify event failed") }, + { (code: "ADB1123", message: "remote usb: 1 - write terminated") }, + { (code: "ADB1124", message: "remote usb: 2 - write terminated") }, + { (code: "ADB1125", message: "remote usb: read terminated (message)") }, + { (code: "ADB1126", message: "remote usb: terminated (data)") }, + { (code: "ADB1127", message: "select failed, closing subprocess pipes") }, + { (code: "ADB1128", message: "backup unable to create file") }, + { (code: "ADB1129", message: "cannot create thread") }, + { (code: "ADB1130", message: "cannot get executable path") }, + { (code: "ADB1131", message: "cannot make handle") }, + { (code: "ADB1132", message: "CreatePipe failed") }, + { (code: "ADB1133", message: "CreateProcessW failed") }, + { (code: "ADB1134", message: "error while reading for") }, + { (code: "ADB1135", message: "execl returned") }, + { (code: "ADB1136", message: "failed to duplicate file descriptor for") }, + { (code: "ADB1137", message: "failed to get file descriptor for") }, + { (code: "ADB1138", message: "failed to open duplicate stream for") }, + { (code: "ADB1139", message: "failed to open file") }, + { (code: "ADB1140", message: "failed to read command") }, + { (code: "ADB1141", message: "failed to read data from") }, + { (code: "ADB1142", message: "failed to read from") }, + { (code: "ADB1143", message: "failed to read package block") }, + { (code: "ADB1144", message: "failed to seek to package block") }, + { (code: "ADB1145", message: "failed to set binary mode for duplicate of") }, + { (code: "ADB1146", message: "failed to stat file") }, + { (code: "ADB1147", message: "failed to stat") }, + { (code: "ADB1148", message: "failed to unbuffer") }, + { (code: "ADB1149", message: "adb_socket_accept: accept on fd") }, + { (code: "ADB1150", message: "unable to open file") }, + { (code: "ADB1151", message: "unexpected result waiting for threads") }, + { (code: "ADB1152", message: "aio: got error event on") }, + { (code: "ADB1153", message: "aio: got error submitting") }, + { (code: "ADB1154", message: "aio: got error waiting") }, + { (code: "ADB1155", message: "cannot open bulk-in endpoint") }, + { (code: "ADB1156", message: "cannot open bulk-out endpoint") }, + { (code: "ADB1157", message: "cannot open control endpoint") }, + { (code: "ADB1158", message: "Can't load") }, + { (code: "ADB1159", message: "could not read ok from ADB Server") }, + { (code: "ADB1160", message: "couldn't allocate state_info") }, + { (code: "ADB1161", message: "Couldn't read") }, + { (code: "ADB1162", message: "cannot write to emulator") }, + { (code: "ADB1163", message: "error reading output FD") }, + { (code: "ADB1164", message: "error reading protocol FD") }, + { (code: "ADB1165", message: "error reading stdin FD") }, + { (code: "ADB1166", message: "write failure during connection") }, + { (code: "ADB1167", message: "failed to fcntl(F_GETFL) for fd") }, + { (code: "ADB1168", message: "failed to fcntl(F_SETFL) for fd") }, + { (code: "ADB1169", message: "failed to inotify_add_watch on path") }, + { (code: "ADB1170", message: "Failed to listen on") }, + { (code: "ADB1171", message: "failed to open directory") }, + { (code: "ADB1172", message: "Failed to write public key to") }, + { (code: "ADB1173", message: "failure closing FD") }, + { (code: "ADB1174", message: "pipe failed in launch_server") }, + { (code: "ADB1175", message: "poll() }, ret =") }, + { (code: "ADB1176", message: "remote usb: read overflow") }, + { (code: "ADB1177", message: "received framework auth socket connection again") }, + { (code: "ADB1178", message: "failed to claim adb interface for device") }, + { (code: "ADB1179", message: "failed to clear halt on device") }, + { (code: "ADB1180", message: "failed to get active config descriptor for device at") }, + { (code: "ADB1181", message: "failed to get device descriptor for device at") }, + { (code: "ADB1182", message: "failed to get serial from device at") }, + { (code: "ADB1183", message: "failed to open usb device at") }, + { (code: "ADB1184", message: "failed to set interface alt setting for device") }, + { (code: "ADB1185", message: "failed to submit zero-length write") }, + { (code: "ADB1186", message: "failed to submit") }, + { (code: "ADB1187", message: "Ignoring unknown shell service argument") }, + { (code: "ADB1188", message: "transfer failed:") }, + { (code: "ADB1189", message: "received empty serial from device at") }, + { (code: "ADB1190", message: "refusing to recurse into directory") }, + { (code: "ADB1191", message: "unmonitored event for") }, + { (code: "ADB1192", message: "Failed to open") }, + { (code: "ADB1193", message: "failed to write") }, + }; + } +} diff --git a/src/Xamarin.Android.Build.Debugging.Tasks/Tasks/FastDeploy2JsonSerializerContext.cs b/src/Xamarin.Android.Build.Debugging.Tasks/Tasks/FastDeploy2JsonSerializerContext.cs new file mode 100644 index 00000000000..933625c5dd5 --- /dev/null +++ b/src/Xamarin.Android.Build.Debugging.Tasks/Tasks/FastDeploy2JsonSerializerContext.cs @@ -0,0 +1,11 @@ +using System.Collections.Generic; +using System.Text.Json.Serialization; + +namespace Xamarin.Android.Tasks +{ + [JsonSourceGenerationOptions (WriteIndented = true)] + [JsonSerializable (typeof (FastDeploy2.ManifestData))] + internal partial class FastDeploy2JsonSerializerContext : JsonSerializerContext + { + } +} diff --git a/src/Xamarin.Android.Build.Debugging.Tasks/Xamarin.Android.Common.Debugging.targets b/src/Xamarin.Android.Build.Debugging.Tasks/Xamarin.Android.Common.Debugging.targets index dc270a09cda..7adc38cb273 100644 --- a/src/Xamarin.Android.Build.Debugging.Tasks/Xamarin.Android.Common.Debugging.targets +++ b/src/Xamarin.Android.Build.Debugging.Tasks/Xamarin.Android.Common.Debugging.targets @@ -23,6 +23,7 @@ Copyright (C) 2016 Xamarin. All rights reserved. + @@ -321,7 +322,26 @@ Copyright (C) 2016 Xamarin. All rights reserved. <_ReInstall Condition=" '$(_ReInstall)' == '' ">False <_AndroidIsTestOnlyPackage Condition=" '$(_AndroidIsTestOnlyPackage)' == '' ">False <_FastDeploymentDiagnosticLogging Condition=" '$(_FastDeploymentDiagnosticLogging)' == '' ">False + <_AndroidFastDevStrategy Condition=" '$(_AndroidFastDevStrategy)' == '' ">FastDeploy2 + <_AndroidFastDeployAppFileTransferMode Condition=" '$(_AndroidFastDeployAppFileTransferMode)' == '' And '$(_AndroidFastDevStrategy)' == 'FastDeploy2' ">Symlink + <_AndroidFastDeployAppFileTransferMode Condition=" '$(_AndroidFastDeployAppFileTransferMode)' == '' ">Copy + any + + <_AndroidFastDeployStaleFileRemovalBatchSize Condition=" '$(_AndroidFastDeployStaleFileRemovalBatchSize)' == '' ">100 + <_AndroidFastDeployCopyBatchSize Condition=" '$(_AndroidFastDeployCopyBatchSize)' == '' ">25 + <_AndroidFastDeployMaxShellCommandLength Condition=" '$(_AndroidFastDeployMaxShellCommandLength)' == '' ">900 + <_AndroidFastDeployMaxAdbCommandLength Condition=" '$(_AndroidFastDeployMaxAdbCommandLength)' == '' ">4096 + + @@ -331,6 +351,7 @@ Copyright (C) 2016 Xamarin. All rights reserved. <_EnvironmentFiles Include="@(AndroidEnvironment);@(LibraryEnvironments)" /> + { + ProfileTask (builder, "FastDeploy2", 20, b => { b.Uninstall (proj); b.Install (proj); });