Repository navigation
Fix URL SCM mirror upload - #724
Merged
Merged
Conversation
Allow the caller to pass additional request headers to openDownload(). This enables conditional downloads with "If-Modified-Since".
A "304 Not Modified" response is reported as WebdavNotModifiedError so that it can be distinguished from real errors.
Replace the hand-rolled urllib requests for http and https URLs by the WebDav class. This adds support for HTTP basic authentication with the credentials embedded in the URL and keeps them out of error messages. Uploads are done with "If-None-Match" now. Losing the race against a concurrent upload is not an error. The ftp downloads still use urllib directly. The retries are still handled by UrlScm to keep the delay between the attempts and the warnings.
Add optional HTTP basic authentication to the HTTP server mock. Use it to verify that UrlScm downloads and mirror uploads work with credentials embedded in the URL. The password contains reserved characters to check the proper unquoting. Wrong credentials must fail.
Quoting RFC 4918: A PUT that would result in the creation of a resource without an appropriately scoped parent collection MUST fail with a 409 (Conflict). Like the HttpArchive, make sure that the parent directories exist before uploading a file to a HTTP mirror.
Let the HTTP server mock fail a PUT with 409 if the parent collection is missing, as mandated by RFC 4918, instead of silently creating it. Add a test that uploads to a HTTP mirror into a missing sub-directory.
Uploading to a mirror might not always be possible, e.g. if the mirror is not always reachable or writable. Add an "ignoreUploadErrors" property to the UrlScm mirror entries. If set, a failed upload is only reported as warning instead of failing the build. As a side effect, errors when copying to local mirrors are reported properly instead of being raised as unhandled exceptions.
Verify that failed uploads to HTTP and local mirrors fail the build by default and that they are ignored if "ignoreUploadErrors" is set.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #724 +/- ##
==========================================
- Coverage 89.96% 89.96% -0.01%
==========================================
Files 50 50
Lines 16993 17022 +29
==========================================
+ Hits 15287 15313 +26
- Misses 1706 1709 +3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The HTTP upload of URL mirrors was not spec compliant. We cannot rely on the server to create the parent directories.
While at it, add an
ignoreUploadErrorsproperty to make error non-fatal.