Repository navigation
Setting up Azure pipelines #1
Description
Activity
Checking in, that all sounds good. 👍
Reacted by Amin YaSomeone at upstream was kind enough to put my branches up at upstream and run it themselves. I'll look into if there is a free tier of Azure Pipelines though, so we can set it up here without paying some sort of subscription.
Edit: This looks like the place to do it. "Start free with GitHub" https://azure.microsoft.com/en-us/services/devops/pipelines/
Kinda makes sense now that Microsoft owns Github...
Reacted by Amin YaSomeone at upstream was kind enough to put my branches up at upstream and run it themselves. I'll look into if there is a free tier of Azure Pipelines though, so we can set it up here without paying some sort of subscription.
Edit: This looks like the place to do it. "Start free with GitHub" https://azure.microsoft.com/en-us/services/devops/pipelines/
Kinda makes sense now that Microsoft owns Github...
There is a free tier, but how do we get the same configuration that upstream is using? Is there a duplicate button or something, or is there a config file we need to use somewhere? I could not find anything in the repository itself.
Ah yeah, the config is in a pretty obscure place. I only found it because it mentions Python, and I did some PRs about being ready for Python3.
https://github.com/atom/atom/tree/master/script/vsts
(VSTS is short for Visual Studio Team System, which got renamed to Azure Devops Server... according to Wikipedia.)According to upstream's nice
README.mdfor the CI, VSTS = Visual Studio Team Services, which dot renamed to Azure DevOps.Reacted by Amin YaParticularly here: https://github.com/atom/atom/tree/master/script/vsts/platforms
Reacted by Amin YaEditing this as I go to be as complete as I can manage:
- Visit here https://azure.microsoft.com/services/devops/pipelines/ and press "Start free with GitHub"
- If you already have a Microsoft account, and you don't want to make one linked to your GitHub account, you can just press "Start free".
- Visit https://dev.azure.com
- Click the "+ New project" button (it's blue for me)
- Fill in some details (name, description) and press "Create"
- In the new project, click "Pipelines" in the side nav area
- Press "New pipeline"
- It asks you "Where is your code"
- Press "GitHub"
- Choose this repo
atom-ide-community:atom - Authenticate Azure Pipelines to GitHub via OAuth
- Jump through a couple more hoops of authentication and which accounts to use to authorize, etc.
- Select the
atomrepository - Press "Existing Azure Piplines YAML File"
- Find the three config files in
script/vsts/*.yml- Add all three of these one by one
- Install tasks if necessary, via https://marketplace.visualstudio.com/
- It asks you "Where is your code"
- After adding the three pipelines ("Atom Nightly", "Atom Production Branches", and "Atom Pull Requests"), manual tweaking might be required.
- They all seem to have "Pull Request" and "Continuous Integration" triggers enabled by default, even though they shouldn't.
- Visit the Pipelines view, from the Pipelines sidebar nav entry
- Press the specific pipeline you want to adjust
- Press "Edit"
- Press the "vertical three dots" button (this appears gray for me with black dots)
- Press "Triggers" from the dropdown menu
- Depending on which pipeline you are editing, use the "Triggers" tab to:
- disable the "Pull request validation" trigger (for the "Atom Production Branches" pipeline)
- disable the "Continuous integration" trigger (for the "Atom Pull requests" pipeline)
- disable both of these triggers and add a scheduled trigger for every night at midnight (for the "Atom Nightly" pipeline)
- Optionally use the "YAML" tab to rename the pipeline and make sure the correct yaml file is selected.
- Don't change the "Default agent pool for YAML" setting for now, or it might break subsequent runs at the moment.
- Press the "Save & queue" button, then "Save" or "Save & queue" to save these changes and update the pipeline.
- Optionally enter a message explaining the edits you made to the pipeline settings.
- Visit here https://azure.microsoft.com/services/devops/pipelines/ and press "Start free with GitHub"
I have created the repository: https://dev.azure.com/atomcommunity/atomcommunity
I am now trying to see how I can set up build using the files in VSTs 🤔I'm curious what happens if you add a pipeline from the side nav bar area, and after authenticating to GitHub with OAuth so it can officially link up with/access this repo. (Trying to write out the instructions in my above comment, but that's where I get stuck.)
Reacted by Amin YaI'm trying it on my own personal fork now to see how far I can get and update my instructions/steps above.
I added pull requests and release builds for now: https://dev.azure.com/atomcommunity/atomcommunity/_build.
We might have to edit the release configuration once we wanted to release things with the new name.Reacted by DeeDeeGI'm a bit confused about what this means, but there are "Missing tasks" required to run the CI, and supposedly these are installable via https://marketplace.visualstudio.com/ according to the error message.
Hmm, maybe we need to get some VMs running? In the Pipelines sidebar there is a subsection "Environments", which is empty by default.Disregard, I'll update if I figure something new out.
I created the "starter pipeline" with no errors.
Full starter pipeline yaml (click to expand)
# Starter pipeline # Start with a minimal pipeline that you can customize to build and deploy your code. # Add steps that build, run tests, deploy, and more: # https://aka.ms/yaml trigger: - master pool: vmImage: 'ubuntu-latest' steps: - script: echo Hello, world! displayName: 'Run a one-line script' - script: | echo Add other tasks to build, test, and deploy your project. echo See https://aka.ms/yaml displayName: 'Run a multi-line script'
I can't tell if it's stalled waiting on an Azure Pipelines worker/VM, or because there are no actual steps in the script, so it never finishes... https://dev.azure.com/DeeDeeG/b/_build/results?buildId=3&view=logs&j=12f1170f-54f2-53f3-20dd-22fc7dff55f9
Now to read the docs and see if I can make one that actually does something. Working toward eventually running the Linux/Windows/macOS tests from upstream Atom.
66 remaining items
Now that #46 is merged, and the node/npm install parts of CI aren't in
platforms/$(os), we should probably addplatforms/templates/preparation.ymlandbootstrap.ymlto the cache identifier.This is for the hypothetical case where we have updated Node, NPM, or some environment variables or config relevant to how the bootstrap and build should proceed.
CI isn't passing on
master. Hmm. Not sure why.- linked a pull request that will close this issueCI: GITHUB_TOKEN to fix vscode-ripgrep downloading issues #45
on Jul 25, 2020 - linked a pull request that will close this issuecollect console log in renderer tests #63
on Jul 25, 2020 Glad that CI is working.
We should disable or delete the CI steps that try to publish artifacts to Amazon S3 buckets, since we don't own an Amazon S3 account, and that step has been erroring out at the very end, making our "Release Branch Build" pipeline always look red.
Other than that, CI is functionally working (tests passing) so that is great, thank you!
Edit: This is a PR now: #66
Doing
npm installis slower on Windows, due to NTFS's poor performance writing lots of small files versus EXT4.Interesting discussion here:
https://github.com/rust-lang/rustup/issues/1540If possible/if it's okay with you, I'd like to revert #3 (or explicitly set to use Linux) for the
GetReleaseVersionandReleaseCI jobs, and use forward-slashes for cross-platform/Linux compatibility: DeeDeeG@fbc3742Status: Will make a PR once I'm done with the stuff from my comment directly above this one.
Reacted by Amin YaThat sounds like a PR. We will not revert things on master anymore. It is OK if you want to do it in your PR.
Reacted by DeeDeeG- added a commit that references this issue
on Aug 11, 2020 Linux Build in CI has been failing by exiting out after the two package formats are apparently successfully built.
I also had a similar experience outside of CI on my personal machine, where the
dpkg-debcommand building the .deb package errors out.I think this is flakiness, not a hard "100% of the time" issue, but it's still weird.
Also, just a heads up that there are some more hard-coded URLs pointing to
github.com/atom/atomin various parts of the repo, and semi-relatedly, if we fall behind with which tags are on ouratom-nightly-releasesrepo vs upstream'satom-nightly-relleasesrepo, it can cause the Windows build to fail on building differential/"update from previous version" partial .nupkg files.I do think if we fix the hard-coded URLs like we did in #70, then this build failure scenario should also go away when using non-hard-coded URLs pointing to our own repos.
I will close this as this is mostly done.
We need to set up Azure pipelines to get CI similar to upstream.