tidy and consolidate CI

[anish-mudaraddi]

Description

setup distinct CI jobs that can be reused.

1. separate image build jobs from testing and linting jobs to promote code reuse.

2. "Create Dev Image" action runs when a PR is created - to build images and upload them to harbor in the stfc-cloud-staging project.

3. "Create Release Image" action runs when a PR is merged into master

4. These actions use scripts in .github/scripts/build_dev_image.sh and .github/scripts/build_release_image.sh which can be invoked locally for testing purposes

5. See updated readme on how to use the new CI jobs

Motivation

Build jobs and CI are complex - there's a lot of repeated code between CI jobs - ultimately performing similar build jobs
To make it easier to add new docker images into this repo - "generic" build scripts need to be created to build and push images without explicitly needing to write another CI action.

To add new docker images - we now only need to write an action to test and lint source code - much easier to write and maintain

Changelog

simplified the CI such that:

1. have 2 scripts to handle building and pushing images - used by github action and can be invoked locally for testing. Easy to test the scripts outside of github actions

2. created a shared action that builds and pushes a dev image when a PR is opened that changes the contents of a image subdirectory.

3. created an action that build a new release image when a PR is merged to master

4. modified existing CI that would regularly publish a PR bumping the patch version for rabbit-consumer docker image. Made this apply to all docker images so that we can force dependencies to updated and the docker build step

5. simplified individual image CI jobs to just testing and linting - not also building the images. This reduces duplicated code.

Submitter:

Have you done the following?:

• Labeled the pull request from the following? major | minor | patch or documentation | workflow
• Updated the documentation?
• Added unit tests for new / untested code?
• Deployed the changes onto dev-chatops.nubes.rl.ac.uk or staging for this new image for code changes?

New / Existing features:

• Written integration tests?
• Configured the Slack Application with new commands or permission scope changes?
• Updated the version.txt and docker-compose.yml files?

Have you checked the following?:

• Version change is appropriate to the code changes? Semantic versioning documentation here
• Unit test code coverage is acceptable?
• Do the CI jobs pass?

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.14%. Comparing base (62bf4c7) to head (7fe0afc).
⚠️ Report is 19 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #175      +/-   ##
==========================================
+ Coverage   97.24%   99.14%   +1.89%     
==========================================
  Files          16       32      +16     
  Lines        1198     1871     +673     
==========================================
+ Hits         1165     1855     +690     
+ Misses         33       16      -17     

Flag	Coverage Δ
rabbit_consumer	?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.