Complete shell assignment

.github/pull_request_template.md

@@ -1,3 +1,5 @@
+<!-- PULL REQUEST TITLE: UofT-DSI | <Module Name> - Assignment <assignment number>-->
+
 ## What changes are you trying to make? (e.g. Adding or removing code, refactoring existing code, adding reports)
 
 ## What did you learn from the changes you have made?

.github/workflows/autograder.yml

@@ -0,0 +1,48 @@
+name: Assignment autograder
+on: 
+  pull_request:
+    branches:
+      - main
+    types: [opened, synchronize, reopened]
+
+  # Allows you to run this workflow manually from the Actions tab
+  workflow_dispatch:
+
+permissions:
+  # Read repository contents for checkout and grading.
+  contents: read
+  # Pull request comments use the Issues API.
+  issues: write
+  # Required to create REQUEST_CHANGES reviews when checks fail.
+  pull-requests: write
+
+jobs:
+  autograder:
+    name: Assignment autograder
+    if: startsWith(github.head_ref, 'assignment') && github.repository_owner != 'UofT-DSI'
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+      - name: Setup working directory for script
+        run: |
+          UUID=$(uuidgen)
+          echo "UUID=$UUID" >> $GITHUB_ENV
+          mkdir /tmp/$UUID
+          cp $GITHUB_WORKSPACE/02_activities/assignments/assignment_script.sh /tmp/$UUID
+      - name: Run assignment script
+        run: |
+          bash -c "bash assignment_script.sh 2>&1 | tee /tmp/${{ env.UUID }}_output.txt"
+        working-directory: /tmp/${{ env.UUID }}
+      - name: Grade responses
+        id: grade
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          REPO_OWNER: ${{ github.repository_owner }}
+          REPO_NAME: ${{ github.event.repository.name }}
+          PR_NUMBER: ${{ github.event.pull_request.number }}
+          REPO_BRANCH: ${{ github.event.pull_request.head.ref }}
+          WORKING_DIR: /tmp/${{ env.UUID }}
+        run: |
+          wget -O /tmp/autograder.py https://github.com/UofT-DSI/shell/raw/refs/heads/main/03_instructional_team/autograder/autograder.py
+          uv run --python 3.12 --with pandas,tabulate,requests /tmp/autograder.py

.github/workflows/deploy_github_pages.yml

@@ -2,9 +2,7 @@
 name: Deploy static content to Pages
 
 on:
-  # Runs on pushes targeting the default branch
   push:
-    branches: ["main"]
 
   # Allows you to run this workflow manually from the Actions tab
   workflow_dispatch:
@@ -24,6 +22,9 @@ concurrency:
 jobs:
   # Single deploy job since we're just deploying
   deploy:
+    if: |
+      (github.repository == 'UofT-DSI/shell' && github.ref == 'refs/heads/main') ||
+      (github.repository != 'UofT-DSI/shell' && github.ref == 'refs/heads/githubpages')
     environment:
       name: github-pages
       url: ${{ steps.deployment.outputs.page_url }}

.gitignore

@@ -1,4 +1,29 @@
 .DS_Store
 .vscode/
-/03_instructional_team/node_modules
-.venv
+03_instructional_team/lessons/pdf/*
+03_instructional_team/lessons/html/*
+
+# local node modules
+node_modules/
+package-lock.json
+package.json
+
+### Running the assignment script unzips a large number of files into the tree
+# Ignore all files in shell/02_activities/assignments/
+02_activities/assignments/*
+
+# Un-ignore the specific files
+!02_activities/assignments/assignment.sh
+!02_activities/assignments/github_actions.png
+!02_activities/assignments/assignment_instructions.md
+
+# Python
+.venv/
+__pycache__/
+*.py[cod]
+*$py.class
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+
+newproject/

01_materials/bash_quick_reference.md

@@ -0,0 +1,84 @@
+# Bash Commands Reference
+
+## 📋 Commands Overview
+
+```bash
+$ echo Rachael
+Rachael
+
+$ date
+Tue, Nov 19, 2024 6:28:10PM
+
+$ asdkfjhgoipwhr
+Command not found
+
+$ man ls
+# Opens the manual for `ls`
+# To exit, press 'q'
+
+$ help
+# Lists available shell commands
+```
+
+---
+
+## 📁 File & Directory Navigation
+
+| Command                | Description                                                |
+|------------------------|------------------------------------------------------------|
+| `pwd`                 | Show current working directory                             |
+| `ls`                  | List folders & files in the working directory              |
+| `cd`                  | Move to home directory                                     |
+| `cd Desktop`          | Move into Desktop directory                                |
+| `cd <path>`           | Move into a specific path                                  |
+| `cd ..`               | Move one level up (parent directory)                       |
+
+---
+
+## 🛠 File & Directory Management
+
+| Command                             | Description                                         |
+|-------------------------------------|-----------------------------------------------------|
+| `mkdir <dir_name>`                 | Make new directories                                |
+| `touch <file_name>`               | Create a file                                       |
+| `rm <directory_name>`             | Remove a directory                                  |
+| `rm <file_name>`                  | Remove a file                                       |
+| `cp <source> <destination>`       | Copy file from source to destination                |
+| `cp <dir_source>/* <dir_dest>`    | Copy all files from source dir to destination dir   |
+| `mv <file1> <file2>`              | Rename/move file1 to file2                          |
+| `cat <file_name>`                 | Display content of the file                         |
+| `echo "text" > file.txt`          | Replace content in file with text                   |
+| `echo "text" >> file.txt`         | Append text to file                                 |
+
+---
+
+## ✏️ Editing Files
+
+| Command            | Description                        |
+|--------------------|------------------------------------|
+| `nano <file_name>` | Open file in nano text editor      |
+| `code <file_name>` | Open file in VS Code               |
+
+---
+
+## 🧰 Useful Options
+
+| Option | Description                                               |
+|--------|-----------------------------------------------------------|
+| `-i`   | Prompt before overwriting an existing file                |
+| `-r`   | Recursively copy directories and their contents           |
+| `-v`   | Show informative messages during operations               |
+| `-f`   | Force action (overrides `-i` if both are used together)   |
+
+---
+
+## 🔧 Hash-bang (Shebang)
+
+Place this at the top of every Bash script to indicate which shell to use:
+
+```bash
+#!/bin/bash
+```
+
+---
+

02_activities/assignments/assignment_instructions.md

@@ -0,0 +1,104 @@
+# Shell / Git Assignment
+You work in the data team at a consulting firm, and one of your team's products is helping companies optimize and manage their cloud hosting expenditures.
+
+Your team has an existing bash script that initializes an analysis directory for each new client. This script is shared amongst members of the team.
+
+You've been asked to update this script to also automate the initial organization of data files provided by clients.
+
+## Instructions
+
+### Setup
+1. **Forking the Repository**: If you have not already done so, fork this Shell learning module repository following these [instructions](https://github.com/UofT-DSI/onboarding/blob/main/onboarding_documents/submissions.md#setting-up). 
+   - Forking creates a copy of the main repository in your GitHub account. This allows you to work on your version without affecting the original repository.
+
+2. **Enable GitHub Actions**: Click on the **Actions** tab in your repository and enable workflows if prompted.  
+<img src="./github_actions.png" width="500px">
+
+2. **Clone Your Fork To Your Computer**: To make edits and test your script.
+   ```bash
+   cd <sensible directory on your computer>
+   git clone <url to your repo>
+   ```
+
+2. **Create a Branch for Your Work**: To keep your changes organized, create a new branch named `assignment_shell`:
+   ```bash
+   git switch -c assignment_shell
+   ```
+
+---
+
+### Part 1: Update the Data Ingest Script
+1. **Modify the Script**: Using the template in ``02_activities/assignments/assignment_script.sh``, fill in the correct commands as described by the comments.
+    - It may help to paste your commands into the Terminal as you write your script (or vice versa) to test as you go
+
+2. **Test Your Script Locally**: Execute your script to ensure it works as expected. You may need to make additional tweaks and re-run it until you are satisfied with the results.
+    ```bash
+    bash assignment_script.sh
+    ```
+
+   - Check if the expected directories are created. (eg. by browsing the directory with `cd` and `ls`)
+   - Verify that files are moved or copied as expected.
+   - Ensure that files that should be deleted are no longer present.
+   - Ensure your script works regardless
+
+3. **Commit Your Changes**: As you complete parts of your script and confirmed that it is working correctly, stage and commit your changes to ensure your progress is saved.
+    ```bash
+    git add assignment_script.sh
+    git commit
+    ```
+
+    - Remember that the script is shared amongst your team members. Your script should run properly in any directory and should not have any hardcoded absolute paths or rely on advanced functionality not available in standard Bash.
+    - *Note*: Your assignment is graded solely based on `assignment_script.sh`. 
+    - ‼️ You **must not** include any data files created during testing of your script and you **must not** change any other files.
+
+4. **Push Your Changes to GitHub**
+   ```bash
+   git status                   # check that your working directory is clean (all changes committed)
+   git push origin assignment_shell   # push your changes
+   ```
+   - ‼️ **Do not** push to `main`
+
+5. **Create a pull request**
+   - Open a pull request from your `assignment_shell` branch to your repository's `main` branch.
+   - The autograder will run automatically and post your assignment grade as a comment.
+   - Ensure the pull request is **NOT** made to the `UofT-DSI` organization
+   - ‼️ **Do not merge** your pull request until the end of the module. Your teaching team will only grade *open* pull requests.
+
+6. **Repeat steps 1-4 as needed**: As you make tweaks to your assignment, push new changes to your `assignment_shell` branch to rerun the autograder
+   - There's no need to create a new pull request
+   - Pushes to the `assignment_shell` branch will trigger additional runs
+
+---
+
+### Part 2: Merge in Updates from Your Coworkers
+Your coworkers have made some other changes to the script. You'll need to incorporate their updates and resolve any conflicts.
+1. **Merge the Updates**: Use the following command to pull in changes from the coworker's branch:
+   ```bash
+   git pull https://github.com/UofT-DSI/shell coworker-changes --no-rebase
+   ```
+2. **Resolve Merge Conflicts**: If there are any conflicts, use ```git status``` to see which files are affected, resolve the conflicts manually, and then mark them as resolved.
+3. **Commit the Merge**: Once all conflicts are resolved, commit the merge.
+1. **Re-test Your Script**: Make sure that your script still works after merging the updates and make any changes as necessary
+
+---
+
+## Submission Information
+
+🚨 **Please review our [Assignment Submission Guide](https://github.com/UofT-DSI/onboarding/blob/main/onboarding_documents/submissions.md)** 🚨 for detailed instructions on how to format, branch, and submit your work. Following these guidelines is crucial for your submissions to be evaluated correctly.
+
+### Grading guidelines:
+This assignment is considered complete if all autograder tests pass successfully and your Pull Request is approved by the github-actions bot, and incomplete otherwise.
+
+### Submission Parameters:
+* The branch name for your repo should be: `assignment_shell`
+* What to submit for this assignment:
+    * One or more commits that update the `assignment_script.sh` script
+* What the pull request link should look like for this assignment: `https://github.com/<your_github_username>/shell/pull/<pr_id>`
+    * Open a private window in your browser. Copy and paste the link to your pull request into the address bar. Make sure you can see your pull request properly. This helps the technical facilitator and learning support staff review your submission easily.
+
+Checklist:
+- [ ] Create a branch called `assignment_shell`.
+- [ ] Ensure that your repository is public.
+- [ ] Review [the PR description guidelines](https://github.com/UofT-DSI/onboarding/blob/main/onboarding_documents/submissions.md#guidelines-for-pull-request-descriptions) and adhere to them.
+- [ ] Verify that your link is accessible in a private browser window.
+- [ ] Verify the autograder has run and approved your pull request

02_activities/assignments/assignment_script.sh

@@ -0,0 +1,50 @@
+#!/bin/bash
+set -x
+
+############################################
+# DSI CONSULTING INC. Project setup script #
+############################################
+# This script creates standard analysis and output directories
+# for a new project. It also creates a README file with the
+# project name and a brief description of the project.
+# Then it unzips the raw data provided by the client.
+
+if [ -d newproject ]; then
+  echo "Recreating the newproject directory"
+  rm -rf newproject
+fi
+mkdir newproject
+cd newproject
+
+mkdir analysis output
+touch README.md
+echo "# Project Name: DSI Consulting Inc." > README.md
+touch analysis/main.py
+
+# download client data
+curl -Lo rawdata.zip https://github.com/UofT-DSI/shell/raw/refs/heads/main/02_activities/assignments/rawdata.zip
+unzip -q rawdata.zip
+
+
+###########################################
+# Complete assignment here
+
+mkdir data
+mv rawdata data/raw
+ls data/raw
+
+mkdir data/processed
+mkdir data/processed/server_logs data/processed/user_logs data/processed/event_logs
+
+cp data/raw/*server*.log data/processed/server_logs/
+cp data/raw/*user_log_*.log data/processed/user_logs/
+cp data/raw/*event_log_*.log data/processed/event_logs/
+
+rm data/raw/*ipaddr*
+rm data/processed/user_logs/*ipaddr*
+
+find data/processed -type f > data/inventory.txt
+
+###########################################
+
+echo "Project setup is complete!"