Fix hard-coded venv path in Gateway property templates

docs/itential_gateway_guide.md

@@ -296,8 +296,13 @@ You can also use the following tags:
 * `install_gateway_build_packages` - Only install Gateway build packages
 * `install_python_dependencies` - Only install Python dependencies
 * `configure_ansible` - Only configure Ansible collections and config for IAG
+* `configure_gateway` - Only set venv ownership/permissions, render `properties.yml`, and write the `automation-gateway.service` unit
 * `uninstall_gateway_build_packages` - Only uninstall Gateway build packages
 
+**Note:** The task that restarts and enables the `automation-gateway` service is tagged `always`, so it
+runs on every execution of the playbook regardless of which `--tags`/`--skip-tags` you pass (unless you
+explicitly `--skip-tags always`).
+
 To execute only certificate management tasks:
 
 ```bash

roles/gateway/CLAUDE.md

@@ -18,17 +18,17 @@ Installs and configures Itential Automation Gateway (IAG). Handles Python virtua
 10. Install build packages (`gateway_build_packages`) — online only (removed in `always` block)
 11. Install Python dependencies (`install-python-dependencies.yml`)
 12. Configure Ansible (when `gateway_enable_ansible: true`): install collections → create `ansible.cfg` → create empty inventory/vault files
-13. Check if IAG is already installed (`stat` on `venv/automation-gateway`)
+13. Check if IAG is already installed (`stat` on `venv/bin/automation-gateway`)
 14. Copy or download IAG wheel file when not already installed
 15. `pip install` IAG wheel into `gateway_install_dir/venv`
-16. Set ownership (`chown -R`) and permissions (`chmod -R 775`) on venv
-17. Create `properties.yml` from versioned template
+16. Set ownership (`chown -R`) and permissions (`chmod -R 775`) on venv (tagged `configure_gateway`)
+17. Create `properties.yml` from versioned template (tagged `configure_gateway`)
 18. Create Nornir inventory/config files (when `gateway_enable_nornir: true`)
-19. Write `automation-gateway.service` systemd unit
+19. Write `automation-gateway.service` systemd unit (tagged `configure_gateway`)
 20. Open firewalld port (HTTP or HTTPS depending on `gateway_https_enabled`)
 21. `configure-selinux.yml`
 22. Copy test scripts to `gateway_install_dir/scripts/`
-23. Start and enable `automation-gateway` service
+23. Start and enable `automation-gateway` service (tagged `always` — runs even when other tags are selected via `--tags`)
 24. `update-release-file.yml`
 25. Remove temp working directory
 26. `always` block: remove build packages that were installed; assert service is active
@@ -176,8 +176,10 @@ Only one of `gateway_whl_file` or `gateway_archive_download_url` should be set.
 ## Gotchas
 
 - Build packages (`gcc`, `libssh-devel`, `make`, etc.) are installed to compile Python wheel dependencies, then removed in the `always` block — even if the play fails. This cleanup runs unconditionally.
-- The IAG install is skipped if `{{ gateway_install_dir }}/venv/automation-gateway` already exists (`stat` check). To force reinstall, remove this file/symlink first.
+- The IAG install is skipped if `{{ gateway_python_venv }}/bin/automation-gateway` already exists (`stat` check). To force reinstall, remove this file/symlink first.
+- The "Set ownership/permissions and create properties.yml" block and the "Write automation-gateway.service to host" task are no longer gated by `not gateway_installed.stat.exists` — they now run on every playbook execution (tagged `configure_gateway`), so `properties.yml` and the systemd unit are regenerated and ownership/permissions are reapplied on re-runs, not just on first install.
 - `gateway_http_server_threads` defaults to `ansible_processor_cores * 4`. On systems where `ansible_processor_cores` is unavailable or 0, this will produce 0 threads. Verify the value is sensible on target hardware.
 - `gateway_pki_copy_certs: true` runs unconditionally when set, regardless of `gateway_https_enabled`. This means if you disable HTTPS but leave `gateway_pki_copy_certs: true`, the role will still try to copy certs (and fail if `gateway_pki_src_dir` is empty).
 - The Ansible `ansible.cfg` written to `/etc/ansible/ansible.cfg` sets the collections path to `gateway_ansible_collections_path`. If `/etc/ansible/ansible.cfg` already exists, it is backed up but replaced.
 - The `gateway_release` variable maps to both a vars file (`vars/gateway-release-<N>.yml`) and a properties template (`templates/properties.<N>.yml.j2`). Adding a new release requires both files.
+- `gateway_venv_name` is a fixed name (`venv`) shared across all releases — it is not suffixed with `gateway_release`. Because different releases pin different `gateway_python_version` values (e.g. `3.9` for 4.3, `3.12` for 4.4), `install-python.yml` checks the existing venv's Python version on every run. If it doesn't match the release's `gateway_python_version`, the existing venv is moved to a `.bak.py<old_version>.<timestamp>` sibling path and a fresh venv is created, which also causes the IAG install/properties steps in `main.yml` to re-run (since the `{{ gateway_python_venv }}/bin/automation-gateway` stat check will report not-installed). This logic only runs via the main install flow (`main.yml` → `install-python.yml`) — `upgrade-gateway.yml` (used by `patch_gateway.yml`) does not check or rebuild the venv, consistent with patch upgrades not supporting major version changes.

roles/gateway/defaults/main/gateway.yml

@@ -56,5 +56,5 @@ gateway_tlsv1_2: false
 gateway_http_server_threads: "{{ ansible_processor_cores * 4 }}"
 
 # Python virtual environment settings
-gateway_venv_name: "venv_gateway_{{ gateway_release }}"
+gateway_venv_name: venv
 gateway_python_venv: "{{ gateway_install_dir }}/{{ gateway_venv_name }}"

roles/gateway/tasks/install-python.yml

@@ -19,6 +19,41 @@
         offline_rpms_path: "{{ gateway_offline_control_node_rpms_dir }}/python"
       when: offline_install_enabled | bool
 
+    - name: Check for existing Gateway Python virtual environment
+      ansible.builtin.stat:
+        path: "{{ gateway_python_venv }}/bin/python3"
+      register: gateway_venv_python_stat
+
+    - name: Backup Gateway virtual environment on Python version change
+      when: gateway_venv_python_stat.stat.exists
+      block:
+        - name: Get Python version used by the existing virtual environment
+          ansible.builtin.command:
+            cmd: "{{ gateway_python_venv }}/bin/python3 -V"
+          register: gateway_existing_python_version_raw
+          changed_when: false
+
+        - name: Set fact for existing virtual environment Python version
+          ansible.builtin.set_fact:
+            gateway_existing_python_version: '{{ gateway_existing_python_version_raw.stdout
+              | regex_search("[0-9]+\.[0-9]+") }}'
+
+        - name: Move existing virtual environment to a backup path
+          when: gateway_existing_python_version != gateway_python_version | string
+          block:
+            - name: Set backup path for existing virtual environment
+              ansible.builtin.set_fact:
+                gateway_venv_backup_path: "{{ gateway_python_venv }}.bak.py{{ gateway_existing_python_version
+                  }}.{{ ansible_date_time.iso8601_basic_short }}"
+
+            - name: Move existing virtual environment to backup path
+              ansible.builtin.command:
+                argv:
+                  - mv
+                  - "{{ gateway_python_venv }}"
+                  - "{{ gateway_venv_backup_path }}"
+              changed_when: true
+
     - name: Setup Python virtual environment
       ansible.builtin.command:
         chdir: "{{ gateway_install_dir }}"

roles/gateway/tasks/main.yml

@@ -195,7 +195,7 @@
 
     - name: Check if Automation Gateway is already installed
       ansible.builtin.stat:
-        path: "{{ gateway_python_venv }}/automation-gateway"
+        path: "{{ gateway_python_venv }}/bin/automation-gateway"
       register: gateway_installed
 
     - name: Propagate IAG archive
@@ -252,7 +252,7 @@
       when: not offline_install_enabled | bool
 
     - name: Set ownership/permissions and create properties.yml
-      when: not gateway_installed.stat.exists
+      tags: configure_gateway
       block:
         # Using chown and chmod is a faster way to enforce the file ownership and
         # permissions. The file module in ansible checks each and every file/dir
@@ -305,6 +305,7 @@
         dest: /etc/systemd/system/automation-gateway.service
         mode: "0644"
         backup: true
+      tags: configure_gateway
 
     # Check if firewalld is running, if it is then open the appropriate ports
     - name: Gather service facts
@@ -359,6 +360,7 @@
         enabled: true
         state: restarted
         daemon_reload: true
+      tags: always
 
     - name: Update release file
       ansible.builtin.include_tasks:

roles/gateway/templates/properties.2021.1.yml.j2

@@ -116,18 +116,18 @@ no_cleanup: false
 # NOTE: Use only the site-packages paths you need for your installation to avoid
 #       cross environment issues in the case where multiple of these paths exist
 module_path:
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/ansible/modules/network"
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/ansible/modules"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/ansible/modules/network"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/ansible/modules"
 
 # Path(s) to the Ansible collections that should be discovered by Automation Gateway and exclusively used in Ansible's execution environment.
 # Due to differences in collections before/after Ansible 2.9, these will be the only paths relevant during discovery AND execution.
 collection_path:
   - "{{ gateway_install_dir }}/ansible/collections"
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/ansible_collections"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/ansible_collections"
 
 # Path(s) to the Ansible roles that should be discovered by Automation Gateway and appended to Ansible's execution environment.
 role_path:
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/automation_gateway/integrations/roles"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/automation_gateway/integrations/roles"
 
 # Discovery behavior for Ansible playbooks. Determines whether or not to
 # recursively search the directories found in the 'playbook_path' parameter, or
@@ -136,7 +136,7 @@ playbook_recursive: true
 
 # Path(s) to the Ansible playbooks that should be discovered by Automation Gateway and appended to Ansible's execution environment.
 playbook_path:
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/automation_gateway/integrations/playbooks"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/automation_gateway/integrations/playbooks"
 
 #################
 # HTTP_Requests #

roles/gateway/templates/properties.2021.2.yml.j2

@@ -128,23 +128,23 @@ inventory_file: "{{ gateway_install_dir }}/ansible/inventory/hosts"
 # NOTE: Use only the site-packages paths you need for your installation to avoid
 #       cross environment issues in the case where multiple of these paths exist
 module_path:
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/ansible/modules/network"
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/ansible/modules"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/ansible/modules/network"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/ansible/modules"
 
 # Path(s) to the Ansible collections that should be discovered by Automation Gateway and exclusively used in Ansible's execution environment.
 # Due to differences in collections before/after Ansible 2.9, these will be the only paths relevant during discovery AND execution.
 collection_path:
   - "{{ gateway_install_dir }}/ansible/collections"
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/ansible_collections"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/ansible_collections"
 
 # Path(s) to the Ansible roles that should be discovered by Automation Gateway and appended to Ansible's execution environment.
 role_path:
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/automation_gateway/integrations/roles"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/automation_gateway/integrations/roles"
 
 # Path(s) to customized roles that extend device support of the Itential roles found in the release, i.e., itential_cli, itential_get_config.
 #extended_device_role_path:
   #This is a sample path to roles that extend device support
-  #- "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/automation_gateway/integrations/extensible_device_roles"
+  #- "{{ gateway_python_venv }}/lib/python3.9/site-packages/automation_gateway/integrations/extensible_device_roles"
 
 
 # Discovery behavior for Ansible playbooks. Determines whether or not to
@@ -154,7 +154,7 @@ playbook_recursive: true
 
 # Path(s) to the Ansible playbooks that should be discovered by Automation Gateway and appended to Ansible's execution environment.
 playbook_path:
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/automation_gateway/integrations/playbooks"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/automation_gateway/integrations/playbooks"
 
 #################
 # HTTP_Requests #

roles/gateway/templates/properties.2022.1.yml.j2

@@ -128,23 +128,23 @@ inventory_file: "{{ gateway_install_dir }}/ansible/inventory/hosts"
 # NOTE: Use only the site-packages paths you need for your installation to avoid
 #       cross environment issues in the case where multiple of these paths exist
 module_path:
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/ansible/modules/network"
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/ansible/modules"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/ansible/modules/network"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/ansible/modules"
 
 # Path(s) to the Ansible collections that should be discovered by Automation Gateway and exclusively used in Ansible's execution environment.
 # Due to differences in collections before/after Ansible 2.9, these will be the only paths relevant during discovery AND execution.
 collection_path:
   - "{{ gateway_install_dir }}/ansible/collections"
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/ansible_collections"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/ansible_collections"
 
 # Path(s) to the Ansible roles that should be discovered by Automation Gateway and appended to Ansible's execution environment.
 role_path:
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/automation_gateway/integrations/roles"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/automation_gateway/integrations/roles"
 
 # Path(s) to customized roles that extend device support of the Itential roles found in the release, i.e., itential_cli, itential_get_config.
 #extended_device_role_path:
   #This is a sample path to roles that extend device support
-  #- "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/automation_gateway/integrations/extensible_device_roles"
+  #- "{{ gateway_python_venv }}/lib/python3.9/site-packages/automation_gateway/integrations/extensible_device_roles"
 
 # Discovery behavior for Ansible playbooks. Determines whether or not to
 # recursively search the directories found in the 'playbook_path' parameter, or
@@ -153,7 +153,7 @@ playbook_recursive: true
 
 # Path(s) to the Ansible playbooks that should be discovered by Automation Gateway and appended to Ansible's execution environment.
 playbook_path:
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/automation_gateway/integrations/playbooks"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/automation_gateway/integrations/playbooks"
 
 #################
 # HTTP_Requests #

roles/gateway/templates/properties.2023.1.yml.j2

@@ -249,7 +249,7 @@ inventory_file: "{{ gateway_install_dir }}/ansible/inventory/hosts"
 # NOTE: Use only the site-packages paths you need for your installation to avoid
 #       cross environment issues in the case where multiple of these paths exist
 module_path:
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/ansible/modules"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/ansible/modules"
   - "{{ gateway_install_dir }}/ansible/modules"
 
 # Path(s) to the Ansible collections that should be discovered by Automation Gateway and exclusively used in Ansible's execution environment.
@@ -259,13 +259,13 @@ collection_path:
 
 # Path(s) to the Ansible roles that should be discovered by Automation Gateway and appended to Ansible's execution environment.
 role_path:
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/automation_gateway/integrations/roles"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/automation_gateway/integrations/roles"
   - "{{ gateway_install_dir }}/ansible/roles"
 
 # Path(s) to customized roles that extend device support of the Itential roles found in the release, i.e., itential_cli, itential_get_config.
 #extended_device_role_path:
   #This is a sample path to roles that extend device support
-  #- "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/automation_gateway/integrations/extensible_device_roles"
+  #- "{{ gateway_python_venv }}/lib/python3.9/site-packages/automation_gateway/integrations/extensible_device_roles"
 
 # Discovery behavior for Ansible playbooks. Determines whether or not to
 # recursively search the directories found in the 'playbook_path' parameter, or
@@ -274,7 +274,7 @@ playbook_recursive: true
 
 # Path(s) to the Ansible playbooks that should be discovered by Automation Gateway and appended to Ansible's execution environment.
 playbook_path:
-  - "{{ gateway_install_dir }}/venv/lib/python3.9/site-packages/automation_gateway/integrations/playbooks"
+  - "{{ gateway_python_venv }}/lib/python3.9/site-packages/automation_gateway/integrations/playbooks"
   - "{{ gateway_install_dir }}/ansible/playbooks"
 
 #################

roles/gateway/templates/properties.2023.2.yml.j2

@@ -249,7 +249,7 @@ inventory_file: "{{ gateway_install_dir }}/ansible/inventory/hosts"
 # NOTE: Use only the site-packages paths you need for your installation to avoid
 #       cross environment issues in the case where multiple of these paths exist
 module_path:
-  - "{{ gateway_install_dir }}/venv/lib/python{{ gateway_python_version }}/site-packages/ansible/modules"
+  - "{{ gateway_python_venv }}/lib/python{{ gateway_python_version }}/site-packages/ansible/modules"
   - "{{ gateway_install_dir }}/ansible/modules"
 
 # Path(s) to the Ansible collections that should be discovered by Automation Gateway and exclusively used in Ansible's execution environment.
@@ -259,13 +259,13 @@ collection_path:
 
 # Path(s) to the Ansible roles that should be discovered by Automation Gateway and appended to Ansible's execution environment.
 role_path:
-  - "{{ gateway_install_dir }}/venv/lib/python{{ gateway_python_version }}/site-packages/automation_gateway/integrations/roles"
+  - "{{ gateway_python_venv }}/lib/python{{ gateway_python_version }}/site-packages/automation_gateway/integrations/roles"
   - "{{ gateway_install_dir }}/ansible/roles"
 
 # Path(s) to customized roles that extend device support of the Itential roles found in the release, i.e., itential_cli, itential_get_config.
 #extended_device_role_path:
   #This is a sample path to roles that extend device support
-  #- "{{ gateway_install_dir }}/venv/lib/python{{ gateway_python_version }}/site-packages/automation_gateway/integrations/extensible_device_roles"
+  #- "{{ gateway_python_venv }}/lib/python{{ gateway_python_version }}/site-packages/automation_gateway/integrations/extensible_device_roles"
 
 # Discovery behavior for Ansible playbooks. Determines whether or not to
 # recursively search the directories found in the 'playbook_path' parameter, or
@@ -274,7 +274,7 @@ playbook_recursive: true
 
 # Path(s) to the Ansible playbooks that should be discovered by Automation Gateway and appended to Ansible's execution environment.
 playbook_path:
-  - "{{ gateway_install_dir }}/venv/lib/python{{ gateway_python_version }}/site-packages/automation_gateway/integrations/playbooks"
+  - "{{ gateway_python_venv }}/lib/python{{ gateway_python_version }}/site-packages/automation_gateway/integrations/playbooks"
   - "{{ gateway_install_dir }}/ansible/playbooks"
 
 #################