Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
name: Bug report
description: Report a reproducible Connector Hub defect.
title: "[Bug]: "
labels: [bug]
body:
- type: markdown
attributes:
value: Never include credentials, tokens, private URLs, or customer data.
- type: textarea
id: description
attributes:
label: Description
description: What happened, and what did you expect?
validations:
required: true
- type: textarea
id: reproduction
attributes:
label: Reproduction
description: Provide the smallest safe sequence of commands and redacted output.
validations:
required: true
- type: input
id: version
attributes:
label: Version or commit
validations:
required: true
- type: dropdown
id: integration
attributes:
label: Uses an external integration?
options: ["No", "Yes"]
validations:
required: true
5 changes: 5 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
blank_issues_enabled: false
contact_links:
- name: Security vulnerability
url: https://github.com/CodeWithJuber/connector-hub/security/advisories/new
about: Report vulnerabilities privately; do not open a public issue.
20 changes: 20 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
version: 2
updates:
- package-ecosystem: pip
directory: "/"
schedule:
interval: weekly
day: monday
open-pull-requests-limit: 5
groups:
python-dependencies:
patterns: ["*"]
- package-ecosystem: github-actions
directory: "/"
schedule:
interval: weekly
day: monday
open-pull-requests-limit: 5
groups:
github-actions:
patterns: ["*"]
17 changes: 17 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
## Summary

<!-- Explain the user-visible outcome and why this change is needed. -->

## Validation

- [ ] Formatting and lint checks pass.
- [ ] Type checks pass.
- [ ] Unit tests pass.
- [ ] Integration tests were run, or are not applicable.
- [ ] Documentation and data-source references are updated.

## Security and operations

- [ ] No credentials, tokens, customer data, or generated `.env` files are included.
- [ ] New network calls have timeouts, bounded retries, validation, and rate-limit handling.
- [ ] Breaking changes, deployment steps, and rollback steps are documented.
77 changes: 64 additions & 13 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,37 +9,88 @@ on:
permissions:
contents: read

concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
quality:
name: Quality / Python ${{ matrix.python-version }}
runs-on: ubuntu-24.04
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
python-version: ["3.11", "3.14"]
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: astral-sh/setup-uv@d0d8abe699bfb85fec6de9f7adb5ae17292296ff # v6
with:
version: "0.8.4"
python-version: ${{ matrix.python-version }}
enable-cache: true
cache-dependency-glob: uv.lock
- name: Install locked dependencies
run: uv sync --frozen --all-groups
- name: Check formatting
run: uv run ruff format --check .
- name: Lint
run: uv run ruff check .
- name: Type check
run: uv run mypy
- name: Test without external network integrations
run: uv run pytest -m 'not integration'
- name: Build distributions
run: uv build

security:
name: Security
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v6
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
fetch-depth: 0
- uses: astral-sh/setup-uv@d0d8abe699bfb85fec6de9f7adb5ae17292296ff # v6
with:
version: "0.8.4"
enable-cache: true
cache-dependency-glob: uv.lock
- run: uv sync --frozen --all-groups
- run: uv run ruff format --check .
- run: uv run ruff check .
- run: uv run mypy
- run: uv run pytest -m 'not integration' tests/unit
- run: uv build
- run: uv run pip-audit
- uses: gitleaks/gitleaks-action@v2
- name: Audit locked dependencies
run: |
uv export --frozen --no-dev --no-emit-project --output-file /tmp/runtime-requirements.txt
uv run pip-audit --require-hashes --no-deps -r /tmp/runtime-requirements.txt
- name: Install Gitleaks
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITLEAKS_VERSION: "8.30.1"
GITLEAKS_SHA256: "551f6fc83ea457d62a0d98237cbad105af8d557003051f41f3e7ca7b3f2470eb"
run: |
archive="gitleaks_${GITLEAKS_VERSION}_linux_x64.tar.gz"
curl --fail --show-error --silent --location \
--retry 3 --retry-all-errors --connect-timeout 10 --max-time 60 \
--output "/tmp/${archive}" \
"https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VERSION}/${archive}"
echo "${GITLEAKS_SHA256} /tmp/${archive}" | sha256sum --check --strict
tar --extract --gzip --file "/tmp/${archive}" --directory /tmp gitleaks
sudo install --owner=root --group=root --mode=0755 /tmp/gitleaks /usr/local/bin/gitleaks
- name: Scan repository for secrets
run: gitleaks git --redact --no-banner --log-opts="--all"

protected-integration:
name: Protected real-data integration
if: github.event_name == 'workflow_dispatch'
environment: protected-integration
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v6
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: astral-sh/setup-uv@d0d8abe699bfb85fec6de9f7adb5ae17292296ff # v6
with:
version: "0.8.4"
- run: uv sync --frozen --all-groups
- run: uv run pytest -m integration tests/integration
- name: Run opt-in integrations
run: uv run pytest -m integration tests/integration
env:
RUN_INTEGRATION: "1"
HETZNER_API_TOKEN: ${{ secrets.HETZNER_API_TOKEN }}
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,11 @@ __pycache__/
.pytest_cache/
.vendor/
vendor-manifest.json
.venv/
dist/
build/
*.egg-info/
.coverage
htmlcov/
.mypy_cache/
.ruff_cache/
24 changes: 24 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Contributing

## Development workflow

1. Create a focused branch from `main` and never commit credentials or `.env` files.
2. Install the reproducible environment with `uv sync --frozen --all-groups`.
3. Add or update tests with each behavior change. Keep real-network tests marked `integration` and gated by `RUN_INTEGRATION=1`.
4. Run the local pull-request checks:

```bash
uv run ruff format --check .
uv run ruff check .
uv run mypy
uv run pytest -m 'not integration'
uv build
uv export --frozen --no-dev --no-emit-project --output-file /tmp/runtime-requirements.txt
uv run pip-audit --require-hashes --no-deps -r /tmp/runtime-requirements.txt
```

5. Open a pull request using the repository template. Describe operational impact, data sources, deployment, and rollback when applicable.

## Engineering expectations

Validate untrusted inputs at system boundaries. Network requests must use bounded timeouts, retries with exponential backoff and jitter where safe, respectful concurrency, and defensive response parsing. Do not log secrets or sensitive request bodies. Use conventional, imperative commit subjects and keep generated artifacts out of commits.
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2026 Omni Connector Hub contributors

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -381,3 +381,20 @@ mechanism and is ignored. Output is bounded and credentials are redacted.
See `.env.template` for a minimal policy example. Keep approval identifiers in
your change-management system and inject policy through deployment secrets;
never commit live approvals or credentials.

## Pull-request quality gates

Every pull request runs locked dependency installation, formatting, linting,
strict type checking, non-integration tests on the oldest and newest supported
Python versions, package builds, runtime dependency auditing, security linting,
and secret scanning. Real-provider tests remain opt-in behind the protected
`protected-integration` environment so untrusted pull requests never receive
credentials. Dependency updates are proposed weekly for both Python packages
and GitHub Actions.

See [CONTRIBUTING.md](CONTRIBUTING.md) for the local check sequence and
[SECURITY.md](SECURITY.md) for private vulnerability reporting. Repository
administrators should protect `main`, require the `Quality / Python 3.11`,
`Quality / Python 3.14`, and `Security` checks, require one
review, dismiss stale approvals, require conversation resolution, and disallow
force pushes and branch deletion.
11 changes: 11 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Security Policy

## Reporting a vulnerability

Do not disclose vulnerabilities in public issues. Use GitHub's private **Security > Report a vulnerability** flow for this repository. Include affected versions, impact, a minimal reproduction, and suggested mitigations without including live credentials or customer data.

Maintainers should acknowledge a complete report within five business days, coordinate remediation and disclosure with the reporter, and publish an advisory when users need to take action. No bounty or response-time guarantee is implied.

## Supported versions

Security fixes are applied to the latest release on `main`. Users should upgrade to the newest published version. Never submit production secrets in reports, logs, tests, or pull requests; rotate any credential that may have been exposed.
37 changes: 0 additions & 37 deletions connectors/ops/browser.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,6 @@ def __init__(self, config=None):
self.missing_env = []
self.security = SecurityPolicy(self.config)

read_only_actions = frozenset(['fetch', 'check_status'])
mutating_actions = frozenset(['screenshot'])
destructive_actions = frozenset([])
dry_run_actions = frozenset(['screenshot'])

def actions(self):
return ["fetch", "check_status", "screenshot"]

Expand Down Expand Up @@ -139,38 +134,6 @@ def probe(u):
# then require an explicit browser capability before using one.
self.security.validate_url(url)
try:
from playwright.sync_api import sync_playwright # noqa: F401
with sync_playwright() as p:
browser = p.chromium.launch()
page = browser.new_page(user_agent=USER_AGENT)
page.goto(url, timeout=DEFAULT_TIMEOUT * 1000)
page.screenshot(path=out_path, full_page=True)
browser.close()
return {"ok": True, "url": url, "path": out_path, "tool": "playwright"}
except ImportError:
pass
except Exception as e:
raise ConnectorError(f"{self.name}: playwright screenshot failed: {e}")

# Fallback: wkhtmltoimage binary if present.
if shutil.which("wkhtmltoimage"):
import subprocess
proc = subprocess.run(
["wkhtmltoimage", "--quiet", url, out_path],
capture_output=True, text=True, timeout=60,
)
if proc.returncode == 0:
return {"ok": True, "url": url, "path": out_path,
"tool": "wkhtmltoimage"}
raise ConnectorError(
f"{self.name}: wkhtmltoimage failed: {proc.stderr[:300]}"
)

# No rendering tool available: explicitly report non-execution.
return {
"ok": False,
"executed": False,
"state": "dependency_required",
self.security.require_capability(self.name, "browser_render")
except SecurityError as e:
raise ConnectorError(f"{self.name}: {e}")
Expand Down
Loading
Loading