Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 2 additions & 56 deletions .github/workflows/bazel_example.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,67 +21,13 @@ jobs:
submodules: 'recursive'
persist-credentials: false

- name: Bazel File Cache Setup
id: cache-bazel
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6
with:
path: ~/.cache/bazel
key: ${{ runner.os }}-bazel-20210105-${{ secrets.CACHE_VERSION }}

- name: Bazel Cache Not Found
if: steps.cache-bazel.outputs.cache-hit != 'true'
run: |
echo "No cache found."
- name: Bazel Cache Found
if: steps.cache-bazel.outputs.cache-hit == 'true'
run: |
echo -n "Cache found. Cache size: "
du -sh ~/.cache/bazel
echo "If the cache seems broken, update the CACHE_VERSION secret in"
echo "https://github.com/googleapis/gapic-generator-php/settings/secrets/actions"
echo "(use any random string, any GUID will work)"
echo "and it will start over with a clean cache."
echo "The old one will disappear after 7 days."

- name: Show bazel version
run: |
bazel --version

#- name: Build example package; all targets
# run: |
# cd bazel_example;
# bazel build ...

#- name: Extract Bazel package output
# run: |
# cd bazel_example;
# tar -xf ./bazel-bin/example_php_pkg.tar.gz

## grep returns 1 if no matches are found; and 0 if >0 matches are found.
#- name: Verify grpc_service_config.json was honoured
# run: |
# grep '"timeout_millis": 942000' ./bazel_example/example_php_pkg/src/resources/example_client_config.json
#
#- name: Verify gapic.yaml was honoured
# run: |
# grep "'initialPollDelayMillis' => '42042'" ./bazel_example/example_php_pkg/src/resources/example_descriptor_config.php

- name: Integration Tests
- name: Bazel Smoke Test
run: |
bazel --batch test tests/Integration:asset \
tests/Integration:compute_small \
tests/Integration:container \
tests/Integration:dataproc \
tests/Integration:functions \
tests/Integration:kms \
tests/Integration:iam \
tests/Integration:logging \
tests/Integration:redis \
tests/Integration:retail \
tests/Integration:speech \
tests/Integration:securitycenter \
tests/Integration:talent \
tests/Integration:videointelligence --noshow_progress --cache_test_results=no --test_output=errors
bazel --batch test tests/Integration:asset --noshow_progress --cache_test_results=no --test_output=errors

- uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5
if: ${{ failure() }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/phpunit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,4 @@ jobs:
composer dump-autoload;

- name: Run tests
run: USE_TOOLS_PROTOC=true ./vendor/bin/phpunit --bootstrap tests/Unit/autoload.php tests/Unit
run: USE_TOOLS_PROTOC=true ./vendor/bin/phpunit
69 changes: 15 additions & 54 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,52 +64,29 @@ We'll be using **PHP 7.4** for the setup.

## Running tests

- Unit tests
- All tests (unit and integration)

```
./vendor/bin/phpunit --bootstrap tests/Unit/autoload.php tests/Unit
./vendor/bin/phpunit
```

If you do not have `protoc` installed, run with `USE_TOOLS_PROTOC=true`.

```
USE_TOOLS_PROTOC=true ./vendor/bin/phpunit --bootstrap tests/Unit/autoload.php tests/Unit
USE_TOOLS_PROTOC=true ./vendor/bin/phpunit
```

This uses the Linux-only `protoc` binary checked into the repository.

If you run into an error: `Error: Call to undefined function Google\Protobuf\Internal\bccomp()`, that is because the [BC Math](https://www.php.net/manual/en/book.bc.php) extension is not always included by default (see tracking bug here: https://github.com/protocolbuffers/protobuf/issues/4465). You can get around this by installing BC Math with the command `sudo apt install php-bcmath`.

- Bazel integration tests.
- Running a specific test suite or integration case:

- Running:

```
bazel test tests/Integration:asset
```

- Running all tests:

```
bash tests/scripts/run_bazel_tests.sh
```

_Note: Running `bazel` commands may require removing the `composer.lock` and
`vendor/` directory._

- Debugging in `googleapis`:

In [`googleapis/WORKSPACE`](https://github.com/googleapis/googleapis/blob/86fa44cc5ee2136e87c312f153113d4dd8e9c4de/WORKSPACE#L397-L401),
replace the `http_archive` downloading the generator with a
`local_repository` target pointing to the locally modified version of the
generator:

```
local_repository(
name = "gapic_generator_php",
path = "/absolute/path/to/local/generator",
)
```
```
USE_TOOLS_PROTOC=true ./vendor/bin/phpunit --testsuite "Unit Tests"
USE_TOOLS_PROTOC=true ./vendor/bin/phpunit --testsuite "Integration Tests"
USE_TOOLS_PROTOC=true ./vendor/bin/phpunit --testsuite "Integration Tests" --filter asset
```

## Updating tests

Expand All @@ -118,7 +95,7 @@ You will need to update the golden test files if you change something in the gen
- Updating unit test goldens:

```
php tests/Unit/ProtoTests/GoldenUpdateMain.php
USE_TOOLS_PROTOC=true php tests/Unit/ProtoTests/GoldenUpdateMain.php
```

Then follow the prompts for which tests to update.
Expand All @@ -128,41 +105,25 @@ You will need to update the golden test files if you change something in the gen
- Updating integration test goldens:

```
bash tests/scripts/run_bazel_updates.sh --expunge
USE_TOOLS_PROTOC=true php tests/Integration/GoldenUpdateMain.php
```

Integration tests can be updated individually as well:
Integration tests can be updated individually by passing a filter argument:

```
bazel run tests/Integration:asset_update
USE_TOOLS_PROTOC=true php tests/Integration/GoldenUpdateMain.php asset
```

__NOTE__: If a new integration test case is added, make sure to add it the list in [run_bazel_tests.sh](tests/scripts/run_bazel_tests.sh) and [run_bazel_updates.sh](tests/scripts/run_bazel_updates.sh).
__NOTE__: If a new integration test case is added, make sure to add it to `TESTS` in [IntegrationTest.php](tests/Integration/IntegrationTest.php).

- Update all goldens (unit and integration)

Run the composer script `update-all-tests` to update all goldens at once:

```
composer update-all-tests
USE_TOOLS_PROTOC=true composer update-all-tests
```

## Rotating the bazel cache key

The GitHub Actions that run the `bazel`-based integration tests hold a cache of
the build in order to speed up testing pull requests that do not impact
generated surface. If proposing a change that modifies the integration test
golden files, one must also rotate the cache key using the
[`gh` CLI](https://cli.github.com/). Use the following command before opening
your pull request:

```
uuidgen | gh secret set CACHE_VERSION -r https://github.com/googleapis/gapic-generator-php
```

If you don't have the proper permissions to rotate the cache key, request that
the reviewers do so and rerun the Action checks.

## Updating the `googleapis` submodule

To update the `googleapis` submodule, change into the directory and pull:
Expand Down
6 changes: 5 additions & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -48,12 +48,16 @@
"scripts": {
"update-all-tests": [
"Google\\Generator\\Tests\\Unit\\ProtoTests\\GoldenUpdateMain::updateAll",
"bash tests/scripts/run_bazel_updates.sh --expunge"
"Google\\Generator\\Tests\\Integration\\GoldenUpdateMain::updateAll"
]
},
"config": {
"platform": {
"php": "8.2"
},
"allow-plugins": {
"php-http/discovery": false,
"tbachert/spi": false
}
}
}
2 changes: 1 addition & 1 deletion googleapis
Submodule googleapis updated 4098 files
4 changes: 4 additions & 0 deletions phpunit.xml.dist
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@
<testsuite name="Unit Tests">
<directory>./tests/Unit</directory>
</testsuite>
<testsuite name="Integration Tests">
<directory>./tests/Integration</directory>
<exclude>./tests/Integration/goldens</exclude>
</testsuite>
</testsuites>
<php>
<ini name="memory_limit" value="2048M"/>
Expand Down
74 changes: 1 addition & 73 deletions rules_php_test/integration_test.bzl
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
def _php_diff_integration_goldens_impl(ctx):
# Extract the Java source files from the generated 3 srcjars from API bazel target,
# Extract the source files from the generated srcjar from API bazel target,
# and put them in the temporary folder `codegen_tmp`.
# Compare the `codegen_tmp` with the goldens folder (e.g tests/Integration/goldens/asset)
# and save the differences in output file `diff_output.txt`.
Expand Down Expand Up @@ -76,75 +76,3 @@ def php_integration_test(name, target, data):
gapic_library = target,
srcs = data,
)

def _php_overwrite_golden_impl(ctx):
# Extract source files from the srcjar generated from php_gapic_library into
# a temporary folder (codegen_tmp) and zip into goldens_output_zip.
# Overwrite the goldens dir (e.g tests/Integration/goldens/asset) with newly generated code.
gapic_library = ctx.attr.gapic_library
srcs = ctx.files.srcs

# Convert the name of bazel rules e.g. `redis_update` to `redis`
# because we will need to overwrite the goldens files in `redis` folder.
api_name = "_".join(ctx.attr.name.split("_")[:-1])
goldens_output_zip = ctx.outputs.goldens_output_zip

script = """
mkdir codegen_tmp
unzip {input_srcs} -d codegen_tmp
cd codegen_tmp
zip -r ../{goldens_output_zip} .
""".format(
goldens_output_zip = goldens_output_zip.path,
input_srcs = gapic_library[DefaultInfo].files.to_list()[0].path,
)

ctx.actions.run_shell(
inputs = srcs + [
gapic_library[DefaultInfo].files.to_list()[0],
],
outputs = [goldens_output_zip],
command = script,
)

# Overwrite the goldens.
golden_update_script_content = """
cd ${{BUILD_WORKSPACE_DIRECTORY}}
# Filename pattern-based removal is needed to preserve the BUILD.bazel file.
find tests/Integration/goldens/{api_name}/ -name \\*.txt -type f -delete
find tests/Integration/goldens/{api_name}/ -name \\*.php -type f -delete
find tests/Integration/goldens/{api_name}/ -name \\*.json -type f -delete
unzip -ao {goldens_output_zip} -d tests/Integration/goldens/{api_name}
""".format(
goldens_output_zip = goldens_output_zip.path,
api_name = api_name,
)
ctx.actions.write(
output = ctx.outputs.golden_update_script,
content = golden_update_script_content,
is_executable = True,
)
return [DefaultInfo(executable = ctx.outputs.golden_update_script)]

php_overwrite_golden = rule(
attrs = {
"gapic_library": attr.label(),
"srcs": attr.label_list(
allow_files = True,
mandatory = True,
),
},
outputs = {
"goldens_output_zip": "%{name}.zip",
"golden_update_script": "%{name}.sh",
},
executable = True,
implementation = _php_overwrite_golden_impl,
)

def php_golden_update(name, target, data):
php_overwrite_golden(
name = name,
gapic_library = target,
srcs = data,
)
Loading
Loading