[DO NOT MERGE] Simplified API by johroj · Pull Request #128 · JuliaIO/gRPCClient.jl

johroj · 2026-07-31T14:46:17Z

Ok, this is a basic draft of what was discussed in #122

Codegen

This feels pretty much like what I had in mind. Some generic function signatures are written explicitly, with type assertions for readability. Some types are left abstract to leave room for future changes. All valid usecases (unary sync, unary async, unary channel and streams) are supported. You can see the new code in the updated test_pb.jl.

Internal logic

The API called directly from the generated code should probably be considered public, but does not need to be exported. The logic is highly based on the traits defined in the generated code - I found this easy to work with and meant that all properties of an RPC could be found by using the function type typeof(MyService.MyRPC) as a single type parameter.

The handle type

For all asynchronous calls, a handle is returned, with types depending on the kind of RPC(gRPCUnaryHandle, gRPCStreamResponseHandle) etc. This should be a single object with methods for all operations one may need after opening a request. I'm still a bit hesitant on the current names, but the following functionality is necessary:

Tell if put! will block (isfull, not available in 1.10)
put! (or equivalent).
Tell if a response is available (isready).
Take a response without removing it (fetch).
Take a response and remove it (take!).
Cancel an RPC, no questions asked (kill)
Gracefully close the RPC and return unary responses (which would also imply blocking) (close).
Tell if the whole RPC is done. (isopen)

The approach I was going for was to overload methods from Base to make operation as similar to a channel as possible. This is good because it allows using short simple names without cluttering the namespace. But there are some cases where there is no clear choice, for example isopen and could very well refer to both the response channel or the gRPCRequest. Same problem with wait. If you have any thoughts, please let me know.

Remaining items:

codecov-commenter · 2026-07-31T14:48:35Z

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 97.74775% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.76%. Comparing base (3155063) to head (1719fc7).

Files with missing lines	Patch %	Lines
src/CallHandles.jl	96.80%	3 Missing ⚠️
src/gRPCClient.jl	60.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #128      +/-   ##
==========================================
+ Coverage   90.67%   92.76%   +2.08%     
==========================================
  Files           7        8       +1     
  Lines         708      926     +218     
==========================================
+ Hits          642      859     +217     
- Misses         66       67       +1

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

johroj · 2026-08-02T09:37:58Z

I have spent some time thinking about the naming of operations on the handle type and realized the key issue is that I originally wanted to make it clear how the simple API wraps the inner API. For example, if a method on handle would be missing, we could redirect the user to call a method on handle.request_channel instead. On the other hand, if the new API can be made complete (or such that any new functionality can be added easily), there is no need to be clear about how the two APIs relate. This would simplify naming and in its turn also mean that the new API can be made complete. So I'm going for this second approach.

I've outlined all operations on the handle object in the table below.

Purpose	Potential names	Throws req.ex	Unary	Client stream	Server stream	Bidirectional
Check if ready for a request	`Base.isfull`, `gRPCClient.iscongested`	Yes	N/A	`isfull(request_c)`	N/A	`isfull(request_c)`
Send request	`put!(rpc, msg)`	Yes	N/A	`put!(request_c, msg)`	N/A	`put!(request_c, msg)`
Signal done with requests	`Base.put!(rpc[, msg], done = true)`	Yes	N/A	`close(request_c)`	N/A	`close(request_c)`
Check if response available	`Base.isready`, `gRPCClient.hasresponse`	Yes	`req.completed && isnothing(req.ex)`	`req.completed && isnothing(req.ex)`	`isready(response_c)`	`isready(response_c)`
Wait until response available	`Base.wait`	Yes	`wait(req.ready)`	`wait(req.ready)`	`wait(response_c)`	`wait(response_c)`
Take response	`Base.take!`	Yes	N/A	N/A	`take!(resp_c)`	`take!(resp_c)`
Take response without removing it	`Base.fetch`	Yes	`grpc_async_await(req, TResponse)`	`grpc_async_await(req, Tresponse)`	`fetch(response_c)`	`fetch(response_c)`
Check if request still active	`Base.isopen`	No	`!req.completed`	`!req.completed`	`!req.completed`	`!req.completed`
wait for server shutdown and report errors	`Base.close`, `gRPCClient.grpc_async_await`	Yes	`grpc_async_await(req)`	`close(request_c); grpc_async_await(req)`	`grpc_async_await(req)`	`close(request_c); grpc_async_await(req)`
cancel	`Base.detach`, `Base.kill`, `gRPCClient.grpc_cancel`	Before call only	`grpc_cancel(req)`	`grpc_cancel(req)`	`grpc_cancel(req)`	`grpc_cancel(req)`

For some of these operations, we could either overload functions in Base or export new functions from gRPCClient. I'm in favor of using Base as much as possible. Nevertheless, the important part is purpose of each operation. As long as all of these operations are available, it looks to me as if it would be possible to do all the things the current interface supports. @csvance I would appreciate if you could also give this a look and see if I missed anything. It is central for the remaining work.

Some notes:

I think it is useful for most functions to be wrapped in a try-catch block and if an exception (e.g. channel closed) occurs, req.ex is also thrown. Highly inspired by take_or_diagnose which I found as a helper in the tests.
For example fetch would have different implementation for unary responses and streaming responses - although that the purpose is common. For unary, it will return the response and ensure everything is cleaned up. Calling fetch multiple times will give the same result. For streaming requests, it will give one result from the stream and not cause any cleanup.

I'm also wondering about the safety of checking req.completed,req.ex or if a unary response is available without any locks or atomics. Is this still safe? Is it necessary to run these checks in a particular order? If there are any gotchas, perhaps it would make more sense to provide some helpers for these purposes that are implemented in Curl.jl?

johroj · 2026-08-04T14:45:06Z

I added some examples of the lifecycles of different types of calls. Basic examples are in the auto-generated docstrings at the end of protobuf.jl and some more advanced examples can be found in runtests.jl.

johroj · 2026-08-05T09:41:40Z

@csvance I think this is at the point where all functionality (except trival things like forwarding keywords) is implemented. Remaining work looks straightforward to me (to-do-list in description updated), but for example the level of documentation depends on to what extent this can be considered a replacement or not as we discussed previously. I would appreciate if you could have a look on the overall design decisions before I go too far.

As an update on previous comments, I took a closer look at how to safely poll the status of gRPCRequest and implemented meaningful error reporting on put! and take!. Despite the extra try-block, the benchmark shows no difference between the old/new api:

╭─────────────────────────────────────────────┬─────────────┬────────────────┬────────────┬──────────────┬─────────┬─────┬─────╮
│                                   Benchmark │  Avg Memory │     Avg Allocs │ Throughput │ Avg duration │ Std-dev │ Min │ Max │
│                                             │ KiB/message │ allocs/message │ messages/s │           μs │      μs │  μs │  μs │
├─────────────────────────────────────────────┼─────────────┼────────────────┼────────────┼──────────────┼─────────┼─────┼─────┤
│                    workload_smol_simple_api │        3.38 │           80.7 │      11909 │           84 │     7.0 │  79 │ 128 │
│                               workload_smol │        3.36 │           79.6 │      11437 │           87 │    8.21 │  80 │ 140 │
│ workload_streaming_bidirectional_simple_api │         2.0 │           26.5 │     350817 │            3 │     1.5 │   2 │  28 │
│            workload_streaming_bidirectional │         2.0 │           26.5 │     361892 │            3 │    1.45 │   2 │  31 │
╰─────────────────────────────────────────────┴─────────────┴────────────────┴────────────┴──────────────┴─────────┴─────┴─────╯

csvance · 2026-08-07T16:40:55Z

@johroj we will push out 1.1.0 so people have the improved stability / cancellation without having to work from [sources]. Once I have that registered I'm going to take a look at this and getting it refactored. We can then do a 1.2.0 release sometime in the next week or so. I think its justified to have two different minor releases, one that basically fixed almost all historical stability problems and made streaming work on all supported julia versions, and one that significantly improves the interface.

johroj · 2026-08-10T07:28:51Z

 end
-PB.default_values(::Type{TestResponse}) = (; data = Vector{UInt64}())
-PB.field_numbers(::Type{TestResponse}) = (; data = 1)
+PB.default_values(::Type{TestResponse}) = (;data = Vector{UInt64}())


I think this is just about ProtoBuf.jl not generating files with Runic formatting.

We should just add a Runic pre commit hook so its not an issue going forward. Although I would say just hold off on that till we rebase ontop of main since it risks making the rebase more difficult.

johroj · 2026-08-10T07:34:33Z

    end
 end

+Base.isopen(req::gRPCRequest) = !(@atomic req.ready.set) # TODO  dont rely on fieldnames of Base.Event?


I think it would be good to have some better mechanic to poll the status of the request, acquiring the full lock would be too inefficient. If we want to check private fields of Base.Event, would it be ok to add an atomic field req.isdone or similar? Or can you see some other way forward?

Same thing regarding checking if req has an exception. Could make sense to have a helper function in Curl.jl.

That's a reasonable trade off. Pretty sure it won't increase allocations either.

CI should also warn if this field is removed in an upcoming release.

johroj · 2026-08-10T07:39:32Z

+                # Until julia gets a dedicated syntax for importing from parent module without
+                # knowing its name, we need to use `parentmodule`. Otherwise the generated file
+                # will only work if included from the correct generated toplevel package file. 
+                push!(import_mod_list, "const $(modname)::Module = Base.parentmodule($service_name).$(modname)")


We could enforce the better syntax import ..A: B if we require that protojl runs with always_use_modules = true and users always include the top-level package. But I noticed that this is not used in e.g. the unit tests of gRPCClient.jl, where the _pb file is included directly. I took that as a signal that this is something users might want to do.

Maybe this can be a target for 2.0.0 combined with improved documentation / examples on the subject. When working with a very small number of protobufs its nice to be able to do include("myproto_pb.jl") because its easy to understand without reading anything.

johroj · 2026-08-10T08:00:08Z

+    host::String
+    port::Int
+    grpc::gRPCCURL
+    function gRPCChannel(host::AbstractString, port::Integer; grpc = gRPCCURL())


I think we should change grpc = gRPCCURL() to just use grpc_global_handle() as default instead. Setting up a new gRPCCURL is not cheap, the benchmarks did not look good until ensured a new instance is not spawned in each workload.

@johroj I believe the is the default behavior in general with 1.0.0 - 1.1.0; we re-use by default, and let the user put things on different multi if they want to. That should make the additional overhead associated with creating a gRPCChannel quite minimal.

Yes, exactly. I think what made me initially go for independent instances is that I have mainly used the channel sort of as a global for all communication to a specific server. I think this it would be more intuitive that if you then set up multiple channels, you want them to be able to operate independently. But if someone really cares about this, they will probably have had a look in the documentation, so lets make the default simpler.

johroj · 2026-08-10T08:01:59Z

@johroj we will push out 1.1.0 so people have the improved stability / cancellation without having to work from [sources]. Once I have that registered I'm going to take a look at this and getting it refactored. We can then do a 1.2.0 release sometime in the next week or so.

Thanks, then we can give these interface changes the time it takes.

I added some comments about behavior where I'm still not certain what the best choice is. You may of course find more things. TODO-list in description is updated.

johroj · 2026-08-16T13:30:56Z

+    import gRPCClient
+    import Base
+
+Base.@static if Base.:!(Base.isless(Base.pkgversion(gRPCClient), Base.VersionNumber("1.2.0-rc1")))


@csvance I added this to the feature list. Let me know if you think it is overkill. This if-statement will only be included if both APIs are generated (default, for backwards compatibility). It means new generated code will run with older versions of gRPCClient without complaining about missing functions. Unfortunately, VSCode completion does not work for methods inside a @static, so it will only work if only the new API is generated (which still can be recommended in documentation or default in a 2.0).

Then there's also gRPCClient.check_codegen_compat which can be used to error or warn on incompatibilities. Even if this function does not do anything today, its probably good to include the capability already now.

johroj · 2026-08-18T09:25:52Z

How to handle partially encoded messages?

@csvance I realized I previously overlooked the ability to bypass encoding/decoding and just send raw Vector{UInt8} messages. AFAIK this is the only thing preventing the updated API from being a pure upgrade, so I spent some thought on different ways this could be implemented. The key problem is that this currently needs to be predetermined so that the request/response channels have the correct element type in a type-stable way. I can think of the following options:

1. Keyword arguments

Optional keyword arguments TResponse = response_type(Trpc), but may be set to Vector{UInt8}. This is how the old API works, but it is not type stable since julia does not dispatch on keyword arguments (it is type stable if the default option is chosen, however).

2. Singleton struct flags

A type stable way to could control the message types is to use singleton structs such as MyService.MyRPC(..., gRPCRawRequest(), gRPCRawResponse()). But using positional arguments quickly becomes messy since one may want to control request/response independently.

3. A combination approach

A good, but slightly more segmented approach is to control the types in different ways depending on the types of request/response.

For unary, encoding/decoding is not done in a separate task so there is no channel which needs a predetermined type. We can simply provide a Vector{UInt8} as request or enable fetch(rpc, Vector{UInt8}) to get the raw response.

For streaming requests, we can use a Channel{Union{TResponse, Vector{UInt8}}). Thanks to union splitting in the compiler, this will only add a single ~30 byte allocation to each message, but it is still type stable. Effect on throughput seems negligible.

For streaming response, using a Channel{Union{...}} is not a good option, since we do not want to spend time decoding unless necessary. So adding a singleton struct argument e.g. MyService.MyRPC(..., gRPCRawResponse()) would be required here.

4. Encode/decode in main thread

May need some input on the overall design choices to tell whether this option is feasible.

I can see that if the current API wants a user-facing Channel{TResponse} which is forwarded to a Channel{IOBuffer} for CURL, it makes sense to run the encoding conversion in the pump task. But if we have our custom versions of put! and take!, encoding/decoding could take place in those methods instead. This would allow the alternative syntaxes put!(rpc, msg::Vector{UInt8}) and take!(rpc, Vector{UInt8}), which is hard to beat in terms of clarity and type stability. But is there also a performance aspect of running the encoding/decoding in the pump tasks that we cannot work around with this option?

5. Dont implement it for now

Also an option, of course.

johroj · 2026-08-23T19:29:10Z

@csvance I took a closer look on how the the streams work and understood that the messages need to be encoded in the pump task since multiple requests may be sent to Curl in the same buffer. But I could not see anything similar for responses. So I went ahead and used a Union{TRequest, Vector{UInt8} channel for the requests (requires one extra alloc due to GC) so that the choice is made at the call, not the setup. For responses, I added the ability to get the raw IOBuffer as intermediate response from the old API (to avoid allocations), from which the user can later decide which response to obtain when calling fetch or take!. After some additional attention to inlining and fixing a type stability issue in Curl.jl, the impact on performance is negligible.

╭─────────────────────────────────────────────┬─────────────┬────────────────┬────────────┬──────────────┬─────────┬──────┬──────╮
│                                   Benchmark │  Avg Memory │     Avg Allocs │ Throughput │ Avg duration │ Std-dev │  Min │  Max │
│                                             │ KiB/message │ allocs/message │ messages/s │           μs │      μs │   μs │   μs │
├─────────────────────────────────────────────┼─────────────┼────────────────┼────────────┼──────────────┼─────────┼──────┼──────┤
│                               workload_smol │        3.34 │           77.7 │      12052 │           83 │    2.26 │   79 │   91 │
│                    workload_smol_simple_api │        3.34 │           77.6 │      11865 │           84 │    4.01 │   79 │   96 │
│                   workload_32_224_224_uint8 │       637.5 │           86.2 │        419 │         2388 │  712.74 │ 2094 │ 5398 │
│        workload_32_224_224_uint8_simple_api │       637.4 │           86.0 │        410 │         2440 │  700.24 │ 2080 │ 5400 │
│                  workload_streaming_request │        1.28 │            6.5 │     663842 │            2 │    1.61 │    1 │   53 │
│       workload_streaming_request_simple_api │        1.33 │            7.5 │     660154 │            2 │    1.65 │    1 │   56 │
│                 workload_streaming_response │       13.06 │           28.7 │     127434 │            8 │     5.5 │    4 │   34 │
│      workload_streaming_response_simple_api │       13.06 │           28.7 │     124673 │            8 │    5.71 │    4 │   34 │
│            workload_streaming_bidirectional │         2.0 │           26.5 │     361243 │            3 │    2.92 │    2 │   71 │
│ workload_streaming_bidirectional_simple_api │        2.03 │           27.5 │     355212 │            3 │    2.97 │    2 │   71 │
╰─────────────────────────────────────────────┴─────────────┴────────────────┴────────────┴──────────────┴─────────┴──────┴──────╯

The only thing remaining is documentation. Do you think it would be the easiest for you to review this as-is and we take the documentation in a follow-up PR? In order to write the full documentation, I would first need your overall input on which direction to take as a whole, but I could probably add some parts during the upcoming week if it would be helpful for a review.

csvance · 2026-08-26T15:46:07Z

@johroj I should have time to work on this next week. I'm extremely close to registering gRPCServer.jl. Once that is pushed through, I will have the bandwidth for gRPCClient 1.2.0.

I agree we should review it as is and push the documentation changes into a follow up PR.

johroj changed the title ~~Feature/simple api~~ Simplified API Jul 31, 2026

johroj changed the title ~~Simplified API~~ [DO NOT MERGE] Simplified API Aug 5, 2026

johroj marked this pull request as ready for review August 5, 2026 09:42

johroj commented Aug 10, 2026

View reviewed changes

johroj commented Aug 16, 2026

View reviewed changes

johroj added 16 commits August 30, 2026 16:38

Initial draft of simpler api

7b36571

Renaming and error handling

5e3bfef

Fix missed renamings

12ecd0d

Use wider types in codegen

42003bf

Check exception on close

d08171b

Place _put! in codegen, add detach

5ad0922

Reorganize handle methods & add print

ea95968

Docstrings and put!(rpc, done = true)

5e1fd6e

Test coverage

fde8b5c

More implementation and documentation

10a2f4c

Complete auto-generated docstrings with lifecycle examples

9cfc44d

Correct polling of gRPCRequest status

0bf4cf3

Remove commented end

c7fe697

Relocate handle and associated methods

9248360

Renaming Handle => Call

ef9399b

Make Service a baremodule

13a162d

johroj added 14 commits August 30, 2026 16:38

Add updated proto

55de524

Add support for connection options

53797db

Allow choosing API in codegen registration

876fd1a

Add codegen compat check

e3cc0d2

Add codegen tests

5b1877f

Docstring for grpc_register_service_codegen

85588ac

Exports

838a770

Break tests into testsets

e31cc1e

Implement partial decoding with union splitting

a7ae888

Fix instability in gRPCRequest constructor

518b708

Add @inline

50fe296

Add benchmark utils

45cbffc

Fully support partial encoding

ded1261

Fix broken tests

7854444

johroj force-pushed the feature/simple_api branch from ea331e1 to 7854444 Compare August 30, 2026 14:42

johroj added 8 commits August 30, 2026 16:57

Fix 1.10 (?)

e7434a5

Test isfull by acquiring grpc lock

b31163e

Use "client stream" instead of "stream request" terminology

f3345f4

Clean up exports

dea3c02

Improve test coverage

3a5f8aa

Improve show when rpc is part of other container

57e3274

Runic

31c915b

Runic runtests.jl

1719fc7

Uh oh!

Conversation

johroj commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codegen

Internal logic

The handle type

Uh oh!

codecov-commenter commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

johroj commented Aug 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

johroj commented Aug 4, 2026

Uh oh!

johroj commented Aug 5, 2026

Uh oh!

csvance commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

johroj commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

johroj commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

How to handle partially encoded messages?

1. Keyword arguments

2. Singleton struct flags

3. A combination approach

4. Encode/decode in main thread

5. Dont implement it for now

Uh oh!

johroj commented Aug 23, 2026

Uh oh!

csvance commented Aug 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

johroj commented Jul 31, 2026 •

edited

Loading

codecov-commenter commented Jul 31, 2026 •

edited

Loading

johroj commented Aug 2, 2026 •

edited

Loading

csvance commented Aug 7, 2026 •

edited

Loading

johroj commented Aug 10, 2026 •

edited

Loading

johroj commented Aug 18, 2026 •

edited

Loading

csvance commented Aug 26, 2026 •

edited

Loading