[backend/client] add ruleApplyAsync method (#11626) by JeremyCloarec · Pull Request #14034 · OpenCTI-Platform/opencti

Jeremy Cloarec (JeremyCloarec) · 2026-01-15T10:23:23Z

Proposed changes

add ruleApplyAsync method to handle long running ruleApply calls

Related issues

Change rule activation in background tasks #11626

Checklist

I consider the submitted work as finished
I tested the code for its functionality
I wrote test cases for the relevant uses case (coverage and e2e)
I added/update the relevant documentation (either on github or on notion)
Where necessary I refactored code to improve the overall quality

Further comments

codecov · 2026-01-15T11:05:44Z

Codecov Report

❌ Patch coverage is 62.72727% with 41 lines in your changes missing coverage. Please review.
✅ Project coverage is 32.03%. Comparing base (3306191) to head (60b2120).
⚠️ Report is 12 commits behind head on master.

Files with missing lines	Patch %	Lines
...-python/pycti/entities/opencti_stix_core_object.py	7.14%	13 Missing ⚠️
...latform/opencti-graphql/src/manager/ruleManager.ts	35.29%	11 Missing ⚠️
...cti-platform/opencti-graphql/src/database/utils.ts	81.39%	8 Missing ⚠️
client-python/pycti/utils/opencti_stix2.py	36.36%	7 Missing ⚠️
client-python/pycti/api/opencti_api_work.py	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #14034      +/-   ##
==========================================
- Coverage   32.06%   32.03%   -0.03%     
==========================================
  Files        3006     3011       +5     
  Lines      205960   208976    +3016     
  Branches    37676    37977     +301     
==========================================
+ Hits        66039    66955     +916     
- Misses     139921   142021    +2100

Flag	Coverage Δ
opencti-client-python	`42.54% <18.51%> (-2.74%)`	⬇️
opencti-front	`2.50% <ø> (-0.01%)`	⬇️
opencti-graphql	`68.42% <77.10%> (-0.12%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot

Pull request overview

This pull request adds a new ruleApplyAsync GraphQL mutation to handle long-running rule application operations asynchronously. The implementation creates a Work object to track the operation's progress, uses a Promise.race pattern with a 1-second timeout to determine whether to wait for completion or return immediately with the ongoing work, and updates both the backend GraphQL API and the Python client to support this new functionality.

Changes:

Added ruleApplyAsync mutation to GraphQL schema that returns a Work object instead of a boolean
Implemented async rule application with timeout-based decision logic in the backend rule manager
Updated Python client to include rule_apply_async method and modified the existing rule_apply to use the async endpoint
Enhanced Python work API to properly initialize work status before polling

Reviewed changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
opencti-platform/opencti-graphql/src/resolvers/rule.js	Adds resolver mapping for the new `ruleApplyAsync` mutation
opencti-platform/opencti-graphql/src/manager/ruleManager.ts	Implements `ruleApplyAsync` with Work tracking, timeout logic, and background execution handling
opencti-platform/opencti-graphql/src/generated/graphql.ts	Generated TypeScript types for the new GraphQL mutation and its arguments
opencti-platform/opencti-graphql/config/schema/opencti.graphql	GraphQL schema definition for `ruleApplyAsync` mutation returning Work object
opencti-platform/opencti-front/src/schema/relay.schema.graphql	Frontend Relay schema update to include the new mutation
client-python/pycti/utils/opencti_stix2.py	Changes `rule_apply` method to call `rule_apply_async` instead of `rule_apply`
client-python/pycti/entities/opencti_stix_core_object.py	Adds new `rule_apply_async` method that creates work and waits for completion
client-python/pycti/api/opencti_api_work.py	Fixes work status initialization in `wait_for_work_to_finish` to properly handle initial state

Comments suppressed due to low confidence (1)

client-python/pycti/api/opencti_api_work.py:261

The infinite while loop lacks any timeout or maximum iteration mechanism. If a work gets stuck in a non-complete status (e.g., due to a backend issue), this will loop forever, potentially hanging the client indefinitely. Consider adding a timeout parameter or maximum retry count to prevent infinite loops.

        while status != "complete":
            state = self.get_work(work_id=work_id)
            if len(state) > 0:
                status = state["status"]

                if state["errors"]:
                    self.api.app_logger.error(
                        "Unexpected connector error", {"state_errors": state["errors"]}
                    )
                    return ""

            time.sleep(1)

Copilot · 2026-01-21T09:13:32Z

+                "Apply rule stix_core_object", {"id": element_id}
+            )
+            query = """
+                    mutation StixCoreApplyRule($elementId: ID!, $ruleId: ID!) {


The GraphQL mutation name 'StixCoreApplyRule' doesn't match the actual mutation being called ('ruleApplyAsync'). While this is just a query name and doesn't affect functionality, it could cause confusion during debugging or when reading GraphQL operation logs. Consider renaming it to 'StixCoreApplyRuleAsync' for consistency.

Suggested change

mutation StixCoreApplyRule($elementId: ID!, $ruleId: ID!) {

mutation StixCoreApplyRuleAsync($elementId: ID!, $ruleId: ID!) {

Copilot · 2026-01-21T09:13:32Z

+        state = self.get_work(work_id=work_id)
        status = ""
+        if len(state) > 0:
+            status = state["status"]


The initial status check (line 246) retrieves the work state but immediately overwrites the status variable with an empty string on line 247, making the initial fetch pointless. This is redundant and could be simplified by initializing status based on the initial state directly, or starting with the loop if the first check isn't needed.

Copilot

Pull request overview

Copilot reviewed 8 out of 9 changed files in this pull request and generated 12 comments.

Copilot · 2026-02-04T08:55:46Z

+            while not rule_apply_complete:
+                self.opencti.app_logger.info(
+                    "Apply rule async stix_core_object", {"id": element_id}
+                )
+                query = """
+                        mutation StixCoreApplyRule($elementId: ID!, $ruleId: ID!, $executionId: ID!) {
+                            ruleApplyAsync(elementId: $elementId, ruleId: $ruleId, executionId: $executionId)
+                        }
+                    """
+                result = self.opencti.query(
+                    query,
+                    {
+                        "elementId": element_id,
+                        "ruleId": rule_id,
+                        "executionId": execution_id,
+                    },
+                )
+                rule_apply_complete = result["data"]["ruleApplyAsync"]
+                self.opencti.app_logger.info(
+                    "Apply rule async stix_core_object complete", {"id": element_id}
+                )
+                if not rule_apply_complete:
+                    time.sleep(10)


The infinite while loop has no maximum retry count or total timeout. If the backend continuously returns false due to an issue, this will loop forever, making repeated GraphQL calls every 10 seconds. Consider adding a maximum retry count or a total timeout duration to prevent infinite loops and resource exhaustion.

Copilot · 2026-02-04T08:55:46Z

+export const ruleApplyAsync = async (context: AuthContext, user: AuthUser, elementId: string, ruleId: string, executionId: string) => {
+  const rule = await getRule(context, user, ruleId) as RuleRuntime;
+  if (!rule) {
+    throw FunctionalError('Cant find rule to scan', { id: ruleId });


Spelling error: "Cant" should be "Cannot" to match the error message style used in the Python client and elsewhere in the codebase.

Copilot · 2026-02-04T08:55:47Z

+export const ruleApplyAsync = async (context: AuthContext, user: AuthUser, elementId: string, ruleId: string, executionId: string) => {
+  const rule = await getRule(context, user, ruleId) as RuleRuntime;
+  if (!rule) {
+    throw FunctionalError('Cant find rule to scan', { id: ruleId });
+  }
+  const dataCanonicalize = jsonCanonicalize({ elementId, ruleId, executionId }) as string;
+  const ruleApplyId = uuidv5(dataCanonicalize, OPENCTI_NAMESPACE);
+  const currentAsyncCall = await redisGetAsyncCall(ruleApplyId);
+  if (currentAsyncCall) {
+    return currentAsyncCall !== '0';
+  }
+  await redisInitializeAsyncCall(ruleApplyId);
+  const timeoutPromise = new Promise((_, reject) =>
+    setTimeout(() => reject(new Error('Timeout')), 10000),
+  );
+  const ruleApplyPromise = ruleApply(context, user, elementId, ruleId);
+  try {
+    await Promise.race([ruleApplyPromise, timeoutPromise]);
+    // If rule apply promise is the first to finish, we move the work to complete here
+    await redisFinishAsyncCall(ruleApplyId);
+    return true;
+  } catch {
+    // If timeout promise is the first to finish, we do not await the rule apply but instead we return the ongoing work
+    // The work will be moved to complete when the rule apply is finished
+    ruleApplyPromise.catch((err) => {
+      logApp.error('[OPENCTI] Error during rule apply', { cause: err });
+    }).finally(async () => {
+      await redisFinishAsyncCall(ruleApplyId);
+    });
+  }
+
+  return false;
+};


The new ruleApplyAsync functionality lacks test coverage. The repository has comprehensive test suites in tests/20-rules/ and tests/03-integration/, and similar async operations would typically have unit and integration tests. Consider adding tests to verify: 1) the async behavior with timeouts, 2) the Redis tracking mechanism, 3) concurrent calls with the same parameters, 4) error handling scenarios, and 5) the Python client polling logic.

Copilot · 2026-02-04T08:55:48Z

+  } catch {
+    // If timeout promise is the first to finish, we do not await the rule apply but instead we return the ongoing work
+    // The work will be moved to complete when the rule apply is finished
+    ruleApplyPromise.catch((err) => {
+      logApp.error('[OPENCTI] Error during rule apply', { cause: err });
+    }).finally(async () => {
+      await redisFinishAsyncCall(ruleApplyId);
+    });
+  }


The catch block here doesn't distinguish between timeout errors and actual rule execution errors. When the timeout occurs, the ruleApplyPromise continues executing in the background, but if it then fails with an actual error (not a timeout), that error is logged but the function still returns false, which could be misleading. The client will interpret false as "still processing" when it might actually mean "failed". Consider returning an error state or throwing an exception for non-timeout errors to provide clearer feedback to the caller.

Copilot · 2026-02-04T08:55:48Z

+  const timeoutPromise = new Promise((_, reject) =>
+    setTimeout(() => reject(new Error('Timeout')), 10000),


The timeout value of 10000ms (10 seconds) is hardcoded. Since this is part of a system designed to handle long-running operations, the timeout should be configurable through the configuration system (similar to other timeouts in the codebase like SCHEDULE_TIME which uses conf.get). Consider adding a configuration option like 'rule_engine:async_timeout' with 10000 as a default value.

Copilot · 2026-02-04T08:55:48Z

+                self.opencti.app_logger.info(
+                    "Apply rule async stix_core_object complete", {"id": element_id}
+                )
+                if not rule_apply_complete:
+                    time.sleep(10)


The log message "Apply rule async stix_core_object complete" is misleading because it's logged immediately after receiving the response from the API, before checking whether rule_apply_complete is actually True. If the API returns False, this message will still log "complete" even though the operation is not complete. Move this log statement inside a conditional check or change the message to reflect the actual status returned.

Suggested change

self.opencti.app_logger.info(

"Apply rule async stix_core_object complete", {"id": element_id}

)

if not rule_apply_complete:

time.sleep(10)

if not rule_apply_complete:

time.sleep(10)

self.opencti.app_logger.info(

"Apply rule async stix_core_object complete", {"id": element_id}

)

Copilot · 2026-02-04T08:55:48Z

+                result = self.opencti.query(
+                    query,
+                    {
+                        "elementId": element_id,
+                        "ruleId": rule_id,
+                        "executionId": execution_id,
+                    },
+                )


The polling loop doesn't handle potential GraphQL query errors. If the query call at lines 1900-1907 throws an exception (e.g., network error, authentication failure, or backend error), the entire function will fail without any retry logic or graceful degradation. Consider wrapping the query in a try-except block with appropriate error handling, logging, and potentially a backoff strategy.

Copilot · 2026-02-04T08:55:49Z

+                    "Apply rule async stix_core_object complete", {"id": element_id}
+                )
+                if not rule_apply_complete:
+                    time.sleep(10)


The function doesn't return a value when rule_apply_complete is True. The function should return a value (presumably True or some status) to indicate successful completion, otherwise it implicitly returns None which could be confusing for callers.

Suggested change

time.sleep(10)

time.sleep(10)

return rule_apply_complete

Xavier Fournet (xfournet) · 2026-02-10T09:51:46Z

+    const currentAsyncCall = await redisGetAsyncCall(ruleApplyId);
+    // If a call is already processing, return the call status
+    if (currentAsyncCall) {
+      result = currentAsyncCall !== '0';


if call is in progress, we should also wait some time before returning the result

I don't fully get what you mean. Do you want the ruleApplyAsync function to always wait the same amount of time after the first call ? Can't we instead return directly and let the caller decide on the wait time between each call to ruleApplyAsync to check if it is done?

Xavier Fournet (xfournet) · 2026-02-10T09:54:18Z

+        ruleApplyPromise.catch((err) => {
+          logApp.error('[OPENCTI] Error during rule apply', { cause: err });
+        }).finally(async () => {
+          await redisFinishAsyncCall(ruleApplyId);
+        });


mixing promise style and try/catch/finally is a bit complicated, you could stick to try/catch/finally everywhere

Xavier Fournet (xfournet) · 2026-02-10T20:19:35Z

@@ -9951,6 +9951,7 @@ type Mutation {
  ruleSetActivation(id: ID!, enable: Boolean!): Rule!
  ruleManagerClean(eventId: ID): RuleManager!
  ruleApply(elementId: ID!, ruleId: ID!): Boolean


i wonder if we could merge ruleApply and ruleApplyAsync by specifing an optional asyncExecutionId: ID parameter ?

Xavier Fournet (xfournet) · 2026-02-10T20:23:12Z

+                if not rule_apply_complete:
+                    time.sleep(10)


we should rather have an active wait in the backend

Xavier Fournet (xfournet) · 2026-02-11T22:22:21Z

  return executeRuleApply(context, user, rule, elementId);
 };

+export const ruleApplyAsync = async (context: AuthContext, user: AuthUser, elementId: string, ruleId: string, executionId: string) => {


In case the node executing the rule apply fail, it won't be able to resume before 1h (in worst case)

The challenges are:

being able to detect a 'dead' ruleApply due to node failure

being able to make active wait, even from a node that is not executing this ruleApply

I propose to rely on redis status + redis lock + local map of async task promise:

keep the lock during all the ruleApply execution

keep the lock alive with a // async task

Draft algo : when receiving a request

check the local promise map -> if present go to step 3.

try to get the lock with timeout
2.1. if lock cannot be acquired during the timeout, task is still in progress, then return false
2.2. if lock can be acquired, check status
2.2.1. status = done, task is terminated, return true
2.2.2. no status or status = in progress (that mean dead task) then create the async call promise

wait the call promise with race timeout, if timeout win the race then return false, else true

I pushed a version of the draft algo. I'm not sure we need the local map of async tasks though, can't we just always try to get the lock?

github-actions Bot added the filigran team Item from the Filigran team. label Jan 15, 2026

Jeremy Cloarec (JeremyCloarec) force-pushed the issue/11626_async_version branch from 7ccc65d to 8451465 Compare January 20, 2026 15:31

Jeremy Cloarec (JeremyCloarec) marked this pull request as ready for review January 20, 2026 15:31

Jeremy Cloarec (JeremyCloarec) force-pushed the issue/11626_async_version branch from 8451465 to a635db7 Compare January 20, 2026 15:33

Jeremy Cloarec (JeremyCloarec) requested a review from Copilot January 21, 2026 09:08

Copilot started reviewing on behalf of Jeremy Cloarec (JeremyCloarec) January 21, 2026 09:08

View session

Copilot AI reviewed Jan 21, 2026

View reviewed changes

Jeremy Cloarec (JeremyCloarec) linked an issue Jan 27, 2026 that may be closed by this pull request

Change rule activation in background tasks #11626

Closed

Jeremy Cloarec (JeremyCloarec) force-pushed the issue/11626_async_version branch from 08b015f to b08e831 Compare February 3, 2026 16:49

Jeremy Cloarec (JeremyCloarec) requested a review from Copilot February 4, 2026 08:48

Copilot started reviewing on behalf of Jeremy Cloarec (JeremyCloarec) February 4, 2026 08:49

View session

Copilot AI reviewed Feb 4, 2026

View reviewed changes

Jeremy Cloarec (JeremyCloarec) force-pushed the issue/11626_async_version branch 2 times, most recently from 3359987 to 1973420 Compare February 4, 2026 14:19

Xavier Fournet (xfournet) reviewed Feb 10, 2026

View reviewed changes

Xavier Fournet (xfournet) reviewed Feb 11, 2026

View reviewed changes

Jeremy Cloarec (JeremyCloarec) added 13 commits February 12, 2026 10:37

[backend/client] add ruleApplyAsync method (#11626)

db3faf8

[backend/client] add ruleApplyAsync method (#11626)

f66af9f

[backend] rework ruleApplyAsync

27692fc

[backend] rework ruleApplyAsync

a8e66ee

[backend] change work to redis only async status

efce988

[client] black format

3a1b2f3

[backend] atomic ruleApplyAsync

9cd1590

[backend] lint

60774b3

[backend] rework ruleApplyAsync

2f45d2c

[client] use bundle_id as execution_id in rule_apply_async

848d70b

[client] use bundle_id as execution_id in rule_apply_async

8c48f86

[backend] PR comments

32f3706

[backend] PR comments

e4ecc45

Jeremy Cloarec (JeremyCloarec) force-pushed the issue/11626_async_version branch from 0aaa808 to e4ecc45 Compare February 12, 2026 09:54

Jeremy Cloarec (JeremyCloarec) added 6 commits February 12, 2026 11:00

[backend] linter

2146f18

[backend] review changes

01c2968

[backend] fix rule apply promise

a7c24c2

[client] modified logs

fdd45e9

[client] linter

a3aebe6

[backend] refactor method into utils + tests

60b2120

Xavier Fournet (xfournet) approved these changes Feb 17, 2026

View reviewed changes

Jeremy Cloarec (JeremyCloarec) merged commit 0caa286 into master Feb 17, 2026
38 checks passed

Jeremy Cloarec (JeremyCloarec) deleted the issue/11626_async_version branch February 17, 2026 13:32

	mutation StixCoreApplyRule($elementId: ID!, $ruleId: ID!) {
	mutation StixCoreApplyRuleAsync($elementId: ID!, $ruleId: ID!) {

		const timeoutPromise = new Promise((_, reject) =>
		setTimeout(() => reject(new Error('Timeout')), 10000),

Conversation

Jeremy Cloarec (JeremyCloarec) commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Related issues

Checklist

Further comments

Uh oh!

codecov Bot commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Xavier Fournet (xfournet) Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Jeremy Cloarec (JeremyCloarec) Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Xavier Fournet (xfournet) Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Xavier Fournet (xfournet) Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Xavier Fournet (xfournet) Feb 10, 2026

Choose a reason for hiding this comment

Jeremy Cloarec (JeremyCloarec) commented Jan 15, 2026 •

edited

Loading

codecov Bot commented Jan 15, 2026 •

edited

Loading