Conversation
This was referenced Aug 13, 2026
Two related delegate-lifecycle fixes in createModel: 1. Skip null delegates: delegate factories can legitimately return nullptr (e.g. TfLiteCoreMlDelegateCreate on devices without a Neural Engine when enabled_devices is ANE-only). Registering that nullptr with TfLiteInterpreterOptionsAddDelegate crashes/corrupts the interpreter. Now a null delegate is skipped so the model falls back to CPU, and getDelegates() reports only the delegates that were actually registered. 2. Free delegates: TFLite's C API does not transfer delegate ownership to the interpreter - the caller must delete delegates itself after the interpreter is destroyed. They were never freed, so every model destruction leaked the delegate's compiled kernels / driver contexts (GPU: TfLiteGpuDelegateV2Delete, NNAPI: TfLiteNnapiDelegateDelete, CoreML: TfLiteCoreMlDelegateDelete). Each delegate is now held in a unique_ptr with its own delete function; ownership moves into the interpreter's shared_ptr deleter, so delegates are freed right after TfLiteInterpreterDelete (they must outlive the interpreter) and on every failure path in createModel via normal unwinding.
jslok
force-pushed
the
fix/delegate-lifecycle
branch
from
September 14, 2026 05:46
ad4fec0 to
68dfaf0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two delegate-lifecycle bugs in
createModel, rebased onto the RAII ownership from #205.Delegate factories can legitimately return
nullptr(e.g.TfLiteCoreMlDelegateCreateon devices without a Neural Engine, since the defaultenabled_devicesis ANE-only), and that null went straight intoTfLiteInterpreterOptionsAddDelegate. Null delegates are now skipped so the model falls back to CPU, andgetDelegates()reports only what was actually registered.TFLite's C API also doesn't transfer delegate ownership to the interpreter, and nothing ever freed them, so every model destruction leaked the delegate's compiled kernels / driver contexts. Each delegate now lives in a
unique_ptrwith its own delete function, and ownership moves into the interpreter'sshared_ptrdeleter, so delegates are freed right afterTfLiteInterpreterDelete(they must outlive the interpreter) and on everycreateModelfailure path by plain unwinding.HybridTfliteModelitself is untouched.We run the pre-#205 form of this as a patch-package fix in production (Android GPU + CoreML), where model load/release cycles no longer accumulate native memory in heapprofd. The rebased version is compile-checked against the NDK; iOS not rebuilt yet.