diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f35edc0..895a6c4 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -96,13 +96,17 @@ Please include: Before submitting your PR, make sure you've: - [ ] Written clear and concise commit messages -- [ ] Followed existing code style and naming conventions +- [ ] Followed the [code style](docs/code-style.md), in particular: code is self-explaining and comments are a last resort - [ ] Added or updated relevant documentation (if applicable) - [ ] Added or updated unit tests (if applicable) - [ ] Verified that all existing tests pass (`npm test`) - [ ] Run linting and formatting (`npm run lint`, `npm run prettier`) -- [ ] Updated the documentation site if needed +- [ ] Updated the documentation site if needed, and for any public API change also `website/ai-usage.md`, the one-page contract agents read (take signatures from the code, not from memory) +- [ ] Added a consumer-perspective test in `package-tests/consumer-app` for any new `global` surface - [ ] Checked [API evolution rules](docs/api-evolution.md) if you touched a `global` type or added an extension point +- [ ] Added every new field to its page layout in `force-app/main/default/layouts` and, for `AsyncResult__c`, to `AsyncResultAccess` (a field missing from either is invisible to admins) +- [ ] Added any new `extras/` class to `extras/README.md` and to the tables in `website/introduction/packaged-install.md` +- [ ] Added any new website page to the sidebar in `website/.vitepress/config.mts` (`llms.txt` is generated from it) ## 📝 Types of Contributions diff --git a/README.md b/README.md index 4b576da..c598f1b 100644 --- a/README.md +++ b/README.md @@ -43,13 +43,35 @@ Visit https://async.beyondthecloud.dev/ to view the full documentation. - **Custom Metadata Configuration**: Configure the QueueableJob settings using the `QueueableJobSettings__mdt` custom metadata type to enable or disable jobs, and to control the creation of Async Result records. - **Custom Object for Async Results**: The `AsyncResult__c` custom object is created for each processed queueable job, allowing you to track the chained job status and details. -## Deploy to Salesforce +## Installation - +Two ways in. Pick one. + +### Unlocked package + + + Install Unlocked Package + + +Versioned, uninstallable, upgrades by installing the next version. Every class carries the `btcdev.` prefix and a few features need a class copied from [`extras/`](./extras). Guide: [Installing as a Package](https://async.beyondthecloud.dev/introduction/packaged-install). + +### Source deploy + + Deploy to Salesforce +Or with the CLI: + +```bash +git clone https://github.com/beyond-the-cloud-dev/async-lib.git +cd async-lib +sf project deploy start --source-dir force-app --target-org your-org +``` + +No namespace, nothing extra to set up, upgrades by redeploying the next tag. Guide: [Deploying the Source](https://async.beyondthecloud.dev/introduction/source-deploy). + ## Contributors diff --git a/docs/code-style.md b/docs/code-style.md new file mode 100644 index 0000000..812b5eb --- /dev/null +++ b/docs/code-style.md @@ -0,0 +1,93 @@ +# Code Style + +## Comments + +**Code must be self-explaining. A comment is a last resort, not a default.** + +Classes, methods, fields and variables carry the meaning. If a comment feels +necessary, that is almost always a naming or structure problem, so fix the code +instead: + +| Instead of a comment saying | Do this | +| -------------------------------------------- | ------------------------------------------------------------ | +| what a block does | extract it into a method whose name says it | +| why a `catch` swallows | name the handler method, `reportWithoutAffectingTheJob(...)` | +| that a static resets each transaction | name the field, `loggerCacheForThisTransaction` | +| that a class must be `global` to be resolved | name the method, `newInstanceOfGlobalClass(...)` | +| what a flag means | name the variable, `retryWillRestore` | + +Delete on sight: comments restating the code, section banners, commented-out +code, narration ("first we...", "now handle..."), and ApexDoc that only echoes +the signature. + +### The bar for keeping one + +A comment earns its place only when a competent Apex developer would be +**surprised or misled** without it, and no name or structure can carry it. In +practice that means a platform quirk or a deliberate choice that looks wrong: + +```apex +// A failed cast is the only way to read the runtime type with its namespace. +String.valueOf((DateTime) job); +``` + +```apex +// List.sort() does not define the order of equal elements, so equal priority needs an +// explicit tiebreak to keep jobs running in the order they were chained. +``` + +Rules of thumb that stay in prose belong in `website/explanations/`, not in the +source. If the reason is about **how consumers use the library**, document it +there and link it from the error message. If it is about **how the framework may +evolve**, it belongs in `docs/api-evolution.md`. + +### The two allowed exceptions + +**PMD suppression justification.** Every `@SuppressWarnings` carries a header +block saying why the rule is a false positive here. Without it a suppression is +indistinguishable from hiding a defect. + +```apex +/** + * PMD False Positives: + * - ExcessivePublicCount: one fluent method per job option + **/ +@SuppressWarnings('PMD.ExcessivePublicCount') +``` + +**A member that must never be deleted.** Where the reason for keeping +dead-looking code is invisible, say so, because the next maintainer will +otherwise remove it. + +```apex +/** + * Superseded by Async.Retryable.resetBeforeRetry(Integer). Nothing calls this any more. + * It cannot be deleted: dropping a global member makes the package install fail in every + * subscriber org that referenced it. See docs/api-evolution.md. + **/ +``` + +## Why ApexDoc is not enforced + +`pmd/ruleset.xml` deliberately excludes `category/apex/documentation.xml`. +Requiring `@description` and `@param` on every member produces exactly the +restatement this policy exists to remove. Editor plugins ship that rule on by +default, so expect warnings; ignore them. + +## Design + +Ordinary clean-code expectations apply, and they matter more here than in an org +codebase because this is a library whose public surface is +[frozen once shipped](/docs/api-evolution.md): + +- **KISS.** The smallest thing that solves the actual problem. No configuration + nobody asked for. +- **DRY, within reason.** Duplication in tests is often clearer than a shared + helper. Duplication in framework logic is a bug waiting to diverge. +- **SOLID.** Most relevant here is interface segregation: many small capability + interfaces beat one fat one, because Apex has no default methods, so a fat + interface can never gain a member. +- **Composition over inheritance.** A consumer has one inheritance slot. Do not + spend it. Prefer a marker interface the consumer can add to any class. +- **Guard clauses over nesting.** Early return, and let the shape of the method + show the flow. diff --git a/extras/README.md b/extras/README.md index ec03e2b..7c4a654 100644 --- a/extras/README.md +++ b/extras/README.md @@ -5,6 +5,16 @@ namespace, which is exactly why they cannot ship inside the package. Copy what you need. Rename anything to suit your project. +| File | Needed for | +| ---- | ---------- | +| `BaseQueueableJob`, `BaseChunkJob` | `deepClone()`, `restoreStateOnRetry()`, `restoreStateOnNextChunk()` | +| `AsyncJobSerializer` | `Async.requeue()` | + +All of them exist for one reason: JSON cannot cross a namespace boundary, so the conversion has to +run in your namespace. See +[Installing as a Package](https://async.beyondthecloud.dev/introduction/packaged-install) for the +full checklist. + ## `BaseQueueableJob` Only needed when Async Lib is installed as a **namespaced package**. If you deployed the source @@ -69,3 +79,22 @@ public class OddJob extends BaseQueueableJob { See [Deep Clone in Packages](https://async.beyondthecloud.dev/explanations/deep-clone-in-packages) for the full explanation and for the error messages that point back here. + +## `AsyncJobSerializer` + +Only needed when Async Lib is installed as a **namespaced package** and you use `Async.requeue()`. + +Requeue stores a snapshot of a job on `AsyncResult__c` and rebuilds it later. Both the store and +the rebuild are JSON conversions, and JSON cannot cross a namespace boundary in either direction, +so both have to run in your code. This class is that code. + +Register it once, on the `All` record of `QueueableJobSetting__mdt`: + +``` +JobSerializerClass__c = AsyncJobSerializer +``` + +It must stay `global`. Async Lib resolves it by name from inside its own namespace, and +`Type.forName` reaches nothing else across the boundary. The methods stay `public`. + +See [Requeue](https://async.beyondthecloud.dev/explanations/requeue). diff --git a/extras/classes/AsyncJobSerializer.cls b/extras/classes/AsyncJobSerializer.cls new file mode 100644 index 0000000..cfd80f4 --- /dev/null +++ b/extras/classes/AsyncJobSerializer.cls @@ -0,0 +1,20 @@ +/** + * Copy this when Async Lib is installed as a namespaced package and you use Async.requeue(). + * Register it once: QueueableJobSetting__mdt.JobSerializerClass__c = 'AsyncJobSerializer' on the + * All record. + * + * JSON cannot cross a namespace boundary in either direction, so Async Lib can neither store nor + * rebuild your job from inside its own namespace. Both halves run here instead, in yours. + * + * `global` is required. Async Lib resolves this class by name, and Type.forName reaches nothing + * else across the boundary. The methods stay public. + **/ +global class AsyncJobSerializer implements btcdev.Async.JobSerializer { + public String serialize(btcdev.QueueableJob job) { + return JSON.serialize(job); + } + + public btcdev.QueueableJob deserialize(String className, String payload) { + return (btcdev.QueueableJob) JSON.deserialize(payload, Type.forName(className)); + } +} diff --git a/extras/classes/AsyncJobSerializer.cls-meta.xml b/extras/classes/AsyncJobSerializer.cls-meta.xml new file mode 100644 index 0000000..cad713d --- /dev/null +++ b/extras/classes/AsyncJobSerializer.cls-meta.xml @@ -0,0 +1,5 @@ + + + 66.0 + Active + diff --git a/force-app/main/default/classes/Async.cls b/force-app/main/default/classes/Async.cls index 75165ce..b3d0f26 100644 --- a/force-app/main/default/classes/Async.cls +++ b/force-app/main/default/classes/Async.cls @@ -58,6 +58,14 @@ public inherited sharing class Async { QueueableManager.get().skipJob(customJobId); } + public static RequeueSummary requeue(Id resultId) { + return AsyncRequeue.run(new Set{ resultId }); + } + + public static RequeueSummary requeue(Set resultIds) { + return AsyncRequeue.run(resultIds); + } + // Inside a QueueableJob subclass the inherited `backoff` field shadows the Backoff type, so a // bare `Backoff.exponential(1)` will not compile there. Reaching the factories through Async // is collision-free in both packaged and source deployments. @@ -67,6 +75,27 @@ public inherited sharing class Async { } } + public interface OnJobEnqueued { + void onJobEnqueued(JobContext ctx); + } + + public interface OnJobSucceeded { + void onJobSucceeded(JobContext ctx); + } + + public interface OnJobFailed { + void onJobFailed(FailureContext ctx); + } + + public interface OnRetryEnqueued { + void onRetryEnqueued(FailureContext ctx); + } + + public interface JobSerializer { + String serialize(QueueableJob job); + QueueableJob deserialize(String className, String payload); + } + public interface Retryable { void resetBeforeRetry(Integer attempt); } @@ -171,6 +200,13 @@ public inherited sharing class Async { } } + @JsonAccess(serializable='always' deserializable='always') + public class RequeueSummary { + public List requeued = new List(); + public Map skipReasonByResultId = new Map(); + public Result enqueueResult; + } + public enum AsyncType { QUEUEABLE, BATCHABLE, @@ -189,6 +225,27 @@ public inherited sharing class Async { EXHAUSTED } + @JsonAccess(serializable='always' deserializable='always') + public class JobContext { + public String customJobId; + public String className; + public Id salesforceJobId; + public String chainId; + public Integer priority; + public Integer retryAttempt; + public Map info; + + public JobContext(QueueableJob job) { + this.customJobId = job.customJobId; + this.className = job.className; + this.salesforceJobId = job.salesforceJobId; + this.chainId = job.chainId; + this.priority = job.priority; + this.retryAttempt = job.retryAttempt; + this.info = job.info ?? new Map(); + } + } + @JsonAccess(serializable='always' deserializable='always') public class FailureContext { public RetryOutcome retryOutcome; @@ -198,6 +255,8 @@ public inherited sharing class Async { public Integer retryAttempt; public Integer maxRetries; public String retryHistory; + public Integer nextAttemptDelayMinutes; + public Map info; public FailureContext(QueueableJob job) { this.retryOutcome = retryOutcomeFor(job); @@ -207,6 +266,7 @@ public inherited sharing class Async { this.retryAttempt = job.retryAttempt; this.maxRetries = job.maxRetries; this.retryHistory = job.retryHistory; + this.info = job.info ?? new Map(); } private Async.RetryOutcome retryOutcomeFor(QueueableJob job) { diff --git a/force-app/main/default/classes/AsyncTest.cls b/force-app/main/default/classes/AsyncTest.cls index 79dd9e9..aeca4ae 100644 --- a/force-app/main/default/classes/AsyncTest.cls +++ b/force-app/main/default/classes/AsyncTest.cls @@ -17,11 +17,13 @@ private class AsyncTest implements Database.Batchable { private static Integer chunkCalloutPages = 0; private static Integer chunkPageFailures = 0; private static List chunkPagesRun = new List(); + private static List loggedEvents = new List(); private static final String DUPLICATE_SIGNATURE_ERROR_MESSAGE = 'Attempt to enqueue job with duplicate queueable signature'; private static final String TEST_SIGNATURE_NAME = 'SignatureName'; private static final String TEST_SCHEDULABLE_JOB_NAME = 'SchedulableTestJob'; private static final String PRIMITIVE_VALUE_INITIAL = 'INITIAL_VALUE'; private static final String CHUNK_PROCESSED = 'CHUNK_PROCESSED'; + private static final String LOGGER_CONSTRUCTOR_FAILURE = 'the logger needs a setting that is not there'; @IsTest private static void shouldEnqueue60QueueablesSuccessfully() { @@ -383,7 +385,7 @@ private class AsyncTest implements Database.Batchable { job3.uniqueName = 'job3'; QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = new Map{ + QueueableChain.jobSettingByName = new Map{ QueueableManager.QUEUEABLE_JOB_SETTING_ALL => new QueueableJobSetting__mdt( DeveloperName = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, IsDisabled__c = true @@ -412,7 +414,7 @@ private class AsyncTest implements Database.Batchable { QueueableJobTest8 job8 = new QueueableJobTest8(); QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = new Map{ + QueueableChain.jobSettingByName = new Map{ getClassNameWithNamespaceDotPrefix( 'AsyncTest.QueueableJobTest1' ) => new QueueableJobSetting__mdt( @@ -463,7 +465,7 @@ private class AsyncTest implements Database.Batchable { QueueableJobTest2 job2 = new QueueableJobTest2(); QueueableChain chain1 = new QueueableChain(); - chain1.queueableJobSettingByJobName = new Map{ + QueueableChain.jobSettingByName = new Map{ getClassNameWithNamespaceDotPrefix( 'AsyncTest.QueueableJobTest1' ) => new QueueableJobSetting__mdt( @@ -472,7 +474,7 @@ private class AsyncTest implements Database.Batchable { ) }; QueueableChain chain2 = new QueueableChain(); - chain2.queueableJobSettingByJobName = new Map{ + QueueableChain.jobSettingByName = new Map{ getClassNameWithNamespaceDotPrefix( 'AsyncTest.QueueableJobTest1' ) => new QueueableJobSetting__mdt( @@ -2229,22 +2231,299 @@ private class AsyncTest implements Database.Batchable { } @IsTest - private static void shouldGateAJobThatGetsItsRetryFromCustomMetadata() { + private static void shouldRouteToALoggerRegisteredThroughAsyncMock() { + AsyncMock.jobSettings( + new List{ + new QueueableJobSetting__mdt( + QueueableJobName__c = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, + LoggerClass__c = loggerName('RecordingLogger') + ) + } + ); + QueueableChain chain = new QueueableChain(); + QueueableManager.get().setChain(chain); + + chain.addJob(new SuccessfulQueueableTest()); + + Assert.isTrue( + loggedEvents.contains('enqueued:' + loggerName('SuccessfulQueueableTest') + ':null'), + 'A consumer must be able to register a logger without reaching into the framework: ' + + loggedEvents + ); + } + + @IsTest + private static void shouldApplyRetryDefaultsRegisteredThroughAsyncMock() { + AsyncMock.jobSettings( + new List{ + new QueueableJobSetting__mdt( + QueueableJobName__c = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, + MaxRetries__c = 3, + BackoffStrategy__c = 'FIXED', + BackoffBaseMinutes__c = 5 + ) + } + ); + QueueableChain chain = new QueueableChain(); + StateCarryingRetryJob job = new StateCarryingRetryJob(); + + chain.addJob(job); + + Assert.areEqual(3, job.maxRetries, 'Retry settings must reach the chain through the mock.'); + Assert.areEqual(5, job.backoff.delayMinutes(1), 'Backoff settings must reach it too.'); + } + + @IsTest + private static void shouldForgetMockedJobSettingsOnReset() { + AsyncMock.jobSettings( + new List{ + new QueueableJobSetting__mdt( + QueueableJobName__c = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, + MaxRetries__c = 3 + ) + } + ); + + AsyncMock.reset(); + + QueueableChain chain = new QueueableChain(); + StateCarryingRetryJob job = new StateCarryingRetryJob(); + chain.addJob(job); + + Assert.areEqual( + 0, + job.maxRetries, + 'reset() must clear mocked settings with everything else.' + ); + } + + @IsTest + private static void shouldSendEveryLifecycleEventToTheRegisteredLogger() { + QueueableChain chain = chainWithLogger(loggerName('RecordingLogger')); + QueueableManager.get().setChain(chain); + StateCarryingRetryJob job = (StateCarryingRetryJob) Async.queueable( + new StateCarryingRetryJob() + ) + .info('team', 'A') + .chain() + .job; + job.maxRetries = 1; + job.retryDecision = true; + job.continueOnJobExecuteFail = true; + + Test.startTest(); + chain.executeCurrentJob(new AsyncMock.MockQueueableContext()); + chain.enqueueNextJobIfAnyFromFinalizer(committed()); + chain.executeCurrentJob(new AsyncMock.MockQueueableContext()); + chain.enqueueNextJobIfAnyFromFinalizer(committed()); + List events = new List(loggedEvents); + Test.stopTest(); + + Assert.isTrue( + events.contains('enqueued:' + loggerName('StateCarryingRetryJob') + ':A'), + 'The enqueue event must fire and carry info(), but was: ' + events + ); + Assert.isTrue( + events.contains('retry:0:delay=null'), + 'A queued retry must fire onRetryEnqueued, but was: ' + events + ); + Assert.isTrue( + events.contains('failed:' + loggerName('StateCarryingRetryJob') + ':1'), + 'The terminal failure must fire onJobFailed, but was: ' + events + ); + } + + @IsTest + private static void shouldLetALoggerSubscribeToOneEventOnly() { + QueueableChain chain = chainWithLogger(loggerName('FailureOnlyLogger')); + QueueableManager.get().setChain(chain); + SuccessfulQueueableTest job = new SuccessfulQueueableTest(); + chain.addJob(job); + + Test.startTest(); + chain.executeCurrentJob(new AsyncMock.MockQueueableContext()); + chain.enqueueNextJobIfAnyFromFinalizer(committed()); + List events = new List(loggedEvents); + Test.stopTest(); + + Assert.isTrue(events.isEmpty(), 'A logger must only receive what it implements: ' + events); + } + + @IsTest + private static void shouldFireTheJobsOwnListenerAndTheGlobalLoggerTogether() { + QueueableChain chain = chainWithLogger(loggerName('RecordingLogger')); + QueueableManager.get().setChain(chain); + chain.addJob(new SelfLoggingJob()); + + Test.startTest(); + chain.executeCurrentJob(new AsyncMock.MockQueueableContext()); + chain.enqueueNextJobIfAnyFromFinalizer(committed()); + List events = new List(loggedEvents); + Test.stopTest(); + + Assert.isTrue( + events.contains('self:' + loggerName('SelfLoggingJob')), + 'The job own listener must fire, but was: ' + events + ); + Assert.isTrue( + events.contains('succeeded:' + loggerName('SelfLoggingJob')), + 'The registered logger must fire too, but was: ' + events + ); + } + + @IsTest + private static void shouldNotFailTheJobWhenTheLoggerThrows() { + QueueableChain chain = chainWithLogger(loggerName('ThrowingLogger')); + QueueableManager.get().setChain(chain); + SuccessfulQueueableTest job = new SuccessfulQueueableTest(); + chain.addJob(job); + + Test.startTest(); + chain.executeCurrentJob(new AsyncMock.MockQueueableContext()); + chain.enqueueNextJobIfAnyFromFinalizer(committed()); + Test.stopTest(); + + Assert.isFalse(job.hasFailed, 'A logger that throws must not fail the job it observed.'); + } + + @IsTest + private static void shouldRunNormallyWhenNoLoggerIsConfigured() { + QueueableChain chain = new QueueableChain(); + QueueableManager.get().setChain(chain); + SuccessfulQueueableTest job = new SuccessfulQueueableTest(); + chain.addJob(job); + + Test.startTest(); + chain.executeCurrentJob(new AsyncMock.MockQueueableContext()); + chain.enqueueNextJobIfAnyFromFinalizer(committed()); + Test.stopTest(); + + Assert.isFalse(job.hasFailed, 'No logger configured must be a silent no-op.'); + Assert.isTrue(loggedEvents.isEmpty(), 'Nothing should have been logged.'); + } + + @IsTest + private static void shouldWarnRatherThanHaltWhenTheLoggerClassCannotBeResolved() { + QueueableChain chain = chainWithLogger('NoSuchLoggerClass'); + QueueableManager.get().setChain(chain); + SuccessfulQueueableTest job = new SuccessfulQueueableTest(); + + chain.addJob(job); + + Assert.isTrue( + job.retryHistory.contains('NoSuchLoggerClass'), + 'The unresolvable class must be named, but was: ' + job.retryHistory + ); + Assert.isTrue( + job.retryHistory.contains('global'), + 'The warning must say the class has to be global, but was: ' + job.retryHistory + ); + } + + @IsTest + private static void shouldPreferAJobSpecificLoggerOverTheOrgWideDefault() { QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = new Map{ + QueueableChain.jobSettingByName = new Map{ QueueableManager.QUEUEABLE_JOB_SETTING_ALL => new QueueableJobSetting__mdt( QueueableJobName__c = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, - MaxRetries__c = 2 + LoggerClass__c = loggerName('RecordingLogger') + ), + loggerName('SuccessfulQueueableTest') => new QueueableJobSetting__mdt( + QueueableJobName__c = loggerName('SuccessfulQueueableTest'), + LoggerClass__c = loggerName('FailureOnlyLogger') ) }; + QueueableManager.get().setChain(chain); + + chain.addJob(new SuccessfulQueueableTest()); + + Assert.isTrue( + loggedEvents.isEmpty(), + 'The job specific logger only implements OnJobFailed, so enqueue logs nothing: ' + + loggedEvents + ); + } + + @IsTest + private static void shouldNotApplyCustomMetadataRetryToAJobThatDeclaresNoReset() { + QueueableChain chain = chainWithSettings( + new QueueableJobSetting__mdt( + QueueableJobName__c = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, + MaxRetries__c = 2 + ) + ); + UngatedRetryJob job = new UngatedRetryJob(); + + chain.addJob(job); + + Assert.areEqual( + 0, + job.maxRetries, + 'An admin turning retry on org-wide must not enable it for a job that never declared a reset.' + ); + Assert.isTrue( + job.retryHistory.contains('Async.Retryable'), + 'The job must carry the reason retry was skipped, but was: ' + job.retryHistory + ); + } + + @IsTest + private static void shouldNotHaltAJobWhenCustomMetadataNamesAnUnknownBackoffStrategy() { + QueueableChain chain = chainWithSettings( + new QueueableJobSetting__mdt( + QueueableJobName__c = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, + MaxRetries__c = 2, + BackoffStrategy__c = 'expnential' + ) + ); + StateCarryingRetryJob job = new StateCarryingRetryJob(); + + chain.addJob(job); + + Assert.areEqual(2, job.maxRetries, 'Retry still applies; only the bad backoff is dropped.'); + Assert.isNull(job.backoff, 'An unknown strategy must leave backoff unset, not throw.'); + Assert.isTrue( + job.retryHistory.contains('expnential'), + 'The typo must be named in the job history, but was: ' + job.retryHistory + ); + Assert.isTrue( + job.retryHistory.contains('EXPONENTIAL_JITTER'), + 'The valid strategies must still be listed, but was: ' + job.retryHistory + ); + } + @IsTest + private static void shouldClampCustomMetadataRetriesAboveTheCapInsteadOfThrowing() { + QueueableChain chain = chainWithSettings( + new QueueableJobSetting__mdt( + QueueableJobName__c = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, + MaxRetries__c = QueueableManager.MAX_RETRY_CAP + 40 + ) + ); + StateCarryingRetryJob job = new StateCarryingRetryJob(); + + chain.addJob(job); + + Assert.areEqual( + QueueableManager.MAX_RETRY_CAP, + job.maxRetries, + 'A number above the cap must clamp, not stop the job from enqueueing.' + ); + Assert.isTrue( + job.retryHistory.contains(String.valueOf(QueueableManager.MAX_RETRY_CAP)), + 'The clamp must be recorded, but was: ' + job.retryHistory + ); + } + + @IsTest + private static void shouldStillThrowWhenRetryIsAskedForInApexWithoutAReset() { try { - chain.addJob(new UngatedRetryJob()); - Assert.fail('Retry configured by CMDT must be gated the same as retry().'); + Async.queueable(new UngatedRetryJob()).retry(2).enqueue(); + Assert.fail('Retry written in Apex is the developer own code and must still throw.'); } catch (IllegalArgumentException ex) { Assert.isTrue( ex.getMessage().contains('Async.Retryable'), - 'CMDT-configured retry must reach the same gate, but was: ' + ex.getMessage() + 'Code-triggered misconfiguration keeps the loud gate, but was: ' + ex.getMessage() ); } } @@ -2391,7 +2670,7 @@ private class AsyncTest implements Database.Batchable { QueueableJobTest1 job = new QueueableJobTest1(); QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = new Map{ + QueueableChain.jobSettingByName = new Map{ QueueableManager.QUEUEABLE_JOB_SETTING_ALL => new QueueableJobSetting__mdt( DeveloperName = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, MaxRetries__c = 3, @@ -2414,7 +2693,7 @@ private class AsyncTest implements Database.Batchable { job.maxRetries = 5; QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = new Map{ + QueueableChain.jobSettingByName = new Map{ QueueableManager.QUEUEABLE_JOB_SETTING_ALL => new QueueableJobSetting__mdt( DeveloperName = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, MaxRetries__c = 3 @@ -2425,29 +2704,6 @@ private class AsyncTest implements Database.Batchable { Assert.areEqual(5, job.maxRetries, 'Explicit fluent config must win over CMDT.'); } - @IsTest - private static void shouldRejectCmdtMaxRetriesAboveCap() { - QueueableJobTest1 job = new QueueableJobTest1(); - - QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = new Map{ - QueueableManager.QUEUEABLE_JOB_SETTING_ALL => new QueueableJobSetting__mdt( - DeveloperName = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, - MaxRetries__c = QueueableManager.MAX_RETRY_CAP + 1 - ) - }; - - try { - chain.addJob(job); - Assert.fail('Should reject a CMDT retry count above the framework cap.'); - } catch (Exception ex) { - Assert.areEqual( - QueueableManager.ERROR_MESSAGE_MAX_RETRIES_EXCEEDS_CAP, - ex.getMessage() - ); - } - } - @IsTest private static void shouldReEnqueueRetryJobOnFailure() { FailureQueueableTest job = new FailureQueueableTest(); @@ -2505,13 +2761,6 @@ private class AsyncTest implements Database.Batchable { ); } - private static Async.Dependency dependencyOn(String customJobId, Async.Outcome outcome) { - Async.Dependency dependency = new Async.Dependency(); - dependency.resolvedTargetCustomJobId = customJobId; - dependency.requiredOutcome = outcome; - return dependency; - } - @IsTest private static void shouldRunDependentJobWhenRequiredOutcomeMatches() { SuccessfulQueueableTest jobA = new SuccessfulQueueableTest(); @@ -2808,21 +3057,12 @@ private class AsyncTest implements Database.Batchable { ); } - private static Map resultsEnabledForAll() { - return new Map{ - QueueableManager.QUEUEABLE_JOB_SETTING_ALL => new QueueableJobSetting__mdt( - DeveloperName = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, - CreateResult__c = true - ) - }; - } - @IsTest private static void shouldCreateCompletedResultWithIdentityFields() { SuccessfulQueueableTest job = new SuccessfulQueueableTest(); QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = resultsEnabledForAll(); + QueueableChain.jobSettingByName = resultsEnabledForAll(); chain.addJob(job); QueueableManager.get().setChain(chain); @@ -2852,7 +3092,7 @@ private class AsyncTest implements Database.Batchable { SuccessfulQueueableTest jobB = new SuccessfulQueueableTest(); QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = resultsEnabledForAll(); + QueueableChain.jobSettingByName = resultsEnabledForAll(); chain.addJob(jobA); jobB.dependencies = new List{ dependencyOn(jobA.customJobId, Async.Outcome.SUCCESS) @@ -2911,7 +3151,7 @@ private class AsyncTest implements Database.Batchable { IsDisabled__c = true ) ); - chain.queueableJobSettingByJobName = settings; + QueueableChain.jobSettingByName = settings; chain.addJob(job); Test.startTest(); @@ -2927,17 +3167,6 @@ private class AsyncTest implements Database.Batchable { Assert.isNotNull(result.SkipReason__c); } - private static MarkerJob markerJob(String tag, Boolean shouldFail) { - MarkerJob job = new MarkerJob(); - job.tag = tag; - job.shouldFail = shouldFail; - return job; - } - - private static Integer accountCount(String name) { - return [SELECT COUNT() FROM Account WHERE Name = :name]; - } - @IsTest private static void shouldGateChainOnDependencyOutcomesEndToEnd() { Test.startTest(); @@ -2954,7 +3183,7 @@ private class AsyncTest implements Database.Batchable { @IsTest private static void shouldRecordChainResultsEndToEnd() { QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = resultsEnabledForAll(); + QueueableChain.jobSettingByName = resultsEnabledForAll(); QueueableManager.get().setChain(chain); Test.startTest(); @@ -3031,7 +3260,7 @@ private class AsyncTest implements Database.Batchable { @IsTest private static void shouldRecordRetryHistoryOnExhaustionEndToEnd() { QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = resultsEnabledForAll(); + QueueableChain.jobSettingByName = resultsEnabledForAll(); QueueableManager.get().setChain(chain); Test.startTest(); @@ -3560,30 +3789,6 @@ private class AsyncTest implements Database.Batchable { ); } - @IsTest - private static void shouldFailFastOnAnUnknownBackoffStrategyInMetadata() { - QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = new Map{ - QueueableManager.QUEUEABLE_JOB_SETTING_ALL => new QueueableJobSetting__mdt( - DeveloperName = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, - QueueableJobName__c = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, - MaxRetries__c = 2, - BackoffStrategy__c = 'EXPONENTAIL' - ) - }; - - try { - chain.addJob(new SuccessfulQueueableTest()); - Assert.fail('A typo in BackoffStrategy__c must not silently disable backoff.'); - } catch (IllegalArgumentException ex) { - Assert.isTrue(ex.getMessage().contains('EXPONENTAIL'), 'The bad value is named.'); - Assert.isTrue( - ex.getMessage().contains('EXPONENTIAL_JITTER'), - 'The valid strategies are listed.' - ); - } - } - @IsTest private static void shouldRecordWhyAFailureWasNotRetriedWhenItsTypeIsNotListed() { SuccessfulQueueableTest job = new SuccessfulQueueableTest(); @@ -3602,7 +3807,7 @@ private class AsyncTest implements Database.Batchable { @IsTest private static void shouldKeepEveryAttemptInRetryHistoryAfterDiscardingChainChanges() { QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = resultsEnabledForAll(); + QueueableChain.jobSettingByName = resultsEnabledForAll(); QueueableManager.get().setChain(chain); Test.startTest(); @@ -3692,7 +3897,7 @@ private class AsyncTest implements Database.Batchable { @IsTest private static void shouldKeepChainRunningWhenOnFinalFailureThrows() { QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = resultsEnabledForAll(); + QueueableChain.jobSettingByName = resultsEnabledForAll(); QueueableManager.get().setChain(chain); Test.startTest(); @@ -3728,49 +3933,18 @@ private class AsyncTest implements Database.Batchable { Assert.areEqual(2, accountCount('HOOK'), 'Each failed page is its own final failure.'); } - private static Account hookRecord() { - return [SELECT Description FROM Account WHERE Name = 'HOOK' LIMIT 1]; - } + @IsTest + private static void shouldOrderFinalizerBeforeRegularJob() { + QueueableJobTest1 finalizerJob = new QueueableJobTest1(); + finalizerJob.parentCustomJobId = 'parent'; + QueueableJobTest1 regularJob = new QueueableJobTest1(); - private static String getClassNameWithNamespaceDotPrefix(String className) { - return getNamespaceDotPrefix() + className; - } - - private static String getNamespaceDotPrefix() { - String className = AsyncTest.class.getName(); - return className.contains('.') ? className.substringBefore('.') + '.' : ''; - } - - public Iterable start(Database.BatchableContext bc) { - // This is just a placeholder to start the batch. - return new List{ new Account() }; - } - - public void execute(Database.BatchableContext ctx, List scope) { - for (Account acc : scope) { - acc.Description = 'Processed by: ' + ctx.getJobId(); - } - if (!scope.isEmpty() && scope[0].Id != null) { - update scope; - } - } - - public void finish(Database.BatchableContext bc) { - insert new Account(Name = 'Batch Complete', Description = 'Job: ' + bc.getJobId()); - } - - @IsTest - private static void shouldOrderFinalizerBeforeRegularJob() { - QueueableJobTest1 finalizerJob = new QueueableJobTest1(); - finalizerJob.parentCustomJobId = 'parent'; - QueueableJobTest1 regularJob = new QueueableJobTest1(); - - Assert.areEqual(-1, finalizerJob.compareTo(regularJob), 'Finalizer sorts first.'); - Assert.areEqual( - 1, - regularJob.compareTo(finalizerJob), - 'Regular job sorts after finalizer.' - ); + Assert.areEqual(-1, finalizerJob.compareTo(regularJob), 'Finalizer sorts first.'); + Assert.areEqual( + 1, + regularJob.compareTo(finalizerJob), + 'Regular job sorts after finalizer.' + ); } @IsTest @@ -3862,7 +4036,7 @@ private class AsyncTest implements Database.Batchable { QueueableJobTest1 job = new QueueableJobTest1(); QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = new Map{ + QueueableChain.jobSettingByName = new Map{ QueueableManager.QUEUEABLE_JOB_SETTING_ALL => new QueueableJobSetting__mdt( DeveloperName = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, MaxRetries__c = 2, @@ -4093,1653 +4267,2474 @@ private class AsyncTest implements Database.Batchable { } } - private static AsyncResult__c insertAsyncResultWithAge(String status, Integer ageDays) { - AsyncResult__c result = new AsyncResult__c(Status__c = status); - insert result; - Test.setCreatedDate(result.Id, System.now().addDays(-ageDays)); - return result; - } + @IsTest + private static void shouldProcessFirstChunkThroughEnqueue() { + List accounts = createAccounts(5); - private static Integer followUpsLeftIn(QueueableChain chain) { - Integer followUps = 0; - for (QueueableJob job : chain.jobs) { - if (job instanceof MarkerJob) { - followUps++; - } - } - return followUps; - } + Test.startTest(); + Async.Result result = Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) + .chunkSize(5) + .enqueue(); + Test.stopTest(); - private static QueueableChain chainRunning(QueueableJob job) { - QueueableChain chain = new QueueableChain(); - chain.addJob(job); - QueueableManager.get().setChain(chain); - return chain; + Assert.areNotEqual(null, result.salesforceJobId); + Assert.areEqual( + 5, + [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], + 'The single chunk should process every record in the source.' + ); } - private static QueueableChain chainWithEnqueuedJob(QueueableJob job) { - QueueableChain chain = chainRunning(job); - job.chain = chain; - return chain; - } + @IsTest + private static void shouldProcessAllInMemoryChunksAcrossPages() { + List accounts = createAccounts(6); - private static AsyncMock.MockFinalizerContext rolledBack() { - return new AsyncMock.MockFinalizerContext().setResult(ParentJobResult.UNHANDLED_EXCEPTION); - } + Test.startTest(); + Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)).chunkSize(3).enqueue(); + Test.stopTest(); - private static AsyncMock.MockFinalizerContext committed() { - return new AsyncMock.MockFinalizerContext().setResult(ParentJobResult.SUCCESS); + Assert.areEqual( + 6, + [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], + 'Self-chaining must process every page of the run.' + ); } - private class DeepCloneFailJob extends QueueableJob { - public override void work() { - } - public override QueueableJob cloneForDeepCopy() { - throw new JSONException('forced deep clone failure'); - } - } + @IsTest + private static void shouldProcessAllCursorChunksAcrossPages() { + createAccounts(6); - public class SelfReferencingJob extends QueueableJob { - public SelfReferencingJob self; - public override void work() { - } - } + Test.startTest(); + Async.chunk(new MarkingChunkJob(), ChunkSource.query('SELECT Id FROM Account')) + .chunkSize(3) + .enqueue(); + Test.stopTest(); - public class AbstractFieldHoldingJob extends QueueableJob { - public Comparable held; - public override void work() { - } + Assert.areEqual( + 6, + [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], + 'A cursor must survive re-enqueue and fetch every page.' + ); } - public class AsyncLibTypeHoldingJob extends QueueableJob { - public Async.Result heldResult; - public Backoff heldBackoff; - public Async.Dependency heldDependency; - public override void work() { - } - } + @IsTest + private static void shouldCarryJobMemberStateAcrossChunks() { + List accounts = createAccounts(6); - public class DeepCloneRetryJob extends QueueableJob implements Async.Retryable { - public List attempts = new List(); - public override void work() { - attempts.add('attempt' + retryAttempt); - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); - } + Test.startTest(); + Async.chunk(new LabelingChunkJob('RUN_LABEL'), ChunkSource.of(accounts)) + .chunkSize(2) + .enqueue(); + Test.stopTest(); - public void resetBeforeRetry(Integer attempt) { - } + Assert.areEqual( + 6, + [SELECT COUNT() FROM Account WHERE Site = 'RUN_LABEL'], + 'Job member state must survive cloning and serialization across every page.' + ); } - public class DeepCloneFailRetryJob extends QueueableJob implements Async.Retryable { - public void resetBeforeRetry(Integer attempt) { - } + @IsTest + private static void shouldContinueToNextChunkAfterAFailedChunk() { + List accounts = accountsWithOneFailingRecord(); - public override void work() { - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); - } - public override QueueableJob cloneForDeepCopy() { - throw new JSONException('forced deep clone failure'); - } - } + Test.startTest(); + Async.chunk(new FailMarkedChunkJob(), ChunkSource.of(accounts)).chunkSize(2).enqueue(); + Test.stopTest(); - private class SuccessfulQueueableTest extends QueueableJob { - public override void work() { - insert new Account(Name = Async.getQueueableJobContext()?.currentJob?.uniqueName); - } + Assert.areEqual( + 4, + [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], + 'A failed chunk must not stop the remaining chunks by default.' + ); } - private class FailureQueueableTest extends QueueableJob.AllowsCallouts implements Async.Retryable { - public void resetBeforeRetry(Integer attempt) { - } + @IsTest + private static void shouldStopAfterFailedChunkWhenConfigured() { + List accounts = new List{ + new Account(Name = 'FAIL first'), + new Account(Name = 'ok second'), + new Account(Name = 'ok third'), + new Account(Name = 'ok fourth') + }; + insert accounts; - public override void work() { - insert new Account(Name = Async.getQueueableJobContext()?.currentJob?.uniqueName); - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); - } - } + Test.startTest(); + Async.chunk(new FailMarkedChunkJob(), ChunkSource.of(accounts)) + .chunkSize(2) + .stopRemainingChunksOnFailure() + .enqueue(); + Test.stopTest(); - private class SelfConfiguringRetryJob extends QueueableJob implements Async.Retryable { - public void resetBeforeRetry(Integer attempt) { - } + Assert.areEqual( + 0, + [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], + 'stopRemainingChunksOnFailure must halt the run after a failed chunk.' + ); + } - private SelfConfiguringRetryJob() { - this.maxRetries = 2; - this.backoff = Async.Backoff.fixed(4); - this.continueOnJobExecuteFail = true; - } + @IsTest + private static void shouldApplyAllChunkBuilderOptions() { + List accounts = createAccounts(2); - public override void work() { - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); - } - } + Test.startTest(); + Async.Result result = Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) + .chunkSize(2) + .priority(3) + .delay(1) + .delayBetweenChunks(1) + .retry(2) + .backoff(Backoff.fixed(1)) + .retryOn(CustomException.class) + .mockId('chunk-mock') + .keepChunkPages() + .stopRemainingChunksOnFailure() + .enqueue(); + Test.stopTest(); - private class ChainStoppingJob extends QueueableJob { - public override void work() { - Async.queueable(new StopChainFinalizer()).attachFinalizer(); - } + Assert.areNotEqual( + null, + result.salesforceJobId, + 'Every builder option should still enqueue.' + ); } - private class MarkerJob extends QueueableJob implements Async.Retryable { - public String tag; - public Boolean shouldFail = false; - public override void work() { - insert new Account(Name = tag); - if (shouldFail) { - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); - } - } + @IsTest + private static void shouldProcessFirstChunkInBulk() { + List accounts = createAccounts(200); - public void resetBeforeRetry(Integer attempt) { - } - } + Test.startTest(); + Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)).chunkSize(200).enqueue(); + Test.stopTest(); - private class StopChainFinalizer extends QueueableJob.Finalizer { - public override void work() { - Async.stopChain(); - } + Assert.areEqual( + 200, + [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], + 'A 200-record chunk should process in one bulk-safe page.' + ); } - private class FinalizerAttachingRetryJob extends QueueableJob implements Async.Retryable { - public void resetBeforeRetry(Integer attempt) { - } - - public override void work() { - insert new Account(Name = 'RETRY-PARENT'); - Async.queueable(new MarkerFinalizer()).attachFinalizer(); - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); - } - } + @IsTest + private static void shouldFetchCorrectOffsetForLaterChunk() { + List accounts = createAccounts(10); + MarkingChunkJob job = new MarkingChunkJob(); + job.getRun().configure(ChunkSource.of(accounts), 5, false, null, false); + job.getRun().position = 5; - private class MarkerFinalizer extends QueueableJob.Finalizer { - public override void work() { - insert new Account(Name = 'RETRY-FINALIZER'); - } - } + job.work(); - private abstract class HookRecordingJob extends QueueableJob implements Async.Retryable { - public void resetBeforeRetry(Integer attempt) { + Set laterHalf = new Set(); + for (Integer i = 5; i < 10; i++) { + laterHalf.add(accounts[i].Id); } - - public override void onFinalFailure(Async.FailureContext failureCtx) { - insert new Account( - Name = 'HOOK', - Description = failureCtx.retryOutcome.name() + - '|' + - (String.isBlank(failureCtx.failure?.stackTrace) ? 'NO-TRACE' : 'HAS-TRACE') + - '|' + - failureCtx.retryAttempt + - '/' + - failureCtx.maxRetries + List processed = [SELECT Id FROM Account WHERE Site = :CHUNK_PROCESSED]; + Assert.areEqual( + 5, + processed.size(), + 'Only the second page of records should be processed.' + ); + for (Account processedAccount : processed) { + Assert.isTrue( + laterHalf.contains(processedAccount.Id), + 'Offset fetch must return records from position 5 onward.' ); } } - private class FailureHookJob extends HookRecordingJob { - public override void work() { - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + @IsTest + private static void shouldRejectChunkJobRunningOutsideAChunkRun() { + try { + new MarkingChunkJob().work(); + Assert.fail('A ChunkJob without a configured run must not execute.'); + } catch (IllegalArgumentException ex) { + Assert.areEqual(ChunkRun.ERROR_MESSAGE_RUN_NOT_STARTED, ex.getMessage()); } } - private class SucceedingHookJob extends HookRecordingJob { - public override void work() { - insert new Account(Name = 'SUCCEEDED'); - } - } + @IsTest + private static void shouldCreateResultRowForChunk() { + List accounts = createAccounts(3); + QueueableChain chain = new QueueableChain(); + QueueableChain.jobSettingByName = resultsEnabledForAll(); - private class ExplodingHookJob extends QueueableJob { - public override void work() { - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); - } + Test.startTest(); + QueueableManager.get().setChain(chain); + Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)).chunkSize(3).enqueue(); + Test.stopTest(); - public override void onFinalFailure(Async.FailureContext failureCtx) { - throw new CustomException('hook exploded'); - } + List results = [ + SELECT Id, Status__c, ChainId__c, ClassName__c + FROM AsyncResult__c + ]; + Assert.areEqual(1, results.size(), 'A completed chunk page should record one AsyncResult.'); + Assert.areEqual(QueueableManager.STATUS_COMPLETED, results[0].Status__c); + Assert.areNotEqual(null, results[0].ChainId__c); } - private class FailingChunkHookJob extends ChunkJob implements Async.ChunkResettable { - public void resetBeforeNextChunk(Integer pageNumber) { - } + @IsTest + private static void shouldReturnNextChunkWhenRecordsRemain() { + MarkingChunkJob job = new MarkingChunkJob(); + job.getRun().configure(ChunkSource.of(createAccounts(10)), 5, false, null, false); - public override void work(List chunk) { - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); - } + ChunkJob nextPage = job.nextPageOrNull(); - public override void onFinalFailure(Async.FailureContext failureCtx) { - insert new Account(Name = 'HOOK', Description = failureCtx.retryOutcome.name()); - } + Assert.areNotEqual(null, nextPage, 'A next page is expected while records remain.'); + Assert.areEqual(5, nextPage.getRun().position, 'The next page must advance by chunkSize.'); } - private class JobChainingRetryJob extends QueueableJob implements Async.Retryable { - public void resetBeforeRetry(Integer attempt) { - } + @IsTest + private static void shouldNotCarryInitialDelayToLaterChunks() { + MarkingChunkJob job = new MarkingChunkJob(); + job.getRun().configure(ChunkSource.of(createAccounts(10)), 5, false, null, false); + job.delay = 5; - public override void work() { - insert new Account(Name = 'CHAIN-PARENT'); - Async.queueable(markerJob('CHAIN-FOLLOWUP', false)).chain(); - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); - } - } + ChunkJob nextPage = job.nextPageOrNull(); - private class ChainingFailureJob extends QueueableJob { - public override void work() { - Async.queueable(markerJob('ROLLED-BACK-FOLLOWUP', false)).chain(); - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); - } + Assert.areEqual(null, nextPage.delay, 'An initial delay must not repeat on every page.'); } - private class FinalizerAndJobChainingFailureJob extends QueueableJob { - public override void work() { - Async.queueable(new MarkerFinalizer()).attachFinalizer(); - Async.queueable(markerJob('ROLLED-BACK-FOLLOWUP', false)).chain(); - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); - } - } + @IsTest + private static void shouldApplyDelayBetweenChunks() { + MarkingChunkJob job = new MarkingChunkJob(); + job.getRun().configure(ChunkSource.of(createAccounts(10)), 5, false, 3, false); - private class ChainStoppingFailureJob extends QueueableJob implements Async.Retryable { - public void resetBeforeRetry(Integer attempt) { - } + ChunkJob nextPage = job.nextPageOrNull(); - public override void work() { - Async.stopChain(); - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); - } + Assert.areEqual(3, nextPage.delay, 'delayBetweenChunks must throttle each later page.'); } - private class JobSkippingFailureJob extends QueueableJob { - public String targetCustomJobId; - public override void work() { - Async.skipJob(targetCustomJobId); - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); - } - } + @IsTest + private static void shouldRejectDeepCloneChunkJob() { + MarkingChunkJob job = new MarkingChunkJob(); + job.deepClone = true; - private class NonRetryableJob extends QueueableJob { - public override void work() { - } - public override Boolean isRetryable(Exception ex) { - return false; + try { + Async.chunk(job, ChunkSource.of(createAccounts(2))); + Assert.fail('A deepClone ChunkJob should be rejected at build time.'); + } catch (Exception ex) { + Assert.areEqual(ChunkBuilder.ERROR_MESSAGE_DEEP_CLONE_UNSUPPORTED, ex.getMessage()); } } - private class MessageVetoJob extends QueueableJob { - public override void work() { - } - public override Boolean isRetryable(Exception ex) { - return !ex.getMessage().containsIgnoreCase('permanent'); - } - } + @IsTest + private static void shouldPruneSettledPagesByDefault() { + MarkingChunkJob job = new MarkingChunkJob(); + job.getRun().configure(ChunkSource.of(createAccounts(4)), 2, false, null, false); - private class ThrowingClassifierJob extends QueueableJob { - public override void work() { - } - public override Boolean isRetryable(Exception ex) { - throw new CustomException('classifier blew up'); - } + Assert.isFalse(job.getRun().keepPages, 'Settled pages are pruned by default.'); } - private class ResettableRetryJob extends QueueableJob implements Async.Retryable { - public Boolean wasReset = false; - public Integer resetAttempt; - public override void work() { - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); - } - public void resetBeforeRetry(Integer attempt) { - this.wasReset = true; - this.resetAttempt = attempt; - } - } + @IsTest + private static void shouldKeepSettledPagesWhenRequested() { + List accounts = createAccounts(6); - private class UngatedRetryJob extends QueueableJob { - public override void work() { - } + Test.startTest(); + Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) + .chunkSize(2) + .keepChunkPages() + .enqueue(); + Test.stopTest(); + + Assert.areEqual( + 6, + [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], + 'A kept-pages run still processes every page.' + ); } - public class RetryingChunkJob extends ChunkJob implements Async.Retryable, Async.ChunkResettable { - public List touched = new List(); + @IsTest + private static void shouldReturnNoNextChunkAtEndOfSource() { + MarkingChunkJob job = new MarkingChunkJob(); + job.getRun().configure(ChunkSource.of(createAccounts(10)), 5, false, null, false); + job.getRun().position = 5; - public override void work(List chunk) { - String firstOfPage = (String) chunk[0].get('Name'); - AsyncTest.chunkPagesRun.add(firstOfPage + ':' + touched.size()); - touched.add('page'); - if (firstOfPage == 'P3' && AsyncTest.chunkPageFailures == 0) { - AsyncTest.chunkPageFailures++; - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); - } - } + Assert.areEqual(null, job.nextPageOrNull(), 'No page should follow the final chunk.'); + } - public void resetBeforeRetry(Integer attempt) { - } + @IsTest + private static void shouldHaltRunAfterFailedPageWhenConfigured() { + MarkingChunkJob stopping = new MarkingChunkJob(); + stopping.getRun().configure(ChunkSource.of(createAccounts(10)), 5, true, null, false); - public void resetBeforeNextChunk(Integer pageNumber) { - } + Assert.isTrue( + stopping.getRun().isHaltedBy(true), + 'A failed page must halt the run when stopRemainingChunksOnFailure is set.' + ); + Assert.isFalse( + stopping.getRun().isHaltedBy(false), + 'A page that succeeded must not halt the run.' + ); } - private class GateTrippingParentJob extends QueueableJob { - public override void work() { - insert new Account(Name = 'GATE-PARENT-RAN'); - Async.queueable(new UngatedRetryJob()).retry(2).chain(); - } - } + @IsTest + private static void shouldContinueRemainingChunksOnFailureByDefault() { + MarkingChunkJob continuing = new MarkingChunkJob(); + continuing.getRun().configure(ChunkSource.of(createAccounts(10)), 5, false, null, false); - public class UngatedChunkJob extends ChunkJob { - public override void work(List chunk) { - } + Assert.isFalse( + continuing.getRun().isHaltedBy(true), + 'A failed chunk must not halt the run by default.' + ); + Assert.areNotEqual( + null, + continuing.nextPageOrNull(), + 'The run continues to the next page after a failed page.' + ); } - private class OkCalloutMock implements HttpCalloutMock { - public HttpResponse respond(HttpRequest request) { - HttpResponse response = new HttpResponse(); - response.setStatusCode(200); - return response; - } - } + @IsTest + private static void shouldCarryFailedPageFlagToLaterPages() { + MarkingChunkJob job = new MarkingChunkJob(); + job.getRun().configure(ChunkSource.of(createAccounts(6)), 2, false, null, false); + job.hasFailed = true; - public class MarkerCalloutChunkJob extends ChunkJob implements Database.AllowsCallouts, Async.ChunkResettable { - public override void work(List chunk) { - HttpRequest request = new HttpRequest(); - request.setEndpoint('https://example.com'); - request.setMethod('GET'); - AsyncTest.chunkCalloutStatus = new Http().send(request).getStatusCode(); - AsyncTest.chunkCalloutPages++; - } + ChunkJob nextPage = job.nextPageOrNull(); - public void resetBeforeNextChunk(Integer pageNumber) { - } + Assert.isTrue( + nextPage.getRun().hasFailedPage, + 'The run remembers a failed page so the run outcome stays FAILURE.' + ); + Assert.isFalse(nextPage.hasFailed, 'The next page starts clean.'); } - public class StateCarryingRetryJob extends QueueableJob implements Async.Retryable { - public List seen = new List(); - - public override void work() { - insert new Account(Name = 'RETRY-SIZE-' + seen.size()); - seen.add('attempt'); - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); - } + @IsTest + private static void shouldTreatEmptySourceAsNoOp() { + Async.Result result = Async.chunk( + new MarkingChunkJob(), + ChunkSource.of(new List()) + ) + .enqueue(); - public void resetBeforeRetry(Integer attempt) { - } + Assert.areEqual(null, result.salesforceJobId, 'An empty source should enqueue nothing.'); + Assert.isTrue(result.queueableChainState.jobs.isEmpty()); } - public class StateCarryingChunkJob extends ChunkJob implements Async.ChunkResettable { - public List seen = new List(); - public List pagesReset = new List(); - - public override void work(List chunk) { - insert new Account(Name = 'CHUNK-SIZE-' + seen.size()); - seen.add('page'); - } + @IsTest + private static void shouldStillRunChainedJobsWhenSourceIsEmpty() { + Test.startTest(); + Async.queueable(new ProcessedCountMarkerJob('EMPTY_SOURCE')) + .chunk(new MarkingChunkJob(), ChunkSource.of(new List())) + .enqueue(); + Test.stopTest(); - public void resetBeforeNextChunk(Integer pageNumber) { - pagesReset.add(pageNumber); - } + Assert.areEqual( + 1, + [SELECT COUNT() FROM Account WHERE Name = 'EMPTY_SOURCE'], + 'An empty chunk source must not swallow the jobs already chained.' + ); } - private class LegacyResetForRetryJob extends QueueableJob implements Async.Retryable { - public Boolean legacyHookRan = false; - public override void work() { - throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); - } - public override void resetForRetry() { - this.legacyHookRan = true; - } - public void resetBeforeRetry(Integer attempt) { + @IsTest + private static void shouldRejectInvalidChunkSize() { + ChunkBuilder builder = Async.chunk( + new MarkingChunkJob(), + ChunkSource.of(createAccounts(1)) + ); + for (Integer invalid : new List{ 0, -1, null }) { + try { + builder.chunkSize(invalid); + Assert.fail('chunkSize ' + invalid + ' should be rejected.'); + } catch (Exception ex) { + Assert.areEqual(ChunkBuilder.ERROR_MESSAGE_INVALID_CHUNK_SIZE, ex.getMessage()); + } } } - private class QueueableTestFinalizer extends QueueableJob.Finalizer { - public override void work() { - FinalizerContext finalizerCtx = Async.getQueueableJobContext()?.finalizerCtx; - insert new Account( - Name = Async.getQueueableJobContext()?.currentJob?.uniqueName, - Description = finalizerCtx?.getResult() == ParentJobResult.SUCCESS - ? 'Success' - : finalizerCtx?.getException()?.getMessage() + @IsTest + private static void shouldRejectChunkSizeAboveTheCursorFetchCap() { + createAccounts(1); + ChunkBuilder builder = Async.chunk( + new MarkingChunkJob(), + ChunkSource.query('SELECT Id FROM Account') + ); + + try { + builder.chunkSize(2001); + Assert.fail('A cursor source cannot page more than 2000 records at a time.'); + } catch (Exception ex) { + Assert.areEqual( + 'chunkSize must be between 1 and 2000 for this source', + ex.getMessage() ); } + Assert.areNotEqual( + null, + builder.chunkSize(2000), + 'A chunk size at the cursor fetch cap is allowed.' + ); } - private class FinalizerErrorQueueableTest extends QueueableJob { - public override void work() { - Async.queueable(new SuccessfulQueueableTest()).attachFinalizer(); - } - } + @IsTest + private static void shouldNotCapChunkSizeForAnInMemorySource() { + ChunkBuilder builder = Async.chunk( + new MarkingChunkJob(), + ChunkSource.of(createAccounts(1)) + ); - private class ChainedQueueableJob extends QueueableJob { - private Integer chainDepthLimit = 1; - private Integer currentChainDepth = 0; + Assert.areEqual( + null, + ChunkSource.of(new List()).maxChunkSize(), + 'An in-memory source has no cursor fetch ceiling.' + ); + Assert.areNotEqual( + null, + builder.chunkSize(5000), + 'An in-memory source is bounded by the job body, not by a page ceiling.' + ); + } - public ChainedQueueableJob(Integer chainDepthLimit) { - this.chainDepthLimit = chainDepthLimit; - } + @IsTest + private static void shouldExposeCursorFetchCapOnCursorSource() { + createAccounts(1); - public override void work() { - currentChainDepth++; - insert new Account(Name = Async.getQueueableJobContext()?.currentJob?.uniqueName); - if (currentChainDepth < chainDepthLimit) { - Async.queueable(this).enqueue(); - } - } + Assert.areEqual(2000, ChunkSource.query('SELECT Id FROM Account').maxChunkSize()); } - public class QueueableJobTest1 extends QueueableJob implements Async.Retryable { - public QueueableJobTest2 complexMember = new QueueableJobTest2(); - public override void work() { + @IsTest + private static void shouldRejectNullJobAndSource() { + try { + Async.chunk(null, ChunkSource.of(createAccounts(1))); + Assert.fail('A null job should be rejected.'); + } catch (Exception ex) { + Assert.areEqual(ChunkBuilder.ERROR_MESSAGE_NULL_JOB, ex.getMessage()); } - - public void resetBeforeRetry(Integer attempt) { + try { + Async.chunk(new MarkingChunkJob(), null); + Assert.fail('A null source should be rejected.'); + } catch (Exception ex) { + Assert.areEqual(ChunkBuilder.ERROR_MESSAGE_NULL_SOURCE, ex.getMessage()); } } - private class QueueableJobTest2 extends QueueableJob { - public String primitiveMember = PRIMITIVE_VALUE_INITIAL; - public override void work() { + @IsTest + private static void shouldRejectRetryAboveCapOnChunk() { + ChunkBuilder builder = Async.chunk( + new MarkingChunkJob(), + ChunkSource.of(createAccounts(1)) + ); + try { + builder.retry(QueueableManager.MAX_RETRY_CAP + 1); + Assert.fail('Retry above the cap should be rejected.'); + } catch (Exception ex) { + Assert.areEqual( + QueueableManager.ERROR_MESSAGE_MAX_RETRIES_EXCEEDS_CAP, + ex.getMessage() + ); } } - private class QueueableJobTest3 extends QueueableJob { - public override void work() { - } - } + @IsTest + private static void shouldAccumulateJobStateAcrossChunks() { + List accounts = createAccounts(6); - private class QueueableJobTest4 extends QueueableJob { - public override void work() { - } - } + Test.startTest(); + Async.chunk(new TotallingChunkJob(), ChunkSource.of(accounts)).chunkSize(2).enqueue(); + Test.stopTest(); - private class QueueableJobTest5 extends QueueableJob { - public override void work() { + Set runningTotals = new Set(); + for (Account marker : [SELECT Site FROM Account WHERE Name = 'RUNNING_TOTAL']) { + runningTotals.add(marker.Site); } + Assert.areEqual( + new Set{ '2', '4', '6' }, + runningTotals, + 'A member mutated in work() keeps accumulating on every later page.' + ); } - private class QueueableJobTest6 extends QueueableJob { - public override void work() { - } - } - - private class QueueableJobTest7 extends QueueableJob { - public override void work() { - } - } - - private class QueueableJobTest8 extends QueueableJob { - public override void work() { - } - } - - private class SchedulableTest implements Schedulable { - public void execute(SchedulableContext ctx) { - } - } - - private class CustomException extends Exception { - } - - public class ParentJobWithFinalizer extends QueueableJob { - private String mockId; - - public ParentJobWithFinalizer(String mockId) { - this.mockId = mockId; - } - - public override void work() { - Async.queueable(new ErrorHandlerFinalizer()).mockId(mockId).attachFinalizer(); - } - } - - public class ErrorHandlerFinalizer extends QueueableJob.Finalizer { - public override void work() { - FinalizerContext ctx = this.finalizerCtx; - if (ctx?.getResult() == ParentJobResult.UNHANDLED_EXCEPTION) { - insert new Account( - Name = 'Error Log', - Description = ctx.getException()?.getMessage() - ); - } - } - } - - public class AccountCreatorJob extends QueueableJob { - private String accountName; - - public AccountCreatorJob(String accountName) { - this.accountName = accountName; - } - - public override void work() { - Id jobId = this.queueableCtx?.getJobId(); - insert new Account(Name = accountName, Description = 'Job: ' + jobId); + @IsTest + private static void shouldPageRecordsFromAConsumerDefinedSource() { + Test.startTest(); + Async.chunk(new CountingChunkJob(), new SyntheticChunkSource(2500)) + .chunkSize(1000) + .enqueue(); + Test.stopTest(); + + Set runningTotals = new Set(); + for (Account marker : [SELECT Site FROM Account WHERE Name = 'SYNTHETIC_TOTAL']) { + runningTotals.add(marker.Site); } + Assert.areEqual( + new Set{ '1000', '2000', '2500' }, + runningTotals, + 'A custom ChunkSource pages 2500 fabricated records without any DML behind it.' + ); } @IsTest - private static void shouldProcessFirstChunkThroughEnqueue() { - List accounts = createAccounts(5); + private static void shouldInjectQueueableMockIntoAChunkPage() { + List accounts = createAccounts(2); + Id mockJobId = fakeAsyncApexJobId(); + AsyncMock.whenQueueable('chunk-page') + .thenReturn(new AsyncMock.MockQueueableContext().setJobId(mockJobId)); Test.startTest(); - Async.Result result = Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) - .chunkSize(5) + Async.chunk(new ContextReadingChunkJob(), ChunkSource.of(accounts)) + .chunkSize(2) + .mockId('chunk-page') .enqueue(); Test.stopTest(); - Assert.areNotEqual(null, result.salesforceJobId); Assert.areEqual( - 5, - [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], - 'The single chunk should process every record in the source.' + String.valueOf(mockJobId), + [SELECT Site FROM Account WHERE Name = 'MOCKED_CONTEXT' LIMIT 1].Site, + 'A chunk page reads the mocked QueueableContext like any other job.' ); } @IsTest - private static void shouldProcessAllInMemoryChunksAcrossPages() { - List accounts = createAccounts(6); + private static void shouldFailAJobFromAQueueableMock() { + QueueableChain chain = new QueueableChain(); + QueueableChain.jobSettingByName = resultsEnabledForAll(); + AsyncMock.whenQueueable('mocked-failure') + .thenThrow(new CustomException(CUSTOM_ERROR_MESSAGE)); Test.startTest(); - Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)).chunkSize(3).enqueue(); + QueueableManager.get().setChain(chain); + Async.queueable(new AccountCreatorJob('never runs')) + .mockId('mocked-failure') + .continueOnJobExecuteFail() + .enqueue(); Test.stopTest(); + AsyncResult__c result = [ + SELECT Status__c, ExceptionType__c, ExceptionMessage__c + FROM AsyncResult__c + LIMIT 1 + ]; Assert.areEqual( - 6, - [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], - 'Self-chaining must process every page of the run.' + QueueableManager.STATUS_FAILED, + result.Status__c, + 'A mocked failure settles through the same path a real one does.' + ); + Assert.areEqual(CUSTOM_ERROR_MESSAGE, result.ExceptionMessage__c); + Assert.areEqual( + 0, + [SELECT COUNT() FROM Account], + 'The job body never runs when its execution is mocked to throw.' ); } @IsTest - private static void shouldProcessAllCursorChunksAcrossPages() { - createAccounts(6); + private static void shouldFailOnlyTheMockedChunkPage() { + List accounts = createAccounts(6); + AsyncMock.whenQueueable('recalc-run') + .thenReturn(new AsyncMock.MockQueueableContext()) + .thenThrow(new CustomException(CUSTOM_ERROR_MESSAGE)) + .thenReturn(new AsyncMock.MockQueueableContext()); Test.startTest(); - Async.chunk(new MarkingChunkJob(), ChunkSource.query('SELECT Id FROM Account')) - .chunkSize(3) + Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) + .chunkSize(2) + .mockId('recalc-run') .enqueue(); Test.stopTest(); Assert.areEqual( - 6, + 4, [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], - 'A cursor must survive re-enqueue and fetch every page.' + 'The mock queue picks which page fails; the rest of the run still processes.' ); } @IsTest - private static void shouldCarryJobMemberStateAcrossChunks() { + private static void shouldHaltRunFromAMockedPageFailure() { List accounts = createAccounts(6); + AsyncMock.whenQueueable('recalc-run') + .thenReturn(new AsyncMock.MockQueueableContext()) + .thenThrow(new CustomException(CUSTOM_ERROR_MESSAGE)); Test.startTest(); - Async.chunk(new LabelingChunkJob('RUN_LABEL'), ChunkSource.of(accounts)) + Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) .chunkSize(2) + .mockId('recalc-run') + .stopRemainingChunksOnFailure() .enqueue(); Test.stopTest(); Assert.areEqual( - 6, - [SELECT COUNT() FROM Account WHERE Site = 'RUN_LABEL'], - 'Job member state must survive cloning and serialization across every page.' + 2, + [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], + 'A mocked page failure halts the run like a real one.' ); } @IsTest - private static void shouldContinueToNextChunkAfterAFailedChunk() { - List accounts = accountsWithOneFailingRecord(); + private static void shouldMockFinalizerOutcomeForAChunkPage() { + List accounts = createAccounts(2); + AsyncMock.whenFinalizer('chunk-error-handler').thenThrow(new DmlException('Page blew up')); Test.startTest(); - Async.chunk(new FailMarkedChunkJob(), ChunkSource.of(accounts)).chunkSize(2).enqueue(); + Async.chunk(new FinalizerAttachingChunkJob('chunk-error-handler'), ChunkSource.of(accounts)) + .chunkSize(2) + .enqueue(); Test.stopTest(); Assert.areEqual( - 4, - [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], - 'A failed chunk must not stop the remaining chunks by default.' + 'Page blew up', + [SELECT Description FROM Account WHERE Name = 'Error Log' LIMIT 1].Description, + 'A page finalizer can be told its page failed without the page actually failing.' ); } @IsTest - private static void shouldStopAfterFailedChunkWhenConfigured() { - List accounts = new List{ - new Account(Name = 'FAIL first'), - new Account(Name = 'ok second'), - new Account(Name = 'ok third'), - new Account(Name = 'ok fourth') - }; - insert accounts; + private static void shouldFailAndRetryAPageWhenTheSourceThrows() { + QueueableChain chain = new QueueableChain(); + QueueableChain.jobSettingByName = resultsEnabledForAll(); Test.startTest(); - Async.chunk(new FailMarkedChunkJob(), ChunkSource.of(accounts)) + QueueableManager.get().setChain(chain); + Async.chunk(new MarkingChunkJob(), new ExplodingChunkSource()) .chunkSize(2) - .stopRemainingChunksOnFailure() + .retry(1) .enqueue(); Test.stopTest(); + AsyncResult__c result = [ + SELECT Status__c, ExceptionMessage__c, RetryAttempts__c + FROM AsyncResult__c + LIMIT 1 + ]; Assert.areEqual( - 0, - [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], - 'stopRemainingChunksOnFailure must halt the run after a failed chunk.' + QueueableManager.STATUS_FAILED, + result.Status__c, + 'A source that throws fails the page instead of silently ending the run.' ); + Assert.areEqual(CUSTOM_ERROR_MESSAGE, result.ExceptionMessage__c); + Assert.areEqual(1, result.RetryAttempts__c, 'The page retried before it settled.'); } @IsTest - private static void shouldApplyAllChunkBuilderOptions() { + private static void shouldChainRunWithoutEnqueueingIt() { List accounts = createAccounts(2); - Test.startTest(); Async.Result result = Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) .chunkSize(2) - .priority(3) - .delay(1) - .delayBetweenChunks(1) - .retry(2) - .backoff(Backoff.fixed(1)) - .retryOn(CustomException.class) - .mockId('chunk-mock') - .keepChunkPages() - .stopRemainingChunksOnFailure() - .enqueue(); - Test.stopTest(); + .chain(); Assert.areNotEqual( null, - result.salesforceJobId, - 'Every builder option should still enqueue.' + result.customJobId, + 'chain() returns the run handle so later jobs can depend on it.' + ); + Assert.areEqual( + 0, + [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], + 'chain() adds the run to the chain without enqueuing anything.' ); } @IsTest - private static void shouldProcessFirstChunkInBulk() { - List accounts = createAccounts(200); + private static void shouldRunASecondChunkRunAfterTheFirstOneFinishes() { + List accounts = createAccounts(4); Test.startTest(); - Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)).chunkSize(200).enqueue(); + Async.chunk(new LabelingChunkJob('FIRST'), ChunkSource.of(accounts)) + .chunkSize(2) + .chunk(new LabelingChunkJob('SECOND'), ChunkSource.of(accounts)) + .chunkSize(2) + .enqueue(); Test.stopTest(); Assert.areEqual( - 200, - [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], - 'A 200-record chunk should process in one bulk-safe page.' + 4, + [SELECT COUNT() FROM Account WHERE Site = 'SECOND'], + 'The second run overwrites the first, so it ran after every page of run one.' ); } @IsTest - private static void shouldFetchCorrectOffsetForLaterChunk() { - List accounts = createAccounts(10); - MarkingChunkJob job = new MarkingChunkJob(); - job.getRun().configure(ChunkSource.of(accounts), 5, false, null, false); - job.getRun().position = 5; + private static void shouldGateASecondChunkRunOnTheFirstRunOutcome() { + List accounts = accountsWithOneFailingRecord(); - job.work(); + Test.startTest(); + Async.chunk(new FailMarkedChunkJob(), ChunkSource.of(accounts)) + .chunkSize(2) + .chunk(new LabelingChunkJob('SECOND'), ChunkSource.of(accounts)) + .chunkSize(2) + .dependsOn(Async.afterPrevious().succeeded()) + .enqueue(); + Test.stopTest(); - Set laterHalf = new Set(); - for (Integer i = 5; i < 10; i++) { - laterHalf.add(accounts[i].Id); - } - List processed = [SELECT Id FROM Account WHERE Site = :CHUNK_PROCESSED]; Assert.areEqual( - 5, - processed.size(), - 'Only the second page of records should be processed.' + 0, + [SELECT COUNT() FROM Account WHERE Site = 'SECOND'], + 'dependsOn(previous) resolves against the first run outcome, which failed.' ); - for (Account processedAccount : processed) { - Assert.isTrue( - laterHalf.contains(processedAccount.Id), - 'Offset fetch must return records from position 5 onward.' - ); - } } @IsTest - private static void shouldRejectChunkJobRunningOutsideAChunkRun() { - try { - new MarkingChunkJob().work(); - Assert.fail('A ChunkJob without a configured run must not execute.'); - } catch (IllegalArgumentException ex) { - Assert.areEqual(ChunkRun.ERROR_MESSAGE_RUN_NOT_STARTED, ex.getMessage()); - } - } - - @IsTest - private static void shouldCreateResultRowForChunk() { - List accounts = createAccounts(3); - QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = resultsEnabledForAll(); - - Test.startTest(); - QueueableManager.get().setChain(chain); - Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)).chunkSize(3).enqueue(); - Test.stopTest(); - - List results = [ - SELECT Id, Status__c, ChainId__c, ClassName__c - FROM AsyncResult__c - ]; - Assert.areEqual(1, results.size(), 'A completed chunk page should record one AsyncResult.'); - Assert.areEqual(QueueableManager.STATUS_COMPLETED, results[0].Status__c); - Assert.areNotEqual(null, results[0].ChainId__c); - } - - @IsTest - private static void shouldReturnNextChunkWhenRecordsRemain() { - MarkingChunkJob job = new MarkingChunkJob(); - job.getRun().configure(ChunkSource.of(createAccounts(10)), 5, false, null, false); - - ChunkJob nextPage = job.nextPageOrNull(); - - Assert.areNotEqual(null, nextPage, 'A next page is expected while records remain.'); - Assert.areEqual(5, nextPage.getRun().position, 'The next page must advance by chunkSize.'); - } - - @IsTest - private static void shouldNotCarryInitialDelayToLaterChunks() { - MarkingChunkJob job = new MarkingChunkJob(); - job.getRun().configure(ChunkSource.of(createAccounts(10)), 5, false, null, false); - job.delay = 5; + private static void shouldOpenACursorInTheRequestedAccessLevel() { + createAccounts(2); - ChunkJob nextPage = job.nextPageOrNull(); + ChunkSource userMode = ChunkSource.query('SELECT Id FROM Account', AccessLevel.USER_MODE); + ChunkSource systemMode = ChunkSource.query( + 'SELECT Id FROM Account WHERE Name LIKE :prefix', + new Map{ 'prefix' => 'Chunk%' }, + AccessLevel.SYSTEM_MODE + ); - Assert.areEqual(null, nextPage.delay, 'An initial delay must not repeat on every page.'); + Assert.areEqual(2, userMode.getNumRecords(), 'The admin running the test sees both.'); + Assert.areEqual(2, systemMode.getNumRecords(), 'Binds and access level combine.'); } @IsTest - private static void shouldApplyDelayBetweenChunks() { - MarkingChunkJob job = new MarkingChunkJob(); - job.getRun().configure(ChunkSource.of(createAccounts(10)), 5, false, 3, false); - - ChunkJob nextPage = job.nextPageOrNull(); + private static void shouldChainNothingWhenSourceIsEmpty() { + Async.Result result = Async.chunk( + new MarkingChunkJob(), + ChunkSource.of(new List()) + ) + .chain(); - Assert.areEqual(3, nextPage.delay, 'delayBetweenChunks must throttle each later page.'); + Assert.areEqual(null, result.customJobId, 'An empty source chains no run.'); + Assert.isTrue(result.queueableChainState.jobs.isEmpty()); } @IsTest - private static void shouldRejectDeepCloneChunkJob() { - MarkingChunkJob job = new MarkingChunkJob(); - job.deepClone = true; - - try { - Async.chunk(job, ChunkSource.of(createAccounts(2))); - Assert.fail('A deepClone ChunkJob should be rejected at build time.'); - } catch (Exception ex) { - Assert.areEqual(ChunkBuilder.ERROR_MESSAGE_DEEP_CLONE_UNSUPPORTED, ex.getMessage()); - } - } + private static void shouldRunChunkRunWhenItsDependencySucceeded() { + List accounts = createAccounts(4); - @IsTest - private static void shouldPruneSettledPagesByDefault() { - MarkingChunkJob job = new MarkingChunkJob(); - job.getRun().configure(ChunkSource.of(createAccounts(4)), 2, false, null, false); + Test.startTest(); + Async.queueable(new ProcessedCountMarkerJob('GATE')) + .chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) + .chunkSize(2) + .dependsOn(Async.afterPrevious().succeeded()) + .enqueue(); + Test.stopTest(); - Assert.isFalse(job.getRun().keepPages, 'Settled pages are pruned by default.'); + Assert.areEqual( + 4, + [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], + 'The run starts once the job it depends on succeeded.' + ); } @IsTest - private static void shouldKeepSettledPagesWhenRequested() { - List accounts = createAccounts(6); + private static void shouldSkipChunkRunWhenItsDependencyFailed() { + List accounts = createAccounts(4); Test.startTest(); - Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) + Async.queueable(new FailureQueueableTest()) + .continueOnJobExecuteFail() + .chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) .chunkSize(2) - .keepChunkPages() + .dependsOn(Async.afterPrevious().succeeded()) .enqueue(); Test.stopTest(); Assert.areEqual( - 6, + 0, [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], - 'A kept-pages run still processes every page.' + 'The whole run is skipped when the job it depends on failed.' ); } @IsTest - private static void shouldReturnNoNextChunkAtEndOfSource() { - MarkingChunkJob job = new MarkingChunkJob(); - job.getRun().configure(ChunkSource.of(createAccounts(10)), 5, false, null, false); - job.getRun().position = 5; - - Assert.areEqual(null, job.nextPageOrNull(), 'No page should follow the final chunk.'); + private static void shouldRejectInvalidDependencyOnChunk() { + ChunkBuilder builder = Async.chunk( + new MarkingChunkJob(), + ChunkSource.of(createAccounts(1)) + ); + try { + builder.dependsOn(null); + Assert.fail('A dependency without an outcome should be rejected.'); + } catch (Exception ex) { + Assert.areEqual(QueueableManager.ERROR_MESSAGE_INVALID_DEPENDENCY, ex.getMessage()); + } + try { + builder.dependsOn(Async.afterPrevious().succeeded()); + Assert.fail('afterPrevious() without a previously chained job should be rejected.'); + } catch (Exception ex) { + Assert.areEqual( + QueueableManager.ERROR_MESSAGE_DEPENDS_ON_PREVIOUS_WITHOUT_JOB, + ex.getMessage() + ); + } } @IsTest - private static void shouldHaltRunAfterFailedPageWhenConfigured() { - MarkingChunkJob stopping = new MarkingChunkJob(); - stopping.getRun().configure(ChunkSource.of(createAccounts(10)), 5, true, null, false); + private static void shouldYieldIdOnlyShellsWhenSourceIsBuiltFromIds() { + List accounts = createAccounts(5); + ChunkSource source = ChunkSource.ofIds(new Map(accounts).keySet()); - Assert.isTrue( - stopping.getRun().isHaltedBy(true), - 'A failed page must halt the run when stopRemainingChunksOnFailure is set.' - ); - Assert.isFalse( - stopping.getRun().isHaltedBy(false), - 'A page that succeeded must not halt the run.' + List page = source.fetch(0, 5); + + Assert.areEqual(5, page.size()); + Assert.areEqual( + new Map(accounts).keySet(), + new Map(page).keySet(), + 'A job reads ids straight off the page with new Map(chunk).keySet().' ); } @IsTest - private static void shouldContinueRemainingChunksOnFailureByDefault() { - MarkingChunkJob continuing = new MarkingChunkJob(); - continuing.getRun().configure(ChunkSource.of(createAccounts(10)), 5, false, null, false); + private static void shouldExposeInMemorySourceRecordsAndSize() { + List accounts = createAccounts(5); + ChunkSource source = ChunkSource.of(accounts); - Assert.isFalse( - continuing.getRun().isHaltedBy(true), - 'A failed chunk must not halt the run by default.' - ); - Assert.areNotEqual( - null, - continuing.nextPageOrNull(), - 'The run continues to the next page after a failed page.' + Assert.areEqual(5, source.getNumRecords()); + Assert.areEqual(2, source.fetch(0, 2).size()); + Assert.areEqual( + 1, + source.fetch(4, 5).size(), + 'A short final page must clamp to the remainder.' ); } @IsTest - private static void shouldCarryFailedPageFlagToLaterPages() { - MarkingChunkJob job = new MarkingChunkJob(); - job.getRun().configure(ChunkSource.of(createAccounts(6)), 2, false, null, false); - job.hasFailed = true; + private static void shouldBuildIdSourceRecordsWithIds() { + List accounts = createAccounts(3); + Set ids = new Map(accounts).keySet(); + ChunkSource source = ChunkSource.ofIds(ids); - ChunkJob nextPage = job.nextPageOrNull(); + Assert.areEqual(3, source.getNumRecords()); + Set fetched = new Set(); + for (SObject record : source.fetch(0, 3)) { + fetched.add(record.Id); + } + Assert.areEqual(ids, fetched); + } - Assert.isTrue( - nextPage.getRun().hasFailedPage, - 'The run remembers a failed page so the run outcome stays FAILURE.' - ); - Assert.isFalse(nextPage.hasFailed, 'The next page starts clean.'); + @IsTest + private static void shouldTreatNullRecordsAsEmptySource() { + Assert.areEqual(0, ChunkSource.ofIds(null).getNumRecords()); + Assert.areEqual(0, ChunkSource.of(null).getNumRecords()); } @IsTest - private static void shouldTreatEmptySourceAsNoOp() { - Async.Result result = Async.chunk( - new MarkingChunkJob(), - ChunkSource.of(new List()) - ) - .enqueue(); + private static void shouldDelegateToCursorSource() { + createAccounts(5); + ChunkSource source = ChunkSource.cursor(Database.getCursor('SELECT Id FROM Account')); - Assert.areEqual(null, result.salesforceJobId, 'An empty source should enqueue nothing.'); - Assert.isTrue(result.queueableChainState.jobs.isEmpty()); + Assert.areEqual(5, source.getNumRecords()); + Assert.areEqual(2, source.fetch(0, 2).size()); } @IsTest - private static void shouldStillRunChainedJobsWhenSourceIsEmpty() { - Test.startTest(); - Async.queueable(new ProcessedCountMarkerJob('EMPTY_SOURCE')) - .chunk(new MarkingChunkJob(), ChunkSource.of(new List())) - .enqueue(); - Test.stopTest(); + private static void shouldOpenCursorFromQuery() { + createAccounts(4); + ChunkSource source = ChunkSource.query('SELECT Id FROM Account'); - Assert.areEqual( - 1, - [SELECT COUNT() FROM Account WHERE Name = 'EMPTY_SOURCE'], - 'An empty chunk source must not swallow the jobs already chained.' - ); + Assert.areEqual(4, source.getNumRecords()); } @IsTest - private static void shouldRejectInvalidChunkSize() { - ChunkBuilder builder = Async.chunk( - new MarkingChunkJob(), - ChunkSource.of(createAccounts(1)) + private static void shouldOpenCursorFromQueryWithBinds() { + createAccounts(4); + ChunkSource source = ChunkSource.query( + 'SELECT Id FROM Account WHERE Name = :accountName', + new Map{ 'accountName' => 'Chunk 1' } ); - for (Integer invalid : new List{ 0, -1, null }) { - try { - builder.chunkSize(invalid); - Assert.fail('chunkSize ' + invalid + ' should be rejected.'); - } catch (Exception ex) { - Assert.areEqual(ChunkBuilder.ERROR_MESSAGE_INVALID_CHUNK_SIZE, ex.getMessage()); - } - } + + Assert.areEqual(1, source.getNumRecords(), 'Bind variables must reach the cursor.'); } @IsTest - private static void shouldRejectChunkSizeAboveTheCursorFetchCap() { - createAccounts(1); - ChunkBuilder builder = Async.chunk( - new MarkingChunkJob(), - ChunkSource.query('SELECT Id FROM Account') - ); - + private static void shouldRejectNullCursorAndBlankQuery() { try { - builder.chunkSize(2001); - Assert.fail('A cursor source cannot page more than 2000 records at a time.'); - } catch (Exception ex) { - Assert.areEqual( - 'chunkSize must be between 1 and 2000 for this source', - ex.getMessage() - ); + ChunkSource.cursor(null); + Assert.fail('A null cursor should be rejected.'); + } catch (Exception ex) { + Assert.areEqual(ChunkSource.ERROR_MESSAGE_NULL_CURSOR, ex.getMessage()); } - Assert.areNotEqual( - null, - builder.chunkSize(2000), - 'A chunk size at the cursor fetch cap is allowed.' - ); - } - - @IsTest - private static void shouldNotCapChunkSizeForAnInMemorySource() { - ChunkBuilder builder = Async.chunk( - new MarkingChunkJob(), - ChunkSource.of(createAccounts(1)) - ); - - Assert.areEqual( - null, - ChunkSource.of(new List()).maxChunkSize(), - 'An in-memory source has no cursor fetch ceiling.' - ); - Assert.areNotEqual( - null, - builder.chunkSize(5000), - 'An in-memory source is bounded by the job body, not by a page ceiling.' - ); - } - - @IsTest - private static void shouldExposeCursorFetchCapOnCursorSource() { - createAccounts(1); - - Assert.areEqual(2000, ChunkSource.query('SELECT Id FROM Account').maxChunkSize()); - } - - @IsTest - private static void shouldRejectNullJobAndSource() { try { - Async.chunk(null, ChunkSource.of(createAccounts(1))); - Assert.fail('A null job should be rejected.'); + ChunkSource.query(' '); + Assert.fail('A blank query should be rejected.'); } catch (Exception ex) { - Assert.areEqual(ChunkBuilder.ERROR_MESSAGE_NULL_JOB, ex.getMessage()); + Assert.areEqual(ChunkSource.ERROR_MESSAGE_BLANK_QUERY, ex.getMessage()); } try { - Async.chunk(new MarkingChunkJob(), null); - Assert.fail('A null source should be rejected.'); + ChunkSource.query(' ', new Map()); + Assert.fail('A blank query with binds should be rejected.'); } catch (Exception ex) { - Assert.areEqual(ChunkBuilder.ERROR_MESSAGE_NULL_SOURCE, ex.getMessage()); + Assert.areEqual(ChunkSource.ERROR_MESSAGE_BLANK_QUERY, ex.getMessage()); } } @IsTest - private static void shouldRejectRetryAboveCapOnChunk() { - ChunkBuilder builder = Async.chunk( - new MarkingChunkJob(), - ChunkSource.of(createAccounts(1)) - ); - try { - builder.retry(QueueableManager.MAX_RETRY_CAP + 1); - Assert.fail('Retry above the cap should be rejected.'); - } catch (Exception ex) { - Assert.areEqual( - QueueableManager.ERROR_MESSAGE_MAX_RETRIES_EXCEEDS_CAP, - ex.getMessage() - ); - } + private static void shouldOrderJobsOfEqualPriorityByChainSequence() { + QueueableJobTest1 first = new QueueableJobTest1(); + first.uniqueName = 'first'; + first.chainSequence = 1; + QueueableJobTest2 second = new QueueableJobTest2(); + second.uniqueName = 'second'; + second.chainSequence = 2; + + List jobs = new List{ second, first }; + jobs.sort(); + + Assert.areEqual('first', jobs[0].uniqueName, 'Equal priority keeps the chained order.'); + Assert.areEqual('second', jobs[1].uniqueName); } @IsTest - private static void shouldAccumulateJobStateAcrossChunks() { + private static void shouldRunEveryChunkBeforeTheNextChainedJob() { List accounts = createAccounts(6); Test.startTest(); - Async.chunk(new TotallingChunkJob(), ChunkSource.of(accounts)).chunkSize(2).enqueue(); + Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) + .chunkSize(2) + .chain(new ProcessedCountMarkerJob('AFTER_RUN')) + .enqueue(); Test.stopTest(); - Set runningTotals = new Set(); - for (Account marker : [SELECT Site FROM Account WHERE Name = 'RUNNING_TOTAL']) { - runningTotals.add(marker.Site); - } Assert.areEqual( - new Set{ '2', '4', '6' }, - runningTotals, - 'A member mutated in work() keeps accumulating on every later page.' + '6', + processedCountSeenBy('AFTER_RUN'), + 'A job chained after the run must wait for every page.' ); } @IsTest - private static void shouldPageRecordsFromAConsumerDefinedSource() { + private static void shouldRunChunkRunAfterAnEarlierChainedJob() { + List accounts = createAccounts(4); + Test.startTest(); - Async.chunk(new CountingChunkJob(), new SyntheticChunkSource(2500)) - .chunkSize(1000) + Async.queueable(new ProcessedCountMarkerJob('BEFORE_RUN')) + .chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) + .chunkSize(2) .enqueue(); Test.stopTest(); - Set runningTotals = new Set(); - for (Account marker : [SELECT Site FROM Account WHERE Name = 'SYNTHETIC_TOTAL']) { - runningTotals.add(marker.Site); - } Assert.areEqual( - new Set{ '1000', '2000', '2500' }, - runningTotals, - 'A custom ChunkSource pages 2500 fabricated records without any DML behind it.' + '0', + processedCountSeenBy('BEFORE_RUN'), + 'A job chained before the run must run first.' ); + Assert.areEqual(4, [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED]); } @IsTest - private static void shouldInjectQueueableMockIntoAChunkPage() { - List accounts = createAccounts(2); - Id mockJobId = fakeAsyncApexJobId(); - AsyncMock.whenQueueable('chunk-page') - .thenReturn(new AsyncMock.MockQueueableContext().setJobId(mockJobId)); + private static void shouldLetHigherPriorityJobPreemptTheNextChunk() { + List accounts = createAccounts(6); + PreemptingChunkJob job = new PreemptingChunkJob(); + job.preemptPriority = 1; Test.startTest(); - Async.chunk(new ContextReadingChunkJob(), ChunkSource.of(accounts)) - .chunkSize(2) - .mockId('chunk-page') - .enqueue(); + Async.chunk(job, ChunkSource.of(accounts)).chunkSize(2).priority(5).enqueue(); Test.stopTest(); Assert.areEqual( - String.valueOf(mockJobId), - [SELECT Site FROM Account WHERE Name = 'MOCKED_CONTEXT' LIMIT 1].Site, - 'A chunk page reads the mocked QueueableContext like any other job.' + '2', + processedCountSeenBy('PREEMPT'), + 'A higher priority job added mid-run runs before the next page.' + ); + Assert.areEqual( + 6, + [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], + 'The run resumes after the preempting job.' ); } @IsTest - private static void shouldFailAJobFromAQueueableMock() { - QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = resultsEnabledForAll(); - AsyncMock.whenQueueable('mocked-failure') - .thenThrow(new CustomException(CUSTOM_ERROR_MESSAGE)); + private static void shouldRunLowerPriorityJobAfterTheWholeRun() { + List accounts = createAccounts(6); + PreemptingChunkJob job = new PreemptingChunkJob(); + job.preemptPriority = 9; Test.startTest(); - QueueableManager.get().setChain(chain); - Async.queueable(new AccountCreatorJob('never runs')) - .mockId('mocked-failure') - .continueOnJobExecuteFail() - .enqueue(); + Async.chunk(job, ChunkSource.of(accounts)).chunkSize(2).priority(5).enqueue(); Test.stopTest(); - AsyncResult__c result = [ - SELECT Status__c, ExceptionType__c, ExceptionMessage__c - FROM AsyncResult__c - LIMIT 1 - ]; - Assert.areEqual( - QueueableManager.STATUS_FAILED, - result.Status__c, - 'A mocked failure settles through the same path a real one does.' - ); - Assert.areEqual(CUSTOM_ERROR_MESSAGE, result.ExceptionMessage__c); Assert.areEqual( - 0, - [SELECT COUNT() FROM Account], - 'The job body never runs when its execution is mocked to throw.' + '6', + processedCountSeenBy('PREEMPT'), + 'A lower priority job added mid-run waits for the whole run.' ); } @IsTest - private static void shouldFailOnlyTheMockedChunkPage() { + private static void shouldRunDependentJobWhenEveryChunkSucceeded() { List accounts = createAccounts(6); - AsyncMock.whenQueueable('recalc-run') - .thenReturn(new AsyncMock.MockQueueableContext()) - .thenThrow(new CustomException(CUSTOM_ERROR_MESSAGE)) - .thenReturn(new AsyncMock.MockQueueableContext()); Test.startTest(); Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) .chunkSize(2) - .mockId('recalc-run') + .chain(new ProcessedCountMarkerJob('DEPENDENT')) + .dependsOn(Async.afterPrevious().succeeded()) .enqueue(); Test.stopTest(); Assert.areEqual( - 4, - [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], - 'The mock queue picks which page fails; the rest of the run still processes.' + '6', + processedCountSeenBy('DEPENDENT'), + 'The run outcome is SUCCESS when every page passed.' ); } @IsTest - private static void shouldHaltRunFromAMockedPageFailure() { - List accounts = createAccounts(6); - AsyncMock.whenQueueable('recalc-run') - .thenReturn(new AsyncMock.MockQueueableContext()) - .thenThrow(new CustomException(CUSTOM_ERROR_MESSAGE)); + private static void shouldSkipDependentJobWhenAnyChunkFailed() { + List accounts = accountsWithOneFailingRecord(); Test.startTest(); - Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) + Async.chunk(new FailMarkedChunkJob(), ChunkSource.of(accounts)) .chunkSize(2) - .mockId('recalc-run') - .stopRemainingChunksOnFailure() + .chain(new ProcessedCountMarkerJob('DEPENDENT')) + .dependsOn(Async.afterPrevious().succeeded()) .enqueue(); Test.stopTest(); Assert.areEqual( - 2, - [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], - 'A mocked page failure halts the run like a real one.' + 0, + [SELECT COUNT() FROM Account WHERE Name = 'DEPENDENT'], + 'One failed page makes the whole run outcome FAILURE.' ); } @IsTest - private static void shouldMockFinalizerOutcomeForAChunkPage() { - List accounts = createAccounts(2); - AsyncMock.whenFinalizer('chunk-error-handler').thenThrow(new DmlException('Page blew up')); + private static void shouldRunDependentJobOnFinishedRunEvenAfterAFailedChunk() { + List accounts = accountsWithOneFailingRecord(); Test.startTest(); - Async.chunk(new FinalizerAttachingChunkJob('chunk-error-handler'), ChunkSource.of(accounts)) + Async.chunk(new FailMarkedChunkJob(), ChunkSource.of(accounts)) .chunkSize(2) + .chain(new ProcessedCountMarkerJob('DEPENDENT')) + .dependsOn(Async.afterPrevious().finished()) .enqueue(); Test.stopTest(); Assert.areEqual( - 'Page blew up', - [SELECT Description FROM Account WHERE Name = 'Error Log' LIMIT 1].Description, - 'A page finalizer can be told its page failed without the page actually failing.' + 1, + [SELECT COUNT() FROM Account WHERE Name = 'DEPENDENT'], + 'finished() runs the dependent job whatever the run outcome was.' ); } @IsTest - private static void shouldFailAndRetryAPageWhenTheSourceThrows() { + private static void shouldRecordSummaryResultWhenRunHaltsOnFailure() { + List accounts = accountsWithOneFailingRecord(); QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = resultsEnabledForAll(); + QueueableChain.jobSettingByName = resultsEnabledForAll(); Test.startTest(); QueueableManager.get().setChain(chain); - Async.chunk(new MarkingChunkJob(), new ExplodingChunkSource()) + Async.chunk(new FailMarkedChunkJob(), ChunkSource.of(accounts)) .chunkSize(2) - .retry(1) + .stopRemainingChunksOnFailure() .enqueue(); Test.stopTest(); - AsyncResult__c result = [ - SELECT Status__c, ExceptionMessage__c, RetryAttempts__c + List stopped = [ + SELECT SkipReason__c FROM AsyncResult__c - LIMIT 1 + WHERE Status__c = :QueueableManager.STATUS_SKIPPED_CHUNK_STOPPED ]; - Assert.areEqual( - QueueableManager.STATUS_FAILED, - result.Status__c, - 'A source that throws fails the page instead of silently ending the run.' + Assert.areEqual(1, stopped.size(), 'A halted run records one summary result.'); + Assert.isTrue( + stopped[0].SkipReason__c.contains('page 3 of 3'), + 'The summary states where the run stopped: ' + stopped[0].SkipReason__c + ); + Assert.isTrue( + stopped[0].SkipReason__c.contains('2 record(s) were not processed'), + 'The summary states how much work was left: ' + stopped[0].SkipReason__c ); - Assert.areEqual(CUSTOM_ERROR_MESSAGE, result.ExceptionMessage__c); - Assert.areEqual(1, result.RetryAttempts__c, 'The page retried before it settled.'); } @IsTest - private static void shouldChainRunWithoutEnqueueingIt() { - List accounts = createAccounts(2); + private static void shouldRecordSummaryResultWhenChainStopsDuringRun() { + List accounts = createAccounts(6); + QueueableChain chain = new QueueableChain(); + QueueableChain.jobSettingByName = resultsEnabledForAll(); - Async.Result result = Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) - .chunkSize(2) - .chain(); + Test.startTest(); + QueueableManager.get().setChain(chain); + Async.chunk(new ChainStoppingChunkJob(), ChunkSource.of(accounts)).chunkSize(2).enqueue(); + Test.stopTest(); - Assert.areNotEqual( - null, - result.customJobId, - 'chain() returns the run handle so later jobs can depend on it.' - ); Assert.areEqual( - 0, + 2, [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], - 'chain() adds the run to the chain without enqueuing anything.' + 'Async.stopChain() ends the run wherever it is.' + ); + List stopped = [ + SELECT SkipReason__c + FROM AsyncResult__c + WHERE + Status__c = :QueueableManager.STATUS_SKIPPED_CHAIN_STOPPED + AND SkipReason__c LIKE '%not processed%' + ]; + Assert.areEqual(1, stopped.size(), 'A stopped chain records what the run left behind.'); + Assert.isTrue( + stopped[0].SkipReason__c.contains('page 2 of 3'), + 'The summary points at the first page that never ran: ' + stopped[0].SkipReason__c ); } @IsTest - private static void shouldRunASecondChunkRunAfterTheFirstOneFinishes() { - List accounts = createAccounts(4); + private static void shouldStoreAPayloadWhenCustomMetadataTurnsItOn() { + RequeueableJob job = new RequeueableJob('STORED'); - Test.startTest(); - Async.chunk(new LabelingChunkJob('FIRST'), ChunkSource.of(accounts)) - .chunkSize(2) - .chunk(new LabelingChunkJob('SECOND'), ChunkSource.of(accounts)) - .chunkSize(2) - .enqueue(); - Test.stopTest(); + chainStoringPayloads().addJob(job); - Assert.areEqual( - 4, - [SELECT COUNT() FROM Account WHERE Site = 'SECOND'], - 'The second run overwrites the first, so it ran after every page of run one.' + Assert.areEqual(AsyncRequeue.STATUS_STORED, job.requeueStatus); + Assert.isTrue( + job.requeuePayload.contains('STORED'), + 'The payload must carry the state the job was enqueued with: ' + job.requeuePayload ); + Assert.areEqual(job.requeuePayload.length(), job.requeuePayloadSize); } @IsTest - private static void shouldGateASecondChunkRunOnTheFirstRunOutcome() { - List accounts = accountsWithOneFailingRecord(); + private static void shouldNotTouchTheJobWhenPayloadStorageIsOff() { + RequeueableJob job = new RequeueableJob('NOT-STORED'); - Test.startTest(); - Async.chunk(new FailMarkedChunkJob(), ChunkSource.of(accounts)) - .chunkSize(2) - .chunk(new LabelingChunkJob('SECOND'), ChunkSource.of(accounts)) - .chunkSize(2) - .dependsOn(Async.afterPrevious().succeeded()) - .enqueue(); - Test.stopTest(); + new QueueableChain().addJob(job); - Assert.areEqual( - 0, - [SELECT COUNT() FROM Account WHERE Site = 'SECOND'], - 'dependsOn(previous) resolves against the first run outcome, which failed.' - ); + Assert.isNull(job.requeueStatus, 'Storing payloads has to be opted into.'); + Assert.isNull(job.requeuePayload); } @IsTest - private static void shouldOpenACursorInTheRequestedAccessLevel() { - createAccounts(2); - - ChunkSource userMode = ChunkSource.query('SELECT Id FROM Account', AccessLevel.USER_MODE); - ChunkSource systemMode = ChunkSource.query( - 'SELECT Id FROM Account WHERE Name LIKE :prefix', - new Map{ 'prefix' => 'Chunk%' }, - AccessLevel.SYSTEM_MODE + private static void shouldLetAJobRecordOptOutOfOrgWidePayloadStorage() { + RequeueableJob job = new RequeueableJob('SENSITIVE'); + QueueableChain chain = new QueueableChain(); + QueueableChain.jobSettingByName = payloadStorageOn(); + QueueableChain.jobSettingByName.put( + job.className, + new QueueableJobSetting__mdt( + QueueableJobName__c = job.className, + StoreJobPayload__c = 'No' + ) ); - Assert.areEqual(2, userMode.getNumRecords(), 'The admin running the test sees both.'); - Assert.areEqual(2, systemMode.getNumRecords(), 'Binds and access level combine.'); + chain.addJob(job); + + Assert.isNull( + job.requeueStatus, + 'A job carrying sensitive data must be able to opt out of an org-wide Yes.' + ); } @IsTest - private static void shouldChainNothingWhenSourceIsEmpty() { - Async.Result result = Async.chunk( - new MarkingChunkJob(), - ChunkSource.of(new List()) + private static void shouldNotStoreAPayloadForChunkJobs() { + QueueableChain.jobSettingByName = payloadStorageOn(); + + Async.Result chunked = Async.chunk( + new StateCarryingChunkJob(), + ChunkSource.of(new List{ new Account(Name = 'PAGE') }) ) .chain(); - Assert.areEqual(null, result.customJobId, 'An empty source chains no run.'); - Assert.isTrue(result.queueableChainState.jobs.isEmpty()); + Assert.areEqual( + AsyncRequeue.STATUS_NOT_SERIALIZABLE, + chunked.job.requeueStatus, + 'A chunk run holds a source that cannot be stored, and a half-read cursor is not ' + + 'a meaningful thing to replay.' + ); } @IsTest - private static void shouldRunChunkRunWhenItsDependencySucceeded() { - List accounts = createAccounts(4); + private static void shouldRecordTooLargeRatherThanBreakTheInsert() { + BulkyRequeueableJob job = new BulkyRequeueableJob(); + for (Integer i = 0; i < 70; i++) { + job.payloadParts.add('x'.repeat(2000)); + } + + chainStoringPayloads().addJob(job); + + Assert.areEqual(AsyncRequeue.STATUS_TOO_LARGE, job.requeueStatus); + Assert.isNull( + job.requeuePayload, + 'An oversized payload would fail the insert and lose the failure record with it.' + ); + Assert.isTrue( + job.requeuePayloadSize > JobPayload.MAX_CHARS, + 'The size is recorded even when the payload is not, so it can be reported on.' + ); + } + + @IsTest + private static void shouldWriteAFailureRowEvenWhenResultCreationIsOff() { + FailingRequeueableJob job = new FailingRequeueableJob(); + job.continueOnJobExecuteFail = true; + QueueableChain chain = chainStoringPayloads(); + chain.addJob(job); + QueueableManager.get().setChain(chain); Test.startTest(); - Async.queueable(new ProcessedCountMarkerJob('GATE')) - .chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) - .chunkSize(2) - .dependsOn(Async.afterPrevious().succeeded()) - .enqueue(); + chain.executeCurrentJob(new AsyncMock.MockQueueableContext()); + chain.enqueueNextJobIfAnyFromFinalizer( + new AsyncMock.MockFinalizerContext().setResult(ParentJobResult.UNHANDLED_EXCEPTION) + ); Test.stopTest(); + AsyncResult__c result = [ + SELECT Status__c, RequeueStatus__c, JobPayload__c + FROM AsyncResult__c + WHERE CustomJobId__c = :job.customJobId + ]; + Assert.areEqual(QueueableManager.STATUS_FAILED, result.Status__c); Assert.areEqual( - 4, - [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], - 'The run starts once the job it depends on succeeded.' + AsyncRequeue.STATUS_STORED, + result.RequeueStatus__c, + 'A payload with no row to sit on could never be replayed.' ); + Assert.isNotNull(result.JobPayload__c); } @IsTest - private static void shouldSkipChunkRunWhenItsDependencyFailed() { - List accounts = createAccounts(4); + private static void shouldNotWriteASuccessRowWhenOnlyPayloadStorageIsOn() { + RequeueableJob job = new RequeueableJob('SUCCEEDS'); + QueueableChain chain = chainStoringPayloads(); + chain.addJob(job); + QueueableManager.get().setChain(chain); Test.startTest(); - Async.queueable(new FailureQueueableTest()) - .continueOnJobExecuteFail() - .chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) - .chunkSize(2) - .dependsOn(Async.afterPrevious().succeeded()) - .enqueue(); + chain.executeCurrentJob(new AsyncMock.MockQueueableContext()); + chain.enqueueNextJobIfAnyFromFinalizer( + new AsyncMock.MockFinalizerContext().setResult(ParentJobResult.SUCCESS) + ); Test.stopTest(); Assert.areEqual( 0, - [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], - 'The whole run is skipped when the job it depends on failed.' + [SELECT COUNT() FROM AsyncResult__c], + 'Jobs that succeeded still obey CreateResult__c, so the added volume stays bounded ' + + 'by the failure rate.' ); } @IsTest - private static void shouldRejectInvalidDependencyOnChunk() { - ChunkBuilder builder = Async.chunk( - new MarkingChunkJob(), - ChunkSource.of(createAccounts(1)) + private static void shouldWriteASkippedRowSoAChainWideReplayStaysPossible() { + FailureQueueableTest blocker = new FailureQueueableTest(); + blocker.continueOnJobExecuteFail = true; + RequeueableJob blocked = new RequeueableJob('NEVER-RAN'); + QueueableChain chain = chainStoringPayloads(); + chain.addJob(blocker); + blocked.dependencies = new List{ + dependencyOn(blocker.customJobId, Async.Outcome.SUCCESS) + }; + chain.addJob(blocked); + QueueableManager.get().setChain(chain); + + Test.startTest(); + chain.executeCurrentJob(new AsyncMock.MockQueueableContext()); + chain.enqueueNextJobIfAnyFromFinalizer( + new AsyncMock.MockFinalizerContext().setResult(ParentJobResult.UNHANDLED_EXCEPTION) + ); + Test.stopTest(); + + AsyncResult__c skipped = [ + SELECT Status__c, RequeueStatus__c + FROM AsyncResult__c + WHERE CustomJobId__c = :blocked.customJobId + ]; + Assert.areEqual(QueueableManager.STATUS_SKIPPED_DEPENDENCY, skipped.Status__c); + Assert.areEqual( + AsyncRequeue.STATUS_STORED, + skipped.RequeueStatus__c, + 'A job blocked by a dependency never failed, and it is exactly the one a chain-wide ' + + 'replay has to rebuild.' ); - try { - builder.dependsOn(null); - Assert.fail('A dependency without an outcome should be rejected.'); - } catch (Exception ex) { - Assert.areEqual(QueueableManager.ERROR_MESSAGE_INVALID_DEPENDENCY, ex.getMessage()); - } - try { - builder.dependsOn(Async.afterPrevious().succeeded()); - Assert.fail('afterPrevious() without a previously chained job should be rejected.'); - } catch (Exception ex) { - Assert.areEqual( - QueueableManager.ERROR_MESSAGE_DEPENDS_ON_PREVIOUS_WITHOUT_JOB, - ex.getMessage() - ); - } } @IsTest - private static void shouldYieldIdOnlyShellsWhenSourceIsBuiltFromIds() { - List accounts = createAccounts(5); - ChunkSource source = ChunkSource.ofIds(new Map(accounts).keySet()); + private static void shouldDropThePayloadFromTheJobOnceItIsOnTheRecord() { + FailingRequeueableJob job = new FailingRequeueableJob(); + job.continueOnJobExecuteFail = true; + QueueableChain chain = chainStoringPayloads(); + chain.addJob(job); + QueueableManager.get().setChain(chain); - List page = source.fetch(0, 5); + Test.startTest(); + chain.executeCurrentJob(new AsyncMock.MockQueueableContext()); + chain.enqueueNextJobIfAnyFromFinalizer( + new AsyncMock.MockFinalizerContext().setResult(ParentJobResult.UNHANDLED_EXCEPTION) + ); + Test.stopTest(); - Assert.areEqual(5, page.size()); - Assert.areEqual( - new Map(accounts).keySet(), - new Map(page).keySet(), - 'A job reads ids straight off the page with new Map(chunk).keySet().' + Assert.isNull( + job.requeuePayload, + 'The payload rides the serialized job into every later context, so keeping it would ' + + 'carry the job size twice for the rest of the chain.' ); } @IsTest - private static void shouldExposeInMemorySourceRecordsAndSize() { - List accounts = createAccounts(5); - ChunkSource source = ChunkSource.of(accounts); + private static void shouldRequeueAStoredJobAndLinkItBackToTheOriginal() { + RequeueableJob job = new RequeueableJob('REPLAYED'); + chainStoringPayloads().addJob(job); + QueueableChain.jobSettingByName.get(QueueableManager.QUEUEABLE_JOB_SETTING_ALL) + .CreateResult__c = true; + AsyncResult__c original = storedResultFor(job); - Assert.areEqual(5, source.getNumRecords()); - Assert.areEqual(2, source.fetch(0, 2).size()); + Test.startTest(); + Async.RequeueSummary summary = Async.requeue(original.Id); + Test.stopTest(); + + Assert.areEqual(new List{ original.Id }, summary.requeued); + Assert.isNotNull( + summary.enqueueResult?.salesforceJobId, + 'A requeue is an enqueue, so the caller gets the same handle on the chain.' + ); Assert.areEqual( 1, - source.fetch(4, 5).size(), - 'A short final page must clamp to the remainder.' + [SELECT COUNT() FROM Account WHERE Name = 'REPLAYED'], + 'The rebuilt job has to run with the state it was enqueued with.' + ); + Assert.areEqual( + AsyncRequeue.STATUS_REQUEUED, + [SELECT RequeueStatus__c FROM AsyncResult__c WHERE Id = :original.Id].RequeueStatus__c, + 'Marking the source is what stops a scheduled replay picking it up again.' + ); + Assert.areEqual( + original.Id, + [ + SELECT RequeuedFrom__c + FROM AsyncResult__c + WHERE Id != :original.Id + LIMIT 1 + ] + .RequeuedFrom__c, + 'A replay runs in a new chain, so this lookup is the only link back.' ); } @IsTest - private static void shouldBuildIdSourceRecordsWithIds() { - List accounts = createAccounts(3); - Set ids = new Map(accounts).keySet(); - ChunkSource source = ChunkSource.ofIds(ids); - - Assert.areEqual(3, source.getNumRecords()); - Set fetched = new Set(); - for (SObject record : source.fetch(0, 3)) { - fetched.add(record.Id); + private static void shouldRequeueTwoHundredResultsAsOneChainInOneCall() { + RequeueableJob job = new RequeueableJob('BULK'); + chainStoringPayloads().addJob(job); + List stored = new List(); + for (Integer i = 0; i < 200; i++) { + stored.add( + new AsyncResult__c( + ClassName__c = job.className, + Status__c = QueueableManager.STATUS_FAILED, + JobPayload__c = job.requeuePayload, + PayloadSize__c = job.requeuePayloadSize, + RequeueStatus__c = AsyncRequeue.STATUS_STORED + ) + ); } - Assert.areEqual(ids, fetched); - } - - @IsTest - private static void shouldTreatNullRecordsAsEmptySource() { - Assert.areEqual(0, ChunkSource.ofIds(null).getNumRecords()); - Assert.areEqual(0, ChunkSource.of(null).getNumRecords()); - } + insert stored; + Map storedById = new Map(stored); - @IsTest - private static void shouldDelegateToCursorSource() { - createAccounts(5); - ChunkSource source = ChunkSource.cursor(Database.getCursor('SELECT Id FROM Account')); + Async.RequeueSummary summary = Async.requeue(storedById.keySet()); - Assert.areEqual(5, source.getNumRecords()); - Assert.areEqual(2, source.fetch(0, 2).size()); + Assert.areEqual(200, summary.requeued.size()); + Assert.isTrue(summary.skipReasonByResultId.isEmpty(), '' + summary.skipReasonByResultId); + Assert.areEqual( + 200, + summary.enqueueResult.queueableChainState.jobs.size(), + 'Every replay has to land in the one chain, not in 200 separate enqueues.' + ); + Assert.areEqual( + 200, + [ + SELECT COUNT() + FROM AsyncResult__c + WHERE + Id IN :storedById.keySet() + AND RequeueStatus__c = :AsyncRequeue.STATUS_REQUEUED + ], + 'Every source row has to be marked in the same call.' + ); } @IsTest - private static void shouldOpenCursorFromQuery() { - createAccounts(4); - ChunkSource source = ChunkSource.query('SELECT Id FROM Account'); + private static void shouldNotStoreAPayloadOnTheRowOfAJobThatSucceeded() { + RequeueableJob job = new RequeueableJob('SUCCEEDED'); + QueueableChain chain = chainStoringPayloads(); + QueueableChain.jobSettingByName.get(QueueableManager.QUEUEABLE_JOB_SETTING_ALL) + .CreateResult__c = true; + chain.addJob(job); + QueueableManager.get().setChain(chain); - Assert.areEqual(4, source.getNumRecords()); + Test.startTest(); + chain.executeCurrentJob(new AsyncMock.MockQueueableContext()); + chain.enqueueNextJobIfAnyFromFinalizer( + new AsyncMock.MockFinalizerContext().setResult(ParentJobResult.SUCCESS) + ); + Test.stopTest(); + + AsyncResult__c result = [ + SELECT Status__c, RequeueStatus__c, JobPayload__c + FROM AsyncResult__c + WHERE CustomJobId__c = :job.customJobId + ]; + Assert.areEqual(QueueableManager.STATUS_COMPLETED, result.Status__c); + Assert.isNull( + result.RequeueStatus__c, + 'There is nothing to replay about a job that worked, and a Stored status here would ' + + 'let a bulk replay re-run it.' + ); + Assert.isNull(result.JobPayload__c); } @IsTest - private static void shouldOpenCursorFromQueryWithBinds() { - createAccounts(4); - ChunkSource source = ChunkSource.query( - 'SELECT Id FROM Account WHERE Name = :accountName', - new Map{ 'accountName' => 'Chunk 1' } + private static void shouldKeepThePayloadThroughARetryThatRestoresEnqueuedState() { + QueueableChain.jobSettingByName = payloadStorageOn(); + + Test.startTest(); + Async.queueable(new FailingRequeueableJob()) + .retry(1) + .restoreStateOnRetry() + .continueOnJobExecuteFail() + .enqueue(); + Test.stopTest(); + + AsyncResult__c result = [ + SELECT RetryAttempts__c, RequeueStatus__c, JobPayload__c + FROM AsyncResult__c + LIMIT 1 + ]; + Assert.areEqual(1, result.RetryAttempts__c, 'The retry has to have run.'); + Assert.areEqual( + AsyncRequeue.STATUS_STORED, + result.RequeueStatus__c, + 'A job that exhausted its retries after an outage is the one requeue exists for.' ); + Assert.isNotNull(result.JobPayload__c); + } - Assert.areEqual(1, source.getNumRecords(), 'Bind variables must reach the cursor.'); + @IsTest + private static void shouldKeepThePayloadThroughADeepClonedRetry() { + QueueableChain.jobSettingByName = payloadStorageOn(); + + Test.startTest(); + Async.queueable(new FailingRequeueableJob()) + .deepClone() + .retry(1) + .continueOnJobExecuteFail() + .enqueue(); + Test.stopTest(); + + AsyncResult__c result = [ + SELECT RetryAttempts__c, RequeueStatus__c, JobPayload__c + FROM AsyncResult__c + LIMIT 1 + ]; + Assert.areEqual(1, result.RetryAttempts__c, 'The retry has to have run.'); + Assert.areEqual(AsyncRequeue.STATUS_STORED, result.RequeueStatus__c); + Assert.isNotNull( + result.JobPayload__c, + 'The deep copy strips the payload out of the JSON it makes; the copy still has to get it back.' + ); } @IsTest - private static void shouldRejectNullCursorAndBlankQuery() { - try { - ChunkSource.cursor(null); - Assert.fail('A null cursor should be rejected.'); - } catch (Exception ex) { - Assert.areEqual(ChunkSource.ERROR_MESSAGE_NULL_CURSOR, ex.getMessage()); - } - try { - ChunkSource.query(' '); - Assert.fail('A blank query should be rejected.'); - } catch (Exception ex) { - Assert.areEqual(ChunkSource.ERROR_MESSAGE_BLANK_QUERY, ex.getMessage()); - } - try { - ChunkSource.query(' ', new Map()); - Assert.fail('A blank query with binds should be rejected.'); - } catch (Exception ex) { - Assert.areEqual(ChunkSource.ERROR_MESSAGE_BLANK_QUERY, ex.getMessage()); - } + private static void shouldDropThePayloadFromASupersededAttempt() { + FailureQueueableTest job = new FailureQueueableTest(); + job.maxRetries = 1; + job.continueOnJobExecuteFail = true; + QueueableChain chain = chainStoringPayloads(); + chain.addJob(job); + QueueableManager.get().setChain(chain); + String payload = job.requeuePayload; + + chain.executeCurrentJob(new AsyncMock.MockQueueableContext()); + chain.enqueueNextJobIfAnyFromFinalizer( + new AsyncMock.MockFinalizerContext().setResult(ParentJobResult.UNHANDLED_EXCEPTION) + ); + + Assert.isNull( + job.requeuePayload, + 'The failed attempt stays in the chain; it must not carry the payload too.' + ); + Assert.areEqual( + payload, + chain.jobs[0].requeuePayload, + 'The retry attempt is the one that will produce the row, so it carries the payload.' + ); } @IsTest - private static void shouldOrderJobsOfEqualPriorityByChainSequence() { - QueueableJobTest1 first = new QueueableJobTest1(); - first.uniqueName = 'first'; - first.chainSequence = 1; - QueueableJobTest2 second = new QueueableJobTest2(); - second.uniqueName = 'second'; - second.chainSequence = 2; + private static void shouldDegradeWhenTheRegisteredSerializerReturnsNull() { + QueueableChain chain = chainStoringPayloads(); + QueueableChain.jobSettingByName.get(QueueableManager.QUEUEABLE_JOB_SETTING_ALL) + .JobSerializerClass__c = loggerName('NullReturningJobSerializer'); + RequeueableJob job = new RequeueableJob('NULL-SERIALIZER'); - List jobs = new List{ second, first }; - jobs.sort(); + chain.addJob(job); - Assert.areEqual('first', jobs[0].uniqueName, 'Equal priority keeps the chained order.'); - Assert.areEqual('second', jobs[1].uniqueName); + Assert.areEqual(AsyncRequeue.STATUS_NOT_SERIALIZABLE, job.requeueStatus); + Assert.isTrue( + job.retryHistory.contains(QueueableManager.CAUSE_SERIALIZER_RETURNED_NULL), + 'A registered class is configuration, and configuration mistakes degrade with a ' + + 'reason rather than stopping every enqueue: ' + + job.retryHistory + ); } @IsTest - private static void shouldRunEveryChunkBeforeTheNextChainedJob() { - List accounts = createAccounts(6); + private static void shouldCaptureTheJobBeforeAnyChainBookkeepingLandsOnIt() { + RequeueableJob job = new RequeueableJob('PRISTINE'); + QueueableChain.jobSettingByName = payloadStorageOn(); + QueueableChain.jobSettingByName.put( + job.className, + new QueueableJobSetting__mdt(QueueableJobName__c = job.className, MaxRetries__c = 3) + ); - Test.startTest(); - Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) - .chunkSize(2) - .chain(new ProcessedCountMarkerJob('AFTER_RUN')) - .enqueue(); - Test.stopTest(); + Async.queueable(job).chain(); + RequeueableJob rebuilt = (RequeueableJob) JSON.deserialize( + QueueableManager.get().getChain().getJobs()[0].requeuePayload, + RequeueableJob.class + ); + Assert.isNull(rebuilt.customJobId, 'The replay gets a fresh id.'); + Assert.isNull(rebuilt.chainSequence, 'The replay takes its place in its own chain.'); Assert.areEqual( - '6', - processedCountSeenBy('AFTER_RUN'), - 'A job chained after the run must wait for every page.' + 0, + rebuilt.maxRetries, + 'Custom Metadata defaults are applied again at replay, from the metadata of that day.' ); } @IsTest - private static void shouldRunChunkRunAfterAnEarlierChainedJob() { - List accounts = createAccounts(4); + private static void shouldSkipAJobWhoseClassNoLongerDeclaresHowItsStateResets() { + RequeueableJob job = new RequeueableJob('CHANGED-CLASS'); + chainStoringPayloads().addJob(job); + job.requeuePayload = job.requeuePayload.replace('"maxRetries":0', '"maxRetries":2'); + AsyncResult__c stored = storedResultFor(job); + + Async.RequeueSummary summary = Async.requeue(new Set{ stored.Id }); + + Assert.isTrue(summary.requeued.isEmpty(), 'The gate has to hold on replay too.'); + Assert.isTrue( + summary.skipReasonByResultId.get(stored.Id).contains('Async.Retryable'), + 'One bad class is one skipped row with the gate message, not an abandoned batch: ' + + summary.skipReasonByResultId.get(stored.Id) + ); + } + + @IsTest + private static void shouldStillWriteTheReplayRowWhenTheSourceWasDeletedMeanwhile() { + FailingRequeueableJob job = new FailingRequeueableJob(); + job.continueOnJobExecuteFail = true; + chainStoringPayloads().addJob(job); + AsyncResult__c original = storedResultFor(job); Test.startTest(); - Async.queueable(new ProcessedCountMarkerJob('BEFORE_RUN')) - .chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) - .chunkSize(2) - .enqueue(); + Async.requeue(new Set{ original.Id }); + delete original; Test.stopTest(); + AsyncResult__c replay = [ + SELECT Status__c, RequeuedFrom__c + FROM AsyncResult__c + LIMIT 1 + ]; + Assert.areEqual(QueueableManager.STATUS_FAILED, replay.Status__c); + Assert.isNull( + replay.RequeuedFrom__c, + 'A lookup to a row the cleanup batch removed would fail the insert and lose the row.' + ); + } + + @IsTest + private static void shouldDoNothingWhenAskedToRequeueNothing() { + Async.RequeueSummary summary = Async.requeue(new Set()); + + Assert.isTrue(summary.requeued.isEmpty()); + Assert.isTrue(summary.skipReasonByResultId.isEmpty()); + Assert.isNull(summary.enqueueResult, 'Nothing was replayed, so there is no chain.'); + } + + @IsTest + private static void shouldMergeInfoMapsOnBothBuilders() { + Async.Result queued = Async.queueable(new SuccessfulQueueableTest()) + .info('team', 'platform') + .info(new Map{ 'package' => 'billing', 'team' => 'billing' }) + .chain(); + Async.Result chunked = Async.chunk( + new StateCarryingChunkJob(), + ChunkSource.of(new List{ new Account(Name = 'PAGE') }) + ) + .info('team', 'platform') + .info(new Map{ 'package' => 'billing' }) + .chain(); + Assert.areEqual( - '0', - processedCountSeenBy('BEFORE_RUN'), - 'A job chained before the run must run first.' + new Map{ 'team' => 'billing', 'package' => 'billing' }, + queued.job.info, + 'A later info() call adds keys and overwrites the ones it repeats.' + ); + Assert.areEqual( + new Map{ 'team' => 'platform', 'package' => 'billing' }, + chunked.job.info ); - Assert.areEqual(4, [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED]); } @IsTest - private static void shouldLetHigherPriorityJobPreemptTheNextChunk() { - List accounts = createAccounts(6); - PreemptingChunkJob job = new PreemptingChunkJob(); - job.preemptPriority = 1; + private static void shouldWarnOnceWhenTheLoggerConstructorThrows() { + QueueableChain chain = chainWithLogger(loggerName('UnconstructableLogger')); + QueueableManager.get().setChain(chain); + SuccessfulQueueableTest first = new SuccessfulQueueableTest(); + SuccessfulQueueableTest second = new SuccessfulQueueableTest(); + + chain.addJob(first); + chain.addJob(second); + + Assert.isTrue( + first.retryHistory.contains('could not be constructed'), + 'The warning has to say the class was found but not built: ' + first.retryHistory + ); + Assert.isTrue( + first.retryHistory.contains(LOGGER_CONSTRUCTOR_FAILURE), + 'The constructor message has to reach the job: ' + first.retryHistory + ); + Assert.isNull( + second.retryHistory, + 'The same broken class is reported once per transaction, not once per job.' + ); + } + + @IsTest + private static void shouldExplainEveryResultItRefusedToRequeue() { + AsyncResult__c noPayload = new AsyncResult__c(Status__c = QueueableManager.STATUS_FAILED); + AsyncResult__c alreadyRequeued = new AsyncResult__c( + Status__c = QueueableManager.STATUS_FAILED, + RequeueStatus__c = AsyncRequeue.STATUS_REQUEUED + ); + AsyncResult__c tooLarge = new AsyncResult__c( + Status__c = QueueableManager.STATUS_FAILED, + RequeueStatus__c = AsyncRequeue.STATUS_TOO_LARGE + ); + insert new List{ noPayload, alreadyRequeued, tooLarge }; + AsyncResult__c deleted = new AsyncResult__c(Status__c = QueueableManager.STATUS_FAILED); + insert deleted; + Id deletedId = deleted.Id; + delete deleted; Test.startTest(); - Async.chunk(job, ChunkSource.of(accounts)).chunkSize(2).priority(5).enqueue(); + Async.RequeueSummary summary = Async.requeue( + new Set{ noPayload.Id, alreadyRequeued.Id, tooLarge.Id, deletedId } + ); Test.stopTest(); + Assert.isTrue(summary.requeued.isEmpty(), 'None of these can be replayed.'); Assert.areEqual( - '2', - processedCountSeenBy('PREEMPT'), - 'A higher priority job added mid-run runs before the next page.' + QueueableManager.REQUEUE_SKIPPED_NO_PAYLOAD, + summary.skipReasonByResultId.get(noPayload.Id) ); Assert.areEqual( - 6, - [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], - 'The run resumes after the preempting job.' + QueueableManager.REQUEUE_SKIPPED_ALREADY_REQUEUED, + summary.skipReasonByResultId.get(alreadyRequeued.Id) + ); + Assert.isTrue( + summary.skipReasonByResultId.get(tooLarge.Id).contains(AsyncRequeue.STATUS_TOO_LARGE), + 'The reason has to name the status, but was: ' + + summary.skipReasonByResultId.get(tooLarge.Id) + ); + Assert.areEqual( + QueueableManager.REQUEUE_SKIPPED_NO_RECORD, + summary.skipReasonByResultId.get(deletedId) ); } @IsTest - private static void shouldRunLowerPriorityJobAfterTheWholeRun() { - List accounts = createAccounts(6); - PreemptingChunkJob job = new PreemptingChunkJob(); - job.preemptPriority = 9; + private static void shouldRefuseToRequeueMoreCharactersThanItCanHold() { + List heavy = new List(); + for (Integer i = 0; i < 20; i++) { + heavy.add( + new AsyncResult__c( + Status__c = QueueableManager.STATUS_FAILED, + RequeueStatus__c = AsyncRequeue.STATUS_STORED, + PayloadSize__c = JobPayload.MAX_CHARS + ) + ); + } + insert heavy; + Map heavyById = new Map(heavy); + + try { + Async.requeue(heavyById.keySet()); + Assert.fail('Deserializing this many payloads at once would blow the heap.'); + } catch (Async.IllegalArgumentException expected) { + Assert.isTrue( + expected.getMessage().contains(String.valueOf(AsyncRequeue.MAX_CHARS_PER_REQUEUE)), + 'The caller has to be told the limit, but was: ' + expected.getMessage() + ); + } + } + + @IsTest + private static void shouldRouteBothHalvesThroughARegisteredSerializer() { + QueueableChain chain = chainStoringPayloads(); + QueueableChain.jobSettingByName.get(QueueableManager.QUEUEABLE_JOB_SETTING_ALL) + .JobSerializerClass__c = loggerName('RecordingJobSerializer'); + RequeueableJob job = new RequeueableJob('VIA-SERIALIZER'); + chain.addJob(job); + AsyncResult__c original = storedResultFor(job); Test.startTest(); - Async.chunk(job, ChunkSource.of(accounts)).chunkSize(2).priority(5).enqueue(); + Async.requeue(new Set{ original.Id }); + Test.stopTest(); + + Assert.isTrue( + loggedEvents.contains('serialize:' + job.className), + 'Capture has to run in the consumer namespace too: ' + loggedEvents + ); + Assert.isTrue( + loggedEvents.contains('deserialize:' + job.className), + 'Rebuild has to run in the consumer namespace: ' + loggedEvents + ); + } + + @IsTest + private static void shouldRejectARegisteredClassThatIsNotASerializer() { + QueueableChain chain = chainStoringPayloads(); + QueueableChain.jobSettingByName.get(QueueableManager.QUEUEABLE_JOB_SETTING_ALL) + .JobSerializerClass__c = loggerName('RecordingLogger'); + RequeueableJob job = new RequeueableJob('NOT-A-SERIALIZER'); + + chain.addJob(job); + + Assert.areEqual(AsyncRequeue.STATUS_NOT_SERIALIZABLE, job.requeueStatus); + Assert.isTrue( + job.retryHistory.contains('JobSerializerClass__c'), + 'The misconfigured field has to be named, but was: ' + job.retryHistory + ); + } + + @IsTest + private static void shouldReportWhenTheStoredClassNoLongerExists() { + AsyncResult__c topLevel = orphanResultFor('NoSuchJobClass'); + AsyncResult__c nested = orphanResultFor('NoSuch.JobClass'); + insert new List{ topLevel, nested }; + + Test.startTest(); + Async.RequeueSummary summary = Async.requeue(new Set{ topLevel.Id, nested.Id }); Test.stopTest(); + Assert.isTrue( + summary.skipReasonByResultId.get(topLevel.Id).contains('NoSuchJobClass'), + 'A renamed or deleted class must be named, but was: ' + + summary.skipReasonByResultId.get(topLevel.Id) + ); + Assert.isTrue( + summary.skipReasonByResultId.get(nested.Id).contains('NoSuch.JobClass'), + 'An inner class name goes through the two-argument lookup too: ' + + summary.skipReasonByResultId.get(nested.Id) + ); Assert.areEqual( - '6', - processedCountSeenBy('PREEMPT'), - 'A lower priority job added mid-run waits for the whole run.' + 2, + [ + SELECT COUNT() + FROM AsyncResult__c + WHERE RequeueStatus__c = :AsyncRequeue.STATUS_STORED + ], + 'A row that could not be replayed must stay replayable once the class is back.' + ); + } + + private static QueueableChain chainWithLogger(String loggerClassName) { + return chainWithSettings( + new QueueableJobSetting__mdt( + QueueableJobName__c = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, + LoggerClass__c = loggerClassName + ) ); } - @IsTest - private static void shouldRunDependentJobWhenEveryChunkSucceeded() { - List accounts = createAccounts(6); - - Test.startTest(); - Async.chunk(new MarkingChunkJob(), ChunkSource.of(accounts)) - .chunkSize(2) - .chain(new ProcessedCountMarkerJob('DEPENDENT')) - .dependsOn(Async.afterPrevious().succeeded()) - .enqueue(); - Test.stopTest(); + private static String loggerName(String simpleName) { + return getClassNameWithNamespaceDotPrefix('AsyncTest.' + simpleName); + } + + private static Async.Dependency dependencyOn(String customJobId, Async.Outcome outcome) { + Async.Dependency dependency = new Async.Dependency(); + dependency.resolvedTargetCustomJobId = customJobId; + dependency.requiredOutcome = outcome; + return dependency; + } + + private static Map resultsEnabledForAll() { + return new Map{ + QueueableManager.QUEUEABLE_JOB_SETTING_ALL => new QueueableJobSetting__mdt( + DeveloperName = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, + CreateResult__c = true + ) + }; + } + + private static MarkerJob markerJob(String tag, Boolean shouldFail) { + MarkerJob job = new MarkerJob(); + job.tag = tag; + job.shouldFail = shouldFail; + return job; + } + + private static Integer accountCount(String name) { + return [SELECT COUNT() FROM Account WHERE Name = :name]; + } + + private static Account hookRecord() { + return [SELECT Description FROM Account WHERE Name = 'HOOK' LIMIT 1]; + } + + private static String getClassNameWithNamespaceDotPrefix(String className) { + return getNamespaceDotPrefix() + className; + } + + private static String getNamespaceDotPrefix() { + String className = AsyncTest.class.getName(); + return className.contains('.') ? className.substringBefore('.') + '.' : ''; + } + + public Iterable start(Database.BatchableContext bc) { + // This is just a placeholder to start the batch. + return new List{ new Account() }; + } + + public void execute(Database.BatchableContext ctx, List scope) { + for (Account acc : scope) { + acc.Description = 'Processed by: ' + ctx.getJobId(); + } + if (!scope.isEmpty() && scope[0].Id != null) { + update scope; + } + } + + public void finish(Database.BatchableContext bc) { + insert new Account(Name = 'Batch Complete', Description = 'Job: ' + bc.getJobId()); + } + + private static AsyncResult__c insertAsyncResultWithAge(String status, Integer ageDays) { + AsyncResult__c result = new AsyncResult__c(Status__c = status); + insert result; + Test.setCreatedDate(result.Id, System.now().addDays(-ageDays)); + return result; + } + + private static Integer followUpsLeftIn(QueueableChain chain) { + Integer followUps = 0; + for (QueueableJob job : chain.jobs) { + if (job instanceof MarkerJob) { + followUps++; + } + } + return followUps; + } + + private static QueueableChain chainWithSettings(QueueableJobSetting__mdt setting) { + QueueableChain chain = new QueueableChain(); + QueueableChain.jobSettingByName = new Map{ + setting.QueueableJobName__c => setting + }; + return chain; + } + + private static QueueableChain chainRunning(QueueableJob job) { + QueueableChain chain = new QueueableChain(); + chain.addJob(job); + QueueableManager.get().setChain(chain); + return chain; + } + + private static QueueableChain chainWithEnqueuedJob(QueueableJob job) { + QueueableChain chain = chainRunning(job); + job.chain = chain; + return chain; + } + + private static AsyncMock.MockFinalizerContext rolledBack() { + return new AsyncMock.MockFinalizerContext().setResult(ParentJobResult.UNHANDLED_EXCEPTION); + } + + private static AsyncMock.MockFinalizerContext committed() { + return new AsyncMock.MockFinalizerContext().setResult(ParentJobResult.SUCCESS); + } + + private static String processedCountSeenBy(String tag) { + return [SELECT Site FROM Account WHERE Name = :tag LIMIT 1].Site; + } + + private static Id fakeAsyncApexJobId() { + return AsyncApexJob.SObjectType.getDescribe().getKeyPrefix() + '0'.repeat(11) + '1AAA'; + } + + private static List createAccounts(Integer count) { + List accounts = new List(); + for (Integer i = 0; i < count; i++) { + accounts.add(new Account(Name = 'Chunk ' + i)); + } + insert accounts; + return accounts; + } + + private static List accountsWithOneFailingRecord() { + List accounts = new List{ + new Account(Name = 'ok first'), + new Account(Name = 'ok second'), + new Account(Name = 'FAIL third'), + new Account(Name = 'ok fourth'), + new Account(Name = 'ok fifth'), + new Account(Name = 'ok sixth') + }; + insert accounts; + return accounts; + } + + private static void markProcessed(List chunk, String marker) { + List toUpdate = new List(); + for (SObject record : chunk) { + toUpdate.add(new Account(Id = record.Id, Site = marker)); + } + update toUpdate; + } + + private static Map payloadStorageOn() { + return new Map{ + QueueableManager.QUEUEABLE_JOB_SETTING_ALL => new QueueableJobSetting__mdt( + QueueableJobName__c = QueueableManager.QUEUEABLE_JOB_SETTING_ALL, + StoreJobPayload__c = 'Yes' + ) + }; + } + + private static QueueableChain chainStoringPayloads() { + QueueableChain chain = new QueueableChain(); + QueueableChain.jobSettingByName = payloadStorageOn(); + return chain; + } + + private static AsyncResult__c orphanResultFor(String missingClassName) { + return new AsyncResult__c( + ClassName__c = missingClassName, + Status__c = QueueableManager.STATUS_FAILED, + JobPayload__c = '{}', + PayloadSize__c = 2, + RequeueStatus__c = AsyncRequeue.STATUS_STORED + ); + } + + private static AsyncResult__c storedResultFor(QueueableJob capturedJob) { + AsyncResult__c stored = new AsyncResult__c( + ClassName__c = capturedJob.className, + CustomJobId__c = capturedJob.customJobId, + Status__c = QueueableManager.STATUS_FAILED, + JobPayload__c = capturedJob.requeuePayload, + PayloadSize__c = capturedJob.requeuePayloadSize, + RequeueStatus__c = AsyncRequeue.STATUS_STORED + ); + insert stored; + return stored; + } + + private class DeepCloneFailJob extends QueueableJob { + public override void work() { + } + public override QueueableJob cloneForDeepCopy() { + throw new JSONException('forced deep clone failure'); + } + } + + public class SelfReferencingJob extends QueueableJob { + public SelfReferencingJob self; + public override void work() { + } + } + + public class AbstractFieldHoldingJob extends QueueableJob { + public Comparable held; + public override void work() { + } + } + + public class AsyncLibTypeHoldingJob extends QueueableJob { + public Async.Result heldResult; + public Backoff heldBackoff; + public Async.Dependency heldDependency; + public override void work() { + } + } + + public class DeepCloneRetryJob extends QueueableJob implements Async.Retryable { + public List attempts = new List(); + public override void work() { + attempts.add('attempt' + retryAttempt); + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + + public void resetBeforeRetry(Integer attempt) { + } + } + + public class DeepCloneFailRetryJob extends QueueableJob implements Async.Retryable { + public void resetBeforeRetry(Integer attempt) { + } + + public override void work() { + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + public override QueueableJob cloneForDeepCopy() { + throw new JSONException('forced deep clone failure'); + } + } + + public class RecordingLogger implements Async.OnJobEnqueued, Async.OnJobSucceeded, Async.OnJobFailed, Async.OnRetryEnqueued { + public void onJobEnqueued(Async.JobContext ctx) { + AsyncTest.loggedEvents.add('enqueued:' + ctx.className + ':' + ctx.info.get('team')); + } + public void onJobSucceeded(Async.JobContext ctx) { + AsyncTest.loggedEvents.add('succeeded:' + ctx.className); + } + public void onJobFailed(Async.FailureContext ctx) { + AsyncTest.loggedEvents.add('failed:' + ctx.className + ':' + ctx.retryAttempt); + } + public void onRetryEnqueued(Async.FailureContext ctx) { + AsyncTest.loggedEvents.add( + 'retry:' + ctx.retryAttempt + ':delay=' + ctx.nextAttemptDelayMinutes + ); + } + } + + public class UnconstructableLogger implements Async.OnJobFailed { + public UnconstructableLogger() { + throw new CustomException(LOGGER_CONSTRUCTOR_FAILURE); + } + + public void onJobFailed(Async.FailureContext ctx) { + } + } + + public class FailureOnlyLogger implements Async.OnJobFailed { + public void onJobFailed(Async.FailureContext ctx) { + AsyncTest.loggedEvents.add('failed-only:' + ctx.className); + } + } + + public class ThrowingLogger implements Async.OnJobSucceeded { + public void onJobSucceeded(Async.JobContext ctx) { + throw new CustomException('logger exploded'); + } + } + + private class SelfLoggingJob extends QueueableJob implements Async.OnJobSucceeded { + public override void work() { + } + public void onJobSucceeded(Async.JobContext ctx) { + AsyncTest.loggedEvents.add('self:' + ctx.className); + } + } + + private class SuccessfulQueueableTest extends QueueableJob { + public override void work() { + insert new Account(Name = Async.getQueueableJobContext()?.currentJob?.uniqueName); + } + } + + private class FailureQueueableTest extends QueueableJob.AllowsCallouts implements Async.Retryable { + public void resetBeforeRetry(Integer attempt) { + } + + public override void work() { + insert new Account(Name = Async.getQueueableJobContext()?.currentJob?.uniqueName); + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + } + + private class SelfConfiguringRetryJob extends QueueableJob implements Async.Retryable { + public void resetBeforeRetry(Integer attempt) { + } + + private SelfConfiguringRetryJob() { + this.maxRetries = 2; + this.backoff = Async.Backoff.fixed(4); + this.continueOnJobExecuteFail = true; + } + + public override void work() { + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + } + + private class ChainStoppingJob extends QueueableJob { + public override void work() { + Async.queueable(new StopChainFinalizer()).attachFinalizer(); + } + } + + private class MarkerJob extends QueueableJob implements Async.Retryable { + public String tag; + public Boolean shouldFail = false; + public override void work() { + insert new Account(Name = tag); + if (shouldFail) { + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + } + + public void resetBeforeRetry(Integer attempt) { + } + } + + private class StopChainFinalizer extends QueueableJob.Finalizer { + public override void work() { + Async.stopChain(); + } + } + + private class FinalizerAttachingRetryJob extends QueueableJob implements Async.Retryable { + public void resetBeforeRetry(Integer attempt) { + } + + public override void work() { + insert new Account(Name = 'RETRY-PARENT'); + Async.queueable(new MarkerFinalizer()).attachFinalizer(); + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + } + + private class MarkerFinalizer extends QueueableJob.Finalizer { + public override void work() { + insert new Account(Name = 'RETRY-FINALIZER'); + } + } + + private abstract class HookRecordingJob extends QueueableJob implements Async.Retryable { + public void resetBeforeRetry(Integer attempt) { + } + + public override void onFinalFailure(Async.FailureContext failureCtx) { + insert new Account( + Name = 'HOOK', + Description = failureCtx.retryOutcome.name() + + '|' + + (String.isBlank(failureCtx.failure?.stackTrace) ? 'NO-TRACE' : 'HAS-TRACE') + + '|' + + failureCtx.retryAttempt + + '/' + + failureCtx.maxRetries + ); + } + } + + private class FailureHookJob extends HookRecordingJob { + public override void work() { + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + } + + private class SucceedingHookJob extends HookRecordingJob { + public override void work() { + insert new Account(Name = 'SUCCEEDED'); + } + } + + private class ExplodingHookJob extends QueueableJob { + public override void work() { + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + + public override void onFinalFailure(Async.FailureContext failureCtx) { + throw new CustomException('hook exploded'); + } + } + + private class FailingChunkHookJob extends ChunkJob implements Async.ChunkResettable { + public void resetBeforeNextChunk(Integer pageNumber) { + } + + public override void work(List chunk) { + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + + public override void onFinalFailure(Async.FailureContext failureCtx) { + insert new Account(Name = 'HOOK', Description = failureCtx.retryOutcome.name()); + } + } + + private class JobChainingRetryJob extends QueueableJob implements Async.Retryable { + public void resetBeforeRetry(Integer attempt) { + } + + public override void work() { + insert new Account(Name = 'CHAIN-PARENT'); + Async.queueable(markerJob('CHAIN-FOLLOWUP', false)).chain(); + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + } + + private class ChainingFailureJob extends QueueableJob { + public override void work() { + Async.queueable(markerJob('ROLLED-BACK-FOLLOWUP', false)).chain(); + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + } + + private class FinalizerAndJobChainingFailureJob extends QueueableJob { + public override void work() { + Async.queueable(new MarkerFinalizer()).attachFinalizer(); + Async.queueable(markerJob('ROLLED-BACK-FOLLOWUP', false)).chain(); + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + } + + private class ChainStoppingFailureJob extends QueueableJob implements Async.Retryable { + public void resetBeforeRetry(Integer attempt) { + } + + public override void work() { + Async.stopChain(); + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + } + + private class JobSkippingFailureJob extends QueueableJob { + public String targetCustomJobId; + public override void work() { + Async.skipJob(targetCustomJobId); + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + } + + private class NonRetryableJob extends QueueableJob { + public override void work() { + } + public override Boolean isRetryable(Exception ex) { + return false; + } + } + + private class MessageVetoJob extends QueueableJob { + public override void work() { + } + public override Boolean isRetryable(Exception ex) { + return !ex.getMessage().containsIgnoreCase('permanent'); + } + } + + private class ThrowingClassifierJob extends QueueableJob { + public override void work() { + } + public override Boolean isRetryable(Exception ex) { + throw new CustomException('classifier blew up'); + } + } + + private class ResettableRetryJob extends QueueableJob implements Async.Retryable { + public Boolean wasReset = false; + public Integer resetAttempt; + public override void work() { + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + public void resetBeforeRetry(Integer attempt) { + this.wasReset = true; + this.resetAttempt = attempt; + } + } + + private class UngatedRetryJob extends QueueableJob { + public override void work() { + } + } + + public class RetryingChunkJob extends ChunkJob implements Async.Retryable, Async.ChunkResettable { + public List touched = new List(); + + public override void work(List chunk) { + String firstOfPage = (String) chunk[0].get('Name'); + AsyncTest.chunkPagesRun.add(firstOfPage + ':' + touched.size()); + touched.add('page'); + if (firstOfPage == 'P3' && AsyncTest.chunkPageFailures == 0) { + AsyncTest.chunkPageFailures++; + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + } + + public void resetBeforeRetry(Integer attempt) { + } + + public void resetBeforeNextChunk(Integer pageNumber) { + } + } + + private class GateTrippingParentJob extends QueueableJob { + public override void work() { + insert new Account(Name = 'GATE-PARENT-RAN'); + Async.queueable(new UngatedRetryJob()).retry(2).chain(); + } + } + + public class UngatedChunkJob extends ChunkJob { + public override void work(List chunk) { + } + } + + private class OkCalloutMock implements HttpCalloutMock { + public HttpResponse respond(HttpRequest request) { + HttpResponse response = new HttpResponse(); + response.setStatusCode(200); + return response; + } + } + + public class MarkerCalloutChunkJob extends ChunkJob implements Database.AllowsCallouts, Async.ChunkResettable { + public override void work(List chunk) { + HttpRequest request = new HttpRequest(); + request.setEndpoint('https://example.com'); + request.setMethod('GET'); + AsyncTest.chunkCalloutStatus = new Http().send(request).getStatusCode(); + AsyncTest.chunkCalloutPages++; + } + + public void resetBeforeNextChunk(Integer pageNumber) { + } + } + + public class StateCarryingRetryJob extends QueueableJob implements Async.Retryable { + public List seen = new List(); + + public override void work() { + insert new Account(Name = 'RETRY-SIZE-' + seen.size()); + seen.add('attempt'); + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + + public void resetBeforeRetry(Integer attempt) { + } + } + + public class StateCarryingChunkJob extends ChunkJob implements Async.ChunkResettable { + public List seen = new List(); + public List pagesReset = new List(); + + public override void work(List chunk) { + insert new Account(Name = 'CHUNK-SIZE-' + seen.size()); + seen.add('page'); + } + + public void resetBeforeNextChunk(Integer pageNumber) { + pagesReset.add(pageNumber); + } + } + + private class LegacyResetForRetryJob extends QueueableJob implements Async.Retryable { + public Boolean legacyHookRan = false; + public override void work() { + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + public override void resetForRetry() { + this.legacyHookRan = true; + } + public void resetBeforeRetry(Integer attempt) { + } + } + + private class QueueableTestFinalizer extends QueueableJob.Finalizer { + public override void work() { + FinalizerContext finalizerCtx = Async.getQueueableJobContext()?.finalizerCtx; + insert new Account( + Name = Async.getQueueableJobContext()?.currentJob?.uniqueName, + Description = finalizerCtx?.getResult() == ParentJobResult.SUCCESS + ? 'Success' + : finalizerCtx?.getException()?.getMessage() + ); + } + } - Assert.areEqual( - '6', - processedCountSeenBy('DEPENDENT'), - 'The run outcome is SUCCESS when every page passed.' - ); + private class FinalizerErrorQueueableTest extends QueueableJob { + public override void work() { + Async.queueable(new SuccessfulQueueableTest()).attachFinalizer(); + } } - @IsTest - private static void shouldSkipDependentJobWhenAnyChunkFailed() { - List accounts = accountsWithOneFailingRecord(); + private class ChainedQueueableJob extends QueueableJob { + private Integer chainDepthLimit = 1; + private Integer currentChainDepth = 0; - Test.startTest(); - Async.chunk(new FailMarkedChunkJob(), ChunkSource.of(accounts)) - .chunkSize(2) - .chain(new ProcessedCountMarkerJob('DEPENDENT')) - .dependsOn(Async.afterPrevious().succeeded()) - .enqueue(); - Test.stopTest(); + public ChainedQueueableJob(Integer chainDepthLimit) { + this.chainDepthLimit = chainDepthLimit; + } - Assert.areEqual( - 0, - [SELECT COUNT() FROM Account WHERE Name = 'DEPENDENT'], - 'One failed page makes the whole run outcome FAILURE.' - ); + public override void work() { + currentChainDepth++; + insert new Account(Name = Async.getQueueableJobContext()?.currentJob?.uniqueName); + if (currentChainDepth < chainDepthLimit) { + Async.queueable(this).enqueue(); + } + } } - @IsTest - private static void shouldRunDependentJobOnFinishedRunEvenAfterAFailedChunk() { - List accounts = accountsWithOneFailingRecord(); + public class QueueableJobTest1 extends QueueableJob implements Async.Retryable { + public QueueableJobTest2 complexMember = new QueueableJobTest2(); + public override void work() { + } - Test.startTest(); - Async.chunk(new FailMarkedChunkJob(), ChunkSource.of(accounts)) - .chunkSize(2) - .chain(new ProcessedCountMarkerJob('DEPENDENT')) - .dependsOn(Async.afterPrevious().finished()) - .enqueue(); - Test.stopTest(); + public void resetBeforeRetry(Integer attempt) { + } + } - Assert.areEqual( - 1, - [SELECT COUNT() FROM Account WHERE Name = 'DEPENDENT'], - 'finished() runs the dependent job whatever the run outcome was.' - ); + private class QueueableJobTest2 extends QueueableJob { + public String primitiveMember = PRIMITIVE_VALUE_INITIAL; + public override void work() { + } } - @IsTest - private static void shouldRecordSummaryResultWhenRunHaltsOnFailure() { - List accounts = accountsWithOneFailingRecord(); - QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = resultsEnabledForAll(); + private class QueueableJobTest3 extends QueueableJob { + public override void work() { + } + } - Test.startTest(); - QueueableManager.get().setChain(chain); - Async.chunk(new FailMarkedChunkJob(), ChunkSource.of(accounts)) - .chunkSize(2) - .stopRemainingChunksOnFailure() - .enqueue(); - Test.stopTest(); + private class QueueableJobTest4 extends QueueableJob { + public override void work() { + } + } - List stopped = [ - SELECT SkipReason__c - FROM AsyncResult__c - WHERE Status__c = :QueueableManager.STATUS_SKIPPED_CHUNK_STOPPED - ]; - Assert.areEqual(1, stopped.size(), 'A halted run records one summary result.'); - Assert.isTrue( - stopped[0].SkipReason__c.contains('page 3 of 3'), - 'The summary states where the run stopped: ' + stopped[0].SkipReason__c - ); - Assert.isTrue( - stopped[0].SkipReason__c.contains('2 record(s) were not processed'), - 'The summary states how much work was left: ' + stopped[0].SkipReason__c - ); + private class QueueableJobTest5 extends QueueableJob { + public override void work() { + } } - @IsTest - private static void shouldRecordSummaryResultWhenChainStopsDuringRun() { - List accounts = createAccounts(6); - QueueableChain chain = new QueueableChain(); - chain.queueableJobSettingByJobName = resultsEnabledForAll(); + private class QueueableJobTest6 extends QueueableJob { + public override void work() { + } + } - Test.startTest(); - QueueableManager.get().setChain(chain); - Async.chunk(new ChainStoppingChunkJob(), ChunkSource.of(accounts)).chunkSize(2).enqueue(); - Test.stopTest(); + private class QueueableJobTest7 extends QueueableJob { + public override void work() { + } + } - Assert.areEqual( - 2, - [SELECT COUNT() FROM Account WHERE Site = :CHUNK_PROCESSED], - 'Async.stopChain() ends the run wherever it is.' - ); - List stopped = [ - SELECT SkipReason__c - FROM AsyncResult__c - WHERE - Status__c = :QueueableManager.STATUS_SKIPPED_CHAIN_STOPPED - AND SkipReason__c LIKE '%not processed%' - ]; - Assert.areEqual(1, stopped.size(), 'A stopped chain records what the run left behind.'); - Assert.isTrue( - stopped[0].SkipReason__c.contains('page 2 of 3'), - 'The summary points at the first page that never ran: ' + stopped[0].SkipReason__c - ); + private class QueueableJobTest8 extends QueueableJob { + public override void work() { + } } - private static String processedCountSeenBy(String tag) { - return [SELECT Site FROM Account WHERE Name = :tag LIMIT 1].Site; + private class SchedulableTest implements Schedulable { + public void execute(SchedulableContext ctx) { + } } - private static Id fakeAsyncApexJobId() { - return AsyncApexJob.SObjectType.getDescribe().getKeyPrefix() + '0'.repeat(11) + '1AAA'; + private class CustomException extends Exception { } - private static List createAccounts(Integer count) { - List accounts = new List(); - for (Integer i = 0; i < count; i++) { - accounts.add(new Account(Name = 'Chunk ' + i)); + public class ParentJobWithFinalizer extends QueueableJob { + private String mockId; + + public ParentJobWithFinalizer(String mockId) { + this.mockId = mockId; + } + + public override void work() { + Async.queueable(new ErrorHandlerFinalizer()).mockId(mockId).attachFinalizer(); } - insert accounts; - return accounts; } - private static List accountsWithOneFailingRecord() { - List accounts = new List{ - new Account(Name = 'ok first'), - new Account(Name = 'ok second'), - new Account(Name = 'FAIL third'), - new Account(Name = 'ok fourth'), - new Account(Name = 'ok fifth'), - new Account(Name = 'ok sixth') - }; - insert accounts; - return accounts; + public class ErrorHandlerFinalizer extends QueueableJob.Finalizer { + public override void work() { + FinalizerContext ctx = this.finalizerCtx; + if (ctx?.getResult() == ParentJobResult.UNHANDLED_EXCEPTION) { + insert new Account( + Name = 'Error Log', + Description = ctx.getException()?.getMessage() + ); + } + } } - private static void markProcessed(List chunk, String marker) { - List toUpdate = new List(); - for (SObject record : chunk) { - toUpdate.add(new Account(Id = record.Id, Site = marker)); + public class AccountCreatorJob extends QueueableJob { + private String accountName; + + public AccountCreatorJob(String accountName) { + this.accountName = accountName; + } + + public override void work() { + Id jobId = this.queueableCtx?.getJobId(); + insert new Account(Name = accountName, Description = 'Job: ' + jobId); } - update toUpdate; } private class MarkingChunkJob extends ChunkJob implements Async.ChunkResettable, Async.Retryable { @@ -5906,4 +6901,58 @@ private class AsyncTest implements Database.Batchable { ); } } + + // Type.forName cannot reach a private inner class, and requeue rebuilds jobs by name. + public class RequeueableJob extends QueueableJob { + public String tag; + + public RequeueableJob() { + } + + public RequeueableJob(String tag) { + this.tag = tag; + } + + public override void work() { + insert new Account(Name = tag); + } + } + + public class FailingRequeueableJob extends QueueableJob implements Async.Retryable { + public override void work() { + throw new CustomException(AsyncTest.CUSTOM_ERROR_MESSAGE); + } + + public void resetBeforeRetry(Integer attempt) { + } + } + + public class BulkyRequeueableJob extends QueueableJob { + public List payloadParts = new List(); + + public override void work() { + } + } + + public class NullReturningJobSerializer implements Async.JobSerializer { + public String serialize(QueueableJob job) { + return null; + } + + public QueueableJob deserialize(String className, String payload) { + return null; + } + } + + public class RecordingJobSerializer implements Async.JobSerializer { + public String serialize(QueueableJob job) { + AsyncTest.loggedEvents.add('serialize:' + job.className); + return JSON.serialize(job); + } + + public QueueableJob deserialize(String className, String payload) { + AsyncTest.loggedEvents.add('deserialize:' + className); + return (QueueableJob) JSON.deserialize(payload, Type.forName(className)); + } + } } diff --git a/force-app/main/default/classes/mocks/AsyncMock.cls b/force-app/main/default/classes/mocks/AsyncMock.cls index 27fa6c5..9526b0f 100644 --- a/force-app/main/default/classes/mocks/AsyncMock.cls +++ b/force-app/main/default/classes/mocks/AsyncMock.cls @@ -5,6 +5,14 @@ public class AsyncMock { private static FinalizerMockSetup defaultFinalizerSetup; private static QueueableMockSetup defaultQueueableSetup; + public static void jobSettings(List settings) { + Map settingByJobName = new Map(); + for (QueueableJobSetting__mdt setting : settings) { + settingByJobName.put(setting.QueueableJobName__c, setting); + } + QueueableChain.jobSettingByName = settingByJobName; + } + public static FinalizerMockSetup whenFinalizer(String mockId) { FinalizerMockSetup setup = new FinalizerMockSetup(); finalizerSetups.put(mockId, setup); @@ -58,6 +66,7 @@ public class AsyncMock { queueableSetups.clear(); defaultFinalizerSetup = null; defaultQueueableSetup = null; + QueueableChain.jobSettingByName = null; } public class FinalizerMockSetup { diff --git a/force-app/main/default/classes/queue/AsyncEventDispatcher.cls b/force-app/main/default/classes/queue/AsyncEventDispatcher.cls new file mode 100644 index 0000000..ff39ca1 --- /dev/null +++ b/force-app/main/default/classes/queue/AsyncEventDispatcher.cls @@ -0,0 +1,127 @@ +/** + * PMD False Positives: + * - AvoidDebugStatements: a logger that cannot be reached leaves the debug log as the only channel + * - ExcessiveParameterList: one event carries exactly one of the two context types, and bundling + * them into a wrapper would allocate an object per event for no gain + **/ +@SuppressWarnings('PMD.AvoidDebugStatements,PMD.ExcessiveParameterList') +public inherited sharing class AsyncEventDispatcher { + private enum Event { + ENQUEUED, + SUCCEEDED, + FAILED, + RETRY_ENQUEUED + } + + private static Set alreadyWarnedClassNames = new Set(); + + public static void jobEnqueued(QueueableJob job, String loggerClassName) { + dispatch(job, loggerClassName, Event.ENQUEUED, new Async.JobContext(job), null); + } + + public static void jobSucceeded(QueueableJob job, String loggerClassName) { + dispatch(job, loggerClassName, Event.SUCCEEDED, new Async.JobContext(job), null); + } + + public static void jobFailed(QueueableJob job, String loggerClassName) { + dispatch(job, loggerClassName, Event.FAILED, null, new Async.FailureContext(job)); + } + + public static void retryEnqueued(QueueableJob job, Integer delayMinutes, String loggerName) { + Async.FailureContext ctx = new Async.FailureContext(job); + ctx.nextAttemptDelayMinutes = delayMinutes; + dispatch(job, loggerName, Event.RETRY_ENQUEUED, null, ctx); + } + + private static void dispatch( + QueueableJob job, + String loggerClassName, + Event firedEvent, + Async.JobContext jobCtx, + Async.FailureContext failureCtx + ) { + notifyListener(job, firedEvent, jobCtx, failureCtx); + Object registeredLogger = newLogger(job, loggerClassName); + if (registeredLogger != null) { + notifyListener(registeredLogger, firedEvent, jobCtx, failureCtx); + } + } + + private static void notifyListener( + Object listener, + Event firedEvent, + Async.JobContext jobCtx, + Async.FailureContext failureCtx + ) { + try { + switch on firedEvent { + when ENQUEUED { + if (listener instanceof Async.OnJobEnqueued) { + ((Async.OnJobEnqueued) listener).onJobEnqueued(jobCtx); + } + } + when SUCCEEDED { + if (listener instanceof Async.OnJobSucceeded) { + ((Async.OnJobSucceeded) listener).onJobSucceeded(jobCtx); + } + } + when FAILED { + if (listener instanceof Async.OnJobFailed) { + ((Async.OnJobFailed) listener).onJobFailed(failureCtx); + } + } + when RETRY_ENQUEUED { + if (listener instanceof Async.OnRetryEnqueued) { + ((Async.OnRetryEnqueued) listener).onRetryEnqueued(failureCtx); + } + } + } + } catch (Exception listenerFailure) { + reportWithoutAffectingTheJob(listenerFailure); + } + } + + private static void reportWithoutAffectingTheJob(Exception listenerFailure) { + System.debug( + LoggingLevel.ERROR, + String.format( + QueueableManager.WARNING_LOGGER_THREW, + new List{ + listenerFailure.getTypeName(), + listenerFailure.getMessage(), + listenerFailure.getStackTraceString() + } + ) + ); + } + + private static Object newLogger(QueueableJob job, String loggerClassName) { + Object logger; + try { + logger = GlobalClassFactory.newInstance(loggerClassName); + } catch (Exception instantiationFailure) { + if (alreadyWarnedClassNames.add(loggerClassName)) { + job.recordConfigurationWarning( + String.format( + QueueableManager.WARNING_LOGGER_NOT_USABLE, + new List{ loggerClassName, instantiationFailure.getMessage() } + ) + ); + } + return null; + } + if ( + logger == null && + String.isNotBlank(loggerClassName) && + alreadyWarnedClassNames.add(loggerClassName) + ) { + job.recordConfigurationWarning( + String.format( + QueueableManager.WARNING_LOGGER_NOT_FOUND, + new List{ loggerClassName, QueueableManager.API_PREFIX } + ) + ); + } + return logger; + } +} diff --git a/force-app/main/default/classes/queue/AsyncEventDispatcher.cls-meta.xml b/force-app/main/default/classes/queue/AsyncEventDispatcher.cls-meta.xml new file mode 100644 index 0000000..cad713d --- /dev/null +++ b/force-app/main/default/classes/queue/AsyncEventDispatcher.cls-meta.xml @@ -0,0 +1,5 @@ + + + 66.0 + Active + diff --git a/force-app/main/default/classes/queue/AsyncRequeue.cls b/force-app/main/default/classes/queue/AsyncRequeue.cls new file mode 100644 index 0000000..b5ed189 --- /dev/null +++ b/force-app/main/default/classes/queue/AsyncRequeue.cls @@ -0,0 +1,204 @@ +public inherited sharing class AsyncRequeue { + public static final String STATUS_STORED = 'Stored'; + public static final String STATUS_TOO_LARGE = 'TooLarge'; + public static final String STATUS_NOT_SERIALIZABLE = 'NotSerializable'; + public static final String STATUS_REQUEUED = 'Requeued'; + + public static final Integer MAX_CHARS_PER_REQUEUE = 2000000; + + private static final String STORE_PAYLOAD_YES = 'Yes'; + + public static void captureJobAsHandedOver(QueueableJob job) { + if (!storesPayload(job)) { + return; + } + if (job instanceof ChunkJob || job.isFinalizer) { + job.requeueStatus = STATUS_NOT_SERIALIZABLE; + return; + } + + String payload; + try { + payload = JobPayload.forJob(job); + } catch (Exception captureFailure) { + recordNotSerializable(job, captureFailure.getMessage()); + return; + } + if (payload == null) { + recordNotSerializable(job, QueueableManager.CAUSE_SERIALIZER_RETURNED_NULL); + return; + } + + job.requeuePayloadSize = payload.length(); + if (payload.length() > JobPayload.MAX_CHARS) { + job.requeueStatus = STATUS_TOO_LARGE; + return; + } + job.requeueStatus = STATUS_STORED; + job.requeuePayload = payload; + } + + public static Boolean storesPayload(QueueableJob job) { + return QueueableManager.settingFor( + job.className, + QueueableJobSetting__mdt.StoreJobPayload__c + ) == STORE_PAYLOAD_YES; + } + + public static Async.RequeueSummary run(Set resultIds) { + Async.RequeueSummary summary = new Async.RequeueSummary(); + if (resultIds == null || resultIds.isEmpty()) { + return summary; + } + List replayable = replayableOrExplained(resultIds, summary); + assertPayloadsWillFitInHeap(replayable); + + QueueableBuilder replay = chainRebuilt(readPayloads(replayable), summary); + if (summary.requeued.isEmpty()) { + return summary; + } + summary.enqueueResult = replay.enqueue(); + markRequeued(summary.requeued); + return summary; + } + + private static List replayableOrExplained( + Set resultIds, + Async.RequeueSummary summary + ) { + List replayable = new List(); + for (AsyncResult__c row : lockRowsWithoutPayloads(resultIds)) { + String blocker = reasonItCannotBeRequeued(row); + if (blocker == null) { + replayable.add(row); + } else { + summary.skipReasonByResultId.put(row.Id, blocker); + } + } + for (Id missingId : missingIds(resultIds, summary, replayable)) { + summary.skipReasonByResultId.put(missingId, QueueableManager.REQUEUE_SKIPPED_NO_RECORD); + } + return replayable; + } + + private static QueueableBuilder chainRebuilt( + List lockedRows, + Async.RequeueSummary summary + ) { + QueueableBuilder replay = Async.queueable(); + for (AsyncResult__c row : lockedRows) { + QueueableJob job = rebuildOrExplain(row, summary); + if (job == null) { + continue; + } + job.requeuedFromResultId = row.Id; + replay.chain(job); + summary.requeued.add(row.Id); + } + return replay; + } + + private static void recordNotSerializable(QueueableJob job, String cause) { + job.requeueStatus = STATUS_NOT_SERIALIZABLE; + job.recordConfigurationWarning( + String.format( + QueueableManager.WARNING_PAYLOAD_NOT_CAPTURED, + new List{ job.className, cause } + ) + ); + } + + private static List lockRowsWithoutPayloads(Set resultIds) { + return [ + SELECT Id, ClassName__c, PayloadSize__c, RequeueStatus__c + FROM AsyncResult__c + WHERE Id IN :resultIds + WITH SYSTEM_MODE + FOR UPDATE + ]; + } + + private static List readPayloads(List lockedRows) { + return [ + SELECT Id, ClassName__c, JobPayload__c + FROM AsyncResult__c + WHERE Id IN :lockedRows + WITH SYSTEM_MODE + ]; + } + + private static Set missingIds( + Set requestedIds, + Async.RequeueSummary summary, + List replayable + ) { + Set missing = new Set(requestedIds); + missing.removeAll(summary.skipReasonByResultId.keySet()); + missing.removeAll(new Map(replayable).keySet()); + return missing; + } + + private static String reasonItCannotBeRequeued(AsyncResult__c row) { + if (row.RequeueStatus__c == STATUS_STORED) { + return null; + } + if (row.RequeueStatus__c == STATUS_REQUEUED) { + return QueueableManager.REQUEUE_SKIPPED_ALREADY_REQUEUED; + } + if (String.isBlank(row.RequeueStatus__c)) { + return QueueableManager.REQUEUE_SKIPPED_NO_PAYLOAD; + } + return String.format( + QueueableManager.REQUEUE_SKIPPED_BY_STATUS, + new List{ row.RequeueStatus__c } + ); + } + + private static void assertPayloadsWillFitInHeap(List rows) { + Integer totalChars = 0; + for (AsyncResult__c row : rows) { + totalChars += (row.PayloadSize__c ?? 0).intValue(); + } + if (totalChars > MAX_CHARS_PER_REQUEUE) { + throw new Async.IllegalArgumentException( + String.format( + QueueableManager.ERROR_MESSAGE_REQUEUE_BUDGET_EXCEEDED, + new List{ + String.valueOf(totalChars), + String.valueOf(MAX_CHARS_PER_REQUEUE), + String.valueOf(rows.size()) + } + ) + ); + } + } + + private static QueueableJob rebuildOrExplain(AsyncResult__c row, Async.RequeueSummary summary) { + try { + QueueableJob job = JobPayload.toJob(row.ClassName__c, row.JobPayload__c); + if (job == null) { + summary.skipReasonByResultId.put( + row.Id, + String.format( + QueueableManager.REQUEUE_SKIPPED_CLASS_NOT_FOUND, + new List{ row.ClassName__c, QueueableManager.API_PREFIX } + ) + ); + return null; + } + QueueableManager.assertStateHandlingDeclared(job); + return job; + } catch (Exception rebuildFailure) { + summary.skipReasonByResultId.put(row.Id, rebuildFailure.getMessage()); + return null; + } + } + + private static void markRequeued(List resultIds) { + List toMark = new List(); + for (Id resultId : resultIds) { + toMark.add(new AsyncResult__c(Id = resultId, RequeueStatus__c = STATUS_REQUEUED)); + } + update as system toMark; + } +} diff --git a/force-app/main/default/classes/queue/AsyncRequeue.cls-meta.xml b/force-app/main/default/classes/queue/AsyncRequeue.cls-meta.xml new file mode 100644 index 0000000..cad713d --- /dev/null +++ b/force-app/main/default/classes/queue/AsyncRequeue.cls-meta.xml @@ -0,0 +1,5 @@ + + + 66.0 + Active + diff --git a/force-app/main/default/classes/queue/ChunkBuilder.cls b/force-app/main/default/classes/queue/ChunkBuilder.cls index 37565c7..7dd890b 100644 --- a/force-app/main/default/classes/queue/ChunkBuilder.cls +++ b/force-app/main/default/classes/queue/ChunkBuilder.cls @@ -135,6 +135,18 @@ public inherited sharing class ChunkBuilder { return this; } + public ChunkBuilder info(String key, String value) { + job.info = job.info ?? new Map(); + job.info.put(key, value); + return this; + } + + public ChunkBuilder info(Map info) { + job.info = job.info ?? new Map(); + job.info.putAll(info); + return this; + } + public ChunkBuilder mockId(String mockId) { job.mockId = mockId; return this; diff --git a/force-app/main/default/classes/queue/GlobalClassFactory.cls b/force-app/main/default/classes/queue/GlobalClassFactory.cls new file mode 100644 index 0000000..eb31194 --- /dev/null +++ b/force-app/main/default/classes/queue/GlobalClassFactory.cls @@ -0,0 +1,27 @@ +/** + * Returns null when the name does not resolve, and throws when it resolves but cannot be + * constructed. Callers need those apart, because the first means "not declared global" and the + * second means "no usable constructor", and only they know which setting to name in the message. + **/ +public inherited sharing class GlobalClassFactory { + private static Map instanceByClassName = new Map(); + + public static Object newInstance(String className) { + if (String.isBlank(className)) { + return null; + } + if (!instanceByClassName.containsKey(className)) { + instanceByClassName.put(className, constructOrRememberFailure(className)); + } + return instanceByClassName.get(className); + } + + private static Object constructOrRememberFailure(String className) { + try { + return Type.forName(className)?.newInstance(); + } catch (Exception constructorFailure) { + instanceByClassName.put(className, null); + throw constructorFailure; + } + } +} diff --git a/force-app/main/default/classes/queue/GlobalClassFactory.cls-meta.xml b/force-app/main/default/classes/queue/GlobalClassFactory.cls-meta.xml new file mode 100644 index 0000000..cad713d --- /dev/null +++ b/force-app/main/default/classes/queue/GlobalClassFactory.cls-meta.xml @@ -0,0 +1,5 @@ + + + 66.0 + Active + diff --git a/force-app/main/default/classes/queue/JobPayload.cls b/force-app/main/default/classes/queue/JobPayload.cls new file mode 100644 index 0000000..0635df3 --- /dev/null +++ b/force-app/main/default/classes/queue/JobPayload.cls @@ -0,0 +1,53 @@ +/** + * JSON cannot cross a namespace boundary in either direction, so on a packaged install both + * conversions run in the subscriber's own code through a registered Async.JobSerializer. A source + * deployment has no boundary and needs no registration. + **/ +public inherited sharing class JobPayload { + public static final Integer MAX_CHARS = 131072; + + public static String forJob(QueueableJob job) { + QueueableJob storable = job.copyWithoutRuntimeState(); + Async.JobSerializer serializer = registeredSerializer(job.className); + return serializer == null ? JSON.serialize(storable) : serializer.serialize(storable); + } + + public static QueueableJob toJob(String className, String payload) { + Async.JobSerializer serializer = registeredSerializer(className); + if (serializer != null) { + return serializer.deserialize(className, payload); + } + Type jobType = resolveType(className); + return jobType == null ? null : (QueueableJob) JSON.deserialize(payload, jobType); + } + + public static Type resolveType(String fullName) { + return Type.forName(fullName) ?? resolveTypeInsideNamespace(fullName); + } + + private static Type resolveTypeInsideNamespace(String fullName) { + return fullName.contains('.') + ? Type.forName(fullName.substringBefore('.'), fullName.substringAfter('.')) + : null; + } + + private static Async.JobSerializer registeredSerializer(String jobClassName) { + String serializerClassName = QueueableManager.settingFor( + jobClassName, + QueueableJobSetting__mdt.JobSerializerClass__c + ); + if (String.isBlank(serializerClassName)) { + return null; + } + Object serializer = GlobalClassFactory.newInstance(serializerClassName); + if (!(serializer instanceof Async.JobSerializer)) { + throw new Async.IllegalArgumentException( + String.format( + QueueableManager.ERROR_MESSAGE_SERIALIZER_NOT_USABLE, + new List{ serializerClassName, QueueableManager.API_PREFIX } + ) + ); + } + return (Async.JobSerializer) serializer; + } +} diff --git a/force-app/main/default/classes/queue/JobPayload.cls-meta.xml b/force-app/main/default/classes/queue/JobPayload.cls-meta.xml new file mode 100644 index 0000000..cad713d --- /dev/null +++ b/force-app/main/default/classes/queue/JobPayload.cls-meta.xml @@ -0,0 +1,5 @@ + + + 66.0 + Active + diff --git a/force-app/main/default/classes/queue/QueueableBuilder.cls b/force-app/main/default/classes/queue/QueueableBuilder.cls index 8f4ae07..55820cb 100644 --- a/force-app/main/default/classes/queue/QueueableBuilder.cls +++ b/force-app/main/default/classes/queue/QueueableBuilder.cls @@ -109,6 +109,18 @@ public inherited sharing class QueueableBuilder { return this; } + public QueueableBuilder info(String key, String value) { + job.info = job.info ?? new Map(); + job.info.put(key, value); + return this; + } + + public QueueableBuilder info(Map info) { + job.info = job.info ?? new Map(); + job.info.putAll(info); + return this; + } + public QueueableBuilder mockId(String mockId) { job.mockId = mockId; return this; diff --git a/force-app/main/default/classes/queue/QueueableChain.cls b/force-app/main/default/classes/queue/QueueableChain.cls index c4f9173..ab4fcfc 100644 --- a/force-app/main/default/classes/queue/QueueableChain.cls +++ b/force-app/main/default/classes/queue/QueueableChain.cls @@ -1,9 +1,11 @@ /** * PMD False Positives: - * - CognitiveComplexity / CyclomaticComplexity / StdCyclomaticComplexity: This was intended to - * have all the logic in one class + * - CognitiveComplexity / CyclomaticComplexity / StdCyclomaticComplexity / NcssTypeCount: This + * was intended to have all the logic in one class **/ -@SuppressWarnings('PMD.CognitiveComplexity,PMD.CyclomaticComplexity,PMD.StdCyclomaticComplexity') +@SuppressWarnings( + 'PMD.CognitiveComplexity,PMD.CyclomaticComplexity,PMD.StdCyclomaticComplexity,PMD.NcssTypeCount' +) public inherited sharing class QueueableChain { @TestVisible private List jobs = new List(); @@ -22,20 +24,24 @@ public inherited sharing class QueueableChain { private String chainId; @TestVisible - private Map queueableJobSettingByJobName { + public static Map jobSettingByName { get { - if (Test.isRunningTest()) { - return queueableJobSettingByJobName ?? new Map(); + if (jobSettingByName == null) { + jobSettingByName = fromCustomMetadata(); } - Map jobSettings = new Map(); - for (QueueableJobSetting__mdt jobSetting : QueueableJobSetting__mdt.getAll().values()) { - jobSettings.put(jobSetting.QueueableJobName__c, jobSetting); - } - return jobSettings; + return jobSettingByName; } private set; } + private static Map fromCustomMetadata() { + Map settings = new Map(); + for (QueueableJobSetting__mdt setting : QueueableJobSetting__mdt.getAll().values()) { + settings.put(setting.QueueableJobName__c, setting); + } + return settings; + } + public void execute(QueueableContext ctx) { QueueableManager.get().setChain(this); attachQueueableChainFinalizer(); @@ -212,10 +218,15 @@ public inherited sharing class QueueableChain { } private void notifyFinalFailure(QueueableJob job) { + if (job.skipStatus != null) { + return; + } if (!job.hasFailed) { + AsyncEventDispatcher.jobSucceeded(job, QueueableManager.loggerClassFor(job)); return; } job.safeOnFinalFailure(new Async.FailureContext(job)); + AsyncEventDispatcher.jobFailed(job, QueueableManager.loggerClassFor(job)); } private void recordChunkRunOutcome(ChunkJob settledPage) { @@ -262,10 +273,20 @@ public inherited sharing class QueueableChain { retryJob.retryDecision = false; retryJob.finalizerCtx = null; retryJob.retryHistory = previousJob.retryHistory; + retryJob.requeuePayload = previousJob.requeuePayload; + retryJob.requeueStatus = previousJob.requeueStatus; + retryJob.requeuePayloadSize = previousJob.requeuePayloadSize; + retryJob.requeuedFromResultId = previousJob.requeuedFromResultId; + releasePayload(previousJob); QueueableManager.runRetryReset(retryJob, nextAttempt); revertChainChangesBeforeRetry(); jobs.add(0, retryJob); + AsyncEventDispatcher.retryEnqueued( + previousJob, + delayMinutes, + QueueableManager.loggerClassFor(previousJob) + ); enqueueNextJobIfAny(); } @@ -349,9 +370,17 @@ public inherited sharing class QueueableChain { } private Boolean resultEnabledFor(QueueableJob job) { - return queueableJobSettingByJobName.get(QueueableManager.QUEUEABLE_JOB_SETTING_ALL) - ?.CreateResult__c == true || - queueableJobSettingByJobName.get(job.className)?.CreateResult__c == true; + return createResultEnabledFor(job) || payloadStorageOnAndJobDidNotSucceed(job); + } + + private Boolean createResultEnabledFor(QueueableJob job) { + return jobSettingByName.get(QueueableManager.QUEUEABLE_JOB_SETTING_ALL)?.CreateResult__c == + true || + jobSettingByName.get(job.className)?.CreateResult__c == true; + } + + private Boolean payloadStorageOnAndJobDidNotSucceed(QueueableJob job) { + return job.requeueStatus != null && (job.hasFailed || job.skipStatus != null); } private void createPendingResults() { @@ -374,6 +403,7 @@ public inherited sharing class QueueableChain { toRecord.add(job); } else { job.resultCreated = true; + releasePayload(job); } } if (toRecord.isEmpty()) { @@ -384,14 +414,43 @@ public inherited sharing class QueueableChain { for (QueueableJob job : toRecord) { results.add(buildResult(job)); } + dropLinksToSourcesTheCleanupBatchRemoved(results); insert as system results; for (Integer i = 0; i < toRecord.size(); i++) { toRecord[i].resultCreated = true; resultIdByCustomJobId.put(toRecord[i].customJobId, results[i].Id); + releasePayload(toRecord[i]); } linkDependencyResults(toRecord); } + // The payload rides the serialized job into every later execution context, so a job that kept + // it would carry its own size twice for the rest of the chain. + private void releasePayload(QueueableJob job) { + job.requeuePayload = null; + } + + private void dropLinksToSourcesTheCleanupBatchRemoved(List results) { + Set sourceIds = new Set(); + for (AsyncResult__c result : results) { + if (result.RequeuedFrom__c != null) { + sourceIds.add(result.RequeuedFrom__c); + } + } + if (sourceIds.isEmpty()) { + return; + } + Set stillThere = new Map( + [SELECT Id FROM AsyncResult__c WHERE Id IN :sourceIds WITH SYSTEM_MODE] + ) + .keySet(); + for (AsyncResult__c result : results) { + if (result.RequeuedFrom__c != null && !stillThere.contains(result.RequeuedFrom__c)) { + result.RequeuedFrom__c = null; + } + } + } + private AsyncResult__c buildResult(QueueableJob job) { AsyncResult__c result = new AsyncResult__c( SalesforceJobId__c = job.salesforceJobId, @@ -399,8 +458,14 @@ public inherited sharing class QueueableChain { ChainId__c = job.chainId, ClassName__c = job.className, RetryAttempts__c = job.retryAttempt, - RetryHistory__c = job.retryHistory + RetryHistory__c = job.retryHistory, + RequeuedFrom__c = job.requeuedFromResultId ); + if (payloadStorageOnAndJobDidNotSucceed(job)) { + result.JobPayload__c = job.requeuePayload; + result.PayloadSize__c = job.requeuePayloadSize; + result.RequeueStatus__c = job.requeueStatus; + } if (job.skipStatus != null) { result.Status__c = job.skipStatus; result.SkipReason__c = job.skipReason; @@ -588,6 +653,7 @@ public inherited sharing class QueueableChain { } public void addJob(QueueableJob job) { + AsyncRequeue.captureJobAsHandedOver(job); job.setMainAttributes(); if (chainId == null) { chainId = UUID.randomUUID().toString(); @@ -599,6 +665,7 @@ public inherited sharing class QueueableChain { job.captureEnqueuedState(); jobs.add(job); jobs.sort(); + AsyncEventDispatcher.jobEnqueued(job, QueueableManager.loggerClassFor(job)); } private void keepSlotOfChunkRunOrTakeNextOne(QueueableJob job) { @@ -621,30 +688,18 @@ public inherited sharing class QueueableChain { return; } - Integer settingMaxRetries = setting.MaxRetries__c.intValue(); - if (settingMaxRetries > QueueableManager.MAX_RETRY_CAP) { - throw new IllegalArgumentException( - QueueableManager.ERROR_MESSAGE_MAX_RETRIES_EXCEEDS_CAP + if (!QueueableManager.declaresRetryStateHandling(job)) { + job.recordConfigurationWarning( + String.format( + QueueableManager.WARNING_CMDT_RETRY_WITHOUT_RESET, + new List{ job.className, QueueableManager.API_PREFIX } + ) ); + return; } - job.maxRetries = settingMaxRetries; - - if (job.backoff == null && String.isNotBlank(setting.BackoffStrategy__c)) { - Integer baseMinutes = setting.BackoffBaseMinutes__c?.intValue() ?? 1; - job.backoff = Backoff.fromName(setting.BackoffStrategy__c, baseMinutes); - if (job.backoff == null) { - throw new IllegalArgumentException( - String.format( - QueueableManager.ERROR_MESSAGE_UNKNOWN_BACKOFF_STRATEGY, - new List{ - setting.BackoffStrategy__c, - setting.QueueableJobName__c, - Backoff.KNOWN_STRATEGIES - } - ) - ); - } - } + + job.maxRetries = QueueableManager.cappedRetries(job, setting.MaxRetries__c.intValue()); + QueueableManager.applyBackoffDefault(job, setting); if (job.retryOnExceptionTypes == null) { job.retryOnExceptionTypes = parseExceptionTypes(setting.RetryableExceptions__c); @@ -652,7 +707,7 @@ public inherited sharing class QueueableChain { } private QueueableJobSetting__mdt resolveRetrySetting(String jobClassName) { - Map jobSettings = queueableJobSettingByJobName; + Map jobSettings = jobSettingByName; QueueableJobSetting__mdt setting = jobSettings.get(jobClassName); if (setting == null || setting.MaxRetries__c == null) { setting = jobSettings.get(QueueableManager.QUEUEABLE_JOB_SETTING_ALL); @@ -716,7 +771,7 @@ public inherited sharing class QueueableChain { @TestVisible private void removeJobsThatAreDisabledAndDependentFinalizers() { - Map jobSettings = queueableJobSettingByJobName; + Map jobSettings = jobSettingByName; if (jobSettings.isEmpty()) { return; diff --git a/force-app/main/default/classes/queue/QueueableJob.cls b/force-app/main/default/classes/queue/QueueableJob.cls index 505ae08..8a1132e 100644 --- a/force-app/main/default/classes/queue/QueueableJob.cls +++ b/force-app/main/default/classes/queue/QueueableJob.cls @@ -41,6 +41,12 @@ public without sharing abstract class QueueableJob implements Queueable, Compara public Boolean retryDecision = false; public String retryHistory; + public String requeuePayload; + public String requeueStatus; + public Integer requeuePayloadSize; + public Id requeuedFromResultId; + + public Map info; public String parentCustomJobId; public FinalizerContext finalizerCtx; public String mockId; @@ -50,7 +56,7 @@ public without sharing abstract class QueueableJob implements Queueable, Compara @TestVisible private QueueableJob enqueuedState; - public transient String className { + public String className { get { if (className == null) { className = getFullClassName(this); @@ -103,17 +109,10 @@ public without sharing abstract class QueueableJob implements Queueable, Compara } public virtual QueueableJob cloneForDeepCopy() { - String fullName = this.className; - Type resolvedType = Type.forName(fullName); - - if (resolvedType == null && fullName.contains('.')) { - resolvedType = Type.forName( - fullName.substringBefore('.'), - fullName.substringAfter('.') - ); - } - - return (QueueableJob) JSON.deserialize(JSON.serialize(this), resolvedType); + return (QueueableJob) JSON.deserialize( + JSON.serialize(this), + JobPayload.resolveType(className) + ); } public void execute(QueueableContext ctx) { @@ -296,15 +295,16 @@ public without sharing abstract class QueueableJob implements Queueable, Compara } /** - * JSON cannot round-trip a live job, for three unrelated reasons: + * JSON cannot round-trip a live job, for unrelated reasons: * - `chain` points back at this job: "Cycle detected" * - `queueableCtx` / `finalizerCtx` are platform interfaces: "Cannot deserialize JSON as * abstract type" * - `backoff` / `dependencies` / `failure` are Async Lib types, and JSON.serialize refuses any * graph crossing a namespace, so a subscriber's override hits "Type cannot be serialized" + * - `requeuePayload` would otherwise nest the previous payload inside every new one * Stripping a shallow copy rather than this job also covers subclass overrides. **/ - private QueueableJob deepCloneWithoutRuntimeState() { + public QueueableJob copyWithoutRuntimeState() { QueueableJob stripped = this.clone(); stripped.chain = null; stripped.queueableCtx = null; @@ -313,9 +313,13 @@ public without sharing abstract class QueueableJob implements Queueable, Compara stripped.dependencies = null; stripped.failure = null; stripped.enqueuedState = null; + stripped.requeuePayload = null; stripped.clearFrameworkStateForCopy(); + return stripped; + } - QueueableJob copy = stripped.cloneForDeepCopy(); + private QueueableJob deepCloneWithoutRuntimeState() { + QueueableJob copy = copyWithoutRuntimeState().cloneForDeepCopy(); // Backoff is immutable so sharing is safe. The other two are not, and sharing them // would defeat the isolation deepClone() exists to provide. copy.backoff = this.backoff; @@ -365,6 +369,12 @@ public without sharing abstract class QueueableJob implements Queueable, Compara : (FailureInfo) JSON.deserialize(JSON.serialize(failure), FailureInfo.class); } + @SuppressWarnings('PMD.AvoidDebugStatements') + public void recordConfigurationWarning(String warning) { + appendRetryHistoryLine(warning); + System.debug(LoggingLevel.ERROR, warning); + } + public void recordRetryNotEnqueued(Integer attemptNumber, Exception ex) { appendRetryHistoryLine( 'Attempt ' + diff --git a/force-app/main/default/classes/queue/QueueableManager.cls b/force-app/main/default/classes/queue/QueueableManager.cls index 7f1ece4..9b47479 100644 --- a/force-app/main/default/classes/queue/QueueableManager.cls +++ b/force-app/main/default/classes/queue/QueueableManager.cls @@ -2,8 +2,10 @@ * PMD False Positives: * - CyclomaticComplexity / CognitiveComplexity: one guard clause per documented error message, * and the most complex member scores 7 + * - ExcessivePublicCount: most of the count is error and warning message constants, kept here so + * every message the framework can raise is visible and assertable in one place **/ -@SuppressWarnings('PMD.CyclomaticComplexity,PMD.CognitiveComplexity') +@SuppressWarnings('PMD.CyclomaticComplexity,PMD.CognitiveComplexity,PMD.ExcessivePublicCount') public inherited sharing class QueueableManager { public static final String QUEUEABLE_JOB_SETTING_ALL = 'All'; public static final String STATUS_COMPLETED = 'COMPLETED'; @@ -36,6 +38,37 @@ public inherited sharing class QueueableManager { API_PREFIX + 'Async.ChunkResettable and clear your own state in resetBeforeNextChunk(Integer pageNumber)\n- call restoreStateOnNextChunk() to have Async Lib replay every page from the state the job had when it was enqueued\nAn empty resetBeforeNextChunk() body is a valid answer, and keeps whatever the previous page left behind, exactly as 2.x did.\nSee ' + DOCS_URL_JOB_STATE; + public static final String WARNING_CMDT_RETRY_WITHOUT_RESET = + 'QueueableJobSetting__mdt turns retry on for "{0}", but that job does not say what happens to its state between attempts, so retry was NOT applied and the job ran once. A retry re-runs the same object and would carry the failed attempt state forward. To enable retry for this job, implement {1}Async.Retryable or call restoreStateOnRetry(). See ' + + DOCS_URL_JOB_STATE; + public static final String WARNING_UNKNOWN_BACKOFF_STRATEGY = 'QueueableJobSetting__mdt.BackoffStrategy__c is "{0}" for "{1}", which is not a known strategy, so no backoff was applied and retries run without delay. Use one of: {2}.'; + public static final String WARNING_MAX_RETRIES_EXCEEDS_CAP = 'QueueableJobSetting__mdt.MaxRetries__c is {0} for "{1}", above the framework cap of {2}, so it was clamped to {2}.'; + public static final String DOCS_URL_LOGGING = DOCS_URL + '/explanations/logging'; + public static final String WARNING_LOGGER_NOT_FOUND = + 'QueueableJobSetting__mdt.LoggerClass__c names "{0}", which could not be resolved, so no events were logged. Async Lib can only reach a class declared global. Change it to "global class {0}" and make sure the name is spelled exactly as the class. See ' + + DOCS_URL_LOGGING; + public static final String WARNING_LOGGER_NOT_USABLE = + 'QueueableJobSetting__mdt.LoggerClass__c names "{0}", which could not be constructed, so no events were logged: {1}. The class needs a public no-argument constructor. See ' + + DOCS_URL_LOGGING; + public static final String WARNING_LOGGER_THREW = 'An Async Lib event listener threw {0}: {1}. The job was not affected. {2}'; + public static final String DOCS_URL_REQUEUE = DOCS_URL + '/explanations/requeue'; + public static final String WARNING_PAYLOAD_NOT_CAPTURED = + 'QueueableJobSetting__mdt.StoreJobPayload__c is Yes for "{0}", but the job could not be serialized, so it cannot be requeued: {1}. On a packaged install this is usually a missing JobSerializerClass__c, because JSON cannot cross a namespace boundary. Otherwise the job holds a field JSON cannot write, such as an interface or an abstract type. See ' + + DOCS_URL_REQUEUE; + public static final String ERROR_MESSAGE_SERIALIZER_NOT_USABLE = + 'QueueableJobSetting__mdt.JobSerializerClass__c names "{0}", which is not a usable {1}Async.JobSerializer. Async Lib can only reach a class declared global, and it has to implement that interface. See ' + + DOCS_URL_REQUEUE; + public static final String ERROR_MESSAGE_REQUEUE_BUDGET_EXCEEDED = 'Requeuing these {2} results needs {0} characters of payload, above the limit of {1}. Split the ids into smaller batches. Order by PayloadSize__c to see which results are the heavy ones.'; + public static final String REQUEUE_SKIPPED_NO_RECORD = 'No AsyncResult__c record with this id. It may have been deleted by AsyncResultCleanupBatch.'; + public static final String REQUEUE_SKIPPED_NO_PAYLOAD = + 'No payload was stored for this job, so there is nothing to replay. Set QueueableJobSetting__mdt.StoreJobPayload__c to Yes before the job runs. See ' + + DOCS_URL_REQUEUE; + public static final String REQUEUE_SKIPPED_ALREADY_REQUEUED = 'Already requeued. Requeue the result of the replay instead, so the trail stays readable.'; + public static final String REQUEUE_SKIPPED_BY_STATUS = 'The payload was not stored: {0}.'; + public static final String CAUSE_SERIALIZER_RETURNED_NULL = 'the registered JobSerializer returned null'; + public static final String REQUEUE_SKIPPED_CLASS_NOT_FOUND = + 'The class "{0}" could not be resolved, so the job cannot be rebuilt. It may have been renamed or deleted. On a packaged install, register a {1}Async.JobSerializer in QueueableJobSetting__mdt.JobSerializerClass__c. See ' + + DOCS_URL_REQUEUE; public static final String ERROR_MESSAGE_INVALID_DEPENDENCY = 'A dependency requires a target and an outcome (e.g. Async.after(result).succeeded())'; public static final String ERROR_MESSAGE_DEPENDS_ON_PREVIOUS_WITHOUT_JOB = 'dependsOn(Async.afterPrevious()) requires a previously chained job'; public static final String ERROR_MESSAGE_UNKNOWN_JOB = 'No job in this chain has custom job id "{0}".'; @@ -179,8 +212,61 @@ public inherited sharing class QueueableManager { frameworkFailure.getStackTraceString(); } + public static void applyBackoffDefault(QueueableJob job, QueueableJobSetting__mdt setting) { + if (job.backoff != null || String.isBlank(setting.BackoffStrategy__c)) { + return; + } + Integer baseMinutes = setting.BackoffBaseMinutes__c?.intValue() ?? 1; + job.backoff = Backoff.fromName(setting.BackoffStrategy__c, baseMinutes); + if (job.backoff == null) { + job.recordConfigurationWarning( + String.format( + WARNING_UNKNOWN_BACKOFF_STRATEGY, + new List{ + setting.BackoffStrategy__c, + setting.QueueableJobName__c, + Backoff.KNOWN_STRATEGIES + } + ) + ); + } + } + + public static String loggerClassFor(QueueableJob job) { + return settingFor(job.className, QueueableJobSetting__mdt.LoggerClass__c); + } + + public static String settingFor(String jobClassName, SObjectField field) { + Map settings = QueueableChain.jobSettingByName; + String jobSpecific = (String) settings.get(jobClassName)?.get(field); + return String.isNotBlank(jobSpecific) + ? jobSpecific + : (String) settings.get(QUEUEABLE_JOB_SETTING_ALL)?.get(field); + } + + public static Boolean declaresRetryStateHandling(QueueableJob job) { + return job.restoreStateOnRetry || job instanceof Async.Retryable; + } + + public static Integer cappedRetries(QueueableJob job, Integer requested) { + if (requested <= MAX_RETRY_CAP) { + return requested; + } + job.recordConfigurationWarning( + String.format( + WARNING_MAX_RETRIES_EXCEEDS_CAP, + new List{ + String.valueOf(requested), + job.className, + String.valueOf(MAX_RETRY_CAP) + } + ) + ); + return MAX_RETRY_CAP; + } + public static void assertStateHandlingDeclared(QueueableJob job) { - if (job.maxRetries > 0 && !job.restoreStateOnRetry && !(job instanceof Async.Retryable)) { + if (job.maxRetries > 0 && !declaresRetryStateHandling(job)) { throw new IllegalArgumentException( String.format(ERROR_MESSAGE_RETRY_WITHOUT_RESET, new List{ job.className }) ); diff --git a/force-app/main/default/layouts/AsyncResult__c-Async Result Layout.layout-meta.xml b/force-app/main/default/layouts/AsyncResult__c-Async Result Layout.layout-meta.xml index b548ab6..c9d69d2 100644 --- a/force-app/main/default/layouts/AsyncResult__c-Async Result Layout.layout-meta.xml +++ b/force-app/main/default/layouts/AsyncResult__c-Async Result Layout.layout-meta.xml @@ -11,26 +11,124 @@ Name - Edit - SalesforceJobId__c + Readonly + ClassName__c - Edit - CustomJobId__c + Readonly + Status__c - Edit + Readonly Result__c - Edit + Readonly OwnerId + + Readonly + SalesforceJobId__c + + + Readonly + CustomJobId__c + + + Readonly + ChainId__c + + + + + + false + false + true + + + + Readonly + DependsOnResult__c + + + Readonly + RequiredOutcome__c + + + + + Readonly + ActualOutcome__c + + + Readonly + SkipReason__c + + + false + false + true + + + + Readonly + RetryAttempts__c + + + Readonly + RetryHistory__c + + + + + + false + false + true + + + + Readonly + ExceptionType__c + + + Readonly + ExceptionMessage__c + + + + + + false + false + true + + + + Readonly + RequeueStatus__c + + + Readonly + RequeuedFrom__c + + + Readonly + PayloadSize__c + + + Readonly + JobPayload__c + + + + false false @@ -59,6 +157,22 @@ + + NAME + ClassName__c + Status__c + SkipReason__c + CREATED_DATE + AsyncResult__c.DependsOnResult__c + + + NAME + ClassName__c + Status__c + RequeueStatus__c + CREATED_DATE + AsyncResult__c.RequeuedFrom__c + false false false diff --git a/force-app/main/default/layouts/QueueableJobSetting__mdt-Queueable Job Setting Layout.layout-meta.xml b/force-app/main/default/layouts/QueueableJobSetting__mdt-Queueable Job Setting Layout.layout-meta.xml index 4287a1e..6c4e547 100644 --- a/force-app/main/default/layouts/QueueableJobSetting__mdt-Queueable Job Setting Layout.layout-meta.xml +++ b/force-app/main/default/layouts/QueueableJobSetting__mdt-Queueable Job Setting Layout.layout-meta.xml @@ -39,6 +39,63 @@ + + false + false + true + + + + Edit + MaxRetries__c + + + Edit + RetryableExceptions__c + + + + + Edit + BackoffStrategy__c + + + Edit + BackoffBaseMinutes__c + + + + + + false + false + true + + + + Edit + LoggerClass__c + + + + + + false + false + true + + + + Edit + StoreJobPayload__c + + + Edit + JobSerializerClass__c + + + + false false diff --git a/force-app/main/default/objects/AsyncResult__c/fields/JobPayload__c.field-meta.xml b/force-app/main/default/objects/AsyncResult__c/fields/JobPayload__c.field-meta.xml new file mode 100644 index 0000000..8311631 --- /dev/null +++ b/force-app/main/default/objects/AsyncResult__c/fields/JobPayload__c.field-meta.xml @@ -0,0 +1,12 @@ + + + JobPayload__c + Serialized snapshot of the job taken when it was enqueued, before it ran. Async.requeue() rebuilds the job from this. Written only when QueueableJobSetting__mdt.StoreJobPayload__c is Yes, because it holds whatever data the job carried. + false + + 131072 + false + false + LongTextArea + 10 + diff --git a/force-app/main/default/objects/AsyncResult__c/fields/PayloadSize__c.field-meta.xml b/force-app/main/default/objects/AsyncResult__c/fields/PayloadSize__c.field-meta.xml new file mode 100644 index 0000000..cd888af --- /dev/null +++ b/force-app/main/default/objects/AsyncResult__c/fields/PayloadSize__c.field-meta.xml @@ -0,0 +1,13 @@ + + + PayloadSize__c + Characters in the serialized job, recorded even when the payload was too large to store. Async.requeue() adds these up to stay inside its character budget, and reporting on it shows which job classes are bloating. + false + + 9 + false + 0 + false + Number + false + diff --git a/force-app/main/default/objects/AsyncResult__c/fields/RequeueStatus__c.field-meta.xml b/force-app/main/default/objects/AsyncResult__c/fields/RequeueStatus__c.field-meta.xml new file mode 100644 index 0000000..7be8a14 --- /dev/null +++ b/force-app/main/default/objects/AsyncResult__c/fields/RequeueStatus__c.field-meta.xml @@ -0,0 +1,35 @@ + + + RequeueStatus__c + Whether this job can be requeued, and if not, why. Blank means payload storage was off for this job. Job Payload is a Long Text Area and cannot be filtered on, so this is what Async.requeue() selects by. + + false + false + Picklist + + true + + false + + Stored + false + + + + TooLarge + false + + + + NotSerializable + false + + + + Requeued + false + + + + + diff --git a/force-app/main/default/objects/AsyncResult__c/fields/RequeuedFrom__c.field-meta.xml b/force-app/main/default/objects/AsyncResult__c/fields/RequeuedFrom__c.field-meta.xml new file mode 100644 index 0000000..0dba4ac --- /dev/null +++ b/force-app/main/default/objects/AsyncResult__c/fields/RequeuedFrom__c.field-meta.xml @@ -0,0 +1,13 @@ + + + RequeuedFrom__c + The result this job was requeued from. A requeued job runs in a new chain, so this is the only link back to the failure it replays. + SetNull + + AsyncResult__c + Requeue Attempts + RequeueAttempts + false + false + Lookup + diff --git a/force-app/main/default/objects/QueueableJobSetting__mdt/fields/JobSerializerClass__c.field-meta.xml b/force-app/main/default/objects/QueueableJobSetting__mdt/fields/JobSerializerClass__c.field-meta.xml new file mode 100644 index 0000000..bf1e51a --- /dev/null +++ b/force-app/main/default/objects/QueueableJobSetting__mdt/fields/JobSerializerClass__c.field-meta.xml @@ -0,0 +1,11 @@ + + + JobSerializerClass__c + Apex class that converts a job to and from its stored payload. Required on a packaged install, because JSON cannot cross a namespace boundary in either direction, so both halves have to run in your own code. The class must be declared global and implement Async.JobSerializer. Blank means Async Lib serializes directly, which is what a source deployment wants. + DeveloperControlled + + false + Text + 255 + false + diff --git a/force-app/main/default/objects/QueueableJobSetting__mdt/fields/LoggerClass__c.field-meta.xml b/force-app/main/default/objects/QueueableJobSetting__mdt/fields/LoggerClass__c.field-meta.xml new file mode 100644 index 0000000..1500db3 --- /dev/null +++ b/force-app/main/default/objects/QueueableJobSetting__mdt/fields/LoggerClass__c.field-meta.xml @@ -0,0 +1,11 @@ + + + LoggerClass__c + Apex class that receives job lifecycle events for the matching job. Set it on the All record to route every job to one logger. The class must be declared global and implement at least one of Async.OnJobEnqueued, Async.OnJobSucceeded, Async.OnJobFailed, Async.OnRetryEnqueued. Blank means no logging. + DeveloperControlled + + false + Text + 255 + false + diff --git a/force-app/main/default/objects/QueueableJobSetting__mdt/fields/StoreJobPayload__c.field-meta.xml b/force-app/main/default/objects/QueueableJobSetting__mdt/fields/StoreJobPayload__c.field-meta.xml new file mode 100644 index 0000000..e6a6eba --- /dev/null +++ b/force-app/main/default/objects/QueueableJobSetting__mdt/fields/StoreJobPayload__c.field-meta.xml @@ -0,0 +1,25 @@ + + + StoreJobPayload__c + Store a snapshot of the matching job on AsyncResult__c so Async.requeue() can replay it. A payload holds whatever data the job carried, so this is off unless you turn it on. Blank falls back to the All record; No on a job record overrides Yes on All, which is how you exclude a job that carries sensitive data. Turning this on also writes a result row for failed and skipped jobs even when Create Result is off, because a payload with no row is unreachable. + DeveloperControlled + + false + Picklist + + true + + false + + Yes + false + + + + No + false + + + + + diff --git a/force-app/main/default/permissionsets/AsyncResultAccess.permissionset-meta.xml b/force-app/main/default/permissionsets/AsyncResultAccess.permissionset-meta.xml index 6a6774e..5fc5d6e 100644 --- a/force-app/main/default/permissionsets/AsyncResultAccess.permissionset-meta.xml +++ b/force-app/main/default/permissionsets/AsyncResultAccess.permissionset-meta.xml @@ -1,7 +1,7 @@ - Read access to AsyncResult__c records and all their fields, so admins and reports can see async job outcomes. The framework writes these records in system context and does not require this set. + Read access to AsyncResult__c and all its fields, so admins and reports can see async job outcomes. The framework writes these records in system context and does not need this set. Job Payload is in this set and holds whatever data the job carried. false AsyncResult__c @@ -47,6 +47,26 @@ true false + + AsyncResult__c.JobPayload__c + true + false + + + AsyncResult__c.PayloadSize__c + true + false + + + AsyncResult__c.RequeueStatus__c + true + false + + + AsyncResult__c.RequeuedFrom__c + true + false + AsyncResult__c.RequiredOutcome__c true diff --git a/package-tests/consumer-app/force-app/main/default/classes/NsAsyncLogger.cls b/package-tests/consumer-app/force-app/main/default/classes/NsAsyncLogger.cls new file mode 100644 index 0000000..8a34c61 --- /dev/null +++ b/package-tests/consumer-app/force-app/main/default/classes/NsAsyncLogger.cls @@ -0,0 +1,21 @@ +global class NsAsyncLogger implements btcdev.Async.OnJobEnqueued, btcdev.Async.OnJobSucceeded, btcdev.Async.OnJobFailed, btcdev.Async.OnRetryEnqueued { + public void onJobEnqueued(btcdev.Async.JobContext ctx) { + record('ENQUEUED', ctx.className, ctx.info.get('team')); + } + + public void onJobSucceeded(btcdev.Async.JobContext ctx) { + record('SUCCEEDED', ctx.className, ctx.info.get('team')); + } + + public void onJobFailed(btcdev.Async.FailureContext ctx) { + record('FAILED', ctx.className, String.valueOf(ctx.retryAttempt)); + } + + public void onRetryEnqueued(btcdev.Async.FailureContext ctx) { + record('RETRY', ctx.className, String.valueOf(ctx.retryAttempt)); + } + + private void record(String event, String className, String detail) { + insert new Account(Name = 'NS-LOG-' + event, Description = className + '|' + detail); + } +} diff --git a/package-tests/consumer-app/force-app/main/default/classes/NsAsyncLogger.cls-meta.xml b/package-tests/consumer-app/force-app/main/default/classes/NsAsyncLogger.cls-meta.xml new file mode 100644 index 0000000..cad713d --- /dev/null +++ b/package-tests/consumer-app/force-app/main/default/classes/NsAsyncLogger.cls-meta.xml @@ -0,0 +1,5 @@ + + + 66.0 + Active + diff --git a/package-tests/consumer-app/force-app/main/default/classes/NsJobSerializer.cls b/package-tests/consumer-app/force-app/main/default/classes/NsJobSerializer.cls new file mode 100644 index 0000000..5f5d14d --- /dev/null +++ b/package-tests/consumer-app/force-app/main/default/classes/NsJobSerializer.cls @@ -0,0 +1,11 @@ +global class NsJobSerializer implements btcdev.Async.JobSerializer { + public String serialize(btcdev.QueueableJob job) { + insert new Account(Name = 'NS-SERIALIZE', Description = job.className); + return JSON.serialize(job); + } + + public btcdev.QueueableJob deserialize(String className, String payload) { + insert new Account(Name = 'NS-DESERIALIZE', Description = className); + return (btcdev.QueueableJob) JSON.deserialize(payload, Type.forName(className)); + } +} diff --git a/package-tests/consumer-app/force-app/main/default/classes/NsJobSerializer.cls-meta.xml b/package-tests/consumer-app/force-app/main/default/classes/NsJobSerializer.cls-meta.xml new file mode 100644 index 0000000..cad713d --- /dev/null +++ b/package-tests/consumer-app/force-app/main/default/classes/NsJobSerializer.cls-meta.xml @@ -0,0 +1,5 @@ + + + 66.0 + Active + diff --git a/package-tests/consumer-app/force-app/main/default/classes/NsTestLogging.cls b/package-tests/consumer-app/force-app/main/default/classes/NsTestLogging.cls new file mode 100644 index 0000000..d82484a --- /dev/null +++ b/package-tests/consumer-app/force-app/main/default/classes/NsTestLogging.cls @@ -0,0 +1,99 @@ +@IsTest +public class NsTestLogging { + public class NsLoggingException extends Exception { + } + + public class LoggedJob extends btcdev.QueueableJob { + public override void work() { + insert new Account(Name = 'NS-LOGGED-WORK'); + } + } + + public class LoggedFailingJob extends btcdev.QueueableJob implements btcdev.Async.Retryable { + public override void work() { + throw new NsLoggingException('ns logging failure'); + } + + public void resetBeforeRetry(Integer attempt) { + } + } + + public class SelfListeningJob extends btcdev.QueueableJob implements btcdev.Async.OnJobSucceeded { + public override void work() { + } + + public void onJobSucceeded(btcdev.Async.JobContext ctx) { + insert new Account(Name = 'NS-SELF-LISTENER', Description = ctx.info.get('team')); + } + } + + private static Integer countOf(String name) { + return [SELECT COUNT() FROM Account WHERE Name = :name]; + } + + private static void registerLogger() { + btcdev.AsyncMock.jobSettings( + new List{ + new btcdev__QueueableJobSetting__mdt( + btcdev__QueueableJobName__c = 'All', + btcdev__LoggerClass__c = 'NsAsyncLogger' + ) + } + ); + } + + @IsTest + static void shouldReachAGlobalSubscriberLoggerFromInsideThePackage() { + registerLogger(); + + Test.startTest(); + btcdev.Async.queueable(new LoggedJob()).info('team', 'platform').enqueue(); + Test.stopTest(); + + System.assertEquals( + 1, + countOf('NS-LOG-ENQUEUED'), + 'Type.forName must reach a global subscriber class from package code' + ); + System.assertEquals(1, countOf('NS-LOG-SUCCEEDED'), 'Success must reach the logger'); + + String detail = [SELECT Description FROM Account WHERE Name = 'NS-LOG-ENQUEUED' LIMIT 1] + .Description; + System.assert( + detail.endsWith('|platform'), + 'info() must survive into the context across the namespace, but was: ' + detail + ); + } + + @IsTest + static void shouldReportFailureAndRetryEventsToTheSubscriberLogger() { + registerLogger(); + + Test.startTest(); + btcdev.Async.queueable(new LoggedFailingJob()) + .continueOnJobExecuteFail() + .retry(1) + .enqueue(); + Test.stopTest(); + + System.assertEquals(1, countOf('NS-LOG-RETRY'), 'The queued retry must reach the logger'); + System.assertEquals( + 1, + countOf('NS-LOG-FAILED'), + 'The terminal failure must reach the logger exactly once' + ); + } + + @IsTest + static void shouldFireAJobsOwnListenerWithoutAnyRegistration() { + Test.startTest(); + btcdev.Async.queueable(new SelfListeningJob()).info('team', 'billing').enqueue(); + Test.stopTest(); + + System.assertEquals( + 1, + countOf('NS-SELF-LISTENER'), + 'A job implementing the interface needs no Custom Metadata at all' + ); + } +} diff --git a/package-tests/consumer-app/force-app/main/default/classes/NsTestLogging.cls-meta.xml b/package-tests/consumer-app/force-app/main/default/classes/NsTestLogging.cls-meta.xml new file mode 100644 index 0000000..cad713d --- /dev/null +++ b/package-tests/consumer-app/force-app/main/default/classes/NsTestLogging.cls-meta.xml @@ -0,0 +1,5 @@ + + + 66.0 + Active + diff --git a/package-tests/consumer-app/force-app/main/default/classes/NsTestRequeue.cls b/package-tests/consumer-app/force-app/main/default/classes/NsTestRequeue.cls new file mode 100644 index 0000000..de19e40 --- /dev/null +++ b/package-tests/consumer-app/force-app/main/default/classes/NsTestRequeue.cls @@ -0,0 +1,134 @@ +@IsTest +public class NsTestRequeue { + public class NsRequeueException extends Exception { + } + + public class ReplayableJob extends btcdev.QueueableJob { + public String tag; + + public ReplayableJob() { + } + + public ReplayableJob(String tag) { + this.tag = tag; + } + + public override void work() { + insert new Account(Name = 'NS-REPLAY-RAN', Description = tag); + throw new NsRequeueException('ns requeue failure'); + } + } + + private static void storePayloadsWithoutResults() { + btcdev.AsyncMock.jobSettings( + new List{ + new btcdev__QueueableJobSetting__mdt( + btcdev__QueueableJobName__c = 'All', + btcdev__StoreJobPayload__c = 'Yes', + btcdev__JobSerializerClass__c = 'NsJobSerializer' + ) + } + ); + } + + private static btcdev__AsyncResult__c failedResult() { + return [ + SELECT + Id, + btcdev__Status__c, + btcdev__RequeueStatus__c, + btcdev__PayloadSize__c, + btcdev__JobPayload__c + FROM btcdev__AsyncResult__c + LIMIT 1 + ]; + } + + @IsTest + static void shouldStoreAPayloadWithoutTurningOnRoutineResultRows() { + storePayloadsWithoutResults(); + + Test.startTest(); + btcdev.Async.queueable(new ReplayableJob('NS-FIRST-RUN')) + .continueOnJobExecuteFail() + .enqueue(); + Test.stopTest(); + + btcdev__AsyncResult__c stored = failedResult(); + System.assertEquals( + 'FAILED', + stored.btcdev__Status__c, + 'A failure row has to be written even with Create Result off, or the payload is unreachable' + ); + System.assertEquals('Stored', stored.btcdev__RequeueStatus__c); + System.assert( + stored.btcdev__JobPayload__c.contains('NS-FIRST-RUN'), + 'The payload must carry the state the job was enqueued with, but was: ' + + stored.btcdev__JobPayload__c + ); + System.assertEquals( + stored.btcdev__JobPayload__c.length(), + stored.btcdev__PayloadSize__c, + 'The size has to be filterable, because the payload field is not' + ); + System.assertEquals( + 1, + [SELECT COUNT() FROM Account WHERE Name = 'NS-SERIALIZE'], + 'Capture has to run through the registered serializer, in the subscriber namespace' + ); + } + + @IsTest + static void shouldRebuildAStoredJobThroughTheRegisteredSerializer() { + storePayloadsWithoutResults(); + + Test.startTest(); + btcdev.Async.queueable(new ReplayableJob('NS-FIRST-RUN')) + .continueOnJobExecuteFail() + .enqueue(); + Test.stopTest(); + + btcdev__AsyncResult__c stored = failedResult(); + btcdev.Async.RequeueSummary summary = btcdev.Async.requeue(stored.Id); + + System.assertEquals( + new List{ stored.Id }, + summary.requeued, + 'The stored result has to be replayable from consumer code' + ); + System.assert( + summary.skipReasonByResultId.isEmpty(), + 'Nothing should have been skipped, but was: ' + summary.skipReasonByResultId + ); + System.assertEquals( + 1, + [SELECT COUNT() FROM Account WHERE Name = 'NS-DESERIALIZE'], + 'Rebuild has to run through the registered serializer, in the subscriber namespace' + ); + System.assertEquals( + 'Requeued', + [ + SELECT btcdev__RequeueStatus__c + FROM btcdev__AsyncResult__c + WHERE Id = :stored.Id + ] + .btcdev__RequeueStatus__c, + 'Marking the source is what stops a scheduled replay picking it up twice' + ); + } + + @IsTest + static void shouldExplainAResultThatHasNothingToReplay() { + btcdev__AsyncResult__c noPayload = new btcdev__AsyncResult__c(btcdev__Status__c = 'FAILED'); + insert noPayload; + + btcdev.Async.RequeueSummary summary = btcdev.Async.requeue(noPayload.Id); + + System.assert(summary.requeued.isEmpty(), 'There is nothing to replay'); + System.assert( + summary.skipReasonByResultId.get(noPayload.Id).contains('StoreJobPayload__c'), + 'The reason has to name the setting to turn on, but was: ' + + summary.skipReasonByResultId.get(noPayload.Id) + ); + } +} diff --git a/package-tests/consumer-app/force-app/main/default/classes/NsTestRequeue.cls-meta.xml b/package-tests/consumer-app/force-app/main/default/classes/NsTestRequeue.cls-meta.xml new file mode 100644 index 0000000..cad713d --- /dev/null +++ b/package-tests/consumer-app/force-app/main/default/classes/NsTestRequeue.cls-meta.xml @@ -0,0 +1,5 @@ + + + 66.0 + Active + diff --git a/pmd/ruleset.xml b/pmd/ruleset.xml index be9c320..a1ea9d8 100644 --- a/pmd/ruleset.xml +++ b/pmd/ruleset.xml @@ -8,7 +8,7 @@ diff --git a/release-notes/v3.0.0.md b/release-notes/v3.0.0.md index e1f93e1..302893c 100644 --- a/release-notes/v3.0.0.md +++ b/release-notes/v3.0.0.md @@ -84,8 +84,11 @@ At `.enqueue()` or `.chain()`, synchronously, in your own transaction. It fails tests the first time you run them on a sandbox, not in production. The exception names both options and the type to implement, with the `btcdev.` prefix when you are on a packaged install. -Retry configured through `QueueableJobSetting__mdt` is gated too, so turning retry on in custom -metadata cannot bypass the check. +Retry configured through `QueueableJobSetting__mdt` cannot bypass the check either, but it does +not throw. Custom Metadata is editable in production with no deploy and no test run, and the `All` +record reaches every job, so retry is simply **not applied** to a job that declares no reset and +the reason is recorded. See +[Configuration Safety](https://async.beyondthecloud.dev/explanations/configuration-safety). ### `resetForRetry()` is superseded @@ -132,6 +135,153 @@ jobs. It is a separate file because Apex rejects an inner type inside an inner t | `BaseQueueableJob.Finalizer` | `btcdev.QueueableJob.Finalizer` | | `BaseChunkJob` | `btcdev.ChunkJob` | +### `Async.Backoff` for retry settings inside the job class + +Retry settings often belong next to the job rather than on every enqueue call site. Inside a +`QueueableJob` subclass the inherited `backoff` field shadows the `Backoff` type, so +`Backoff.exponential(1)` does not compile there. `Async.Backoff` exposes the same three factories +without the clash: + +```apex +public class SyncJob extends QueueableJob implements Async.Retryable { + public SyncJob() { + this.maxRetries = 5; + this.backoff = Async.Backoff.exponentialWithJitter(1); + } +} +``` + +Same `Backoff` object, interchangeable with `.backoff(...)` on the builder, and it resolves the +same way on a packaged install. See +[Configuring backoff inside the job class](https://async.beyondthecloud.dev/api/queueable#configuring-backoff-inside-the-job-class). + +### Pluggable logger and job lifecycle events + +Register one class and every async job in the org reports to it. No per-job code, no +`AsyncResult__c` trigger, no shared base class to remember. + +```apex +global class AsyncJobLogger implements Async.OnJobFailed { + public void onJobFailed(Async.FailureContext ctx) { + Logger.error('Async job failed: ' + ctx.className, ctx.failure.message); + Logger.saveLog(); + } +} +``` + +Set `LoggerClass__c` to `AsyncJobLogger` on the `All` record of `QueueableJobSetting__mdt`. Per-job +records override the default, exactly like retry settings. + +Four capability interfaces, implement only what you need: + +| Interface | Fires | +| --------- | ----- | +| `Async.OnJobEnqueued` | a job is added to a chain | +| `Async.OnJobSucceeded` | a job finished without failing | +| `Async.OnJobFailed` | a job failed with no attempts left | +| `Async.OnRetryEnqueued` | an attempt failed and another is queued | + +Separate interfaces rather than one base class, so a fifth event can be added later without +breaking a single existing logger, and so your logger keeps its inheritance slot. The same +interfaces work directly on a `QueueableJob` when you want one job to react, with no Custom +Metadata involved. + +**The registered class must be declared `global`.** Async Lib resolves it by name from inside its +own namespace, and `Type.forName` only reaches a subscriber class that is `global`. Only the class, +not its methods. + +A listener that throws is caught and logged; it never affects the job. A `LoggerClass__c` that +cannot be resolved degrades to no logging and records why, rather than stopping your jobs. + +### `AsyncMock.jobSettings(...)` + +Custom Metadata cannot be inserted in Apex, so until now none of the `QueueableJobSetting__mdt` +behaviour could be tested by a consumer. Inject records for the duration of a test instead: + +```apex +AsyncMock.jobSettings(new List{ + new QueueableJobSetting__mdt( + QueueableJobName__c = 'All', + LoggerClass__c = 'MyAsyncLogger', + MaxRetries__c = 2 + ) +}); +``` + +That covers all six settings, not just the logger: retry defaults, backoff, retryable exceptions, +result creation, disabled jobs and the registered logger. `AsyncMock.reset()` clears them. + +### Requeue a failed job from its `AsyncResult__c` record + +A job failed, you fixed the cause, and now it should run again with the same input. No redeploy, +no hand-written script. + +```apex +Async.requeue(resultId); + +Async.RequeueSummary summary = Async.requeue(failedResultIds); +summary.requeued; // replayed +summary.skipReasonByResultId; // the rest, and why +``` + +Async Lib stores a snapshot of the job as it was **enqueued**, not as the failed attempt left it, +so a replay starts from the same place the original did. It is off until you turn it on, because a +payload is a copy of whatever data the job carried. Set `StoreJobPayload__c` to `Yes` on the `All` +record of `QueueableJobSetting__mdt`. It is a picklist, so a single job record can say `No` and opt +out of an org-wide `Yes`. + +**It does not need `CreateResult__c`.** That setting writes a row for every job on every run, which +is why orgs with volume keep it off, and those are the orgs that want requeue most. Payload storage +writes a row for failed and skipped jobs on its own; successful jobs still obey `CreateResult__c`, +so the extra rows are bounded by your failure rate. + +Bulk replays run as **one chain**, so requeue works from inside a Queueable too. Each replay gets +its own row in a new chain, linked back by `RequeuedFrom__c`, and the source is marked `Requeued`, +so a scheduled replay never picks the same row up twice. There is no depth cap: requeue is a person +acting after a fix, and `retry(n)` already bounds the automated case. + +Four new fields on `AsyncResult__c`: `JobPayload__c`, `PayloadSize__c`, `RequeueStatus__c` and +`RequeuedFrom__c`. Selection goes through `RequeueStatus__c`, because a Long Text Area cannot be +filtered on. + +**On a packaged install, register a serializer.** JSON cannot cross a namespace boundary in either +direction, so both the store and the rebuild have to run in your code. Copy +`extras/classes/AsyncJobSerializer.cls` and name it in `JobSerializerClass__c`. One `global` class +per org, not one per job. Source deployments need none of this. + +**Requeue replays data, not intent.** The payload was written by the class as it was and is rebuilt +by the class as it is now. A renamed field arrives as `null`; a field that kept its name but +changed its meaning replays the old data under the new meaning, silently. If the fix changed the +job's own fields, enqueue it fresh instead. + +`Async.requeue` is not permission-gated, the same as every other `Async` call. If you put it behind +a button, check a custom permission in your controller first: the user picks which stored job +runs, in system context. + +See [Requeue](https://async.beyondthecloud.dev/explanations/requeue). + +### `info(...)` for your own metadata + +Attach arbitrary key/value pairs to a job and read them back on every event, so alerts can be +routed by team, package or owner: + +```apex +Async.queueable(new ImportJob()) + .info('team', 'platform') + .enqueue(); +``` + +```apex +public void onJobFailed(Async.FailureContext ctx) { + String team = ctx.info.get('team'); +} +``` + +It travels on the job, so it survives retries, chunk pages and serialization. + +`Async.FailureContext` also gains `nextAttemptDelayMinutes`, so a logger can report "retrying in +5m" rather than just "failed". + ## Changed guidance: declaring callouts Use the platform marker on any job that calls out: @@ -158,5 +308,88 @@ so it means something a capability marker cannot express. covering both hazards, the migration, and what gets restored. - New: `docs/api-evolution.md`, the internal rules for changing a shipped `global` surface, backed by measured install failures. +- New: [Logging](https://async.beyondthecloud.dev/explanations/logging), covering registration, the + `global` requirement, per-job listeners and a Nebula Logger adapter. +- New: [Configuration Safety](https://async.beyondthecloud.dev/explanations/configuration-safety), + why Custom Metadata mistakes degrade instead of throwing. +- New: [Requeue](https://async.beyondthecloud.dev/explanations/requeue), storing payloads, the + serializer, and how a replay trail reads. +- New: [Async Lib for AI Agents](https://async.beyondthecloud.dev/ai-usage), the whole public + surface on one page with recipes and gotchas, and the same content served as + [`/llms.txt`](https://async.beyondthecloud.dev/llms.txt) and + [`/llms-full.txt`](https://async.beyondthecloud.dev/llms-full.txt) for agents that look for the + standard. +- Installation is now three pages: the + [hub](https://async.beyondthecloud.dev/introduction/installation) with a package-vs-source table, + [Installing as a Package](https://async.beyondthecloud.dev/introduction/packaged-install) with + every namespace-specific step in one checklist, and + [Deploying the Source](https://async.beyondthecloud.dev/introduction/source-deploy), including + what a redeploy does to your `All` record. The Deploy button now pins the release tag instead of + `main`. +- New in `extras/classes/`: `AsyncJobSerializer`, next to the two base classes. + +## Fixed + +### A failed attempt no longer leaks chain changes into the next one + +Jobs chained inside `work()` only ever touched the chain in memory, and the chain travels to the +next transaction inside the finalizer, so a rollback never undid them. A job whose DML was rolled +back still ran the jobs it had chained. The same leak applied to `stopChain()` and `skipJob()`: a +stop from an attempt the framework discarded on retry survived into the next attempt and skipped +the successors of a job that then succeeded. + +An attempt that did not commit now leaves the chain as it found it, matching what Salesforce does +for a plain `System.enqueueJob`. Attached finalizers are kept, matching `System.attachFinalizer`. +See [What a Failed Job Does to the Chain](https://async.beyondthecloud.dev/explanations/failures-and-the-chain). + +### `deepClone()` with `retry()` silently never retried + +`deepClone()` could not copy a job that had already run: the chain points back at the job, the +platform contexts cannot be deserialized, and `backoff`, `dependencies` and `failure` are Async Lib +types that JSON refuses to carry across a namespace. The clone happens inside the finalizer, where +nothing surfaces an exception, so a job configured with both ran once, reported `Completed`, and +never retried. + +The clone now works from a stripped copy, is taken before the failed attempt is touched, and a +clone that still fails settles the job with the reason in `RetryHistory__c`. Any fault inside the +framework's own finalizer now writes an `AsyncResult__c` row with `Status__c = FRAMEWORK_ERROR`, +even when `CreateResult__c` is off, and re-throws. A failed clone names its actual cause and the fix +for it. See [Deep Clone in Packages](https://async.beyondthecloud.dev/explanations/deep-clone-in-packages#error-messages). + +### One Custom Metadata typo could stop every job in the org + +`QueueableJobSetting__mdt` is editable in production with no deploy and no test run, and the `All` +record reaches every job. Three settings could throw from enqueue: `MaxRetries__c` above the cap, +an unknown `BackoffStrategy__c`, and retry turned on for a job that had not declared how its state +resets. One mistake halted every async job at once. + +Configuration is now treated as untrusted input. Each case degrades to the safe behaviour, records +why on the job so it reaches `RetryHistory__c`, and writes it to the debug log: retry not applied, +backoff dropped, the count clamped. Mistakes written in Apex still throw, unchanged, because they +fail in the developer's own tests. See +[Configuration Safety](https://async.beyondthecloud.dev/explanations/configuration-safety). + +### `className` is no longer recomputed on every hop + +It was `transient`, so every transaction in a chain recomputed it for every job by throwing and +catching a `TypeException`. In a 200-job chain that was 200 exceptions per hop. The value is derived +from the runtime type, which never changes for a job, so it is serialized once now. + +### Page layouts were missing almost every field + +`AsyncResult__c` showed 4 of its 15 fields. `Status__c` and `ExceptionMessage__c` were not among +them, so an admin opening a failed job record saw nothing useful. `QueueableJobSetting__mdt` showed +3 of 8, leaving retry, backoff and logger settings invisible in the UI. + +Both layouts now carry every field, grouped, and `AsyncResult__c` gains related lists for +`DependsOnResult__c` and `RequeuedFrom__c` so a dependency chain and a replay trail can be walked +from the record. + +## For contributors + +PMD runs in CI next to Prettier and ESLint, with the shared Beyond The Cloud ruleset, and fails the +build on any finding. `npm run pmd:verify` runs the same locally. The comment policy the ruleset +enforces is written down in `docs/code-style.md`, and the rules for changing a shipped `global` +surface in `docs/api-evolution.md`. diff --git a/scripts/api-surface.sh b/scripts/api-surface.sh index 7850d01..54eac7f 100644 --- a/scripts/api-surface.sh +++ b/scripts/api-surface.sh @@ -73,6 +73,22 @@ revert_internal_wiring() { "force-app/main/default/classes/queue/QueueableJob.cls" api_surface_sed 's/global QueueableJob restoreEnqueuedState(/public QueueableJob restoreEnqueuedState(/g' \ "force-app/main/default/classes/queue/QueueableJob.cls" + api_surface_sed 's/global void recordConfigurationWarning(/public void recordConfigurationWarning(/g' \ + "force-app/main/default/classes/queue/QueueableJob.cls" + api_surface_sed 's/global QueueableJob copyWithoutRuntimeState(/public QueueableJob copyWithoutRuntimeState(/g' \ + "force-app/main/default/classes/queue/QueueableJob.cls" + + # Requeue bookkeeping. AsyncRequeue and QueueableChain write these across class boundaries, but + # a subscriber reads them from AsyncResult__c instead, and a global field can never be withdrawn. + api_surface_sed 's/global String requeuePayload;/public String requeuePayload;/g' \ + "force-app/main/default/classes/queue/QueueableJob.cls" + api_surface_sed 's/global String requeueStatus;/public String requeueStatus;/g' \ + "force-app/main/default/classes/queue/QueueableJob.cls" + api_surface_sed 's/global Integer requeuePayloadSize;/public Integer requeuePayloadSize;/g' \ + "force-app/main/default/classes/queue/QueueableJob.cls" + api_surface_sed 's/global Id requeuedFromResultId;/public Id requeuedFromResultId;/g' \ + "force-app/main/default/classes/queue/QueueableJob.cls" + api_surface_sed 's/global ChunkRun getRun(/public ChunkRun getRun(/g' \ "force-app/main/default/classes/queue/ChunkJob.cls" diff --git a/scripts/release.sh b/scripts/release.sh index 2a867dd..59cb7f8 100755 --- a/scripts/release.sh +++ b/scripts/release.sh @@ -36,6 +36,14 @@ psed() { SFDX_PROJECT="$PROJECT_ROOT/sfdx-project.json" INSTALL_PAGE="$PROJECT_ROOT/website/introduction/installation.md" +# Every file that carries the "Deploy to Salesforce" button or a `git checkout vX.Y.Z` line. +# The button takes a git ref, so pinning it to the release tag keeps it in step with the +# package link instead of deploying whatever is on main. +DEPLOY_REF_PAGES=( + "$PROJECT_ROOT/README.md" + "$PROJECT_ROOT/website/introduction/installation.md" + "$PROJECT_ROOT/website/introduction/source-deploy.md" +) CURRENT_VERSION=$(jq -r '.packageDirectories[0].versionNumber' "$SFDX_PROJECT" | sed 's/\.NEXT$//') PACKAGE_NAME=$(jq -r '.packageDirectories[0].package' "$SFDX_PROJECT") @@ -205,6 +213,17 @@ echo " Old package ID: $OLD_PKG_ID" echo " New package ID: $SUBSCRIBER_PKG_VERSION_ID" echo " Badge version: v$NEW_VERSION" +for page in "${DEPLOY_REF_PAGES[@]}"; do + if ! grep -q 'ref=v[0-9]*\.[0-9]*\.[0-9]*' "$page"; then + fail "Could not find a pinned deploy ref in $page" + fi + psed "s/ref=v[0-9]*\.[0-9]*\.[0-9]*/ref=v$NEW_VERSION/g" "$page" + psed "s/git checkout v[0-9]*\.[0-9]*\.[0-9]*/git checkout v$NEW_VERSION/g" "$page" + psed "s/latest release, \`v[0-9]*\.[0-9]*\.[0-9]*\`/latest release, \`v$NEW_VERSION\`/g" "$page" +done + +ok "Pinned deploy button and checkout lines to v$NEW_VERSION in ${#DEPLOY_REF_PAGES[@]} pages" + # ───────────────────────────────────────────────── # Step 5: Generate release notes # ───────────────────────────────────────────────── diff --git a/website/.vitepress/config.mts b/website/.vitepress/config.mts index 1bbf062..4fdb4f0 100644 --- a/website/.vitepress/config.mts +++ b/website/.vitepress/config.mts @@ -70,7 +70,13 @@ export default defineConfig({ hostname: siteUrl }, vite: { - plugins: [llmstxt({ domain: siteUrl })] + plugins: [ + llmstxt({ + domain: siteUrl, + details: + 'Agents: start with /ai-usage.md, the whole public API on one page with recipes and gotchas. Every other page is depth on one topic.' + }) + ] }, transformPageData(pageData) { const canonicalUrl = `${siteUrl}/${pageData.relativePath}` @@ -123,11 +129,20 @@ export default defineConfig({ collapsed: false, items: [ { text: 'Getting Started', link: '/getting-started' }, + { text: 'For AI Agents', link: '/ai-usage' }, { text: 'Standard Apex vs Async Lib', link: '/introduction/standard-apex-vs-async-lib' }, - { text: 'Installation', link: '/introduction/installation' } + { text: 'Installation', link: '/introduction/installation' }, + { + text: 'Installing as a Package', + link: '/introduction/packaged-install' + }, + { + text: 'Deploying the Source', + link: '/introduction/source-deploy' + } ] }, { @@ -153,6 +168,12 @@ export default defineConfig({ text: 'Job State Between Runs', link: '/explanations/job-state-between-runs' }, + { + text: 'Configuration Safety', + link: '/explanations/configuration-safety' + }, + { text: 'Logging', link: '/explanations/logging' }, + { text: 'Requeue', link: '/explanations/requeue' }, { text: 'Job Cloning', link: '/explanations/job-cloning' }, { text: 'Deep Clone in Packages', diff --git a/website/ai-usage.md b/website/ai-usage.md new file mode 100644 index 0000000..5a24cc4 --- /dev/null +++ b/website/ai-usage.md @@ -0,0 +1,349 @@ +--- +outline: deep +--- + +# Async Lib for AI Agents + +The whole public surface on one page: entry points, every builder method, what you implement, +what you get back, configuration, and the mistakes agents make most. Written to be read once and +then copied from. For humans the rest of the docs go deeper; for agents there is also +[`/llms.txt`](https://async.beyondthecloud.dev/llms.txt) and +[`/llms-full.txt`](https://async.beyondthecloud.dev/llms-full.txt), which the build generates +from every page. + +Names below are for a source deploy. On a packaged install prefix every class with `btcdev.` +(`btcdev.Async`, `btcdev.QueueableJob`) and every object and field with `btcdev__`. Details in +[Installing as a Package](/introduction/packaged-install). + +## Entry points + +| Call | Returns | Use it for | +| ---- | ------- | ---------- | +| `Async.queueable(QueueableJob job)` | `QueueableBuilder` | one job, or the start of a chain | +| `Async.queueable()` | `QueueableBuilder` | an empty builder to `.chain(job)` into; `enqueue()` with nothing added is a no-op | +| `Async.chunk(ChunkJob job, ChunkSource source)` | `ChunkBuilder` | one job over many records, one page per transaction | +| `Async.batchable(Database.Batchable job)` | `BatchableBuilder` | a standard batch, with scope and delay | +| `Async.schedulable(Schedulable job)` | `SchedulableBuilder` | a standard schedulable, with cron helpers | +| `Async.after(Result r)` / `Async.after(String customJobId)` / `Async.afterPrevious()` | `Async.Dependency` | the target of `dependsOn(...)`, finished with `.succeeded()`, `.failed()` or `.finished()` | +| `Async.stopChain()` | | inside a job or finalizer: skip every remaining job in the chain | +| `Async.skipJob(String customJobId)` | | inside a job or finalizer: skip one job and its finalizers | +| `Async.requeue(Id resultId)` / `Async.requeue(Set resultIds)` | `Async.RequeueSummary` | replay failed jobs from their `AsyncResult__c` records | +| `Async.getQueueableJobContext()` | `Async.QueueableJobContext` | inside a job: the current job, `QueueableContext`, `FinalizerContext` | +| `Async.getCurrentQueueableChainState()` | `Async.QueueableChainState` | every job in the chain and what runs next | +| `Async.getQueueableChainSchedulableId()` | `Id` | the scheduled job id when the chain started through the 50-job overflow path | +| `Async.Backoff.fixed(m)` / `.exponential(m)` / `.exponentialWithJitter(m)` | `Backoff` | the same three as `Backoff.*`, safe to call inside a `QueueableJob` subclass | + +## Builders + +Every builder is fluent. `enqueue()` starts a chain, `chain()` adds to it without starting, +`enqueue()` on the last builder starts everything chained before it. + +### QueueableBuilder + +| Method | What it does | +| ------ | ------------ | +| `priority(Integer)` | lower runs first | +| `delay(Integer minutes)` | 0 to 10, the platform cap; cannot combine with `asyncOptions` | +| `asyncOptions(AsyncOptions)` | duplicate-signature control; cannot combine with `delay` | +| `continueOnJobExecuteFail()` | swallow the exception, commit partial DML, chain continues | +| `rollbackOnJobExecuteFail()` | roll back this job's DML on failure, chain continues | +| `continueOnJobEnqueueFail()` | chain continues if this job cannot be enqueued | +| `retry(Integer maxRetries)` | 0 to 10 more attempts after the first; needs `Async.Retryable` or `restoreStateOnRetry()` | +| `backoff(Backoff)` | delay between attempts, minutes, clamped to 10 | +| `retryOn(Type)` / `retryOn(List)` | only these exception types retry; ANDed with `isRetryable()` | +| `restoreStateOnRetry()` | replay every attempt from the job as it was at enqueue | +| `deepClone()` | copy collections and objects, not just references, when cloning the job | +| `dependsOn(Async.Dependency)` | skip this job unless the target had that outcome | +| `info(String key, String value)` / `info(Map)` | metadata that arrives on every lifecycle context | +| `mockId(String)` | key for `AsyncMock` in tests | +| `chain(QueueableJob next)` | add this job to the chain and hold `next` | +| `chunk(ChunkJob, ChunkSource)` | add this job, then continue as a `ChunkBuilder` | +| `asSchedulable()` | continue as a `SchedulableBuilder` | +| `chain()` | add to the chain, do not start it; returns `Async.Result` | +| `attachFinalizer()` | inside `work()`: run this job after the current one, success or failure | +| `enqueue()` | start the chain; returns `Async.Result` | + +### ChunkBuilder + +Everything from `QueueableBuilder` that makes sense for a run, plus: + +| Method | What it does | +| ------ | ------------ | +| `chunkSize(Integer)` | records per page, default 200, capped by the source | +| `delayBetweenChunks(Integer minutes)` | wait between pages | +| `stopRemainingChunksOnFailure()` | a failed page ends the run; default is to continue | +| `keepChunkPages()` | keep every page's job in the chain state instead of dropping recorded ones | +| `restoreStateOnNextChunk()` | replay every page from the job as it was at enqueue | +| `chain(QueueableJob next)` / `chunk(ChunkJob, ChunkSource)` | continue the chain after the run | +| `chain()` / `enqueue()` | as above | + +### ChunkSource + +| Factory | Reads from | +| ------- | ---------- | +| `ChunkSource.of(List)` | records already in memory | +| `ChunkSource.ofIds(Set)` | id-only records in memory; query the fields you need inside `work()` | +| `ChunkSource.query(String soql)` | a `Database.Cursor` over the query, system mode | +| `ChunkSource.query(soql, AccessLevel)` / `query(soql, Map binds)` / `query(soql, binds, AccessLevel)` | the same with user mode or bind variables | +| `ChunkSource.cursor(Database.Cursor)` | a cursor you opened yourself | + +Your own: extend `ChunkSource`, implement `getNumRecords()` and `fetch(Integer position, Integer count)`. + +### BatchableBuilder + +| Method | What it does | +| ------ | ------------ | +| `scopeSize(Integer)` | records per `execute` | +| `execute()` | run now; returns `Async.Result` | +| `asSchedulable()` | continue as a `SchedulableBuilder` | +| `minutesFromNow(Integer)` | only with `asSchedulable().name(...).schedule()`: run once, that many minutes from now, instead of on a cron | + +### SchedulableBuilder and CronBuilder + +| Method | What it does | +| ------ | ------------ | +| `name(String)` | the scheduled job name, required | +| `cronExpression(String)` / `cronExpression(CronBuilder)` / `cronExpression(List)` | when; a list schedules one job per expression | +| `skipWhenAlreadyScheduled()` | no-op if a job with that name exists | +| `schedule()` | returns `List` | + +`CronBuilder` helpers: `everyHour(minute)`, `everyXHours(x, minute)`, `everyDay(hour, minute)`, +`everyXDays(x, hour, minute)`, `everyMonth(day, hour, minute)`, `everyXMonths(x, day, hour, minute)`, +`buildForEveryXMinutes(x)` (returns a list), and raw `second()`, `minute()`, `hour()`, +`dayOfMonth()`, `month()`, `dayOfWeek()`, `optionalYear()`. `getCronExpression()` gives the string. + +## What you implement + +```apex +public class ImportJob extends QueueableJob { + private List recordIds; + + public ImportJob(List recordIds) { + this.recordIds = recordIds; + } + + public override void work() { /* the job */ } +} +``` + +| Member | On | When to override | +| ------ | -- | ---------------- | +| `void work()` | `QueueableJob` | always; the job body | +| `void work(List page)` | `ChunkJob` | always; one page of the run | +| `Boolean isRetryable(Exception ex)` | `QueueableJob` | veto a retry for a specific exception; default `true` | +| `void onFinalFailure(Async.FailureContext ctx)` | `QueueableJob` | once, after the last attempt fails | +| `QueueableJob cloneForDeepCopy()` | `QueueableJob` | packaged installs only; see `extras/BaseQueueableJob` | +| `void resetBeforeRetry(Integer attempt)` | `implements Async.Retryable` | clear state before a retry; required by `retry(n)` unless `restoreStateOnRetry()` | +| `void resetBeforeNextChunk(Integer pageNumber)` | `implements Async.ChunkResettable` | clear state before the next page; required by every `ChunkJob` unless `restoreStateOnNextChunk()` | +| `onJobEnqueued` / `onJobSucceeded` / `onJobFailed` / `onRetryEnqueued` | `implements Async.OnJobEnqueued` etc. | lifecycle events, on the job or on a class registered in `LoggerClass__c` | +| `serialize(QueueableJob)` / `deserialize(String className, String payload)` | `implements Async.JobSerializer` | packaged installs using `requeue()`; see `extras/AsyncJobSerializer` | + +Base classes: `QueueableJob.Finalizer` for a job attached with `attachFinalizer()`. Callouts are a +marker, `implements Database.AllowsCallouts`, on any of them. + +Inside `work()` of a `ChunkJob`, `getRun()` gives `currentPageNumber()`, `hasRemainingPages()`, +`totalSize`, `chunkSize` and `remainingWorkSummary()`. + +## What you get back + +| Type | Fields | +| ---- | ------ | +| `Async.Result` | `salesforceJobId`, `customJobId`, `asyncType`, `job`, `queueableChainState` | +| `Async.QueueableChainState` | `jobs`, `nextSalesforceJobId`, `nextCustomJobId`, `enqueueType` | +| `Async.QueueableJobContext` | `currentJob`, `queueableCtx`, `finalizerCtx` | +| `Async.JobContext` | `customJobId`, `className`, `salesforceJobId`, `chainId`, `priority`, `retryAttempt`, `info` | +| `Async.FailureContext` | `retryOutcome`, `failure` (`type`, `message`, `stackTrace`), `customJobId`, `className`, `retryAttempt`, `maxRetries`, `retryHistory`, `nextAttemptDelayMinutes`, `info` | +| `Async.RequeueSummary` | `requeued`, `skipReasonByResultId`, `enqueueResult` | +| `Async.Outcome` | `SUCCESS`, `FAILURE`, `COMPLETED` | +| `Async.RetryOutcome` | `NOT_CONFIGURED`, `NOT_RETRYABLE`, `EXHAUSTED` | +| `Async.AsyncType` | `QUEUEABLE`, `BATCHABLE`, `SCHEDULABLE` | + +## Configuration: `QueueableJobSetting__mdt` + +One record named `All` applies to every job; a record whose `QueueableJobName__c` is a class name +applies to that job. Wrong values degrade and record why, they never stop a job. + +| Field | Type | Effect | +| ----- | ---- | ------ | +| `IsDisabled__c` | Checkbox | the job is skipped with `SKIPPED_DISABLED` | +| `CreateResult__c` | Checkbox | write an `AsyncResult__c` row for every outcome | +| `MaxRetries__c` | Number | default retries, 0 to 10; only applied to jobs that declare how state resets | +| `BackoffStrategy__c` | Text | `FIXED`, `EXPONENTIAL`, `EXPONENTIAL_JITTER` | +| `BackoffBaseMinutes__c` | Number | base for the strategy | +| `RetryableExceptions__c` | Text | comma-separated exception type names | +| `LoggerClass__c` | Text | a `global` class implementing the lifecycle interfaces | +| `StoreJobPayload__c` | Picklist `Yes`/`No` | store a snapshot for `requeue()`; `No` on a job beats `Yes` on `All` | +| `JobSerializerClass__c` | Text | a `global` `Async.JobSerializer`, packaged installs only | + +In tests, inject them: `AsyncMock.jobSettings(new List{ ... })`. + +## `AsyncResult__c` + +One row per job, written after its last attempt, when `CreateResult__c` is on or a payload is +stored. Read access through the `AsyncResultAccess` permission set. + +| Field | Holds | +| ----- | ----- | +| `Status__c` | `COMPLETED`, `FAILED`, `SKIPPED_DEPENDENCY`, `SKIPPED_CHAIN_STOPPED`, `SKIPPED_CHUNK_STOPPED`, `SKIPPED_EXPLICIT`, `SKIPPED_DISABLED`, `FRAMEWORK_ERROR` | +| `ClassName__c`, `CustomJobId__c`, `SalesforceJobId__c`, `ChainId__c` | identity | +| `Result__c`, `ExceptionType__c`, `ExceptionMessage__c` | outcome | +| `RetryAttempts__c`, `RetryHistory__c` | one line per attempt, plus configuration warnings | +| `DependsOnResult__c`, `RequiredOutcome__c`, `ActualOutcome__c`, `SkipReason__c` | why a dependent job ran or was skipped | +| `JobPayload__c`, `PayloadSize__c`, `RequeueStatus__c`, `RequeuedFrom__c` | requeue | + +Old rows do not delete themselves. Schedule `AsyncResultCleanupBatch` with +`failedOlderThanDays(n)` and/or `othersOlderThanDays(n)`. + +## `AsyncMock` + +| Call | What it does | +| ---- | ------------ | +| `AsyncMock.whenQueueable(mockId).thenReturn(ctx \| jobId)` / `.thenThrow(ex)` | what the job sees, or fails with, when it runs | +| `AsyncMock.whenFinalizer(mockId).thenReturn(ctx \| ParentJobResult)` / `.thenThrow(ex)` | what the finalizer sees | +| `AsyncMock.whenQueueableDefault()` / `whenFinalizerDefault()` | fallback for jobs without a matching `mockId` | +| `AsyncMock.jobSettings(List)` | inject Custom Metadata | +| `AsyncMock.reset()` | clear everything | +| `new AsyncMock.MockQueueableContext().setJobId(id)` / `new AsyncMock.MockFinalizerContext().setResult(r).setException(ex)` | hand-built contexts for calling `work()` directly | + +Chain several `thenReturn` calls to script successive invocations. + +## Recipes + +### One job + +```apex +Async.queueable(new ImportJob(recordIds)).enqueue(); +``` + +### A chain where the second job runs only if the first succeeded + +```apex +Async.queueable(new ExtractJob()) + .chain(new TransformJob()) + .dependsOn(Async.afterPrevious().succeeded()) + .chain(new NotifyJob()) + .dependsOn(Async.afterPrevious().finished()) + .enqueue(); +``` + +### Retry with backoff, state cleared between attempts + +```apex +public class SyncJob extends QueueableJob implements Async.Retryable { + private List synced = new List(); + + public override void work() { /* may throw CalloutException */ } + + public void resetBeforeRetry(Integer attempt) { + synced.clear(); + } + + public override Boolean isRetryable(Exception ex) { + return !ex.getMessage().contains('401'); + } +} + +Async.queueable(new SyncJob()) + .retry(3) + .backoff(Backoff.exponential(1)) + .retryOn(CalloutException.class) + .enqueue(); +``` + +### Many records, one page per transaction + +```apex +public class RecalcJob extends ChunkJob implements Async.ChunkResettable { + public override void work(List page) { + update page; + } + + public void resetBeforeNextChunk(Integer pageNumber) { + } +} + +Async.chunk(new RecalcJob(), ChunkSource.query('SELECT Id FROM Account WHERE Recalc__c = true')) + .chunkSize(200) + .enqueue(); +``` + +### Schedule + +```apex +Async.queueable(new NightlyJob()) + .asSchedulable() + .name('Nightly') + .cronExpression(new CronBuilder().everyDay(2, 0)) + .skipWhenAlreadyScheduled() + .schedule(); +``` + +### React to a final failure, on the job or org-wide + +```apex +public class ImportJob extends QueueableJob { + public override void work() { /* ... */ } + + public override void onFinalFailure(Async.FailureContext ctx) { + insert new IntegrationError__c(Message__c = ctx.failure.message, Attempts__c = ctx.retryAttempt); + } +} + +global class AsyncJobLogger implements Async.OnJobFailed { + public void onJobFailed(Async.FailureContext ctx) { + Logger.error(ctx.className + ' failed: ' + ctx.failure.message); + } +} +// then QueueableJobSetting__mdt.LoggerClass__c = 'AsyncJobLogger' on the All record +``` + +### Test a job + +```apex +@IsTest +static void failsCleanly() { + AsyncMock.whenQueueable('import').thenThrow(new CalloutException('down')); + + Test.startTest(); + Async.queueable(new ImportJob(ids)).mockId('import').continueOnJobExecuteFail().enqueue(); + Test.stopTest(); + + Assert.areEqual(1, [SELECT COUNT() FROM IntegrationError__c]); +} +``` + +## Gotchas + +Things that read as bugs and are not, and things agents get wrong on the first try. + +- **`retry(n)` throws at enqueue unless the job says what happens to its state.** Implement + `Async.Retryable` or call `restoreStateOnRetry()`. An empty `resetBeforeRetry` body is a valid + answer. Same for every `ChunkJob` with `Async.ChunkResettable` or `restoreStateOnNextChunk()`. + [Job State Between Runs](/explanations/job-state-between-runs). +- **More than 50 jobs is fine.** The chain switches to a scheduled starter past the platform's + 50-queueable limit on its own. Do not batch enqueues by hand. +- **Jobs chained inside `work()` join the running chain.** Use `Async.queueable(...).chain()` or + `.enqueue()` from inside a job, never `System.enqueueJob`, or you spend the transaction's single + enqueue slot on a job the chain does not know about. +- **A failed job does not stop the chain.** It stops its own work. Chain control is + `dependsOn(...)`, `Async.stopChain()` or `Async.skipJob(...)`, and the safe place to call the + last two is a finalizer. [Failures and the Chain](/explanations/failures-and-the-chain). +- **`Invalid conversion from runtime type ... to Datetime` in the debug log is expected.** The + framework throws and catches it once per job to read the class name. + [Expected Exceptions](/explanations/expected-exceptions-in-debug-logs). +- **`deepClone()` and `restoreStateOn*()` need a base class on a packaged install.** Copy + `extras/BaseQueueableJob` and `BaseChunkJob`. Source deploys need nothing. + [Deep Clone in Packages](/explanations/deep-clone-in-packages). +- **Inside a `QueueableJob` subclass write `Async.Backoff.exponential(1)`, not `Backoff.exponential(1)`.** + The inherited `backoff` field shadows the type there. +- **Result rows are opt-in.** Nothing is written unless `CreateResult__c` is on, or a payload is + stored and the job failed or was skipped. Do not query `AsyncResult__c` and expect a row. +- **`requeue()` replays data, not intent.** The payload is the job as it was enqueued, rebuilt by + the class as it is now. If the fix renamed a field or changed what one means, enqueue fresh. + [Requeue](/explanations/requeue). +- **Anything registered by name in Custom Metadata is `global` on a packaged install.** + `LoggerClass__c`, `JobSerializerClass__c`. Class only, methods stay `public`. +- **A `ChunkJob` is not a batch.** One page per transaction, in sequence, inside the chain, with + retry and dependencies. `Async.batchable(...)` is a plain `Database.Batchable` with a fluent + wrapper. [Chunk](/api/chunk). +- **`delay()` and `asyncOptions()` are exclusive**, and `delay` tops out at 10 minutes. diff --git a/website/api/async-mock.md b/website/api/async-mock.md index 9dbf859..da18303 100644 --- a/website/api/async-mock.md +++ b/website/api/async-mock.md @@ -318,11 +318,66 @@ Async.chunk(new AccountRecalcJob(), ChunkSource.of(records)) ::: +### Configuration + +#### jobSettings + +Injects `QueueableJobSetting__mdt` records for the duration of a test. + +Custom Metadata cannot be inserted in Apex, so without this there is no way to +test behaviour that depends on it. This makes all of it testable: retry defaults, +backoff, retryable exceptions, result creation, disabled jobs and the registered +logger. + +Records are keyed by `QueueableJobName__c`, so use `All` for the org-wide default +and a class name to override a single job, exactly as in real configuration. + +**Signature** + +```apex +static void jobSettings(List settings); +``` + +**Example** + +```apex +@IsTest +static void shouldRouteFailuresToOurLogger() { + AsyncMock.jobSettings( + new List{ + new QueueableJobSetting__mdt( + QueueableJobName__c = 'All', + LoggerClass__c = 'MyAsyncLogger', + MaxRetries__c = 2 + ) + } + ); + + Test.startTest(); + Async.queueable(new ImportJob()).enqueue(); + Test.stopTest(); + + // assert against whatever MyAsyncLogger recorded +} +``` + +On a packaged install the type and its fields carry the namespace: + +```apex +new btcdev__QueueableJobSetting__mdt( + btcdev__QueueableJobName__c = 'All', + btcdev__LoggerClass__c = 'MyAsyncLogger' +); +``` + +[`reset()`](#reset) clears injected settings along with everything else. + ### Utility #### reset -Clears all mock setups (both specific and default mocks). +Clears all mock setups (both specific and default mocks) and any settings +injected with [`jobSettings`](#jobsettings). **Signature** diff --git a/website/api/queueable.md b/website/api/queueable.md index 263b076..119000b 100644 --- a/website/api/queueable.md +++ b/website/api/queueable.md @@ -126,6 +126,7 @@ The following are methods for using Async with Queueable jobs: - [`dependsOn(Async.Dependency dependency)`](#dependson) - [`deepClone()`](#deepclone) - [`restoreStateOnRetry()`](#restorestateonretry) +- [`info(String key, String value)`](#info) - [`chain()`](#chain) - [`chain(QueueableJob job)`](#chain-next-job) - [`asSchedulable()`](#asschedulable) @@ -154,6 +155,7 @@ The following are methods for using Async with Queueable jobs: - [`resetBeforeRetry(Integer attempt)`](#resetbeforeretry) — `Async.Retryable` - [`resetBeforeNextChunk(Integer pageNumber)`](#resetbeforenextchunk) — `Async.ChunkResettable` - [`onFinalFailure(Async.FailureContext failureCtx)`](#onfinalfailure) +- [`onJobEnqueued` / `onJobSucceeded` / `onJobFailed` / `onRetryEnqueued`](#lifecycle-events) - ~~[`resetForRetry()`](#resetforretry)~~ ### INIT @@ -341,9 +343,11 @@ Async.queueable(new MyQueueableJob()) Opts the job into automatic retry on execution failure. `maxRetries` is the number of retries **after** the first run (so `retry(3)` runs the job up to 4 times total). Retry is **off by default**, so without this call a failed job is -never retried. `maxRetries` must not exceed the framework safety limit of `10`; -a higher value (whether passed to `retry(...)` or configured via -`QueueableJobSetting__mdt`) throws an exception. +never retried. `maxRetries` must not exceed the framework safety limit of `10`. +Passing a higher value to `retry(...)` throws. Configuring one on +`QueueableJobSetting__mdt` clamps to the limit and records why, because a +Custom Metadata mistake must never stop an org's jobs from running. See +[Configuration Safety](/explanations/configuration-safety). On each failed attempt the framework re-enqueues a fresh clone of the job with an incremented attempt counter. Jobs the failed attempt chained, and any @@ -572,6 +576,29 @@ Async.queueable(new MyQueueableJob()) .deepClone(); ``` +#### info + +Attaches arbitrary key/value metadata to the job. It arrives on every lifecycle +context as `ctx.info`, and survives retries, chunk pages and serialization. + +Use it to route alerts by team, package or owner. See +[Logging](/explanations/logging). + +**Signature** + +```apex +QueueableBuilder info(String key, String value); +QueueableBuilder info(Map info); +``` + +**Example** + +```apex +Async.queueable(new ImportJob()) + .info('team', 'platform') + .enqueue(); +``` + #### restoreStateOnRetry Replays every retry from the state the job had when it was enqueued, instead of @@ -931,6 +958,48 @@ void skipJob(String customJobId); Async.skipJob(notificationsResult.customJobId); ``` +#### requeue + +Rebuilds jobs from the payload stored on their `AsyncResult__c` records and runs +them again as one chain. Needs `QueueableJobSetting__mdt.StoreJobPayload__c = Yes` +before the original run, and a registered `Async.JobSerializer` on a packaged +install. See [Requeue](/explanations/requeue). + +Every result that could not be replayed comes back with a reason. Throws when the +call asks for more than 2,000,000 characters of payload. + +::: warning Requeue replays data, not intent + +The payload was written by the class as it was and is rebuilt by the class as it +is now. A renamed field arrives `null`; a field that changed meaning replays the +old data under the new meaning, silently. If the fix changed the job's own fields, +enqueue it fresh. See +[Requeue](/explanations/requeue#what-is-stored). + +::: + +**Signature** + +```apex +RequeueSummary requeue(Id resultId); +RequeueSummary requeue(Set resultIds); +``` + +| `RequeueSummary` | Holds | +| ---------------- | ----- | +| `List requeued` | replayed | +| `Map skipReasonByResultId` | the rest, and why | +| `Async.Result enqueueResult` | the chain they run in, `null` when nothing was replayed | + +**Example** + +```apex +Async.RequeueSummary summary = Async.requeue(failedResultIds); +for (Id skipped : summary.skipReasonByResultId.keySet()) { + System.debug(skipped + ': ' + summary.skipReasonByResultId.get(skipped)); +} +``` + ### Override hooks These are `public virtual` methods you override on your own `QueueableJob` @@ -1054,6 +1123,37 @@ public class ImportChunk extends ChunkJob implements Async.ChunkResettable { } ``` +#### Lifecycle events {#lifecycle-events} + +Four capability interfaces, implement only the ones you need. They work on a job +directly, and on a class registered once in +`QueueableJobSetting__mdt.LoggerClass__c` to cover the whole org. + +| Interface | Fires | Context | +| --------- | ----- | ------- | +| `Async.OnJobEnqueued` | a job is added to a chain | `Async.JobContext` | +| `Async.OnJobSucceeded` | a job finished without failing | `Async.JobContext` | +| `Async.OnJobFailed` | a job failed with no attempts left | `Async.FailureContext` | +| `Async.OnRetryEnqueued` | an attempt failed and another is queued | `Async.FailureContext` | + +**Example** + +```apex +public class ImportJob extends QueueableJob implements Async.OnJobFailed { + public override void work() { ... } + + public void onJobFailed(Async.FailureContext ctx) { + Logger.error(ctx.className + ' failed: ' + ctx.failure.message); + } +} +``` + +A listener that throws never affects the job. Adding a fifth event later is a new +interface, so existing listeners keep compiling. + +See [Logging](/explanations/logging) for org-wide registration, the `global` +requirement and a Nebula adapter. + #### ~~resetForRetry~~ {#resetforretry} ::: danger This method is never called diff --git a/website/explanations/configuration-safety.md b/website/explanations/configuration-safety.md new file mode 100644 index 0000000..d21e9d4 --- /dev/null +++ b/website/explanations/configuration-safety.md @@ -0,0 +1,68 @@ +--- +outline: deep +--- + +# Configuration Safety + +## The rule + +> **Mistakes in Apex throw. Mistakes in Custom Metadata degrade and warn.** + +Async Lib refuses to enqueue a job that is wrong in code. It never refuses to enqueue a job +because a Custom Metadata record is wrong. + +## Why + +| | Wrong Apex | Wrong `QueueableJobSetting__mdt` | +| --- | --- | --- | +| Changed by | a developer | an admin | +| Needs a deploy | yes | **no** | +| Runs your tests first | yes | **no** | +| Reaches | the one job just written | **every job in the org**, via the `All` record | + +A typo in the `All` record would otherwise stop every async job in production, with no test run +and no deploy to catch it first. Losing a retry costs you a behaviour. Refusing to enqueue stops +the business. + +## What happens instead + +Each case degrades, records the reason on the job, and writes it to the debug log at `ERROR`. +Nothing is thrown and the job runs. + +| Configuration mistake | Result | +| --- | --- | +| `MaxRetries__c` set for a job that declares no reset | retry not applied, job runs once | +| `BackoffStrategy__c` is not a known strategy | no backoff, retries run without delay | +| `MaxRetries__c` above the framework cap | clamped to the cap | +| `LoggerClass__c` cannot be resolved | no logger, jobs run normally | +| `JobSerializerClass__c` cannot be resolved, or is not an `Async.JobSerializer` | no payload stored, the result records `NotSerializable`, job runs normally | + +Every fallback degrades toward doing **less**, never toward doing something the developer did not +ask for. Skipping retry is safe, because the job then runs exactly once, which is what its code +was written and tested against. Silently *enabling* retry on a job that never declared how its +state resets would not be. + +Warnings append to the job's retry history, so they reach `AsyncResult__c.RetryHistory__c` when +result creation is enabled. Each one names the record, the job, what was skipped, and the fix. + +## What still throws + +Anything a developer wrote, because it cannot escape their own test run: + +- `retry(n)` or `Async.chunk(...)` without a declared reset, see + [Job State Between Runs](/explanations/job-state-between-runs) +- `retry(-1)`, or a retry count above the cap passed in Apex +- `delay()` combined with `asyncOptions()` +- `dependsOn(Async.afterPrevious())` with no previous job +- `Async.requeue(...)` asked for more payload than it can hold in one call + +These throw at `.enqueue()` or `.chain()`, synchronously, in the caller's transaction. + +## Consequence worth knowing + +Setting `MaxRetries__c` on the `All` record enables retry only for jobs that have declared how +their state resets. The rest keep running as before and say why in their history. + +That is intentional. The alternative is an admin silently enabling state-carrying retries across +an entire org, which is the bug +[Job State Between Runs](/explanations/job-state-between-runs) exists to prevent. diff --git a/website/explanations/deep-clone-in-packages.md b/website/explanations/deep-clone-in-packages.md index 512a4f6..e10475d 100644 --- a/website/explanations/deep-clone-in-packages.md +++ b/website/explanations/deep-clone-in-packages.md @@ -74,6 +74,9 @@ ready to copy. Rename them to suit your project. If you deploy Async Lib **without a namespace** (Deploy button, `sf project deploy`), skip all of this. Everything already works. +Everything else a packaged install needs is on one page: +[Installing as a Package](/introduction/packaged-install). + ## Per-Job Override `cloneForDeepCopy()` on the base class is left `virtual`, so a job with unusual needs can still diff --git a/website/explanations/job-state-between-runs.md b/website/explanations/job-state-between-runs.md index d74db09..d6dddb3 100644 --- a/website/explanations/job-state-between-runs.md +++ b/website/explanations/job-state-between-runs.md @@ -173,5 +173,11 @@ At `.enqueue()` or `.chain()`, synchronously, before anything is sent to the que - a job with `retry(n)` that neither implements `Async.Retryable` nor calls `restoreStateOnRetry()` - a `ChunkJob` that neither implements `Async.ChunkResettable` nor calls `restoreStateOnNextChunk()` -Retry configured through `QueueableJobSetting__mdt` is gated too. Turning retry on for a job in -custom metadata cannot bypass the check. +Retry configured through `QueueableJobSetting__mdt` cannot bypass the check either, but it does +not throw. An admin can edit Custom Metadata in production with no deploy and no test run, and the +`All` record reaches every job in the org, so refusing to enqueue would turn one typo into an +org-wide outage. + +Instead, **retry is simply not applied** to a job that has not declared how its state resets, the +job runs once as its code was written and tested to, and the reason is recorded on the job. See +[Configuration Safety](/explanations/configuration-safety). diff --git a/website/explanations/logging.md b/website/explanations/logging.md new file mode 100644 index 0000000..8f15d70 --- /dev/null +++ b/website/explanations/logging.md @@ -0,0 +1,185 @@ +--- +outline: deep +--- + +# Logging + +## TL;DR + +Register one class, and every async job in the org reports to it. + +```apex +global class AsyncJobLogger implements Async.OnJobFailed { + public void onJobFailed(Async.FailureContext ctx) { + Logger.error('Async job failed: ' + ctx.className, ctx.failure.message); + Logger.saveLog(); + } +} +``` + +Then set `LoggerClass__c` to `AsyncJobLogger` on the `All` record of +`QueueableJobSetting__mdt`. That is the whole setup. + +::: warning The class must be `global` + +Async Lib resolves your class by name from inside its own namespace, and `Type.forName` only +reaches a subscriber class declared `global`. A `public` class resolves to null and nothing is +logged. + +Only the **class** needs `global`. The methods stay `public`. + +On a source deploy there is no boundary, so keep the class `public` like any other. Add `global` +if you later switch to the package; it is on the +[switching list](/introduction/source-deploy#switching-to-the-package-later). + +The full packaged-install checklist is at [Installing as a Package](/introduction/packaged-install). + +::: + +## Events + +Implement only the ones you want. Each is a separate interface, so a logger that only cares about +failures implements one method and nothing else. + +| Interface | Fires | Context | +| --------- | ----- | ------- | +| `Async.OnJobEnqueued` | a job is added to a chain | `JobContext` | +| `Async.OnJobSucceeded` | a job finished without failing | `JobContext` | +| `Async.OnJobFailed` | a job failed with no attempts left | `FailureContext` | +| `Async.OnRetryEnqueued` | an attempt failed and another is queued | `FailureContext` | + +A job that fails with `retry(2)` and never succeeds produces `OnRetryEnqueued`, `OnRetryEnqueued`, +`OnJobFailed`. Every failed attempt fires exactly one event, so the two together tell you whether +to warn or to page. There is no overlap and no double counting. + +Chunk runs fire `OnJobEnqueued` per page, because each page really is queued separately. + +## Two layers, one vocabulary + +The same interfaces work on a **job**, with no Custom Metadata at all: + +```apex +public class ImportJob extends QueueableJob implements Async.OnJobFailed { + public override void work() { ... } + + public void onJobFailed(Async.FailureContext ctx) { + // just this job + } +} +``` + +Both fire for the same event, and the job's own listener runs first. Use the job listener for +one-off behaviour and the registered class for the org-wide sink. Neither needs the other. + +## Attaching your own metadata + +`info(...)` puts arbitrary key/value pairs on a job, and they arrive on every context. This is how +you route alerts by team, package or anything else you own: + +```apex +Async.queueable(new ImportJob()) + .info('team', 'platform') + .info('package', 'billing') + .enqueue(); +``` + +```apex +public void onJobFailed(Async.FailureContext ctx) { + String team = ctx.info.get('team'); +} +``` + +It survives serialization, retries and chunk pages, because it travels on the job. + +## Per-job override + +`LoggerClass__c` resolves job-first, then falls back to the `All` record, the same way retry +settings do: + +| Record | `LoggerClass__c` | Result | +| ------ | ---------------- | ------ | +| `All` | `AsyncJobLogger` | every job goes here | +| `ImportJob` | `ImportJobLogger` | that job goes here instead | +| `ImportJob` | blank | that job falls back to `All` | + +## A logger that throws cannot break a job + +Every listener call is wrapped. If yours throws, the failure is written to the debug log and the +job carries on untouched. By the time most events fire the job has already done its work, so +failing it over a logging problem would turn an observability problem into a data problem. + +The same applies to a `LoggerClass__c` that cannot be resolved: jobs keep running, and the reason +is recorded. See [Configuration Safety](/explanations/configuration-safety). + +## Testing your logger + +Custom Metadata cannot be inserted in Apex, so register the logger through +`AsyncMock` instead. This is the only way to assert that the framework actually routes to you: + +```apex +@IsTest +static void shouldLogFailures() { + AsyncMock.jobSettings( + new List{ + new QueueableJobSetting__mdt( + QueueableJobName__c = 'All', + LoggerClass__c = 'MyAsyncLogger' + ) + } + ); + + Test.startTest(); + Async.queueable(new FailingJob()).continueOnJobExecuteFail().enqueue(); + Test.stopTest(); + + // assert on whatever MyAsyncLogger recorded +} +``` + +The same call covers every other Custom Metadata driven behaviour: retry defaults, backoff, +retryable exceptions, result creation and disabled jobs. See +[AsyncMock.jobSettings](/api/async-mock#jobsettings). + +## Nebula Logger adapter + +```apex +global class NebulaAsyncLogger implements Async.OnJobFailed, Async.OnRetryEnqueued { + public void onJobFailed(Async.FailureContext ctx) { + Logger.error( + String.format( + 'Async job {0} failed after {1} attempt(s): {2}', + new List{ + ctx.className, + String.valueOf(ctx.retryAttempt + 1), + ctx.failure?.message + } + ) + ); + Logger.setScenario(ctx.info.get('team')); + Logger.saveLog(); + } + + public void onRetryEnqueued(Async.FailureContext ctx) { + Logger.warn( + 'Async job ' + + ctx.className + + ' attempt ' + + ctx.retryAttempt + + ' failed, retrying in ' + + ctx.nextAttemptDelayMinutes + + 'm' + ); + Logger.saveLog(); + } +} +``` + +`saveLog()` is called inside the listener on purpose. Each Queueable execution is its own +transaction, so there is no later point at which to flush. + +## Scope + +Queueable only, including chunk runs. Batchable and Schedulable do not fire these events yet, +because the framework does not own their base classes. + +`AsyncResult__c` is untouched and orthogonal. Use either, both, or neither. diff --git a/website/explanations/requeue.md b/website/explanations/requeue.md new file mode 100644 index 0000000..269b055 --- /dev/null +++ b/website/explanations/requeue.md @@ -0,0 +1,209 @@ +--- +outline: deep +--- + +# Requeue + +## TL;DR + +A job failed, you fixed whatever caused it, and now you want it to run again with the same input. +That is what requeue is for. + +```apex +Async.requeue(resultId); +``` + +```apex +Async.RequeueSummary summary = Async.requeue(failedResultIds); +summary.requeued; // the results that were replayed +summary.skipReasonByResultId; // the rest, and why each one was skipped +summary.enqueueResult; // the chain the replays run in +``` + +To make this possible, Async Lib stores a snapshot of the job on its `AsyncResult__c` record. That +is off by default, because the snapshot is a copy of whatever data the job was carrying, and you +should decide whether that belongs on a queryable object in your org. + +## Turning it on + +Set `StoreJobPayload__c` to `Yes` on the `All` record of `QueueableJobSetting__mdt`, or on a +record for a single job. + +It is a picklist rather than a checkbox on purpose. A job record can say `No`, and that beats a +`Yes` on `All`. In practice the decision usually looks like "store payloads for everything, except +the one job that carries sensitive data", and a checkbox cannot express that. + +## You do not need `CreateResult__c` + +`CreateResult__c` writes a row for every job on every run, so most orgs with real volume keep it +off. If requeue depended on it, the orgs that need it most could not use it. + +Instead, turning payload storage on writes a result row for **failed** and **skipped** jobs, no +matter what `CreateResult__c` says. Successful jobs still obey `CreateResult__c`, because there is +nothing to replay about a job that worked. The extra rows you get are bounded by your failure rate. + +Skipped jobs are included because a job skipped over an unmet dependency never actually failed, +and once you fix the blocker it is exactly the one you want back. + +## What the snapshot holds + +The job as you handed it to `enqueue()`, before it ran. Not the failed attempt. A failed attempt +has already mutated its own state, and replaying from that is the bug +[the state gate](/explanations/job-state-between-runs) exists to prevent. + +| Travels with the payload | Does not | +| ------------------------ | -------- | +| your own fields, `retry(n)`, `info(...)` | `backoff(...)`, `dependsOn(...)`, chain position, retry defaults from Custom Metadata | + +So a replay retries the way the original did, but it starts a fresh chain, takes the retry +defaults from today's Custom Metadata, and does not carry dependencies from the old chain. + +Chunk pages and finalizers are never stored. A page needs its source and a finalizer needs its +parent job, and neither of those survives on a record. They get `NotSerializable` on the row so you +can see that it was deliberate. + +## Requeue replays data, not intent + +::: warning Think before you requeue a job whose class you just changed + +The payload was written by the class as it was when the job failed. It is rebuilt by the class as +it is now. If your fix changed the shape or the meaning of a field, the replay is not the job you +tested. + +::: + +There are two ways this goes wrong, and only one of them is loud. + +**A renamed field arrives empty.** You renamed `accountIds` to `recordIds`. The payload still says +`accountIds`, so the rebuilt job starts with `recordIds` as `null`. It processes nothing, or throws +on the first dereference. A changed type, say `String` to `Integer`, fails at rebuild instead and +shows up in `skipReasonByResultId`. Either way, you notice. + +**A field that kept its name and changed its meaning does the opposite.** The job carried +`Set accountIds` meaning "process these". Your fix changed `work()` so the set now means "skip +these". The payload rebuilds cleanly, the set holds the same ids, and the replay skips exactly the +records it was supposed to process. Nothing fails. Nothing is logged. + +Requeue is the right tool when the fix was outside the job: an integration was down, a validation +rule was wrong, a permission was missing. When the fix changed what the job's own fields mean, do +not requeue. Enqueue it fresh with the input you want. + +## On a packaged install, register a serializer + +::: warning Required when Async Lib is installed as a package + +JSON cannot cross a namespace boundary in either direction. Async Lib can neither store your job +nor rebuild it from inside its own namespace, regardless of whether your class is `public` or +`global`. Both halves have to run in your code. + +::: + +The class is ready to copy from +[`extras/classes/AsyncJobSerializer.cls`](https://github.com/beyond-the-cloud-dev/async-lib/tree/main/extras/classes): + +```apex +global class AsyncJobSerializer implements btcdev.Async.JobSerializer { + public String serialize(btcdev.QueueableJob job) { + return JSON.serialize(job); + } + + public btcdev.QueueableJob deserialize(String className, String payload) { + return (btcdev.QueueableJob) JSON.deserialize(payload, Type.forName(className)); + } +} +``` + +Register it once, in `JobSerializerClass__c` on the `All` record. The class has to be `global` for +the same reason `LoggerClass__c` does: Async Lib resolves it by name from its own namespace, and +`Type.forName` reaches nothing else. Only the class, the methods stay `public`. + +If you deployed the source instead, there is no boundary. Leave `JobSerializerClass__c` blank and +Async Lib converts the job itself. The full checklist for a packaged install is at +[Installing as a Package](/introduction/packaged-install). + +## Reading the record + +| Field | Holds | +| ----- | ----- | +| `JobPayload__c` | the serialized job | +| `PayloadSize__c` | its length in characters | +| `RequeueStatus__c` | whether it can be replayed | +| `RequeuedFrom__c` | the result this one was replayed from | + +`JobPayload__c` is a Long Text Area, and Long Text cannot be filtered on. `WHERE JobPayload__c != +null` does not even compile. That is why `RequeueStatus__c` exists, and it is what you select by. + +| Status | Meaning | +| ------ | ------- | +| `Stored` | ready to replay | +| `Requeued` | already replayed | +| `TooLarge` | over 131,072 characters, nothing was stored | +| `NotSerializable` | the job could not be converted, or it is a chunk page or a finalizer. The reason is in `RetryHistory__c` | +| blank | payload storage was off for this job | + +A job that hits `TooLarge` is almost certainly carrying full SObjects. Carry record ids and +re-query them inside `work()`. 131,072 characters holds roughly six thousand ids. + +## Replaying in bulk + +```apex +Set failed = new Map([ + SELECT Id + FROM AsyncResult__c + WHERE RequeueStatus__c = 'Stored' + AND CreatedDate = LAST_N_HOURS:2 +]).keySet(); + +Async.requeue(failed); +``` + +All the replays go into **one chain** and run in sequence. That matters when requeue itself runs +from a Queueable, where only one job can be enqueued per transaction. One chain costs one slot, and +a replay that fails does not stop the ones after it. + +There is a limit of 2,000,000 characters of payload per call, checked before any payload is loaded. +Over that, `Async.requeue` throws instead of dying halfway through. If you have more than that to +replay, order by `PayloadSize__c` and batch. + +## The trail + +``` +R1 (failed) <- R2 (failed) <- R3 +``` + +Each replay gets its own `AsyncResult__c` record in a new chain, linked back to its source by +`RequeuedFrom__c`. The source is marked `Requeued`. That mark is what makes a scheduled "replay +everything that failed" job safe: it never picks the same record up twice. + +There is no cap on how long the trail can get. Requeue is something a person does after fixing +something, and `retry(n)` already covers the automated case with a bound. A cap here would only +punish whoever fixed the bug on the third try. If a replay fails again, that is worth a look +rather than another replay. + +## If you put this behind a button, gate the button + +`Async.requeue` is not permission-gated, the same as every other `Async` call. Called from Apex +that is fine: whoever can write the call could do anything else too. Once you expose it to end +users, through an `@AuraEnabled` method, a Flow action or a screen, the user gets to pick which +stored job runs, in system context, with the data it carried. Check a custom permission in your +controller before you call it. + +## Limits + +- Requeue replays one job, not its chain. The rest of the chain is not rebuilt. +- `AsyncResultCleanupBatch` deletes old records, so a replay is bounded by your retention window. +- `AsyncResultAccess` grants read on `JobPayload__c`. Review who holds that permission set before + you turn storage on. + +## Testing it + +`AsyncMock.jobSettings(...)` injects the settings, so none of this needs real Custom Metadata: + +```apex +AsyncMock.jobSettings(new List{ + new QueueableJobSetting__mdt( + QueueableJobName__c = 'All', + StoreJobPayload__c = 'Yes' + ) +}); +``` diff --git a/website/introduction/installation.md b/website/introduction/installation.md index e7a4fd8..b47bcad 100644 --- a/website/introduction/installation.md +++ b/website/introduction/installation.md @@ -4,6 +4,16 @@ outline: deep # Installation +Two ways to get Async Lib into an org. Pick one, then follow its guide. + +| | Unlocked package | Source deploy | +| --- | --- | --- | +| How | one install link | deploy button, `sf` CLI, or copy the source | +| Namespace | `btcdev.` on every class, `btcdev__` on every field | none | +| Upgrade | install the next version | redeploy from the next tag | +| Extra setup | a few `extras` classes for features that copy or store a job | none | +| Pick it when | you want a versioned, uninstallable unit and an upgrade path you do not maintain | you want to read, vendor or patch the code, or you cannot install packages | + ## Install as Unlocked Package Install the latest version of Async Lib as an unlocked package: @@ -16,24 +26,22 @@ Install the latest version of Async Lib as an unlocked package: https://login.salesforce.com/packaging/installPackage.apexp?p0=04tP6000003fb0HIAQ ``` -::: tip -When installed as a package, all classes use the `btcdev` namespace prefix (e.g., `btcdev.QueueableJob`, `btcdev.Async`). If you use [`.deepClone()`](/api/queueable#deepclone), see [Deep Clone in Packages](/explanations/deep-clone-in-packages) for a required override. -::: - -## Deploy via Button +Then follow [Installing as a Package](/introduction/packaged-install): the prefix, the `extras` +classes, what has to be `global`, and the permission set. -Deploy to your Salesforce org using the deploy button: +## Deploy the Source - + Deploy to Salesforce -## Copy and Deploy - -Or clone the repository and deploy using SFDX: +Or with the Salesforce CLI: ```bash git clone https://github.com/beyond-the-cloud-dev/async-lib.git cd async-lib -sf project deploy start -p force-app -u your-org-alias -``` \ No newline at end of file +sf project deploy start --source-dir force-app --target-org your-org +``` + +Then follow [Deploying the Source](/introduction/source-deploy): deploying a tag rather than +`main`, production test levels, vendoring, and what a redeploy does to your `All` record. diff --git a/website/introduction/packaged-install.md b/website/introduction/packaged-install.md new file mode 100644 index 0000000..056760d --- /dev/null +++ b/website/introduction/packaged-install.md @@ -0,0 +1,134 @@ +--- +outline: deep +--- + +# Installing as a Package + +## TL;DR + +Everything a **packaged** install needs that a source deploy does not, on one page. If you +[deployed the source](/introduction/source-deploy) instead, skip this: there is no namespace +boundary and everything already works. + +Most of the library needs nothing beyond the `btcdev.` prefix. A few features copy or store your +job, and those need one class from [`extras/`](https://github.com/beyond-the-cloud-dev/async-lib/tree/main/extras/classes) +in your namespace. That is the whole story, and the rest of this page is the details. + +## The one rule behind all of it + +Async Lib runs inside its own `btcdev` namespace, and two platform behaviours follow from that: + +| Rule | Consequence | +| ---- | ----------- | +| `JSON.serialize` and `JSON.deserialize` refuse any object graph that crosses a namespace, in either direction | anything that copies or stores your job has to run in **your** code | +| `Type.forName` from package code only resolves a subscriber class declared `global` | anything Async Lib reaches **by name** has to be `global` | + +Every item below is a consequence of one of those two. Neither depends on whether your job class +is `public` or `global`; `global` gets a class past `Type.forName` and buys nothing else. + +## Checklist + +### 1. Install + +The current version and install link are on [Installation](/introduction/installation). + +### 2. Use the prefix + +Every class, object and field carries it. + +| Source deploy | Packaged install | +| ------------- | ---------------- | +| `Async.queueable(...)` | `btcdev.Async.queueable(...)` | +| `extends QueueableJob` | `extends btcdev.QueueableJob` | +| `implements Async.Retryable` | `implements btcdev.Async.Retryable` | +| `AsyncResult__c` | `btcdev__AsyncResult__c` | +| `QueueableJobSetting__mdt.CreateResult__c` | `btcdev__QueueableJobSetting__mdt.btcdev__CreateResult__c` | + +The framework's error messages use the right prefix for the org they run in, so whatever a message +tells you to write can be pasted as is. + +### 3. Copy the `extras` classes you need + +The classes in +[`extras/classes/`](https://github.com/beyond-the-cloud-dev/async-lib/tree/main/extras/classes) +belong in your namespace, which is exactly why they cannot ship inside the package. Copy the ones +for the features you use and rename them however you like. + +| Copy | When you use | Then | +| ---- | ------------ | ---- | +| `BaseQueueableJob` | `deepClone()`, `restoreStateOnRetry()` | `extends BaseQueueableJob` instead of `btcdev.QueueableJob` | +| `BaseQueueableJob.Finalizer` | the same, on a finalizer | `extends BaseQueueableJob.Finalizer` | +| `BaseChunkJob` | `restoreStateOnNextChunk()` | `extends BaseChunkJob` instead of `btcdev.ChunkJob` | +| `AsyncJobSerializer` | `Async.requeue()` | register it, see step 4 | + +Callouts are a marker, not a base class, so `implements Database.AllowsCallouts` works on any of +them. Why the base classes exist is in +[Deep Clone in Packages](/explanations/deep-clone-in-packages), and the serializer in +[Requeue](/explanations/requeue). + +### 4. Declare `global` on anything you register by name + +Two fields on `QueueableJobSetting__mdt` name a class for Async Lib to construct. Both classes +must be `global`, or the name resolves to nothing. Only the class needs it, the methods stay +`public`. + +| Field | Implements | Ships in `extras`? | +| ----- | ---------- | ------------------ | +| `LoggerClass__c` | one or more of `btcdev.Async.OnJobEnqueued`, `OnJobSucceeded`, `OnJobFailed`, `OnRetryEnqueued` | no, that one is yours to write | +| `JobSerializerClass__c` | `btcdev.Async.JobSerializer` | yes, `AsyncJobSerializer` | + +A name that does not resolve degrades rather than throws: no logging, or `NotSerializable` on the +result, plus a warning in `RetryHistory__c` naming the field. It never stops a job. See +[Configuration Safety](/explanations/configuration-safety). + +### 5. Assign the permission set + +`btcdev__AsyncResultAccess` grants read on `btcdev__AsyncResult__c` and every field on it, for +admins and reports. The framework writes those records in system context and does not need it +itself. + +`JobPayload__c` is part of that set. If you turn `StoreJobPayload__c` on, whoever holds the set can +read whatever data your jobs carried, so review it first. + +## Feature by feature + +What each feature needs on a packaged install, and nothing more. + +| Feature | Needs | +| ------- | ----- | +| enqueue, chain, finalizers, `retry(n)`, `backoff`, `dependsOn` | the prefix | +| `Async.Retryable`, `Async.ChunkResettable` | the prefix | +| `deepClone()`, `restoreStateOnRetry()` | `BaseQueueableJob` | +| `restoreStateOnNextChunk()` | `BaseChunkJob` | +| `LoggerClass__c` | your logger class, declared `global` | +| `Async.requeue()` | `AsyncJobSerializer`, copied and registered | + +## Writing tests against the package + +`btcdev.AsyncMock` is part of the package and callable from your tests, so none of the settings +above need real Custom Metadata: + +```apex +btcdev.AsyncMock.jobSettings( + new List{ + new btcdev__QueueableJobSetting__mdt( + btcdev__QueueableJobName__c = 'All', + btcdev__LoggerClass__c = 'MyAsyncLogger', + btcdev__StoreJobPayload__c = 'Yes', + btcdev__JobSerializerClass__c = 'AsyncJobSerializer' + ) + } +); +``` + +The prefixed constructor also works in a source deploy, so a test written this way does not need +to change if you ever switch. + +## Error messages that point back here + +| Message | You skipped | +| ------- | ----------- | +| `deepClone() failed ... Type cannot be serialized` | step 3, `BaseQueueableJob` | +| `LoggerClass__c names "X", which could not be resolved` | step 4, `global` on the logger | +| `StoreJobPayload__c is Yes for "X", but the job could not be serialized` | steps 3 and 4, `AsyncJobSerializer` | +| `JobSerializerClass__c names "X", which is not a usable btcdev.Async.JobSerializer` | step 4, `global` or the interface | diff --git a/website/introduction/source-deploy.md b/website/introduction/source-deploy.md new file mode 100644 index 0000000..c4fb779 --- /dev/null +++ b/website/introduction/source-deploy.md @@ -0,0 +1,116 @@ +--- +outline: deep +--- + +# Deploying the Source + +## TL;DR + +The code lands in your org with no namespace, so there is no prefix, nothing to copy from `extras` +and nothing to declare `global`. What you own instead is the upgrade path, because nothing tracks +the version for you. + +If you installed the [unlocked package](/introduction/packaged-install), this page is not for you. + +## What you get + +| | | +| --- | --- | +| Classes | `Async`, `QueueableJob`, `ChunkJob`, `AsyncMock`, the builders, and their tests | +| Object | `AsyncResult__c` with every field and its page layout | +| Custom Metadata | `QueueableJobSetting__mdt`, its layout, and one record named `All` | +| Permission set | `AsyncResultAccess` | + +All of it `public`, in your default namespace: `Async.queueable(...)`, `extends QueueableJob`, +`AsyncResult__c`. + +## Three ways to deploy + +### Deploy button + + + Deploy to Salesforce + + +The button deploys the latest release, `v2.8.0`, and the link is updated with every release the +same way the package link is. To deploy an older release put its tag in the URL, and for whatever +was merged last use `ref=main`: + +``` +https://githubsfdeploy.herokuapp.com?owner=beyond-the-cloud-dev&repo=async-lib&ref=main +``` + +Tags are on the [releases page](https://github.com/beyond-the-cloud-dev/async-lib/releases). + +### Salesforce CLI + +```bash +git clone https://github.com/beyond-the-cloud-dev/async-lib.git +cd async-lib +git checkout v2.8.0 +sf project deploy start --source-dir force-app --target-org your-org +``` + +Production, and any org that requires tests, needs `--test-level RunLocalTests`. That runs +`AsyncTest`, about 300 tests, **and every test already in your org**. If an unrelated test of yours +is failing, the deploy fails with it. For a sandbox you can use +`--test-level RunSpecifiedTests --tests AsyncTest`; production still needs `RunLocalTests`. + +### Vendor the source + +Copy `force-app/main/default/` into your own repository and deploy it with the rest of your code. +From then on you own upgrades: diff the next tag against what you copied. The PMD suppressions in +the classes travel with them, so a vendored copy passes the same static analysis it passes here. + +## Upgrading + +Redeploy from the new tag, the same way you deployed the first time. Read the +[release notes](https://github.com/beyond-the-cloud-dev/async-lib/releases) first: a major version +means a breaking change, and the notes say what to change. + +::: warning A redeploy replaces the `All` record + +`force-app/main/default/customMetadata/` holds one `QueueableJobSetting__mdt` record, `All`, with +only `IsDisabled__c` and `QueueableJobName__c` set. A metadata deploy **replaces** a Custom +Metadata record rather than merging it. Anything you set on `All` in the org that is not in that +file, `CreateResult__c`, `MaxRetries__c`, `LoggerClass__c`, `StoreJobPayload__c`, is cleared. + +Leave that folder out when you upgrade: + +```bash +sf project deploy start --source-dir force-app/main/default/classes \ + --source-dir force-app/main/default/objects \ + --source-dir force-app/main/default/layouts \ + --source-dir force-app/main/default/permissionsets \ + --target-org your-org +``` + +Or note your `All` values first and put them back after. The per-job records you created yourself +are not in the repository and are untouched either way. + +::: + +## After the deploy + +1. Assign `AsyncResultAccess` to whoever should read `AsyncResult__c` in the UI or in reports. The + framework writes those records in system context and does not need it. +2. Open `QueueableJobSetting__mdt` and decide what `All` should say. Nothing is on by default: no + result rows, no retry, no logger, no payload storage. See + [Configuration Safety](/explanations/configuration-safety) for what a wrong value does. +3. Write your first job. [Getting Started](/getting-started). + +## Switching to the package later + +The code changes are mechanical, and [Installing as a Package](/introduction/packaged-install) is +the checklist. In short: + +- every class reference gains `btcdev.`, every object and field gains `btcdev__` +- any class registered in `LoggerClass__c` or `JobSerializerClass__c` becomes `global`. Keep them + `public` until then, like any other class of yours; a source deploy has no boundary for `global` + to cross +- the features that copy or store a job start needing the + [`extras`](https://github.com/beyond-the-cloud-dev/async-lib/tree/main/extras) base classes + +The data does not move. `AsyncResult__c` and `btcdev__AsyncResult__c` are different objects, so +history stays on the old one and new jobs write to the new one. Uninstall the source classes only +once nothing references them.