OrangePro · Release verification report

Example release verification report

14 pull requests merged. One risky workflow had no test.

Before release, OrangePro mapped the combined changes in this build to the workflows they affect and checked the three riskiest for test protection. Where a test was missing, it wrote one, ran it on the original build, and checked that it fails when that workflow breaks.

Application
Example storefront
Build
rc-2026.10.2 (9f3c2e1)
Stage
After merge, before release
Since
v2026.10.1: 14 pull requests, 41 files, 9 workflows affected

Illustrative example. Everything in this report is synthetic: the application, pull requests, code and results. It shows what a hands-on evaluation returns.

Coverage assessment

3 of 9 workflows

1 gap, 1 existing test, 1 blocked

Test verification

2 of 2 detected

Simulated failures, checkout retry only

Unresolved checks

4 open

Listed below

Patch adoption

Ready for review

Not applied or merged

01 What ran, and who did each step

  1. 1 Pick the buildYour teamrc-2026.10.2: 14 PRs merged since v2026.10.1
  2. 2 Map changes to workflowsOrangePro41 files, 9 workflows, top 3 prioritized
  3. 3 Check existing protectionOrangePro flags, team reviews1 gap, 1 existing test, 1 blocked
  4. 4 Write the missing testOrangePro drafts, team reviews1 test for the gap
  5. 5 Run it on the original buildYour test runnerPasses
  6. 6 Check it against simulated failuresFaults agreed with the team2 of 2 valid trials detected
  7. 7 Return the evidenceOrangeProPatch for review, receipt, logs

02 The 3 riskiest workflows in this build

The risk score sets the order of work. It is not a defect probability.

Priority 1Risk 84

Checkout payment retry

  • #812 Shorten the payment client timeout to 8 seconds
  • #819 Move order status changes into OrderStateMachine

Charges a card. Two merged changes meet here: timeouts become more common, and retries now update the order through new code.

No test found

No test covers a retry after a timeout through the new order state code.

Priority 2Risk 61

Refund to original card

  • #823 Round refund amounts in the currency helper

Moves money back to the customer. One change to the amount calculation.

Existing test

Existing test refunds.integration.spec.ts runs and passes. It was not checked against a simulated failure here.

Priority 3Risk 57

Subscription renewal

  • #808 Retry failed renewals once before emailing the customer

Charges a card on a schedule, with no customer present.

Blocked

Blocked. The test needs a payment-provider sandbox key that was not available. Result: unknown, not low risk.

03 The gap, in plain words

If a card payment times out and is retried, nothing tests that the order stays unpaid until the payment is confirmed.

PR #812 shortened the payment timeout, so retries happen more often. PR #819 changed how a retry updates the order. Each pull request passed its own tests. No test covers the two together. This is missing protection, not a bug found in the code.

04 The new test, and how it was checked

Passes on the original build

It does not flag the build as it stands.

Fails on each simulated failure

At the expected assertion, in an isolated copy.

  • F1 Mark the order paid before the retried payment is confirmed

    src/checkout/payment-retry.ts:57

    Detected
    Original
    if (result.status === 'confirmed') await orders.markPaid(order)
    Simulated
    await orders.markPaid(order)

    Failed at the expected assertion: order status was 'paid', expected 'pending_payment'.

  • F2 Skip the retry after a timeout

    src/checkout/payment-retry.ts:44

    Detected
    Original
    if (isTimeout(error)) return retry()
    Simulated
    if (isTimeout(error)) return { status: 'failed' }

    Failed at the expected assertion: charge was called 1 time, expected 2.

  • F3 Change the retry limit

    src/checkout/payment-retry.ts:12

    Not counted
    Original
    MAX_RETRIES = 1
    Simulated
    MAX_RETRIES = '1'

    Not counted. This simulated change did not compile, so it says nothing about the test.

Valid trials: 2. Detected: 2. Invalid: 1. Blocked: 0. Timed out: 0.

Show the test payment-retry.after-timeout.spec.ts
// SYNTHETIC EXAMPLE for the OrangePro sample report. Not from a real repository.
// Boundary: the payment provider is mocked. The order state machine and store are real, in memory.
import { retryPayment } from '../payment-retry';
import { PaymentTimeoutError } from '../payment-errors';
import { OrderStateMachine } from '../../orders/order-state-machine';
import { inMemoryOrderStore } from '../../orders/testing/in-memory-order-store';

describe('checkout payment retry after a timeout', () => {
  it('keeps the order pending until the retried payment is confirmed', async () => {
    const provider = {
      charge: jest
        .fn()
        .mockRejectedValueOnce(new PaymentTimeoutError())
        .mockResolvedValueOnce({ status: 'pending' }),
    };
    const orders = new OrderStateMachine(inMemoryOrderStore());
    const order = await orders.create({ id: 'order-1', totalCents: 4200 });

    await retryPayment({ order, provider, orders });

    expect(provider.charge).toHaveBeenCalledTimes(2);
    expect(order.status).toBe('pending_payment');
  });
});

What this shows

  • The new test passes on the original build.
  • It fails at the expected assertion for each of the two simulated failures.

What it does not show

  • That checkout is fully protected or free of defects.
  • That a real payment provider behaves the same way. The provider is mocked at the service boundary.
  • That the release is safe to ship. That decision stays with your team.

05 Still open

  • Subscription renewal is blocked. It needs a payment-provider sandbox key. Next step: add the key to the test environment and rerun.
  • 6 lower-priority workflows were not assessed.
  • The existing refund test runs and passes, but it was not checked against a simulated failure.
  • No end-to-end run against a live payment provider.

06 What you get back

Commands and logs
# SYNTHETIC LOG from the OrangePro sample report. Not a real run.
$ git checkout 9f3c2e1            # rc-2026.10.2, 14 PRs since v2026.10.1
orangepro  changes: 41 files  workflows affected: 9
orangepro  1  Checkout payment retry   risk 84  protection: none found     -> write test
orangepro  2  Refund to original card  risk 61  protection: existing test  -> run it
orangepro  3  Subscription renewal     risk 57  protection: blocked        -> needs sandbox key

$ npx jest src/refunds/__tests__/refunds.integration.spec.ts
PASS  src/refunds/__tests__/refunds.integration.spec.ts (4 tests)

$ git apply orangepro-tests.patch
$ npx jest src/checkout/__tests__/payment-retry.after-timeout.spec.ts
PASS  checkout payment retry after a timeout
  ✓ keeps the order pending until the retried payment is confirmed

# Isolated copy F1: src/checkout/payment-retry.ts:57
$ npx jest src/checkout/__tests__/payment-retry.after-timeout.spec.ts
FAIL  keeps the order pending until the retried payment is confirmed
  expect(order.status).toBe('pending_payment')   Received: 'paid'

# Isolated copy F2: src/checkout/payment-retry.ts:44
$ npx jest src/checkout/__tests__/payment-retry.after-timeout.spec.ts
FAIL  keeps the order pending until the retried payment is confirmed
  expect(provider.charge).toHaveBeenCalledTimes(2)   Received: 1

# Isolated copy F3: src/checkout/payment-retry.ts:12
error TS2322: Type 'string' is not assignable to type 'number'.   (not counted)

orangepro  valid trials: 2  detected: 2  invalid: 1  blocked: 0  timed out: 0
orangepro  wrote orangepro-tests.patch, verification-receipt.json

The release decision stays with your team. This is evidence for it, not a certification.

Illustrative example from orangepro.ai