Back to WorkflowBench
🔐KeycloakJava25.2k starsIdentity & Access Management

Zero RBAC tests for a security-critical admin tab in a 25k-star IAM platform

A PR fixing Identity Provider Links tab access had no tests validating who can see the tab — the exact behavior the PR was fixing.

8
Requirements Extracted
5
Verified Gaps
19%
Coverage Before
88%
Coverage After
Trace Matrix

Requirement-to-Test Traceability

Every requirement extracted from the PR, mapped to existing tests and verified against the codebase. Click any row to see the search evidence.

Coverage Distribution

8
5 Gaps
3 Partial
0 Covered

Requirement Coverage

Before19%
After OrangePro88%
+69% coverage improvement
8 requirements traced|5 gaps|3 partial|0 covered
R1GAP

Admin with 'view-users' role should see the IDP Links tab

UserIdentityProviderLinks.tsx
R2GAP

Admin with 'manage-users' role should see the IDP Links tab

UserIdentityProviderLinks.tsx
R3GAP

Admin without required roles should NOT see the IDP Links tab

UserIdentityProviderLinks.tsx
R4PARTIAL

IDP Links tab lists all linked identity providers

UserIdentityProviderLinks.tsx
R5GAP

Admin can link a new identity provider to a user

UserIdentityProviderLinks.tsx
R6GAP

Admin can unlink an identity provider from a user

UserIdentityProviderLinks.tsx
R7PARTIAL

UI displays message when no IDPs are linked

UserIdentityProviderLinks.tsx
R8PARTIAL

Change should not grant access to IDP configurations

UserIdentityProviderLinks.tsx
Critical Findings

Top 3 Verified Gaps

These gaps were verified by searching the repository's test directories. Each finding includes the exact search query and result so you can reproduce it yourself.

1

No role-based access control tests for the IDP Links tab

search evidence

Searched for tests validating tab visibility based on 'view-users' and 'manage-users' roles in testsuite/ and js/apps/admin-ui/test/ — found none.

2

No tests for linking and unlinking identity providers

search evidence

No tests found in the repository that cover the workflow of an admin adding or removing identity provider links from a user's profile.

3

No negative tests for access control

search evidence

No tests found that assert the tab is hidden for users who lack the required permissions — the exact bug this PR was fixing.

Before vs. After

Coverage Comparison

RequirementBeforeAfter
Admin with 'view-users' sees IDP Links tab✗ None✓ Unit + E2E
Admin with 'manage-users' sees IDP Links tab✗ None✓ Unit + E2E
Admin without roles does NOT see tab✗ None✓ Unit + E2E
IDP Links tab lists linked providers~ Partial✓ Unit + E2E
Admin can link new IDP to user✗ None✓ E2E
Admin can unlink IDP from user✗ None✓ E2E
Empty state message displayed~ Partial✓ Unit
No access to IDP configurations~ Partial✓ Unit + E2E
Verifiable Evidence

Don't take our word for it. Verify it yourself.

Every finding in this analysis is based on a public GitHub repository. Clone the repo, run the searches we documented, and see the same results. No black boxes.