Staff System Design Walkthrough - Benefits Platform

How to lay out the Excalidraw canvas

Build one wide canvas with six left-to-right frames:

1 Requirements -> 2 Core Entities -> 3 API / Interface -> 4 Data Flow -> 5 High-Level Design -> 6 Deep Dives

Place a small progress strip at the top of every frame. Highlight the current stage and mute the others. Keep domain colors consistent throughout:

Identity: blue (#a5d8ff, stroke #1971c2)
Benefits: green (#b2f2bb, stroke #2b8a3e)
Wallet: violet (#d0bfff, stroke #7048e8)
Payments: orange (#ffd8a8, stroke #e67700)
Notifications: pink (#fcc2d7, stroke #c2255c)
Shared infrastructure / data: gray (#e9ecef, stroke #495057)
External systems: yellow (#ffec99, stroke #f08c00)
Failure paths: red dashed arrows (#e03131)

Use solid arrows for synchronous calls, dashed arrows for asynchronous events, and double-line arrows for money movement. Add a small legend in the bottom-right corner.

Stage 1 - Functional Requirements

Frame title

1. Functional Requirements - What must the system do?

Boxes and grouping

Create five colored domain groups in a row. Within each group, use one title box followed by compact requirement cards.

Identity

Group label: IDENTITY - Who is acting, and are they allowed?

Cards:

Register / invite employee
Authenticate user
Maintain profile + employment status
Organization membership and roles
Eligibility attributes: country, plan, employment type
Consent, privacy, and account recovery

Small annotation beneath the group:

Identity is the system of record for people, organizations, membership, authentication, and authorization context.

Benefits

Group label: BENEFITS - What is the user entitled to?

Cards:

Employer configures benefit programs
Define eligibility rules and limits
Enroll eligible employees
Track allowance periods and usage
Approve or reject benefit claims

Wallet

Group label: WALLET - What value is available or reserved?

Cards:

Show available balance by benefit
Credit allowance / adjustment
Reserve funds for a purchase
Release or capture a reservation
Immutable transaction history

Payments

Group label: PAYMENTS - How does value move externally?

Cards:

Initiate card / reimbursement payment
Authorize, capture, refund, reverse
Integrate with payment processor / bank
Handle retries and duplicate requests safely
Reconcile internal and external records

Notifications

Group label: NOTIFICATIONS - How are people informed?

Cards:

Transactional email / push / SMS
Benefit and balance reminders
Payment success / failure updates
User preferences and quiet hours
Template, delivery, retry, and audit status

Cross-domain arrows

Identity -> Benefits: label membership + eligibility attributes
Benefits -> Wallet: label grant / limit policy
Wallet -> Payments: label reservation permits spend
Payments -> Wallet: label capture / refund outcome
Dashed arrows from all domains to Notifications: label domain events

Why Identity is separated

Add a blue callout titled Why a separate Identity domain? with these bullets:

Different responsibility: Identity proves who the actor is; Benefits decides what that actor receives.
Security boundary: credentials, sessions, MFA, recovery, and sensitive profile data need tighter controls.
Shared capability: every domain needs the same user, organization, and role context.
Independent lifecycle: a person can exist before enrollment and remain auditable after employment ends.
Avoid coupling: benefit rules can change without changing authentication or canonical person records.
Scale and ownership: login traffic, security operations, and compliance evolve differently from benefit workflows.

Add a warning note:

Separation does not mean duplicating identity data. Downstream domains store stable IDs and only the minimum snapshot required for decisions or audit.

Scope strip

At the bottom, add two columns.

IN SCOPE

Multi-tenant employer and employee model
Benefit allowance and wallet ledger
Payment lifecycle and processor callbacks
Transactional notifications
Auditability, idempotency, and reconciliation

OUT OF SCOPE / CLARIFY

Payroll calculation
Full HRIS ownership
Merchant acquiring network
Tax advice and jurisdiction-specific accounting
Marketing campaigns

Speaker notes

“I begin by separating the nouns that sound similar but have different invariants. Identity answers who the actor is and which organization they belong to. Benefits answers what they are entitled to. Wallet represents internal value and its accounting history. Payments coordinates external money movement. Notifications communicates outcomes but does not own them. This gives us clean boundaries before we discuss technology.”

“I would confirm the primary journey: an eligible employee spends an employer-funded benefit, the system prevents overspend, moves money safely, and communicates the result. I would also clarify geography, payment rails, expected scale, consistency requirements, and regulatory constraints.”

Stage 2 - Core Entities

Frame title

2. Core Entities - What state must exist?

Arrange five domain containers. Put entity boxes inside them; connect relationships with labeled arrows.

Identity entities

User {user_id, status, profile_ref}
Organization {org_id, status}
Membership {user_id, org_id, role, employment_status}
IdentityCredential {credential_id, user_id, provider}
EligibilityProfile {user_id, country, employment_type, attributes_version}

Relationships:

User 1 -> N Membership
Organization 1 -> N Membership
User 1 -> N IdentityCredential

Benefits entities

BenefitProgram {program_id, org_id, type, rules_version}
EligibilityRule {rule_id, program_id, expression}
Enrollment {enrollment_id, user_id, program_id, status}
Allowance {allowance_id, enrollment_id, period, limit}
Claim {claim_id, user_id, amount, status}

Wallet entities

Wallet {wallet_id, owner_id, currency}
BalanceAccount {account_id, wallet_id, benefit_id}
LedgerEntry {entry_id, debit_account, credit_account, amount}
Reservation {reservation_id, account_id, amount, status, expires_at}

Callout: Balance is derived from ledger entries; never overwrite history.

Payments entities

Payment {payment_id, user_id, amount, currency, status}
PaymentAttempt {attempt_id, payment_id, provider_ref, status}
Refund {refund_id, payment_id, amount, status}
ProcessorEvent {provider_event_id, type, received_at}
ReconciliationRecord {date, internal_total, external_total, variance}

Notification entities

Notification {notification_id, user_id, event_id, template_id}
Template {template_id, channel, locale, version}
Preference {user_id, channel, category, enabled}
DeliveryAttempt {attempt_id, notification_id, status}

Cross-domain relationships

User.user_id -> Enrollment.user_id
Enrollment -> Allowance -> BalanceAccount
Payment -> Reservation
Payment outcome -> LedgerEntry
Domain event -> Notification

Add a red boundary note:

Cross-domain references use IDs, APIs, and events - not direct foreign-key joins across service-owned databases.

Speaker notes

“The important modeling decision is the distinction between entitlement, value, and money movement. An allowance describes policy. A wallet account and ledger describe available internal value. A payment describes an external transaction. Keeping these separate prevents a processor retry from silently changing entitlement and gives us a complete audit trail.”

Stage 3 - API / Interface

Frame title

3. API / Interface - What contracts cross boundaries?

Put Web / Mobile Client on the left, API Gateway / BFF in the center, and five domain API columns on the right.

Identity API

POST /v1/sessions
GET /v1/me
POST /v1/organizations/{orgId}/members
PATCH /v1/memberships/{membershipId}

Benefits API

GET /v1/me/benefits
POST /v1/programs
POST /v1/programs/{id}/enrollments
POST /v1/claims
POST /v1/claims/{id}/decision

Wallet API

GET /v1/wallets/{id}/balances
GET /v1/wallets/{id}/transactions?cursor=
POST /v1/wallets/{id}/reservations
POST /v1/reservations/{id}/capture
POST /v1/reservations/{id}/release

Payments API

POST /v1/payments
GET /v1/payments/{id}
POST /v1/payments/{id}/refunds
POST /v1/webhooks/payment-provider

Notifications API

GET /v1/me/notification-preferences
PUT /v1/me/notification-preferences
Internal event consumer: payment.captured.v1

Contract callouts

Add four gray notes near the gateway:

Authorization: tenant + actor + role on every request
Idempotency-Key required for mutating money APIs
Money = integer minor units + ISO currency
Cursor pagination; request ID and trace ID returned

Example request card:

POST /v1/payments
Idempotency-Key: 7b3...
{
  "wallet_id": "wal_123",
  "benefit_id": "ben_456",
  "amount_minor": 4250,
  "currency": "EUR",
  "merchant": {"name": "Example", "category": "FITNESS"}
}

Example response card:

202 Accepted
{
  "payment_id": "pay_789",
  "status": "PROCESSING",
  "reservation_id": "res_321"
}

Speaker notes

“The public contract is resource-oriented, but money-changing operations are commands with explicit idempotency. A 202 response reflects that processor completion may be asynchronous. The client polls or receives a pushed update; it never infers success from request acceptance.”

Stage 4 - Data Flow

Frame title

4. Data Flow - One benefit purchase, end to end

Use a sequence diagram with vertical lanes:

Happy path arrows

Number every arrow visibly.

Employee -> Client: Submit €42.50 purchase
Client -> Gateway: POST /payments + token + idempotency key
Gateway -> Identity: Validate token, membership, tenant
Gateway -> Benefits: Check enrollment + category + current rules
Benefits -> Wallet: Authorize against benefit account
Wallet -> Wallet DB: Atomic create reservation; available -= 42.50
Wallet -> Payments: reservation.created
Payments -> Provider: Authorize / capture with provider key
Provider -> Payments: Success callback / webhook
Payments -> Wallet: Capture reservation idempotently
Wallet -> Ledger: Post balanced immutable entries
Payments -> Event Bus: payment.captured.v1
Event Bus -> Notifications: Consume event
Notifications -> Employee: Receipt / push
Client -> Gateway: GET payment and updated balance

Failure branches

Place three red side branches:

From step 6: Insufficient balance -> reject; no processor call
From step 9: Declined / timeout -> release now or after timeout policy
From step 13: Delivery fails -> retry with backoff; payment remains successful

Add a bottom note:

No distributed transaction spans Wallet and the provider. Use local transactions + durable events + idempotent compensating actions.

Speaker notes

“The correctness boundary is the wallet reservation. It prevents concurrent spends from exceeding the allowance. External payment completion cannot share that database transaction, so we model the workflow as a state machine. Every callback and event can be delivered more than once; every consumer must therefore be idempotent.”

Stage 5 - High-Level Design

Frame title

5. High-Level Design - Services, stores, and asynchronous boundaries

Lay out four horizontal zones.

Zone A - Clients and edge

Boxes:

Employee Web / Mobile
Employer Admin Portal
API Gateway / BFF
WAF + Rate Limiter

Arrows: clients -> WAF -> Gateway.

Zone B - Domain services

Five colored boxes:

Identity Service
Benefits Service
Wallet / Ledger Service
Payments Orchestrator
Notification Service

Add a smaller gray box beside them: Policy / Feature Configuration.

Zone C - Data and messaging

Give each owning service its own store:

Identity -> Identity DB + credential provider
Benefits -> Benefits DB
Wallet -> Ledger DB (strong consistency)
Payments -> Payments DB
Notifications -> Notification DB

Shared infrastructure:

Event Bus
Outbox / CDC publisher
Redis cache (only for non-authoritative reads, rate limits, short-lived state)
Object storage (receipts / evidence)

Zone D - External systems

Yellow boxes:

HRIS / Payroll
Payment Processor / Bank
Email / SMS / Push Providers
Fraud / KYC Provider

Major arrows

HRIS -> Identity: employee lifecycle sync
Gateway -> Identity: authn/authz context
Gateway -> Benefits: program and claim operations
Gateway -> Wallet: balance and history reads
Benefits -> Wallet: credits / adjustments
Wallet <-> Payments: reserve / capture / release
Payments <-> Processor: double-line money workflow
Every domain -> Outbox -> Event Bus
Event Bus -> Notifications
Notifications -> channel providers

Ownership note

Each service is the only writer to its database. Cross-domain read models may be built from versioned events.

Speaker notes

“These boxes are logical ownership boundaries, not a demand to deploy five microservices on day one. A modular monolith could preserve the same contracts initially. I would isolate Wallet and Payments earlier because they have stronger correctness, audit, and operational requirements. The event bus removes Notifications from the critical path and supports read models and reconciliation.”

Stage 6 - Deep Dives

Frame title

6. Deep Dives - Prove correctness, resilience, and operability

Create six mini-panels in a 2 x 3 grid.

A. Identity and tenancy

Boxes:

OIDC / OAuth provider
Access token {sub, tenant, scopes, exp}
Gateway authorization
Domain-level policy check

Notes:

Never trust tenant ID supplied only in the request body.
Short-lived access tokens; rotate signing keys.
Domain services verify resource ownership and role.
Propagate actor_id, tenant_id, and request_id into audit logs.
PII encryption, retention, deletion workflow, and access audit.

Speaker note: “Authentication at the gateway is necessary but insufficient. The domain still checks that the actor may perform this action on this tenant’s resource.”

B. Wallet ledger and concurrency

Draw two accounts and balanced postings:

Employer Funding Account --debit 42.50--> Employee Benefit Account --debit 42.50--> Settlement Account

Notes:

Double-entry ledger; every transaction balances to zero.
Append-only entries; corrections use reversals.
Serializable transaction or row lock for reservation.
Unique (wallet_id, idempotency_key) constraint.
Cached balance is derived and rebuildable.
Reservation lifecycle: PENDING -> CAPTURED | RELEASED | EXPIRED.

Speaker note: “A balance is a view, not the source of truth. The ledger and reservation constraints make concurrent spending safe and auditable.”

C. Payment state machine

Boxes and arrows:

CREATED -> RESERVED -> PROCESSING -> SUCCEEDED

Branches:

PROCESSING -> FAILED -> RELEASED
SUCCEEDED -> REFUND_PENDING -> REFUNDED
PROCESSING -> UNKNOWN -> RECONCILE -> SUCCEEDED | FAILED

Notes:

Store provider request and event IDs.
Verify webhook signatures and timestamps.
Deduplicate provider events.
Never blindly retry an ambiguous money operation; query provider first.

Speaker note: “Unknown is a real payment state. Treating a timeout as failure can cause a second charge.”

D. Reliable events and notifications

Flow:

Domain transaction -> Outbox row -> Publisher -> Event Bus -> Consumer inbox -> Template -> Provider

Side boxes:

Retry queue
Dead-letter queue
Delivery status webhook

Notes:

At-least-once delivery with idempotent consumers.
Event ID is the notification deduplication key.
Version schemas: payment.captured.v1.
Preferences can suppress optional messages, not legally required receipts.

Speaker note: “Notification failure never rolls back a payment. Delivery is independently observable and retryable.”

E. Reconciliation and recovery

Flow:

Provider settlement file/API + Payments DB + Ledger totals -> Reconciliation job -> variance queue -> Operations

Notes:

Compare counts, amounts, currencies, and provider references.
Automatic repair only for known safe cases.
Manual queue for ambiguity; every action audited.
Periodic sweeper expires stale reservations.
Replay events from durable log to rebuild projections.

Speaker note: “Real correctness includes detecting and repairing drift, not only preventing it.”

F. Scale, SLOs, and observability

Cards:

Availability: 99.95% payment initiation; graceful degradation for notifications.
Latency: balance read p95 < 200 ms; reservation p95 < 300 ms excluding provider.
Scale: partition ledger by wallet / tenant; preserve per-wallet ordering.
Signals: payment success rate, unknown states, reservation age, ledger imbalance, outbox lag, webhook lag, notification failure rate.
Tracing: request ID + payment ID + reservation ID + provider reference.
Security: secrets manager, least privilege, tokenization, tamper-evident audit logs.

Speaker note: “I define SLOs around user outcomes and financial invariants. A fast response is not healthy if reservations are stuck or reconciliation is drifting.”

Closing trade-offs frame

Place this as a narrow footer after Stage 6.

Decisions made

Domain separation follows ownership and invariants, not UI screens.
Identity is shared context but not a dumping ground for benefit state.
Ledger entries are authoritative; balances are derived.
Wallet reserves before Payments contacts an external provider.
Local atomicity plus outbox replaces cross-service distributed transactions.
Notifications are event-driven and outside the payment critical path.

Alternatives to mention

Modular monolith vs microservices: start with modules if team/scale is small; retain ownership boundaries.
Build vs buy Identity: prefer managed identity unless differentiation or regulation requires ownership.
Synchronous eligibility check vs local projection: synchronous is fresher; projection is more resilient and faster.
Single ledger vs ledger per benefit: single engine with separate accounts simplifies audit while preserving benefit isolation.

Final speaker note

“The design’s center of gravity is correctness at domain boundaries. Identity establishes the actor and tenant; Benefits establishes entitlement; Wallet safely accounts for value; Payments coordinates uncertain external money movement; Notifications communicates durable outcomes. The architecture can evolve operationally without collapsing those responsibilities.”

Excalidraw build checklist

Create the six stage frames and top progress strip.
Add the Stage 1 domain groups and keep their colors for the full canvas.
Add entity boxes in Stage 2; label cardinalities and domain crossings.
Add client, gateway, APIs, and contract callouts in Stage 3.
Draw the numbered purchase sequence and three failure branches in Stage 4.
Build the four-zone service diagram in Stage 5.
Add the six deep-dive panels and state machines in Stage 6.
Add the shared arrow legend and closing trade-offs footer.
Put speaker notes in small gray boxes below each frame, or keep them in Excalidraw element links/notes for presentation mode.

Assumptions to validate against the original attachments

The exercise concerns an employer-funded benefits platform with employee wallets and payments.
“Identity, Benefits, Wallet, Payments, Notifications” are intended domain boundaries.
The required interview framework is exactly: Requirements, Core Entities, API/Interface, Data Flow, High-Level Design, Deep Dives.
The expected primary journey is spending or reimbursing a benefit allowance.
Exact scale, supported rails, geography, compliance regime, and availability targets were not available in the referenced conversation payload.