Data workflow

Overview

Lean has optimized the data ingestion flow to reduce the need for repeated bank-side calls and improve data availability timing.

Upon receiving the entity.created and entity.updated webhooks, Lean now immediately begins fetching and populating all available data into its internal store. This means clients no longer need to actively poll, implement the async failover for timeouts or trigger fetches for each data type - instead, data becomes queryable once the entity.data.refresh.updated webhook signals a FINISHED status. These changes are fully backward compatible and designed to simplify integration logic while preserving current API behaviour for in-progress and partial data requests.


How the platform works

Lean has simplified the integration process by consuming all available data and populating that data into our own store upon completion of the entity.created and entity.updated events. This reduces the need for roundtrip calls to banks per request and retries failed requests automatically.

These changes have been made to be fully backward compatible with our existing expected implementation flow.


Changes

Expected workflow

sequenceDiagram
    participant Client App
    participant Client Backend
    participant Lean API
    participant Lean Store
    participant Lean Webhooks

    Client App->>Client Backend: Request Customer Access Token
    Client Backend->>Lean API: POST /auth
    Lean API-->>Client Backend: Access Token
    Client Backend-->>Client App: Return Token
    Client App->>Client App: Initiate LinkSDK with Token
    Client App->>Lean API: User completes account connection
    Lean Webhooks-->>Client Backend: entity.created
    Lean Store->>Lean Store: Begin async data fetch + populate store
    loop Until All Data Ready
        Lean Webhooks-->>Client Backend: entity.data.refresh.updated (PENDING)
    end
    Lean Webhooks-->>Client Backend: entity.data.refresh.updated (FINISHED)
    Client Backend->>Lean API: GET /data/v2, /insights/v2
    Lean API->>Lean Store: Fetch requested data
    Lean API-->>Client Backend: Return populated data

  • Client registers a Customer resource for a user on their platform. POST /customers
  • User lands on screen in Client App that can initiate the LinkSDK
  • Client Frontend calls Client Backend for a Customer Access Token. POST auth.leantech.me
  • Client adds the Auth token to the configuration and initiation of the LinkSDK. .Connect()
  • User completes the account connection process.
  • Lean sends entity.created webhook - and starts populating data into store.
  • Lean sends entity.data.refresh.updated webhooks with status PENDING to notify progress of population.
  • Optional: Client begins making requests for completed data
  • Lean sends a final entity.data.refresh.updated webhook with status FINISHED once all requests have been completed and data is available in store.
  • Client begins making requests for data. GET /data/v2 GET /insights/v2

💭

What happens to requests for data still being fetched?

Lean will return PENDING for any requests that do not populate within 50 seconds of the request being made. These will include a results_id, a corresponding results.ready webhook and can be picked up from the /results endpoint when ready as is the behaviour today.

This enables you to queue up relatively safe API calls such as getting accounts and balances - whilst also providing a predictable maximum timeout period.

The 50 second maximum timeout is subject to change.

💭

How long is data available for in the Store?

The Data Store retains data for 30 days before being securely deleted from Lean’s system.

The 30 day period is subject to change.


Webhooks

entity.created/updated handling

Clients should only use this webhook for completion handling moving forward, and not use it to trigger data requests.

entity.data.refresh.updated handling

  • Log into dev.leantech.me and navigate to the webhooks tab
  • Add a subscription to the entity.data.refresh.updated webhook to an endpoint you specify.
  • Trigger all API calls based on the following criteria:
    • Do a full entity pull upon receiving status: FINISHED
    • Do partial data pulls based on mapping types marked as OK / SUCCESS

Webhooks

In all cases the below webhooks will be rate-limited to reduce noise - with the exception of a change of a refresh from PENDING → FINISHED which will always be sent at the point of completion.

{
  "type": "entity.data.refresh.updated",
  "message": "An entity data refresh state has been updated.",
  "payload": {
    "refresh_id": "d4718195-fef6-43ff-a3aa-69fc257752ab",
    "entity_id": "d4718195-fef6-43ff-a3aa-69fc257752ab",
    "customer_id": "d4718195-fef6-43ff-a3aa-69fc257752ab",
    "status": "PENDING/FINISHED",
    "data_status": {
      "accounts": "PENDING/SUCCESS/FAILED/UNSUPPORTED",
      "identity": "PENDING/SUCCESS/FAILED/UNSUPPORTED",
      "account_data": [
	      {
		      "account_id": "d4718195-fef6-43ff-a3aa-69fc257752ab",
		      "balance": "PENDING/SUCCESS/FAILED/UNSUPPORTED",
		      "identity": "PENDING/SUCCESS/FAILED/UNSUPPORTED",
		      "transactions": "PENDING/PARTIAL/SUCCESS/FAILED/UNSUPPORTED",
		      "scheduled_payments": "PENDING/PARTIAL/SUCCESS/FAILED/UNSUPPORTED",
		      "direct_debits": "PENDING/PARTIAL/SUCCESS/FAILED/UNSUPPORTED",
		      "standing_orders": "PENDING/PARTIAL/SUCCESS/FAILED/UNSUPPORTED",
		      "beneficiaries": "PENDING/SUCCESS/FAILED/UNSUPPORTED",
		      "transaction_availability": {
			      "start": "<DateTime>",
			      "end": "<DateTime>"
			    }
		    },{
		      "account_id": "b5098d49-840d-459e-9ea1-d02901af9b8c",
		      "balance": "PENDING/SUCCESS/FAILED/UNSUPPORTED",
		      "identity": "PENDING/SUCCESS/FAILED/UNSUPPORTED",
		      "transactions": "PENDING/PARTIAL/SUCCESS/FAILED/UNSUPPORTED",
		      "scheduled_payments": "PENDING/PARTIAL/SUCCESS/FAILED/UNSUPPORTED",
		      "direct_debits": "PENDING/PARTIAL/SUCCESS/FAILED/UNSUPPORTED",
		      "standing_orders": "PENDING/PARTIAL/SUCCESS/FAILED/UNSUPPORTED",
		      "beneficiaries": "PENDING/SUCCESS/FAILED/UNSUPPORTED",
		      "transaction_availability": {
			      "start": "<DateTime>",
			      "end": "<DateTime>"
			    }
		    }
      ]
    }
  }
}


Refreshing data

Lean will automatically refresh data on a periodic basis where ongoing access to an account is obtained to ensure that data is up-to-date.

Detecting Stale Data

All API responses for Data will now include a refreshed_at DateTime attribute in the response body

Triggering a manual Refresh

🚧

Manual refreshes may become chargeable in future

Data in the Store can be refreshed by calling /data/v2/refreshes if a consent is still available - a status of RECONNECT_REQUIRED will be returned along with a reconnect_id where no available session or consent is active.

POST 'https://api.leantech.me/data/v2/refreshes'
-h 'Authorization: Bearer <env.token>'
--data-raw '{
  "entity_id": "ENTITY_ID"
  }
Successful 
{
  "timestamp": "2025-07-22T05:07:30.030450168Z",
   "status": "Ok",
    "message": "Data refresh was triggered successfully",
}

Reconnect entity is required (UAE)
{
    "status": "RECONNECT_REQUIRED",
    "message": "User input is required to reconnect with the bank",
    "timestamp": "2025-07-24T08:46:17.848098201Z",
    "reconnect_id": "RECONNECT_ID"
}

Consent is expired (KSA)
{
    "timestamp": "2025-07-22T05:07:30.030450168Z",
    "status": "CONSENT_EXPIRED",
    "message": "Consent was suspended, revoked, or expired. New consent is required; please reconnect the entity.",
}

Refresh automation: UAE

Reverse-engineered

  • Data refresh is triggered upon a successful .Reconnect in the LinkSDK.
  • Completion of a refresh is messaged with an entity.data.refresh.updated webhook

Long-lived consent

  • Data refresh is triggered 4 times a day (provisional in-line with off-session refreshes allowed by framework).
  • On-session refreshes will be handled in future functionality (provisionally look at how Plaid handles refresh tokens as an example).
  • Completion of a refresh is messaged with an entity.data.refresh.updated webhook when changes are detected.

Single-use consent(s)

  • Lean will not trigger any refreshes outside of failure handling.

Refresh automation: KSA

Long-lived consent

  • Data refresh is triggered 4 times a day (provisional in-line with off-session refreshes allowed by framework).
  • On-session refreshes will be handled in future functionality (provisionally look at how Plaid handles refresh tokens as an example).
  • Completion of a refresh is messaged with an entity.data.refresh.updated webhook when changes are detected.

Single-use consent(s)

  • Lean will not trigger any refreshes outside of failure handling.
  • Manual refreshes can be triggered - however it likely makes more sense to run .connect() each time.

Failure handling

What happens when a failure occurs?

In case there is any error in communication with bank we will retry the underlying request internally. We apply different retry strategies (different number of attempts and backoff) depending on error type. Retried requests will appear as still pending externally. There may occur additional delay caused by retries, but the design hides retry logic from clients.

How do I handle a failed request?

Retries are handled internally. If result is marked as failed this means that retries were already applied. Status may change only after next data refresh.