Selfie capture

A selfie step checks that a live person is in front of the camera. The inquiry UI draws the step's screens itself and hands the liveness check to a selfie capture provider: a small component that runs a face liveness engine and returns the captured portrait. The SDK decides everything else: which mode to run, the service address and the headers of each run, the texts on the engine's screens, and what happens with the result.

Folio ships one provider on each platform, built on the capture engine of the selfie step:

The liveness engine runs against a service the SDK names per run.

The selfie stepCopy Anchor Link

The inquiry store projects a selfie step as a SelfieStateView. The inquiry UI renders every state; your app does not handle them. The states are:

The cases are spelled by platform:

Liveness and Passive carry the data a provider needs in capture, a SelfieCaptureView:

Both states run the engine in the mode capture.mode names, which comes from the step's liveness mode. Passive also carries title and description; the ready-made inquiry UIs render only the engine, and a UI of your own may show them.

The SDK fills headers for each run with:

The Folio providers send them unchanged with every request to the service; a provider of your own must send them too.

SelfieCaptureTexts has the optional texts holdSteady, turnHead, lookStraight, centerFace and processing.

Photo carries captureSessionId, the capture session of the photo, title, description, hint, cameraErrors (a CameraErrorsView), buttons, captureField, the field the photo is uploaded to, and processing.

Preparing carries only processing, a ProcessingView that is always set.

SelfieLivenessMode is .active and .passive in Swift, and SelfieLivenessMode.ACTIVE and SelfieLivenessMode.PASSIVE in Kotlin and TypeScript. See Inquiry store for the other fields of the inquiry state.

A photo step needs a capture fieldCopy Anchor Link

The SDK checks every selfie photo step it receives. A step whose capture target names no field, or an empty one, is not shown: the SDK rejects it with InquiryError.MissingCaptureField, whose step is the id of the step. It is spelled .missingCaptureField(step:) in Swift and has the type 'MISSING_CAPTURE_FIELD' in TypeScript. The error is a banner without a retry. When this happens on start, the inquiry shows no step: the lifecycle is Started and the model shows the generic failure notice without retry. The step comes from the server, so your app cannot supply the field. See Errors.

The SDK checks each captureCopy Anchor Link

The SDK also checks the capture of every selfie state it receives, before the UI sees it:

• Liveness and Passive need a capture session, the command that prepared it, a liveness mode that matches the state, a service address and a credential for it. The service address is the selfie proxy of the Folio services the SDK talks to: an http or https URL with no user name, query or fragment.
• Photo needs a capture session and the command that prepared it, and carries no service address and no credential.

A service address that does not parse or points elsewhere fails the update with InquiryError.InvalidLink, whose url is that address. Any other mismatch fails it with InquiryError.InvalidState. In both cases the step is not shown and the previous state stays. So serviceUrl is always set in Liveness and Passive.

What happens with the resultCopy Anchor Link

1. The inquiry UI shows a liveness state and calls the provider with its capture.
2. The provider runs the engine. The engine talks to the liveness service itself, with the headers from capture.
3. When liveness passes, the provider returns the portrait as JPEG bytes and the engine's transaction id.
4. The inquiry UI sends InquiryIntent.SubmitCapture with capture.captureSessionId and the transaction id. The intent carries no image. The portrait stays in the UI and is shown on the Review screen.
5. On iOS and Android, when the user closes the engine or the provider fails, the inquiry UI goes back one step. On the web a failed run shows a retry dialog instead, see How a web run ends.

While the step is being checked, or when capture has no serviceUrl, the inquiry UI shows its processing screen in place of the engine. On iOS the provider is optional, and without one the step cannot be completed; the InquiryPlatform sets the default provider unless you pass nil.

In the Photo state the user takes one photo: with the front camera on iOS and the web, and with the system image picker (image/*) on Android. The inquiry UI uploads it with InquiryIntent.UploadCapture with the state's captureSessionId, its captureField and the image bytes, or sends InquiryIntent.FileUnreadable when the image cannot be read. See Selfie capture for the checks the store makes.

iOSCopy Anchor Link

SelfieCaptureProviderCopy Anchor Link

The protocol is in FolioInquiryUI and runs on the main actor:

@MainActor
public protocol SelfieCaptureProvider {
    func startLiveness(
        _ capture: SelfieCaptureView,
        serviceUrl: String,
        presenter: UIViewController
    ) async throws -> SelfieLivenessCapture
}

capture is the SelfieCaptureView of the state, and serviceUrl its service address; the inquiry UI calls the provider only when there is one. presenter is a view controller the inquiry UI has placed on screen; present the engine from it. Any thrown error makes the inquiry UI go back one step.

DefaultSelfieCaptureProviderCopy Anchor Link

DefaultSelfieCaptureProvider is part of FolioInquiryUI, and the package links FolioInquiryUI to the capture engine of the selfie step; see iOS setup. The InquiryPlatform of the inquiry view uses it by default, so you pass only the NFC host:

import FolioSDK
import FolioInquiryUI

FolioInquiry(
    store: inquiry,
    platform: InquiryPlatform(nfc: sdk)
)

Pass selfieCapture: with your own SelfieCaptureProvider to replace it. See iOS setup.

The default provider runs the engine with the front camera, the close button on, camera switching off, the copyright line off and the onboarding and success screens skipped. It colors the engine from FolioSdk.inquiryTheme().selfieCapture, shows the texts the inquiry sends and returns the portrait as JPEG at quality 0.9.

AndroidCopy Anchor Link

SelfieCaptureProviderCopy Anchor Link

The interface is in id.folio:sdk, package id.folio.sdk.inquiry.ui.verification.selfie:

interface SelfieCaptureProvider {
    suspend fun capture(context: Context, capture: SelfieCaptureView): SelfieCaptureResult
}

class SelfieCaptureResult(
    val image: ByteArray,
    val transactionId: String,
)

capture is the SelfieCaptureView of the state; the inquiry UI calls the provider only when it has a serviceUrl. context is the context of the inquiry UI; start the engine from it. Throw an exception when liveness does not pass: any exception other than a coroutine cancellation makes the inquiry UI go back one step.

DefaultSelfieCaptureProviderCopy Anchor Link

The provider is id.folio.sdk.inquiry.ui.verification.selfie.DefaultSelfieCaptureProvider, part of id.folio:sdk. id.folio:sdk depends on the capture engine of the selfie step at fixed versions, which the Folio Maven repository serves; the provider needs no dependency or repository of its own. See Android setup.

The id.folio:sdk manifest already declares the CAMERA permission.

Pass the providerCopy Anchor Link

The InquiryPlatform you give to FolioInquiry.Host or FolioInquiry.present uses DefaultSelfieCaptureProvider by default:

val platform = InquiryPlatform(nfc = runtime)
FolioInquiry.Host(store, platform)

Pass selfieCapture with your own SelfieCaptureProvider to replace it.

The default provider runs the engine with the close button on, camera switching off, the copyright line off and the onboarding and success screens skipped. It keeps the engine's own colors. It shows the texts the inquiry sends and returns the portrait as JPEG at quality 90.

WebCopy Anchor Link

The web inquiry UI in @folio/sdk/inquiry has the liveness engine built in; there is no provider to pass. The web component of the capture engine of the selfie step is a dependency of @folio/sdk at an exact version, so npm installs it with the package; there is nothing to install yourself. The inquiry embed includes it too.

The inquiry UI loads the selfie code, and with it the engine component, as a separate chunk the first time a liveness state is shown, so it does not weigh on the rest of the inquiry. The chunk is the entry point @folio/sdk/inquiry/ui/selfie, which exports:

The inquiry UI uses these itself; your app does not need to import them. While the chunk loads, the inquiry UI shows its processing screen.

The selfie step loads the engine's worker at run time from the engine's own content delivery origin, https://wasm.regulaforensics.com; this release reads it from https://wasm.regulaforensics.com/face/release/8.2/6b01d803-872e3641/. It downloads Liveness.worker.js from there and starts it as a worker from a blob: URL, and the worker downloads Liveness.wasm and Liveness.data from the same place. A page with a strict Content-Security-Policy must allow https://wasm.regulaforensics.com in connect-src, blob: in worker-src and 'wasm-unsafe-eval' in script-src. The engine can load these files from a host of your own through its workerPath setting; the SDK does not set it in this release.

How a web run endsCopy Anchor Link

On the web the engine runs with its start screen, its close button and its copyright line off. The inquiry UI handles the engine's outcome:

The retry dialog shows the localized texts chrome.error.title and chrome.error.description of the inquiry state and one button labeled chrome.error.retry. The button closes the dialog, sends InquiryIntent.retry and shows the engine again. A failed web run does not go back a step.

A liveness service address whose path starts with /api/ is requested from your page's own origin, so the same-origin proxy described in Web must forward it.