Facial Similarity reports

Start here

This guide presents a technical overview of Onfido's Facial Similarity report.

Introduction

Onfido's Facial Similarity reports compare the most recent live photo, live video or motion capture provided by an applicant during an identity verification flow to the face on their most recently captured identity document (such as passports or national ID cards). Where the document has two sides, Onfido will search both sides of the document for a face. Facial Similarity reports are designed to prove identity document ownership, so that only the owner of the identity document can use it to verify their identity and access services.

Onfido offers four types of Facial Similarity report: Photo, Photo Fully Auto, Video and Motion.

Photo and Photo Fully Auto

In Facial Similarity Photo and Photo Fully Auto reports, images and data are extracted from identity documents using machine learning, and then compared to a live photo taken and submitted by the applicant during their identity verification journey. The report assesses whether the holder of the identity document is the same person as the one on the identity document, while also ensuring the selfie image is a live capture taken by the applicant at the time of submission.

The Photo report uses a combination of algorithms and expert manual review by Onfido's super recognizer analysts to return a result.

The Photo Fully Auto report is intended to be paired with manual review by your own team, and does not include expert review by Onfido's super recognizer analysts. This guarantees faster results, but it also means more reports could potentially end with faces not being detected, or with users incorrectly rejected.

Photo and Photo Fully Auto API response breakdown tree

The result for both Photo and Photo Fully Auto reports can be clear or consider, returned in the output object of a Studio workflow run. The report result is determined by the results of individual breakdowns and sub-breakdowns.

The following diagram shows how these breakdowns and sub-breakdowns are mapped to particular results:

Breakdown descriptions

Breakdown	Sub-breakdown(s)	Description
Visual authenticity		Asserts whether the person in the live photo is real and not a spoof.
	Spoofing detection	Contains a score value between 0 and 1 which expresses the likelihood that the live photo is real, where 0 represents a spoofed image and 1 represents a real image.
Face comparison		Asserts whether the face in the document image matches the face in the live photo.
	Face match	Contains a score value between 0 and 1 which expresses how similar the document image face and live photo are, where 1 represents a perfect match.
Image integrity		Asserts whether the quality and integrity of the uploaded files were sufficient to perform a face comparison.
	Face detected	Asserts that a single face of sufficient quality was found in both the document image and the live photo.
	Source integrity	Asserts whether the source of the live photo is trustworthy. For example, it has not been digitally tampered with or was not taken from a fake webcam. This sub-breakdown also asserts whether the applicant's document is issued by a country subject to comprehensive US sanctions.

Complete details regarding the structure of Facial Similarity Photo and Photo Fully Auto reports and example results can be found in our API reference documentation.

Motion

In addition to extracting images and data from identity documents, Facial Similarity Motion provides added security for high-risk users or transactions, in a fully automated fashion. In addition to confirming a user's face matches their document upload, Motion also assesses liveness by asking users to complete a simple head turn pattern in both directions.

Motion API response breakdown tree

The result for Motion reports can be clear or consider, returned in the output object of a Studio workflow run. The report result is determined by the results of individual breakdowns and sub-breakdowns.

The following diagram shows how these breakdowns and sub-breakdowns are mapped to particular results:

Breakdown descriptions

Breakdown	Sub-breakdown(s)	Description
Visual authenticity		Asserts whether the person in the motion capture is real (not a spoof) and live.
	Spoofing detection	Contains a score value between 0 and 1 which expresses the likelihood that the motion capture is real, where 0 represents a spoofed capture and 1 represents a real motion capture.
	Liveness detected	Asserts whether the head turn completion pattern was executed correctly.
Face comparison		Asserts whether the face in the document matches the face in the motion capture.
	Face match	Contains a score value between 0 and 1 which expresses how similar the face in the document image and motion capture are, where 1 represents a perfect match.
Image integrity		Asserts whether the quality and integrity of the uploaded files and the content contained within them were sufficient to perform a face comparison.
	Face detected	Asserts that a single face of sufficient quality was found in both the document image and the motion capture.
	Source integrity	Asserts whether the source of the live photo is trustworthy. For example, it has not been digitally tampered with or was not taken from a fake webcam. This sub-breakdown also asserts whether the applicant's document is issued by a country subject to comprehensive US sanctions.

Complete details regarding the structure of Facial Similarity Motion reports and example results can be found in our API reference documentation.

Video

Facial Similarity Video also provides added security for high-risk users or transactions. In addition to confirming a user's face matches their document upload, the report also assesses liveness by asking the user to record themselves repeating out loud 3 randomized digits displayed on their device screen and performing a randomized head turn movement.

The Facial Similarity Video report uses a combination of algorithms and expert manual review by Onfido's super recognizer analysts to return a result.

SDK localization

We recommend that you localize the strings if you're using one of the Onfido SDKs, to ensure that users understand the liveness head turn and speaking instructions.

By default, the Onfido SDKs detect the language to use from the user's device or browser settings. We recommend, however, you pass the code for the expected language to the locale mechanism, in any of the Onfido SDKs:

• iOS SDK - pass the onfido_locale parameter
• Android SDK - pass the onfido_locale parameter
• Web SDK - pass the locale parameter

The following languages are supported for string localization:

Language name	Language code
English	"en"
Spanish	"es"
Indonesian	"id"
Italian	"it"
German	"de"
French	"fr"
Portuguese	"pt"
Polish	"pl"
Japanese	"ja"
Dutch	"nl"
Romanian	"ro"
Basque	"eu"
Catalan	"ca"
Galician	"gl"
Chinese	"cn"
Turkish	"tr"
Malay	"ms"

Some string localizations are available out of the box, but this differs for each SDK.

You can also provide your own custom translations to your users.

Video API response breakdown tree

The result for Video reports can be clear or consider, returned in the output object of a Studio workflow run. The report result is determined by the results of individual breakdowns and sub-breakdowns.

The following diagram shows how these breakdowns and sub-breakdowns are mapped to particular results:

Breakdown descriptions

Breakdown	Sub-breakdown(s)	Description
Visual authenticity		Asserts whether the person in the live video is real and not a spoof.
	Spoofing detection	Contains a score value between 0 and 1 which expresses the likelihood that the live video is real, where 0 represents a spoofed video and 1 represents a real video.
	Liveness detected	Asserts whether the numbers and head movements were executed correctly during the live video task.
Face comparison		Asserts whether the face in the document image matches the face in the live video.
	Face match	Contains a score value between 0 and 1 which expresses how similar the document image face and live video are, where 1 represents a perfect match.
Image integrity		Asserts whether the quality and integrity of the uploaded files were sufficient to perform a face comparison.
	Face detected	Asserts that a single face of sufficient quality was found in both the document image and the live video.
	Source integrity	Asserts whether the source of the live photo is trustworthy. For example, it has not been digitally tampered with or was not taken from a fake webcam. This sub-breakdown also asserts whether the applicant's document is issued by a country subject to comprehensive US sanctions.

Complete details regarding the structure of Facial Similarity Video reports and example results can be found in our API reference documentation.

Facial Similarity Report task

Onfido Studio features workflow tasks for all four Facial Similarity report variants. When constructing a Studio workflow, a Facial Similarity Report task must always follow the path of a Document Capture task and a Face Capture task (which also has photo, video and motion variants) in order to extract the appropriate task input data to make the comparison.

It is also highly recommended that the workflow include a Document Report task.

Below you will find an illustrated example of a Studio workflow running a Facial Similarity report:

More information about report tasks can be found in our Studio Product Guide.

Facial Similarity Report task results

If you wish to obtain the specific outputs from the Facial Similarity Report Task via the Onfido API (for example, the overall report result or breakdown results), you can manage this by configuring the Workflow Output in the Studio Workflow Builder. You can also retrieve all associated output data from a Facial Similarity report.

You can refer to our Studio product guide for more information on Workflow Output configuration. Once set up, the output can be consumed by making a Retrieve Workflow Run call to the Onfido API. Report results are found in the output property.

Workflow Run results can also be accessed on the "Results" tab of your Studio Dashboard.

Suggested client actions

In this section, we explore a range of specific scenarios for the Facial Similarity Report by examining individual results from the report breakdown and describe suggested actions. We provide suggestions for clients who manually review photos as part of their service, and those who do not.

This section also presents a table highlighting which Facial Similarity report breakdowns can be affected by user environments, and what advice you can pass on to your users.

The details of the Facial Similarity Report breakdown structure are described above in this guide, and are also documented in our API reference.

Example scenarios

Scenario 1: All results are clear

In this scenario, the overall report result will be clear and we recommend that you allow the user to proceed.

Scenario 2: face_detected result is consider

• Ask user to retry (see "User retries")
• Make sure that both the face on the document and the face in the selfie are clearly visible
• Confirm the document image with side labelled as front contains a face

Scenario 3: source_integrity result is consider

Type of client	Contributing breakdown(s)	Suggested client action
ALL clients	digital_tampering	Block user. Evidence has been found that the user deliberately used photo editing software to manipulate their selfie image.
ALL clients	fake_webcam	Block user. Evidence has been found that the user is running software capable of manipulating their webcam feed. This might be deliberate, or might be that the user installed the fake webcam software for other purposes and then forgot to turn it off. If you are not risk averse, consider asking the user to retry.
Clients WITH manual review	time_of_capture	* If you are not allowing users to upload photos, then review the image. If the image looks genuine, ask the user to check date and time is correct on their device and retry. * If you are allowing users to upload photos (not recommended): ask the user to re-upload a new, more recent selfie.
Clients WITHOUT manual review	time_of_capture	* Ask the user to check date and time is correct on their device and retry.

Scenario 4: face_match result is consider

• EITHER block the user OR ask the user to retry
• Suggest the user retries with a more recent document

Scenario 5: spoofing_detection result is consider

• EITHER block the user OR ask the user to retry

Scenario 6: liveness_detected result is consider

• Ask user to retry
• If you are not risk averse, and all other breakdowns are clear, ignore and allow the user to proceed

You should always cap the number of user retries in line with your tolerable business risk.

Breakdowns affected by user environment

The following table highlights which individual breakdowns in the Facial Similarity report can be affected by user environments, and what actions you can suggest to users:

Suggested user actions	face_detected	face_match	spoofing_detection	liveness_detected
Improve lighting conditions	X	X	X
Take both document and selfie images against a plain background	X	X	X
Avoid any glare caused by light sources in the background	X	X	X
Make sure the face is sufficiently close to the camera (on SDK, make sure the face fills the oval)	X	X	X
Make sure the document is sufficiently close to the camera (on SDK, make sure the document fills the region defined by the overlay)	X	X
Take a frontal selfie, with a straight head	X	X
Make sure there are occlusions (e.g. glasses)		X	X
Have a neutral facial expression		X
Make sure your face is not blurry	X	X	X
Do not have anyone else in the picture	X	X	X
Do not use filters or devices with overlays	X	X	X
Do not hold the document in the selfie image			X
Say the digits displayed on the screen				X
Turn your head in the direction displayed on the screen				X