Migrating Users to Cloudentity Identity Pools
Learn how to migrate users from external CIAM platforms into Cloudentity Identity Pools using the dedicated APIs. Change your CIAM platform to persistently store users even at hyper scale!
About Migrating Users
Cloudentity allows you import users into the tenant's Identity Pool. If the CIAM platform you used so far does not meet your needs or requirements, you can migrate external users to Cloudentity Identity Pools for persistent storage even at hyper-scale.
To migrate users, you will use the Cloudentity Identity Tenant Import API.
It is also possible to migrate users between Cloudentity tenants and between different Identity Pools. To do that, you would first use Cloudentity Identity Tenant Export API and, then, the Identity Tenant Import API. At this moment, however, exporting users is not recommended for production environments.
Prerequisites
You must have an Identity Pool to import/export the user data.
You have access to the System workspace.
You have a client application in the System workspace with the
manage_configuration
scope assigned. You will need this client to obtain an access token from Cloudentity for user import purposes.Note
Preconfigure your workspace describes how to create the client application that you need, but for the Admin workspace and admin APIs. What you need to do, is to create similar application in the System workspace and assign the
manage_configuration
scope to it.For import purposes, you must prepare input data conforming to the Import API schema. Prepare import batches of no more than 100 users (with credentials, identifiers, and addresses) per batch. Sending multiple requests is allowed - with 8 parallel requests running, the expected performance is around 1 500 000 users in half an hour.
The key item to prepare in order to achieve API idempotency is the random, unique ID (we strongly recommend using version 4 UUID). You will need to generate 4 sets of such IDs, since user data is kept in 4 separate objects, responsible for users, user credentials, user identifiers, and user addresses. This way, if an import request fails, you can simply run it again and be sure that no record gets added twice. Please check the Import API documentation and check the Request Body Schema for details.
Note that the user ID is especially important, since it must be mapped in all four user-related objects in the payload to correctly bind the entire user record (
user.id
touser_credentials.user_id
, touser_identifiers.user_id
and so on):users
- responsible for the core user information. You need to prepareid
parameter for each record.user_credentials
- data with user credentials. You need to prepareid
parameter for the record itself and map the user viauser_id
.If you want to migrate users without passwords, make sure to add the
expires_at
parameter with a value set to a past date. As a result, after migration, user login attempts with OAuth2 Resource Owner Password flow fail with thecredential expired
hint:{"error_description":"request lacks valid authentication credentials for the target resource","error_hint":"credential expired","status_code":401}
Similar outcome (with
"error_description": "credential expired"
) is expected for attempts to change password or verify password.Upon facing this error, users must request an OTP for password reset (forgot? in the login UI) and confirm resetting their password.
When
expires_at
is not set or set to1900-01-01T00:00:00Z
, the password never expires.user_identifiers
- data with user identifiers (e-mail or phone number used to log in). You need to prepareid
parameter for the record itself and map the user viauser_id
.user_verifiable_addresses
- data with user's addresses (e-mail and/or mobile). You need to prepareid
parameter for the record itself and map the user viauser_id
.
Below you can find a sample payload conforming to the above schema with two users (including credentials, identifiers, and addresses):
{ "user_credentials": [ { "created_at": "2022-08-03T11:03:38.34+02:00", "expires_at": "2019-08-24T14:15:22Z", "id": "e89e254c-f538-4ae2-9150-4d4580bcf886", "payload": { "hashed_password": { "config": { "method": "sha", "sha": { "function": "SHA-256", "salt": "lJgayFHwYelZGmrBnYqt", "salt_length": 20 } }, "value": "eUJBxl+dwVjPgwC2cm1K+hYNWFRly/RdCT/bgmIBowo=" } }, "tenant_id": "default", "type": "password", "updated_at": "2022-08-03T11:03:38.34+02:00", "user_id": "58e6184f-8e38-4193-b889-965b34326c7f", "user_pool_id": "caku6lrphdd3cfqro3mg" }, { "created_at": "2022-08-03T11:03:38.343+02:00", "id": "20f45596-f407-495c-aa45-043609ae2280", "payload": { "hashed_password": { "config": { "method": "sha", "sha": { "function": "SHA-256", "salt": "lJgayFHwYelZGmrBnYqt", "salt_length": 20 } }, "value": "eUJBxl+dwVjPgwC2cm1K+hYNWFRly/RdCT/bgmIBowo=" } }, "tenant_id": "default", "type": "password", "updated_at": "2022-08-03T11:03:38.343+02:00", "user_id": "cdcb7a5b-a726-404c-ba66-b8552d9be8fc", "user_pool_id": "caku6lrphdd3cfqro3mg" } ], "user_identifiers": [ { "created_at": "2022-08-03T11:03:38.341+02:00", "id": "b7108442-7f1e-40b9-a4d9-0c9ab3b0d18b", "identifier": "user0@example.com", "tenant_id": "default", "type": "email", "updated_at": "2022-08-03T11:03:38.341+02:00", "user_id": "58e6184f-8e38-4193-b889-965b34326c7f", "user_pool_id": "caku6lrphdd3cfqro3mg" }, { "created_at": "2022-08-03T11:03:38.343+02:00", "id": "cb1b6524-a769-442a-8b80-fe62ed9ed614", "identifier": "user1@example.com", "tenant_id": "default", "type": "email", "updated_at": "2022-08-03T11:03:38.343+02:00", "user_id": "cdcb7a5b-a726-404c-ba66-b8552d9be8fc", "user_pool_id": "caku6lrphdd3cfqro3mg" } ], "user_verifiable_addresses": [ { "address": "user0@example.com", "created_at": "2022-08-03T11:03:38.342+02:00", "id": "0b24d104-3d0e-40d7-bd2d-ac1e74b8fb34", "status": "active", "tenant_id": "default", "type": "email", "updated_at": "2022-08-03T11:03:38.342+02:00", "user_id": "58e6184f-8e38-4193-b889-965b34326c7f", "user_pool_id": "caku6lrphdd3cfqro3mg", "verified": true }, { "address": "user1@example.com", "created_at": "2022-08-03T11:03:38.343+02:00", "id": "f0eedf44-f7d8-4401-9a0d-4a02d3ed92f4", "status": "active", "tenant_id": "default", "type": "email", "updated_at": "2022-08-03T11:03:38.343+02:00", "user_id": "cdcb7a5b-a726-404c-ba66-b8552d9be8fc", "user_pool_id": "caku6lrphdd3cfqro3mg", "verified": true } ], "users": [ { "created_at": "2022-08-03T11:03:38.33+02:00", "id": "58e6184f-8e38-4193-b889-965b34326c7f", "metadata": { "groups": [ "admins", "users" ] }, "metadata_schema_id": "default_metadata", "payload": { "name": "Keshia Mraz", "given_name": "Keshia", "family_name": "Mraz" }, "payload_schema_id": "default_payload", "status": "active", "status_updated_at": "2022-08-03T11:03:38.33+02:00", "tenant_id": "default", "updated_at": "2022-08-03T11:03:38.33+02:00", "user_pool_id": "caku6lrphdd3cfqro3mg" }, { "created_at": "2022-08-03T11:03:38.343+02:00", "id": "cdcb7a5b-a726-404c-ba66-b8552d9be8fc", "metadata": { "groups": [ "users" ] }, "metadata_schema_id": "default_metadata", "payload": { "name": "Vicente Moen", "given_name": "Vicente", "family_name": "Moen" }, "payload_schema_id": "default_payload", "status": "active", "status_updated_at": "2022-08-03T11:03:38.343+02:00", "tenant_id": "default", "updated_at": "2022-08-03T11:03:38.343+02:00", "user_pool_id": "caku6lrphdd3cfqro3mg" } ] }
Importing/Exporting Tenant's User Data
With Cloudentity system API, you can export and import the users from a specific tenant.
Import Users
Call the OAuth 2.0 token endpoint to get an access token.
Note
If you need help with getting the token, see the Getting started with Cloudentity REST API guide, section Call the token endpoint.
curl --location --request POST 'https://example.com/tenant_id/system/oauth2/token' \ --header 'Content-Type: application/x-www-form-urlencoded' \ --data-urlencode 'grant_type=client_credentials' \ --data-urlencode 'client_id=value' \ --data-urlencode 'client_secret=secret'
Result: Your token is returned. Since you called the system endpoint as a client requesting the
manage_configuration
scope, the token should grant this scope.Call the Import Tenant Configuration endpoint for Identity. Pass user data in the request body.
Request example:
curl --location --request PUT 'https://example.com/api/identity/system/{tid}/configuration' \ -H 'Authorization: Bearer {ACCESS_TOKEN}' \ -d @configuration.json
Note
The
{tid}
path parameter is your tenant identifier.Replace the
{ACCESS_TOKEN}
variable with the access token that you got in the first step.You can either put the payload directly in the request body or point to the file that contains the configuration using the
-d @YourFileName.json
parameter.Result:
204 NO CONTENT
Your tenant configuration is imported and available within Cloudentity.
Export Users
Warning
The Export API is not ready to be used in production environments with a large number of users. Please treat it as a testing/beta functionality.
Call the OAuth 2.0 token endpoint as the System client to get an access token with the
manage_configuration
scope.Note
If you need help with getting the token, see the Getting started with Cloudentity REST API guide, section Call the token endpoint.
curl --location --request POST 'https://example.com/tenant_id/system/oauth2/token' \ --header 'Content-Type: application/x-www-form-urlencoded' \ --data-urlencode 'grant_type=client_credentials' \ --data-urlencode 'client_id=value' \ --data-urlencode 'client_secret=secret'
Result: Your token is returned. Since you called the system endpoint as a client requesting the
manage_configuration
scope, the token should grant this scope.Call the Export Tenant Configuration endpoint for Identity
Request example:
curl --location --request GET 'https://example.com/api/identity/system/{tid}/configuration' \ --header 'Authorization: Bearer {ACCESS_TOKEN}'
Note
The
{tid}
path parameter is your tenant identifier.{ACCESS_TOKEN}
is the token returned in the first step.Result:
200 OK
Your tenant configuration, including Identity Pools with their users, is returned as JSON in the request response body.
Insert Mode
The import endpoint has the optional mode
query parameter available. This parameter defines what happens if there are any conflicts when importing your user records. For example, if a user already exists within Cloudentity and you are trying to import a configuration that already has a user with this ID, there are the following ways Cloudentity can handle the request:
mode
set toignore
(default) - Cloudentity ignores the changes that come from your configuration import.mode
set tofail
- Cloudentity stops import processing and returns an error.mode
set toupdate
- Cloudentity updates the tenant's configuration with the value provided in the request.