By Paul van Gool. More and more organizations that have implemented Identity Governance and Administration (IGA) solutions choose to maintain user and authorization data in business applications using all kinds of integration methods. In the field I see a lot of different methods of integrations that do not use any standards. In most cases developers need to build custom API’s and/or custom connectors so IGA systems are able to feed target applications, which highly increases the cost and lead time of these implementations. Luckily, an integrations standard, SCIM, is on the rise.
SCIM, or System for Cross-domain Identity management, is an open standard for the managing user and authorization lifecycle in target applications. Because SCIM is an open standard, it reduces cost and complexity, resulting in faster and more economical onboardings of target applications onto IGA platforms. This blog will focus on the implementation of the SCIM service provider API, SCIM client implementations are out-of-scope. More in-depth information of the SCIM protocol can be found on the following site: http://www.simplecloud.info/
During my professional career, I have tested around 40-50 SCIM 2.0 API implementations. These tests result in a lot of different issues and deviations from the SCIM standard. This blog is dedicated to the most common implementation mistakes I have found during the testing of SCIM 2.0 compliant APIs, resulting in the following 10 pitfalls:
- Pitfall 1: Testing and troubleshooting with a generic API IDE.
- Pitfall 2: Core schema modifications.
- Pitfall 3: Extending the core schema without using schema extensions.
- Pitfall 4: Confusion between the Id and ExternalId attributes.
- Pitfall 5: Malformed error response.
- Pitfall 6: Missing or incorrect metadata.
- Pitfall 7: Missing configuration endpoints.
- Pitfall 8: Using PATCH as a (required) method to update objects.
- Pitfall 9: Case sensitivity.
- Pitfall 10: Management of a complex or large service provider authorization models.
Before going into the pitfalls, I would like to state that a lot of effort (and of course all the mentioned pitfalls below) can be avoided by using the Traxion SCIM Gateway. This API front is fully SCIM compliant and can easily make your application SCIM complaint, without building and maintaining the API yourself.
Pitfall 1: Testing and troubleshooting with a generic API IDE
Testing and troubleshooting a SCIM Service Provider with an API IDE (Integrated Development Environment) is very straightforward as you can inspect the raw request and response, however such tooling will not take SCIM 2.0 protocol compliancy into account. A common mistake is to assume that a SCIM API is properly tested when all tests run successfully within an IDE. However, an IDE can differ significantly from an actual SCIM client. This will result in running into the most pitfalls mentioned in this blog, so I feel this is the most important pitfall.
Therefore, I would recommend service providers to test their SCIM API with a fully SCIM compliant test tool, like the Traxion SCIM Test Client. Optionally you can use a web debugger tool like Fiddler in between of the SCIM Test Client and the SCIM Service Provider. This will give insight into the actual Requests send by the SCIM client and the API responses.
Pitfall 2: Core schema modifications
The SCIM 2.0 specification describes a core schema. Although this core schema is publicly available, I have seen many developers trying to create the schema from scratch or altering the standard schema heavily, resulting in amongst others:
- Missing attributes.
- Renamed objects and/or attributes.
- Changed attribute types.
- Changed mutability, required and/or read-only flags.
- Dropping canonical types.
- Creating duplicate attributes.
All these small mistakes cause unexpected behavior that need to be analyzed, this takes up precious time that will delay the implementation. I advise to use the unmodified SCIM 2.0 core schema. If some objects and/or attributes are not used by your application, then I recommend to simply leave those attributes empty and do not process incoming data for these attributes. You can find the core schema included in RFC7643.
Pitfall 3: Extending the core schema without using schema extensions
When additional attributes are required, the User, Group and EnterpriseUser schemas should not be modified. The SCIM specifications states you should not add these attributes within the core schema itself but instead use a schema extension. As an example, you can have a look at the definition of the EnterpriseUser extension and use this extension as a reference for your own extension. An example of an attribute which is commonly added via an extension is the description attribute for a group. For more information see RFC7643 “3.3. Attribute Extensions to Resources”. When additional resources need to be made available (e.g. an organizational unit), please follow RFC7643 “3.2. Defining New Resource Types”.
Pitfall 4: Confusion between the Id and ExternalId attributes
Sometimes there is some confusion about the difference between the Id and ExternalId attributes and what system is responsible for which and/or whether ExternalId must be stored. I have seen examples where this leads to the mismanagement of these attributes. The following explains this:
|Id||Service Provider||Must be assigned by the Service Provider during creation of a resource. Each resource must have a unique and immutable id that is unique over all resource types. Most common used format is a GUID.|
|ExternalId||Client||Can be assigned by the client and depends on the processes of the client. In case there is an ID specified it must be stored in the data storage of the Service Provider. A commonly used example is the identifier of the corresponding identity in an Identity Governance & Administration (IGA) system.
An example of the usage of this field is in a rejoiner scenario. A re-activation of a soft-deleted account should not be based upon a username or e-mail address, because some company’s re-use these attribute value for new users after the old user has left the company, but based upon the unique and immutable ExternalId value.
Pitfall 5: Malformed error response
When testing SCIM Service Providers, I see a lot of malformed error responses when issues occur. The SCIM specification dictates what the error response should look like. It is common flaw in implementations that the error response is not compliant, causing SCIM clients to present you a general non-useful error message, this makes troubleshooting more cumbersome. The required definitions for error messages can be found in RFC7643 “3.7.3. Response and Error Handling”.
Pitfall 6: Missing or incorrect metadata
The SCIM protocol requires that each resource is assigned certain meta data (e.g. resourceType, created, lastModified, location, totalResults) which is returned by the Service Provider. It is a common mistake to return incorrect meta data causing unexpected behavior.
The most common mistake here is an incorrect totalResults return value on a filtered GET. Service providers should return the total number of objects of a resource type, not just the results of the filtered GET. For example, if the total number of objects inside the user endpoint is 1912, and the filtered GET states “startIndex=151&count=50”, the totalResults return value should be 1912, not 50. See RFC7643 “3.1. Common Attributes” for a description of all required the metadata types.
Pitfall 7: Missing configuration endpoints
The first 30 SCIM connectors I have tested where all build specifically for the customer/project I was working for at the time. However, now SCIM 2.0 complaint API implementation are becoming more widely used, I also get to test implementations that are already used by other SCIM clients. In practice, most of these already implemented SCIM API’s are developed for an Azure AD integration.
Unfortunately, the Azure AD SCIM component from Microsoft does not require endpoints that are defined as mandatory by the SCIM specification. Most of these missing requirements are covered by the other the pitfalls of this blog, however all of these implementations seem to miss the following 3 required configuration endpoints.
In this endpoint the service provider defines specifications compliance and the types of authentication they support to the API (most commonly used is basic authentication, Bearer token authentication and Oauth 2.0authentication). Authentication for this configuration endpoint is not required, unlike the other endpoints:
The Service provider defines the supported resources types in the ResourceTypes endpoint, like Users, Groups, and all Schema extensions. Most common mistake here is that service providers set the required flag to ‘true’ on non-mandatory resources types in the ResourceTypes endpoint. Best example here is the requirement of the EnterpriseUser extension if no attributes from this extension are supported/required. My advice is to always check if attributes in specific extensions are required for creating users, and only then setting the requirement of the extension to ‘true’.
The endpoint where the service provider defines the fully supported schema, like recommended in Pitfall 2 & 3 it is best to implement the core SCIM schema and only supplement it using schema extensions.
Without these three discovery endpoints most SCIM implementation will be impossible because the connecting side will never be able to retrieve all the variables and requirements set by the service provider.
Pitfall 8: Using PATCH as a (required) method to update objects
The SCIM protocol describes two methods for updating a resource:
1. Replace (PUT), replaces the complete resource with a newer version. The SCIM client needs to provide all (updatable) attributes during the update-operation.
2. PATCH, updates the resource to a newer version with only the provided (updatable)
attributes. The SCIM client needs to provide at least one (updatable) attribute during the
Method 1 (replace) is required for the service provider to implement, method 2 (patch) is optional. My experience until now with several SCIM implementations having Patch implemented, is that the quality of these implementation differ from bad to mediocre. This mostly comes from the fact that changes pushed through by using PATCH are not processed correctly, because processing these type of calls are more complex than replace calls, and a lot of troubleshooting/rework is required to get this to work properly.
Pitfall 9: Case sensitivity
All information (of type “string”) send by SCIM implementations to the Service Provider’s API is
case sensitive, meaning that the SCIM Service Provider should store and return data in the same casing as how it’s provided. Not doing so will results in an indefinite loop of request to the SCIM API with requests to alter the casing of certain attributes.
Pitfall 10: Management of a complex or large service provider authorization models
Using the group endpoint, a SCIM implementation should be in full control over all authorizations grantable in a target application. The service provider may refer to these authorization grantors as groups, authorizations, rights, roles Etc. However, SCIM just refers to anything that grants a user any form of authorization as a group.
If the service provider has multiple authorization grantors, it is advised to implement a RBAC model of groups that can be managed through the SCIM endpoint. If this still leads to a complex set or high number of groups, it is advised to add a ‘category’ like attribute to the groups endpoint, using this attribute you can create grouped groups for easier management by the IGA system. I advise to add no more than 20 groups to a single category.
In conclusion, the most common implementation errors can be avoided by making sure these 10 pitfalls are not present inside of the SCIM 2.0 API Implementation. My advice to service providers would be to use a SCIM specific test tool, like the Traxion SCIM Test Client, to fully test the API and its capabilities. Even though the Test Client is Windows based, it would be my recommendation to consider investing in a Windows system supporting the Test Client. This will save a lot of effort during the implementation of the API, resulting in reduced lead time and costs.
Like mentioned before, a lot of effort (and of course all the mentioned pitfalls) can be avoided by using the Traxion SCIM Gateway. This API front is fully SCIM compliant and can easily make your application SCIM compliant, without building the API yourself. More information on the Traxion SCIM Gateway can be found here.
Paul van Gool is a Consultant at Traxion specializing in creating integrations between IGA platforms and target applications using SCIM and other protocols and standards. You can contact him for questions regarding this blog on his e-mail: firstname.lastname@example.org