Write API vs XML Synchronisation¶
There are a number of key differences between the write API and XML synchronisation jobs, so it is worthwhile carefully considering these and the best approach to use in implementing an integration. Particularly for those familiar with Pure's current XML synchronisation functionality, there are some major differences in paradigm and behaviour that should be considered when implementing using the API. The following sections provide a summary of the considerations with further detail and examples.
Difference of Protocols¶
XML synchronization:
- XML is becoming less common standard
- Includes helper features to match and/or create records if not found
Write API:
- Industry best practice for integrations
- Applied strict logic on how to create or match records
A big reason for implementing the write API is that it is the industry best practice and standard protocol used for building integrations. The XML "load file" format of the Pure synchronisation is a pattern that it still used but for many integration teams there may not have the familiarity or skill-sets to create such XML files, while the CRUD Write API methods with JSON objects is likely to be much more comfortable. Additionally, some "features" of the XML synchronisation (such as the ID lookup and fallback patterns), while helpful for quickly creating records and error handling, are potentially less intuitive and can lead to confusion due to the multiple ways data can be represented and the unintended consequences if this lookup logic isn't well understood.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<organisations xmlns="v1.organisation-sync.pure.atira.dk" xmlns:v3="v3.commons.pure.atira.dk">
<organisation managedInPure="false">
<organisationId>organisation1</organisationId>
<type>university</type>
<name>
<v3:text lang="en" country="GB">Tesla University</v3:text>
</name>
<startDate>2007-10-26</startDate>
<endDate>2050-01-01</endDate>
<visibility>Restricted</visibility>
<nameVariants>
<nameVariant>
<type>shortname</type>
<name>
<v3:text lang="en" country="GB">TeU</v3:text>
</name>
</nameVariant>
</nameVariants>
<profileInfos>
<profileInfo>
<type>organisation_profile</type>
<profileInfo>
<v3:text lang="en" country="GB">
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Mauris faucibus sem vitae nisl dapibus semper. Praesent iaculis lorem turpis, id dapibus mi tincidunt id.
Suspendisse aliquet magna turpis, quis dignissim elit mollis ut. Donec ut imperdiet ligula. Donec luctus malesuada laoreet.
Maecenas interdum dui ornare pharetra tempor. Vestibulum consectetur mollis magna. Vivamus tristique sem in sem imperdiet, laoreet vestibulum turpis commodo.
Duis nulla quam, bibendum eu turpis ut, venenatis imperdiet urna. Praesent sed ante purus.
</v3:text>
</profileInfo>
</profileInfo>
</profileInfos>
<photos>
<photo>
<type>organisation_photo</type>
<photoValue>https://upload.wikimedia.org/wikipedia/commons/thumb/d/d4/N.Tesla.JPG/800px-N.Tesla.JPG</photoValue>
<photoProtocol>http</photoProtocol>
</photo>
</photos>
<phoneNumbers>
<phoneNumber>
<type>phone</type>
<phoneNumber>568425658</phoneNumber>
</phoneNumber>
</phoneNumbers>
<emails>
<email>
<type>email</type>
<email>[email protected]</email>
</email>
</emails>
<webAddresses>
<webAddress>
<type>official_website</type>
<webAddress>
<v3:text lang="en" country="GB">http://elsevier.com</v3:text>
</webAddress>
</webAddress>
</webAddresses>
<addresses>
<address>
<type>visiting_adress</type>
<city>London</city>
<postalCode>SW1V 4NP</postalCode>
<street>Wayne Street</street>
<building>44</building>
<country>us</country>
<subdivision>us/ak</subdivision>
<geospatialPoint>40.64209354784047, -73.78486633300781</geospatialPoint>
<geospatialPolygon>40.64209354784047, -73.78486633300781, 35.64209354784047, 40.64209354784047</geospatialPolygon>
<displayFormat>
Tesla University
44 Wayne Street
London AK, SW1V 4NP
UNITED STATES
</displayFormat>
</address>
</addresses>
<keywords>
<v3:logicalGroup logicalName="keywordContainers">
<v3:structuredKeywords>
<v3:structuredKeyword classification="A" />
<v3:structuredKeyword classification="T/TA" />
<v3:structuredKeyword classification="T/TP">
<v3:freeKeywords>
<v3:freeKeyword>
<v3:text lang="en" country="GB">Inorganic chemistry</v3:text>
<v3:text lang="da" country="DK">Uorganisk kemi</v3:text>
</v3:freeKeyword>
<v3:freeKeyword>
<v3:text lang="da" country="DK">Applied biochemistry</v3:text>
</v3:freeKeyword>
</v3:freeKeywords>
</v3:structuredKeyword>
</v3:structuredKeywords>
</v3:logicalGroup>
</keywords>
<ids>
<id>
<idSource>organisationid</idSource>
<id>organisation1</id>
</id>
</ids>
<costCenters>
<costCenter>costcenter1</costCenter>
<costCenter>costcenter2</costCenter>
</costCenters>
</organisation>
<organisation managedInPure="false">
<organisationId>organisation2</organisationId>
<type>faculty</type>
<name>
<v3:text lang="en" country="GB">Faculty of Science</v3:text>
</name>
<startDate>2007-01-01</startDate>
<visibility>Public</visibility>
<parentOrganisationId>organisation1</parentOrganisationId>
</organisation>
<organisation managedInPure="false">
<organisationId>organisation3</organisationId>
<type>faculty</type>
<name>
<v3:text lang="en" country="GB">Old Faculty of Science</v3:text>
</name>
<startDate>1907-10-26</startDate>
<endDate>2006-12-31</endDate>
<takenOverBy>organisation2</takenOverBy>
<visibility>Public</visibility>
<parentOrganisationId>organisation1</parentOrganisationId>
</organisation>
</organisations>
{
"pureId": 12944506,
"uuid": "187e1d2c-a839-403e-b611-816652235bff",
"createdBy": "atira",
"createdDate": "2019-02-12T12:05:09.038Z",
"modifiedBy": "[email protected]",
"modifiedDate": "2020-02-17T09:03:25.933Z",
"portalUrl": "https://collaborations.elsevierpure.com/en/organisations/187e1d2c-a839-403e-b611-816652235bff",
"version": "e7982d0e32bb630135f4a3c856f1e9aef7fcc5dd",
"name": {
"en_US": "Elsevier Research Groups"
},
"type": {
"uri": "/dk/atira/pure/organisation/organisationtypes/organisation/elsevier_research",
"term": {
"en_US": "Elsevier research",
"da_DK": "!!!Elsevier research",
"nl_NL": "!!!Elsevier research"
}
},
"nameVariants": [
{
"pureId": 12944508,
"type": {
"uri": "/dk/atira/pure/organisation/namevariants/shortname",
"term": {
"en_US": "Short name",
"da_DK": "Forkortet navn",
"nl_NL": "Verkorte naam"
}
}
},
{
"pureId": 12944509,
"type": {
"uri": "/dk/atira/pure/organisation/namevariants/sortname",
"term": {
"en_US": "Sort name",
"da_DK": "Sorteringsnavn",
"nl_NL": "Sorteer naam"
}
}
},
{
"pureId": 12944510,
"type": {
"uri": "/dk/atira/pure/organisation/namevariants/webname",
"term": {
"en_US": "Web name",
"da_DK": "Webnavn",
"nl_NL": "Webnaam"
}
}
}
],
"profileInformations": [
{
"pureId": 12944511,
"type": {
"uri": "/dk/atira/pure/organisation/customfields/organisation_profile",
"term": {
"en_US": "Organization Profile",
"da_DK": "Organisationsprofil",
"nl_NL": "Organisatieprofiel"
}
}
},
{
"pureId": 13004554,
"type": {
"uri": "/dk/atira/pure/organisation/customfields/nature_of_collaboration",
"term": {
"en_US": "Nature of Collaboration",
"da_DK": "!!!Nature of Collaboration",
"nl_NL": "!!!Nature of Collaboration"
}
}
}
],
"photos": [
{
"pureId": 20415605,
"fileId": "MDAxMzc4NDc1",
"fileName": "lion.jpg",
"mimeType": "image/jpeg",
"size": 4007537,
"url": "https://api.elsevierpure.com/ws/api/organizations/187e1d2c-a839-403e-b611-816652235bff/files/MDAxMzc4NDc1/lion.jpg",
"fileStoreLocations": {},
"type": {
"uri": "/dk/atira/pure/organisation/organisationfiles/profilephoto",
"term": {
"en_US": "Profile photo",
"da_DK": "Profilbilleder",
"nl_NL": "Profielfoto"
}
}
}
],
"lifecycle": {
"startDate": "1900-01-01"
},
"visibility": {
"key": "FREE",
"description": {
"en_US": "Public - No restriction",
"da_DK": "Frit tilgængelig - Ingen begrænsninger",
"nl_NL": "Publiek - Geen beperking"
}
},
"customDefinedFields": {},
"systemName": "Organization"
}
Scope of the dataset¶
XML synchronization:
- Deals in complete "current" sets of data
Write API:
- Works on a single record at a time
While the XML synchronisation deals in a single file containing all records, the write API uses a single call to create each record. This means the XML synchronisation processes a complete "dump" of data, whereas the API is used for individual records. An iterative call of the API could be used to process a complete set of the records, but it is not possible to use the XML synchronisation to update just a couple of records (if there is only a couple of changes in the XML then the rest of the content will be ignored, but all records must be present or else they will be "retired" or deleted, depending on the content type). This leads to the next consideration.
Additionally, the synchronisation jobs require all mandatory fields to be specified even if the job isn't updating them. This may mean that data is included in the XML file as placeholder values for which Pure is meant to be the source of truth. This field can be set to Not Sync in the job configuration, but if it was ever accidentally changed, values in Pure would be overwritten with incorrect data. A common example of this would be for an Awards synchronisation for records created in Pure to add budget expenditure records. Even though only the budget values are meant to be updated, the XML file requires a number of mandatory fields, such as title and type, for which Pure is likely the source of truth. The Write API will only update the fields included in the JSON request body, and mandatory checks are performed only when creating new objects, so those fields can be omitted when performing an update. This also means that the content being sent to Pure is cleaner than the XML as there isn't extraneous placeholder content included.
| XML Synchronization expects all records | API can be used to create, update or delete a single record |
|---|---|
 |
 |
Timeliness of integration¶
XML synchronization:
- Normally is scheduled to run daily / weekly
Write API:
- Can be run in a "real-time" fashion
Due to the quantity of records required for XML synchronisation and the processing required to generate the file and then load it into Pure, XML synchronisation jobs are often run on a nightly basis, to collect all changes from the day and update them in Pure. Due to the potential for individual record updates, the Write API can be used in an event driven way, to update records in Pure in real-time. Consideration should be given when building the integration on the volume and relevance of daily changes in the source system. If there is a large volume of relatively meaningless changes to Pure, then this may create unnecessary processing time, while a daily refresh may sufficient.
Locking of records¶
XML synchronization:
- Can restrict which fields can be modified and if records can be deleted
Write API:
- Presently locking of fields and records is not directly supported for write API
The XML Synchronisation jobs are designed to control certain fields of a record where the source of truth is another system (using the "Sync" locking strategy). With the Write API this is presently not possible, meaning that for fields where data has been loaded via the API they can still be edited in Pure. Additionally, if data is manually updated in this fields the integration could overwrite these values with any update. An integration using the write APIs can manage this according to business requirements, but it is a scenario that should be considered as otherwise unexpected behaviour may result and important data may be lost.
There is a workaround for locking fields in the Write API, please refer to this guide for details.
Managing relation lists¶
XML synchronization:
- Only items in list created by synchronisation can be updated
- Users can create new items that are ignored by synchronisation
Write API:
- Can update or delete any item in list regardless of how it was created
- All items must be provided in JSON Array or Pure will assume missing items should be deleted
With XML synchronisation, there is an option to "Lock Relation Lists". If set, this will mean that new items cannot be added via the Pure interface, and the only related items in the list will be in the XML. As mentioned previously, this is presently not supported via the Write API.
If this "Lock Relation Lists" field is set to "Do not lock", then new items can be added manually, and the XML Synchronisation will ignore these items. This means that manually created data in these list relations won't be overwritten by the synchronisation, but it also means that they can't be updated or "taken over". As the Write API doesn't lock or control records, it can update any relation in the list, whether created via the API or manually.
Updating lists via API
The Write API will presume that when you are updating an item in a list that you are updating the list itself as well. This allows you to remove records from the list by not including them, but does mean that items could be accidentally deleted if they aren't included. The full content of the items isn't needed, just the Pure IDs. So for instance, to add a new author to an output, your request body might include the following:
{
"contributors": [
{
"pureId": 6889377
},
{
"pureId": 6889378
},
{
"pureId": 6889381
},
{
"externalPerson": {
"systemName": "ExternalPerson"
},
"name": {
"firstName": "Brian",
"lastName": "Cox"
},
"role": {
"uri": "/dk/atira/pure/researchoutput/roles/contributiontojournal/author"
},
"externalOrganisations": [
{
"uuid": "0ff75777-c32b-4599-8a21-ea15f1cb98f9"
}
],
"correspondingAuthor": false
}
]
}
Flexibility of integration¶
XML synchronization:
- Key use case is synchronising content for another system which is the source of truth
Write API:
- Can be used for creating content from source of truth system
- Can be used to enrich records with certain fields from another system
- Can be used to support driving workflow based updates
By default the XML synchronisation expects to create all the records it manages in Pure. This can be expanded in settings for the synchronisation job to "take control" of a record, but the presumption is that the source of truth is in another system and the XML synchronisation will load those values into Pure. The API can be used for this same use case but also for other options where you might have certain fields that you want to update from another system (for example adding the cost-code to an award once it has been created in the finance system) or performing ad-hoc bulk updates to records where internal processes have changed and fields need to be changed within Pure.
Maturity¶
XML synchronization:
- XML is well established and used by many institutions globally to manage their integrations into Pure
- Changes to synchronisation jobs and xml format are rare
Write API:
- API is in active development
- JSON format can change as development evolves
- All breaking changes are clearly noted in release notes
The XML Synchronisation functionality in Pure has been in the product for many years and used across the world by institutions of all shapes and sizes to manage their integrations. It is well tested in the field, and while it has its limitations it is stable and rarely updated. As the Write API is under current development it is, by nature, more likely to change as improvements and new content types are added. While institutions are beginning to use it for integrations, it doesn't have the same track record that the XML synchronisations have. The Pure development team has worked hard to build a stable and scalable API platform, but it should still be considered by institutions depending on their appetite for risk in the technologies that are used. Any breaking changes to the API format will be clearly outlined in the release notes for that version so make sure to always check this closely if you are building an ongoing integration using the API.
Error handling and logging¶
XML synchronization:
- XML Synchronisation messages are logged internally within Pure
- Searching, export and notification tools provided
Write API:
- Individual API calls return messages to indicate errors
- Responsibility is on caller to handle message
- Allows for integration to standardise logging platforms
Summary logs for synchronisation jobs are kept within Pure with an overview interface showing the number of success, warning and error messages for each run. Additionally, the job can be configured to notify a specified email address if the log breaches a certain threshold of message/error level. There is no such functionality with the Write API, as each create or update call will respond with its own status message. This means when building an integration using the Write API, consideration needs to be given to how these error messages will be handled and surfaced, so that issues can be rectified. This also presents the opportunity for the integration to send the error and status messages to a central logging tool that may be used across all the organisation for other integrations. With the XML synchronisation this is not possible as all logs are stored internally in Pure.
Coverage & Functionality¶
XML synchronization:
- All standard content types in Pure are covered
Write API:
- Content types are being progressively rolled out in new releases
- New functionality being added as rollout continues (e.g. User Roles management which are not supported in XML Synchronisation)
Not all content is presently available in Write API. This may mean that some integrations are not presently possible via the Write API and will have to be via XML. Inversely though, the Write API has the benefit of being under active development, and as such new features are being included that may not have been possible via the XML Synchronisation. A key example of this is the addition of User Roles in 5.23.1, which for the first time, allows programmatic role management against users. This allows for more sophisticated provisioning of users (for instance, roles could be determined based on positions of staff or automatically allocated following approval of a service request). As the write API evolves this criteria is likely to be weighted towards the write API more as more content types are added and additional functionality included.
Broader applications¶
XML synchronization:
- Addresses key use case of synchronising source of truth records from another system
Write API:
- Can be used for adhoc activities, such as bulk changes, data cleaning and merging duplicates
As outlined above, the XML synchronisation is purpose built for loading data from another system into Pure, with a complete set of data provided each time from the source of truth system. The Write API allows for many more applications as it can create and update individual records as well as individual fields on those records. This means that integrations can be more targeted and event driven. For example, an integration might identify newly created applications in Pure, this integration then allocates space for them in the university's cloud storage before calling the write API to add the ID for this storage folder to the application.