Entity Set Sync
Entity Set Sync allows you to transfer a structural portion of your system from one knowledgebase to another. It differs from External System Sync, which is used to keep Agiloft and external systems up-to-date with the same records and data, such as integration with Salesforce, Google, or other third-party systems. Entity set sync is used to transfer the structure of a KB, such as tables, fields, rules, and actions. A common use of entity set sync is to transfer changes from a development or testing KB to your production system.
The Entity Set Sync wizard is a single screen accessed from the admin Setup menu at Setup > Sync > New > Entity Set. When adding a table to the entity set, Agiloft will automatically include portions of other tables which relate to the construction and function of the initial selection. The system will include the specific fields, saved searches, rules, actions and views that are needed to reproduce the functionality of the initial table. This means that selecting one table for the entity set may pull in tables with linked fields from many other parts of the system.
Using a separate development or testing KB allows complex or experimental build without affecting day-to-day use of the system, or unintentionally changing connected but unrelated system behavior. Working with multiple environments might look like this:
|Entity Set||A set of objects, such as tables, rules, color schemes, fields and teams, that is packaged together into a file for transfer to another KB.|
|Source KB||Sync donor knowledgebase; it contains the entity set including tables, field or other structure which you wish to transfer.|
Sync recipient knowledgebase; it will contain the entity set after the sync is completed.
Note: When transferring changes from a development to a production environment, the Source KB is the development KB, and the Target KB is the production KB.
|Direct Sync||Syncing between KBs on the same server.|
|Indirect Sync||Syncing between KBs on using an exported carrier file. This can be done between KBs on the same server or on different servers, but is usually used when KBs are on different servers.|
|Carrier KB file||The knowledgebase carrier file created from the Source KB for import into the Target KB during an indirect sync. Carrier KB files are in XML format, with a file extension of |
Types of Sync
Sync may be direct or indirect. Direct sync is preferred, but to use it, the source KB and target KB must be on the same server. If the KBs are on different servers, you can still sync them, but you need to use the indirect sync method that uses an intermediary carrier file to transfer the entity set. In indirect sync, the carrier file is exported from the source KB and then imported to the target KB.
If the source KB and target KB for the entity set transfer reside on the same server, you can perform a direct sync. You can export an entity set directly from one KB to another by providing login credentials for the target KB in the Sync wizard. This is the simplest way to perform sync and only requires admin level login credentials for both KBs.
See Direct Sync for detailed instructions.
If the source KB and target KB are on different servers, you must perform an indirect sync, also known as sync via carrier file. Note that Carrier KB files can only be imported into other knowledgebases running the same release version or higher. A carrier KB created on a higher release will not import successfully into an older KB.
In an indirect sync, the entity set is created in the source KB and exported to an XML file, known as the carrier KB file. You will download the file and transfer it to the target server. Then, the carrier file is imported into the target KB on the other server.
See Indirect Sync for detailed instructions.
Create an Entity Set
Direct and indirect sync both require an entity set. Start by creating an entity set that contains everything you need to sync.
- As an admin user, navigate to Setup > Sync and select New > Entity Set.
- In the wizard, enter a Name for the entity set.
- Select which entities are included in the set. Available entities include: tables, choice lists, color schemes, pictures/images, groups, teams, attached files, and global variables.
- Customize the entity set as needed by marking entities as Explicit or Implicit, or by adding or removing fields, rules, etc. See Customize the Entity Set.
- If applicable, select the Transfer Only Explicit Entities checkbox. See Explicit and Implicit Entities.
Note: If you do select this option, leave the Update Implicit Linked Fields option selected to prevent broken relationships after sync. This option is used only for advanced troubleshooting.
- Click Finish to save the entity set.
Explicit and Implicit Entities
When you add an entity to the set, it is added explicitly. Explicit entities are always sent to the target KB. If an explicit entity doesn't exist in the target KB, sync will create it; if an explicit entity does exist in the target KB, sync will update it to match the entity in the source KB. Selecting an entity explicitly tells the system to make sure the target KB has an entity configured the same way.
However, many entities have relationships to other entities in the system, and they do not work as expected without those other entities. For this reason, when you add an entity explicitly, the system identifies any other entities that are necessary for the table to work as expected, and the system adds these other entities to the set as Implicit. Implicit entities are sent only if they do not exist at all in the target KB; if sync finds an implicit entity already exists in the target KB, sync will not update it at all.
|Entity Not Found in Target KB||Entity Found in Target KB|
|Explicit Selection||Entity is created in target KB||Entity is updated in target KB|
|Implicit Selection||Entity is created in target KB||No change in target KB|
You can choose to send only the explicit entities in your set. This is useful to minimize the size and duration of the sync, but for large changes, it can be difficult to make sure all the necessary entities are sent. If you do choose to send only explicit entities, carefully review any implicit entities that were added to the set and evaluate whether they need to be made explicit and sent as well.
Customize the Entity Set
Sync takes more time to complete when the set includes more entities, but you might encounter more sync difficulties when essential entities are not included. When you prepare an entity set, you must carefully consider what you need to send and what you can exclude.
Add Tables to the Entity Set
There are three options for adding tables to an entity set:
- Add empty tables.
- Add tables and all fields.
- Add tables and all entities. This option adds all child entities of the table, including fields, saved searches, rules, email templates, inbound email accounts, and other settings.
Tip: To send a small change with minimal build, add the empty table first and then use Customize to select the relevant table items.
Add Other Entities to a Set
- Locate the type of entity to be added, such as choice lists, groups, teams, images, etc. Note that when sending groups, you must add the group explicitly if you want to sync the group's permissions. For more information, see the Group Permissions section.
- Click Add Entities to expand the list of available items.
- Select the entities you want, then click Add Entities or Make Explicit to finish adding them to the set. The screen will refresh and additional entities may be added implicitly.
- You can change the status of previously added entities by clicking Make Explicit or Make Implicit.
When you add an entity to the set, the system automatically adds implicit entities that are necessary to reproduce the table structure and functionality. You can review and, if necessary, override these implicit entities by clicking Remove next to an item, or Customize next to a table to add or remove fields, searches, the table layout, and so on.
Be cautious when removing items that were added by the system. It is usually best to keep all items which are automatically added to an entity set. Removing these items may cause an incorrect sync transfer. The system will warn the user of any dependencies broken by removing an item. For example:
You can use this warning or the Entity Set Dependency Report to investigate why the unwanted item was originally added, and potentially remove the causal relationship. To successfully remove an entity from the set and prevent the system from adding it back, you must identify how your selected set relies on the unwanted entity and make changes to resolve that relationship. For example, you might need to remove an action you don't want to send from a rule you do want to send, to prevent the system from sending both.
If you do remove an implicit entity added by the system, but you do not remove the relationship that caused it to be added, it might be added back automatically when you save the set. See Automatic Refresh of Entity Set Dependencies.
Automatic Refresh of Entity Set Dependencies
Entity sets are automatically updated by the system in certain situations. These might override changes you make when customizing an entity set. For example, you might remove an entity that is later added back automatically to make sure the table is functional after sync.
Dependent items are automatically added to an entity set.
If you add a linked field to an entity set, the system will transfer the associated saved search and action which are part of the linked field settings.
Add an empty table to an entity set, and the system may add additional tables. Even without custom fields, some tables have a relationship to other tables such as Email, Email Marketing, Team, and People, which the system will identify and add to the set.
Changes to items included in an entity set will be refreshed when the entity set is next opened for edit.
An entity set includes a linked field based on a saved search. The saved search is later modified or replaced. When the entity set is reopened, the new or modified saved search is included in the set.
This feature is designed to make working with sync easy: just add the key components needed, and the system identifies and includes related and dependent entities.
Entity Set Dependency Report
This feature is used to examine why additional entities have been added automatically or implicitly to an entity set. Once you understand which relationship is causing the system to add the entities, you can determine whether to keep the entity or change or remove the relationship.
To run a dependency report:
- Go to Setup > Sync.
- Select the entity set and click Report entity set dependencies.
Reports are divided into four tabs:
- Subtypes, or tables
- Columns, or fields
The report shows the logical chain between items added to the entity set by users and those that the system adds implicitly due to dependencies. For example:
Sync is a complicated process that can produce unexpected conflicts and challenges. It is critical to a successful sync to prepare both the target and source KBs for the process, and to do as much as you can to anticipate issues that may come up. This section details all the steps you should take to prepare for sync.
Verify KB Health and Integrity
Use Verify KB to check the overall integrity of both the target KB and the source KB before performing sync. The system will review linked field schema, actions, tables, and teams and generate an on-screen report of any issues.
- Go to Setup > Sync > Verify KB.
- Allow some time for the report to run.
- When the report is finished, review it and resolve any issues.
- Repeat these steps until the KB is verified to have no issues.
From the admin console, you can also perform Linked Field and Schema Repair, which identifies inconsistencies and broken linked fields that may lead to a sync failure. Hosted customers may contact Agiloft Support to perform LF and Schema repair.
- In the admin console, go to Setup > Repair.
- Select a KB for repair.
- Click LF and Schema Repair.
Make sure to perform these steps for both the target and the source.
Disable Linked Field Auto-propagation During Sync
Sync has a higher risk of failure with large KBs and entity sets where tables in the target KB contain a lot of records and massive propagation on linked field columns. In this situation, it can be useful to disable linked field auto-propagation during the sync. This can save time and prevent sync failures.
To do this, open the Using the Setup Assistant and navigate to the Application Server section. Enter the following text into the Jboss options field:
After adding the property, click Apply changes, and after restarting, Agiloft sync will not automatically propagate linked fields.
Note that this property will survive server upgrades. If you wish to disable the property after sync, just add the following property into the same Jboss options field:
Test Sync on a Copy KB
The best way to see what to expect from a sync is to create a copy of the target KB and sync to the copy. This offers you a complete picture of what to expect.
- Create a copy of the target KB. Note that if the target KB is on a different server, it can be useful to perform these steps twice: once with a copy KB on the same server as the source KB, and once with a copy KB on the same server as the target KB. The more thorough your tests can be, the more likely you are to efficiently resolve any issues in the final sync.
Tip: Perform all preparation steps in this section on the copy KB if possible.
- Test the sync operation using the steps in the sections below. Syncing to a copy KB will help discover any inconsistencies between KBs that must be fixed before syncing to a production KB. The KB might contain obsolete actions that should be deleted, such as unused conversion actions with no conversion selected, or broken actions resulting from the removal of a team, group, or saved search. It's important to clean up these objects before performing sync.
- Check for problems and unique attributes. Run the test sync and carefully read error messages to resolve any issues. For example, if the Company Name field is designated ‘unique’ in the source KB but not the copy, sync will fail if it finds duplicate entries in records in the copy KB.
- Resolve issues and take notes as you go. In the copy KB, investigate and resolve the conflicts, taking careful, detailed notes of each step you take. These notes will be your plan for the final sync, and they are critical to making sure everything will work as expected. For more information about conflicts between entities, see UUID Recognition and Sync Conflicts.
- Repeat these steps to test your plan in a new copy of the target KB. After sync, follow your plan exactly, and then review and test the result to make sure the KB works as expected and data appears appropriately. If you find anything in the KB that does not meet your expectations, investigate, plan, and test a solution, and add that solution to your plan for the final sync.
You might need to perform many test syncs before everything is ready for the final sync. Thorough preparation is essential to making sure the target KB is in good condition after the final sync.
Schedule the Final Sync
Treat sync, especially a large sync, like a server upgrade. Secure a generous time window outside business hours, so you will have time to complete the sync, resolve conflicts, and run basic checks. Also make sure to back up the target KB shortly before the final sync.
If possible, it is helpful to:
- Stop background services and emails from the KB, which can cause sync failure
- Log users out to prevent anyone from modifying a record being synced
- Disable linked field propagation, for sync with very large KBs
- Perform the sync with a user account that won't be needed for anything else during the scheduled window
- Perform the sync on a computer with consistent network connectivity
Sync the Entity Set
Once the entity set has been created, all relevant entities have been added, and all necessary preparations have been made, you can begin your direct or indirect sync.
Direct sync is used between KBs on the same server. To perform direct sync, either export from the source KB or import from the target KB.
You do not need to complete both sets of steps below. There is no functional difference between the two. You can perform direct sync from either KB. Choose one set of steps to follow.
To run a direct sync from the source KB:
- Log into the source KB as an admin.
- Go to Setup > Sync.
- Select the entity set.
- Click Export to open the Sync Export wizard.
- On the Source tab, select Transfer to existing KB.
- The drop-down list shows all knowledgebases on the same server. Select the name of the target KB.
- Enter admin-level login credentials for the selected target KB.
Click Export Now to begin exporting changes. It may take several minutes to transfer changes depending on the size and number of entities in the set. You may not log out of the KB while sync is working.
- Log into the target KB as an admin.
- Go to Setup > Sync > Import to open the Sync Import wizard.
- On the Source tab, select Transfer from existing KB.
- The drop-down list shows all knowledgebases on the same server. Select the name of the source KB containing the entity set. It may take some time to load the drop-down list.
Note: if the Security: Show KB Names variable is set to No, this list will be hidden and you will need to add the KB name manually.
- Enter an admin-level login credentials for the selected source KB.
- Click Next.
- On the Entity Set tab, select the entity set from the drop-down provided.
Click Import to begin the sync process. It may take several minutes to transfer changes depending on the size and number of entities in the set.
Indirect sync is used between KBs on different servers. Given the choice, direct sync is usually preferred. Indirect sync is more complex because it involves creating, transferring, and importing a separate carrier KB file.
Create and Export a Carrier KB File (Indirect Sync)
If the source KB and target KB are on different servers, a carrier KB file must be used to transfer the entity set. Follow these instructions to create the carrier KB file using the sync export wizard.
- Log into the source KB as an admin.
- Go to Setup > Sync.
- Select the relevant entity set, then click Export to open the Sync Export wizard.
- In the wizard, select Create export file.
- Click Export Now.
- Leave the progress window open; it may take a few minutes to create the file. Once completed, you will see a confirmation message with an option to download the export file and to view a transfer report.
- Save the file locally and note its location.
Import from a Carrier KB File (Indirect Sync)
You will need the carrier KB file available on your machine in order to complete an indirect sync.
- Log into the target KB as an admin.
- Go to Setup > Sync > Import to open the carrier KB file.
- On the Source tab, click Browse.
- Find and select the carrier KB file and click Upload.
- Click Next to move to the next tab.
- On the Entity Set tab, select the entity set from the drop-down.
- Click Import to finish. A progress message is displayed while the entities are transferred. It may take a few minutes to transfer changes depending on the size and number of entities in the set.
For example, the following file contains the entities for the 2017 default Look and Feel:
After the file has been imported, Setup → Look and Feel → Manage Power Schemes will show an L&F named "2017 Calibri Default"
Setup → Look and Feel → Manage End User Schemes will show an L&F named "2017 Calibri Default EUI"
If this is a test or final sync, use your plan to resolve any issues or conflicts from the sync. When all issues have been resolved, check the affected tables and workflows to make sure they work as expected. If you took any of the suggested precautions before the final sync, reverse them when it is complete:
- Resume background services and emails from the KB
- Make sure users are allowed to log in
- Enable linked field propagation if needed
Once sync is complete, all users in the Admin Team will receive an email with the results of the entity set. The email subject will be: Sync result for KB <KB LABEL>. The email body will be similar to the following message, including a detailed report of all the structures transferred in the sync:
Status -> COMPLETE
Result message Transfer completed successfully.Click here to see a detailed transfer report
Click here to close this window.
In case of sync failure, see Entity Set Sync Troubleshooting with Error Logs.
The Compare feature generates a report of differences between two selected KBs based on a selected entity set. Compare is only available for two KBs hosted on the same server. The Compare feature is available from both the power user interface and the admin console.
Users typically perform a KB comparison in the following cases:
- Transferring changes from a development to a production environment. Use Compare in this case to discover any changes in the KB, what these changes are, and whether sync is necessary. For more information about environment management, see Working with Multiple Environments.
- Checking the results of a just-performed sync. After a full development-to-production sync, use Compare to check that the comparison report is empty and verify that all items were transferred correctly.
- Comparing two completely different KBs to reveal how many differences they have. For example, you can see how far two systems are from each other, and get a sense of how much cleanup would be needed to perform a successful sync operation between them.
Comparing whole KBs from the admin console
- Log in to the Admin Console and go to Setup > Debugging > Compare KBs.
- Select the source and target KBs you want to compare.
- The system will generate a report in .
xlsformat listing the differences and sorting by item type.
Comparing partial KBs (entity sets) from the admin interface
- Log in to the power user interface as an admin user.
- Go to Setup > Sync, select an entity set, and click Compare.
- You will be prompted to select a second KB for comparison.
- Here you can compare only the items of interest in the entity set.
- You can download or open the system-generated report in
Entity Sync Limitations
The following sections describe the known limitations of sync.
In general, sync does not support deleting objects in a target KB.
An object is included in an entity set and synced to a target KB. The item is later deleted in the source KB and entity set. When you run sync again, the object will not be deleted in the target KB. Instead, its status will be marked 'Free' in the transfer report. Thus, 'Free' items are objects which are not found in the source KB due to deletion.
Transfer of groups and permissions during sync is minimized to save time and resources during sync. Group permissions are transferred only if one of these are true:
- The groups are included explicitly in the entity set
- The target KB contains the same groups as the source KB
Even when group permissions are transferred, only a direct sync of explicitly listed groups will sync all the permissions in the group. Indirect sync transfers permissions only for entities that are included in the entity set. For example, let's say you have group A in your source KB with access to tables B, C, and D, and you add group A explicitly to your entity set. You also add table B explicitly, and table C is added implicitly, but table D is not included at all. If you perform a direct sync with this entity set, group A is synced to the target KB with access to tables B, C, and D, even though table D is not included in the set. However, if you perform an indirect sync with this entity set, group A is synced to the target KB with access to tables B and C, but the permission to access table D is not synced. This applies to permissions at all levels.
Groups may be automatically added to an entity set depending on the table and field dependencies. If the number of user groups is large, we recommend performing an entity set sync with all components (tables, fields, related items) except groups. Permissions can be transferred afterwards in a separate direct sync action.
If an entity set contains some (or all) groups selected for sync, all permissions for previously synchronized tables will be correctly transferred for each group.
Sync does not support the deletion of layout tabs. For example, if all fields are moved from one tab to another and the empty tab is deleted in the source KB, it will not be deleted in the target KB. The target KB table will contain the empty deleted tab, but all fields will be correctly removed to the other tab as in the source KB.
Sync does not support renaming layout tabs because layout tabs have no UUIDs. If a tab is renamed in the source KB and then synced, the corresponding table in the target KB will have two tabs: an empty tab with the previous name, and a tab with the new name holding the corresponding fields.
Entity set sync does not support record transfer, except in special cases:
- Question records used in surveys
- Records which are used as default values for linked fields or inbound email settings
To sync records between KBs, you can use a scheduled Import/Export action.
UUID Recognition and Sync Conflicts
UUIDs (universally unique identifiers) are used to identify groups, teams, saved searches and other items within Agiloft. During sync, the UUID helps identify and match objects in the source and target KBs.
If the system is unable to identify a match based on UUID, typically because the KBs are very different or came from different sources, the object's name is used to resolve a conflict.
Let's take a simple case of UUID conflict involving a saved search. Imagine two knowledgebases named "Alpha" and "Beta".
In Alpha, we created a saved search called "Restrict Test" with a filter defined as follows:
Field A = value 1
In Beta, we manually create a saved search with the same name using a different filter:
Field B ! = $formula1
Although these searches have the same name, they have different UUIDs and are different searches. How does sync behave in this case?
When a user syncs the "Restrict Test" saved search from Alpha to Beta, the system compares UUIDs and determines that they are not a pair. In Beta - the target KB - the saved search will be renamed to "Restrict Test (1)" and the "Restrict Test" saved search from Alpha will be transferred as "Restrict Test".
Any settings in the target KB which previously used "Restrict Test' will be correctly updated to use the "Restrict Test (1)" filter. In other words, the changed label is propagated throughout the system, and both saved searches are available in the target KB.
Field conflicts are resolved in a similar manner. If a source and a target KB have two fields with the same name but of different data types, the field in the target KB is renamed with a numeric suffix, for example the field
my_address is renamed to
my_address0. The source KB field is transferred with its original name. Dependencies are transferred correctly: all rules, searches, fields and so on which referenced or depended on the renamed target KB's field will correctly refer to the renamed field, in this case
Conflicts based on mismatched field types can be detected using the Compare feature. They are reported as follows:
- Types of Sync
- Create an Entity Set
- Sync Preparation
- Sync the Entity Set
- After Sync
- Entity Sync Limitations
- UUID Recognition and Sync Conflicts