This is an internal documentation. There is a good chance you’re looking for something else. See Disclaimer.

Duplicate

Config

The way duplicate entities are identified can be configured in the entity Duplicate_config. Duplicate_field entities are grouped into Duplicate_field_group entities. Fields in a group are combined as AND conditions, while groups themselves are combined as OR conditions. For each Duplicate_field a field to check and the Duplicate_comparison_type have to be set. Currently there exist two options:

  • exact

    • check fields for equality of their values

  • fuzzy

    • check fields for the similarity of their values by comparing their trigram scores

    • by default a similarity of 0.6 is required (see documentation of pg_trgm for what that exactly means)

    • required similarity can be set in nice2.duplicate.min.trigramSimilarity in application.properties

    • accents will be ignored (e.g. “ä” will be treated the same way as “a”)

Configs can be activated and deactivated through the actions on their list and detail forms. After activating a config, a search for duplicates in the system is started. The results can be found as Duplicate entities. These are kept up to date as long as the config is active. When a config is deactivated, all duplicates generated by that config are deleted.

For each entity model, a filter can be contirbuted by setting a relation and unique ids to look for. Entities that contain any of the ids in the defined relation are excluded from the duplicate search. See example at the bottom of this page.

At this moment, Duplicate_config can only be created for User and Address.

Enabling duplicate search for new entity models

Note

Check the examples below if one of the steps is unclear.

  1. Add dependency on :core:duplicate module in build.gradle of the corresponding module

  2. Define relation to filter new entity model by in a @Configuration class.

  3. Add initial values for Duplicate_entity_model for the new entity model.

  4. Create relations from Duplicate to new entity model, like Duplicate_relAddress.

  5. Add the above relation to the detail form of the new entity model.

  6. Deny all write rights on this new relation.

  7. Add rights for manager role to see duplicates.

  8. Set DuplicateListener to run for new entity model.

  9. Create Duplicate_config, Duplicate_field_group and Duplicate_field entities.

  10. Activate the new Duplicate_config once you’re satisfied with it.

Initial values example for User
Duplicate_entity_model:
  localized:
    label: entities.User
  fields:
    unique_id: user
    sorting: 10
    active: true
  config:
    identifier: unique_id
    priority: 30
Field example for Duplicate_relUser
<field data="relDuplicate" scopes="read,update" ignore-copy="true"/>
Relation example for User
<?xml version="1.0" encoding="UTF-8"?>
<relation xmlns="http://nice2.tocco.ch/schema/relation.xsd">
  <source entity-model="Duplicate">
    <delete cascade="no"/>
  </source>
  <target entity-model="User">
    <delete cascade="no"/>
  </target>
  <cardinality>n:n</cardinality>
</relation>
ACL example for User
entityPath(Duplicate, relUser):
    deny access(write);

entityPath(User, relDuplicate):
    deny access(write);

entity(Duplicate):
    grant access to usermanager;
Configuration example for User
@Bean
public DuplicateFilterContribution userDuplicateFilterContribution() {
    DuplicateFilterContribution contribution = new DuplicateFilterContribution();
    contribution.setModel("User");
    contribution.setRelationName("relUser_status");
    contribution.setValuesToExclude("archive");
    return contribution;
}

@Bean
public ListenerContribution<EntityFacadeListener> userDuplicateListenerContribution(@Qualifier("duplicateListener") EntityFacadeListener listener) {
    ListenerContribution<EntityFacadeListener> contribution = new ListenerContribution<>();
    contribution.setFilter("User");
    contribution.setListener(listener);
    return contribution;
}