This is an internal documentation. There is a good chance you’re looking for something else. See Disclaimer.

Elasticsearch

Configuration

To configure which search engine should be used the nice2.enterprisesearch.searchEngine application property may be used. Valid values are elasticsearch and solr.

nice2.enterprisesearch.searchEngine=elasticsearch

If elasticsearch is selected, the following properties may be used to configure the connection to the elasticsearch server / elasticsearch index.

nice2.enterprisesearch.elasticsearch.hostName=es1.stage.tocco.cust.vshn.net
nice2.enterprisesearch.elasticsearch.port=443
nice2.enterprisesearch.elasticsearch.indexName=index-test
nice2.enterprisesearch.elasticsearch.username=user-test
nice2.enterprisesearch.elasticsearch.password=xxxxxxx

To connect to our vshn elastic search server you may need to use the following command:

ssh tocco-proxy@git.tocco.ch -L 1443:es1.stage.tocco.cust.vshn.net:443 -N

Increase field limit

When an index is newly created, the total field limit needs to be increased. Due to a bug in the elastic search library this has to be done manually as followed:

PUT https://{hostName}/{index-name}/_settings
{
      "index.mapping.total_fields.limit": 10000
}

For example:

PUT https://es1.stage.tocco.cust.vshn.net/index-test/_settings
{
      "index.mapping.total_fields.limit": 10000
}

Migration

When migrating from solr to elasticsearch the index needs to be recreated. For small customers this can be done at prod migration.

For large customers, this can be done in advance and the pre-built index can be fixed with the FixIndexTask.

See 3.3: Elastichsearch

Changes to Index-Priority

Index priority had to be amended for elastic search. If an entity should not be indexed index-priority=-1 is still correct. For all other entities the index-priority now starts at 1 instead of 0. This may require changes to customer specific entities. If an entity model has index-priority=0 the index-priority should be set to index-priority=1.

Local Development

When developing locally the FakeFulltextIndexService is used by default. This service works with using a direct db search instead of using a real elasticsearch instance. If a real elasticsearch backend is required (e.g. to debug / fix issues with the elasticsearch integration), a connection to a elasticsearch backend needs to be configured as described in Configuration.

Used endpoints

Mappings

The mappings endpoint is used when setting up an index.

PUT https://{hostName}/{index-name}/_mapping
{
    "properties": {
        "pagerank": {
            "type": "rank_features",
            "positive_score_impact": true
        },
        "completion_suggestions": {
            "type": "completion"
        }
    }
}

Search

The Search-API and Count-API are used to search. For searching, tocco always builds a query string for its searches. Per default only 10000 search results are found. If you need all index entries for a given query (e.g. for the index fix task), Integer.MAX_VALUE must be passed as limit. If a query has no limit, multiple search requests are used. For this deep pagination the search-after param is used.

Indexing

If a single entity is saved (inserted or updated) the Index-API endpoint is used to add / update it in the index. If multiple entities need to be added / updated (e.g. when recreating the index) the Bulk-API is used to index multiple entities in a single request.

Deleting

If a single entity is deleted, the Delete-API is used to remove the index entry. If multiple entries need to be removed from the index (e.g. the whole entity model / multiple pks when fixing the index) the Delete-By-Query-API is used.