Skip to content

Chart-related subscription history data access and sampling optimization

There are several performance-related problems with the subscription history data sampling implemented in https://gitlab.simantics.org/simantics/platform/-/blob/master/bundles/org.simantics.charts/src/org/simantics/charts/Charts.java#L103-112.

The worst thing is that the amount of excess work grows linearly with the amount of subscriptions defined in the active model and each Charts.createHistorySamplerItem2 invocation will always read all the subscription metadata .txt files from disk, which is terribly expensive compared to caching them.

Caching of course makes the ItemManager state handling more complex to ensure that up-to-date data for subscriptions is always available. However it is definitely worth it in this case. The simplest way to keep the caches coherent is to invalidate then entirely upon any detection of change (force rebuild). In practice the set of items stored by a HistoryManager doesn't change all that often so cache will hit most often and rebuilds should be few.

Another thing to optimize is the use of ItemManager.search to find the history items whose variableId matches the item searched for. Currently it just always loops over all the item Beans, reads the variableId field through Bean class mechanisms and compares it to the searched for value. If there are hundreds or thousands of items, the time spent in this will linearly grow as well. Again this data is something that hardly changes and can be heavily cached through adding a mechanism to ItemManager to create an multi-map index (key -> List<Bean>) of the item beans by a specific field in the beans. This allows the createHistorySamplerItem2 code to just ensure the index by variableId exists (or create it if it's missing) and directly lookup all the beans whose variableId matches.

Finally the code will create LevelItem instances from all the found item beans. This involves reading id and samplingInterval fields from the beans. The amount of these beans should always be small so there's not much point in trying to cache even these sorted LevelItem[] arrays.

Edited by Tuukka Lehtonen