ExternalRead query update propagation exhibits undesirable O(N^2) behavior
Multiple problems discovered:
- Code using the Variable interface to loop over a set of experiment-context variables with solver
VariableNodestructure and retrieve structure/values for each variable, should catchPendingVariableExceptionfor each variable separately and only after all variables have been processed, re-throw aPendingVariableException.- This is beneficial for performance because it greatly improves the chances of the next run of the request in succeeding to get all the data that was previously still pending. Otherwise the query system might end up re-performing the request as many times as there are variables to process. In asynchronous query mode each query retrieving experiment/solver data will most likely throw
PendingVariableExceptionin turn. - This applies to e.g. org.simantics.structural2.Functions.getProcedural
- This is beneficial for performance because it greatly improves the chances of the next run of the request in succeeding to get all the data that was previously still pending. Otherwise the query system might end up re-performing the request as many times as there are variables to process. In asynchronous query mode each query retrieving experiment/solver data will most likely throw
- Query hierarchies with
IMMEDIATE_UPDATEqueries at the top andExternalReads as leafs work badly with query result change propagation inQueryProcessor.propagateChangesInQueryCache__.- Every single
ExternalReadEntry"primitiveUpdate" was capable of causing a full re-triggering ofIMMEDIATE_UPDATEqueries, such asResourceURIToVariablewhich gets used a lot with e.g. procedural UCs. - Consider having a procedural UC which generates hundreds of children in its substructure. This would end up having the following query hierarchy:
ResourceURIToVariable ProceduralSubstructureRequest NodeStructureRequest (= ExternalRead) - During propagation, every single update of a
NodeStructureRequestwould immediately cause a full recomputation ofResourceURIToVariable. - Here's an example of a call trace that would recur hundreds of times in an actual problem case:
org.simantics.structural2.Functions: getProcedural: http://Projects/Development%20Project/Model/Experiment/d1663eea-e42b-41da-a601-0e3f5ebd2639/NewGenericDiagram/PIPELINE2 java.lang.Exception: trace at org.simantics.structural2.Functions.getProcedural(Functions.java:346) at org.simantics.structural2.Functions$ProceduralSubstructureRequest.perform(Functions.java:408) at org.simantics.structural2.Functions$ProceduralSubstructureRequest.perform(Functions.java:1) at org.simantics.db.impl.query.ReadEntry.computeForEach(ReadEntry.java:121) at org.simantics.db.impl.query.QueryCache.runnerReadEntry(QueryCache.java:697) at org.simantics.db.impl.graph.ReadGraphImpl.syncRequest(ReadGraphImpl.java:1980) at org.simantics.db.impl.graph.ReadGraphImpl.syncRequest(ReadGraphImpl.java:1973) at org.simantics.structural2.Functions$4.getVariable(Functions.java:618) at org.simantics.db.layer0.variable.StandardGraphChildVariable.getPossibleChild(StandardGraphChildVariable.java:77) at org.simantics.db.layer0.variable.AbstractVariable.getChild(AbstractVariable.java:555) at org.simantics.db.layer0.request.ResourceURIToVariable.perform(ResourceURIToVariable.java:34) at org.simantics.db.layer0.request.ResourceURIToVariable.perform(ResourceURIToVariable.java:1) at org.simantics.db.impl.query.ReadEntry$1.recompute(ReadEntry.java:63) at org.simantics.db.impl.query.QueryProcessor.compareTo(QueryProcessor.java:1370) at org.simantics.db.impl.query.QueryProcessor.update(QueryProcessor.java:1466) at org.simantics.db.impl.query.QueryProcessor$8.execute(QueryProcessor.java:1724) at gnu.trove.impl.hash.TObjectHash.forEach(TObjectHash.java:135) at org.simantics.db.impl.query.QueryProcessor.propagateChangesInQueryCache__(QueryProcessor.java:1717) at org.simantics.db.impl.query.QueryProcessor.propagateChangesInQueryCache_(QueryProcessor.java:1531) at org.simantics.db.impl.query.QueryProcessor.propagateChangesInQueryCache(QueryProcessor.java:1522) at org.simantics.db.impl.query.QueryProcessor.releaseWrite(QueryProcessor.java:395) at fi.vtt.simantics.procore.internal.WriteSupportImpl.claim(WriteSupportImpl.java:110) at fi.vtt.simantics.procore.internal.WriteSupportImpl.claim(WriteSupportImpl.java:85) at org.simantics.db.impl.graph.WriteGraphImpl.claim(WriteGraphImpl.java:442) at org.simantics.selectionview.ModelledTabContributor$Tab$1.perform(ModelledTabContributor.java:251) at org.simantics.selectionview.ModelledTabContributor$Tab$1.perform(ModelledTabContributor.java:1) at fi.vtt.simantics.procore.internal.SessionImplSocket$2.run0(SessionImplSocket.java:581) at org.simantics.db.impl.query.QueryProcessor$SessionTask.run(QueryProcessor.java:231) at fi.vtt.simantics.procore.internal.SessionRequestManager$3.run0(SessionRequestManager.java:177) at org.simantics.db.impl.query.QueryProcessor$SessionTask.run(QueryProcessor.java:231) at org.simantics.db.impl.query.QueryThread.run(QueryThread.java:238)
- Every single