Skip to content

ExternalRead query update propagation exhibits undesirable O(N^2) behavior

Multiple problems discovered:

  • Code using the Variable interface to loop over a set of experiment-context variables with solver VariableNode structure and retrieve structure/values for each variable, should catch PendingVariableException for each variable separately and only after all variables have been processed, re-throw a PendingVariableException.
    • This is beneficial for performance because it greatly improves the chances of the next run of the request in succeeding to get all the data that was previously still pending. Otherwise the query system might end up re-performing the request as many times as there are variables to process. In asynchronous query mode each query retrieving experiment/solver data will most likely throw PendingVariableException in turn.
    • This applies to e.g. org.simantics.structural2.Functions.getProcedural
  • Query hierarchies with IMMEDIATE_UPDATE queries at the top and ExternalReads as leafs work badly with query result change propagation in QueryProcessor.propagateChangesInQueryCache__.
    • Every single ExternalReadEntry "primitiveUpdate" was capable of causing a full re-triggering of IMMEDIATE_UPDATE queries, such as ResourceURIToVariable which gets used a lot with e.g. procedural UCs.
    • Consider having a procedural UC which generates hundreds of children in its substructure. This would end up having the following query hierarchy:
      ResourceURIToVariable
        ProceduralSubstructureRequest
          NodeStructureRequest (= ExternalRead)
    • During propagation, every single update of a NodeStructureRequest would immediately cause a full recomputation of ResourceURIToVariable.
    • Here's an example of a call trace that would recur hundreds of times in an actual problem case:
      org.simantics.structural2.Functions: getProcedural: http://Projects/Development%20Project/Model/Experiment/d1663eea-e42b-41da-a601-0e3f5ebd2639/NewGenericDiagram/PIPELINE2
      java.lang.Exception: trace
          at org.simantics.structural2.Functions.getProcedural(Functions.java:346)
          at org.simantics.structural2.Functions$ProceduralSubstructureRequest.perform(Functions.java:408)
          at org.simantics.structural2.Functions$ProceduralSubstructureRequest.perform(Functions.java:1)
          at org.simantics.db.impl.query.ReadEntry.computeForEach(ReadEntry.java:121)
          at org.simantics.db.impl.query.QueryCache.runnerReadEntry(QueryCache.java:697)
          at org.simantics.db.impl.graph.ReadGraphImpl.syncRequest(ReadGraphImpl.java:1980)
          at org.simantics.db.impl.graph.ReadGraphImpl.syncRequest(ReadGraphImpl.java:1973)
          at org.simantics.structural2.Functions$4.getVariable(Functions.java:618)
          at org.simantics.db.layer0.variable.StandardGraphChildVariable.getPossibleChild(StandardGraphChildVariable.java:77)
          at org.simantics.db.layer0.variable.AbstractVariable.getChild(AbstractVariable.java:555)
          at org.simantics.db.layer0.request.ResourceURIToVariable.perform(ResourceURIToVariable.java:34)
          at org.simantics.db.layer0.request.ResourceURIToVariable.perform(ResourceURIToVariable.java:1)
          at org.simantics.db.impl.query.ReadEntry$1.recompute(ReadEntry.java:63)
          at org.simantics.db.impl.query.QueryProcessor.compareTo(QueryProcessor.java:1370)
          at org.simantics.db.impl.query.QueryProcessor.update(QueryProcessor.java:1466)
          at org.simantics.db.impl.query.QueryProcessor$8.execute(QueryProcessor.java:1724)
          at gnu.trove.impl.hash.TObjectHash.forEach(TObjectHash.java:135)
          at org.simantics.db.impl.query.QueryProcessor.propagateChangesInQueryCache__(QueryProcessor.java:1717)
          at org.simantics.db.impl.query.QueryProcessor.propagateChangesInQueryCache_(QueryProcessor.java:1531)
          at org.simantics.db.impl.query.QueryProcessor.propagateChangesInQueryCache(QueryProcessor.java:1522)
          at org.simantics.db.impl.query.QueryProcessor.releaseWrite(QueryProcessor.java:395)
          at fi.vtt.simantics.procore.internal.WriteSupportImpl.claim(WriteSupportImpl.java:110)
          at fi.vtt.simantics.procore.internal.WriteSupportImpl.claim(WriteSupportImpl.java:85)
          at org.simantics.db.impl.graph.WriteGraphImpl.claim(WriteGraphImpl.java:442)
          at org.simantics.selectionview.ModelledTabContributor$Tab$1.perform(ModelledTabContributor.java:251)
          at org.simantics.selectionview.ModelledTabContributor$Tab$1.perform(ModelledTabContributor.java:1)
          at fi.vtt.simantics.procore.internal.SessionImplSocket$2.run0(SessionImplSocket.java:581)
          at org.simantics.db.impl.query.QueryProcessor$SessionTask.run(QueryProcessor.java:231)
          at fi.vtt.simantics.procore.internal.SessionRequestManager$3.run0(SessionRequestManager.java:177)
          at org.simantics.db.impl.query.QueryProcessor$SessionTask.run(QueryProcessor.java:231)
          at org.simantics.db.impl.query.QueryThread.run(QueryThread.java:238)