Time and memory consumption in Gmsh for OCCT operation

Hello, could you please somehow incorporate the code by python or draw or c++?

The issue is strangle and probable related to bug with scope and life time. Probably the translated variable exist on dynamic memory and creates more and more during the loop. And free only after loop breaks.

But that should be not on OCCT Side, on c++ it is a challenge to create bug like that.

Best regards, Commenter-1.

Hello, Dmitrii!

Idea to run C++ code was fruitful. Thanks.
I ran code in VS and got Stack Overflow error. An increase of stack size allows to make more number of translate operations (with huge memory consumption). Will prepare debug library of Gmsh and will trace workflow.

Thank you,
Author.

Thank you, Dmitrii!

I wrote two small test codes in c++: one uses c-kind functions, other one uses c++-kind functions. Both tests produces the same behaviour. Which, however, is a bit different from previous gmsh's script. But these tests also demonstrate rather large memory consumption and fail silently. Test codes names are occ-c.cpp and occ-cpp.cpp. Also there are two similar tests geo-c and geo-cpp which differ only by using gmsh's built-in kernel. Gmsh functions work without remarks in this test.
The text files are attached to the comment.
All directory with libraries and executables could be downloaded from here: https://disk.yandex.ru/d/GmW0OcwMwMLeZw

Best regards, Author.

Author,

I wrote two small test codes in c++: one uses c-kind functions, other one uses c++-kind functions. Both tests produces the same behaviour. Which, however, is a bit different from previous gmsh's script.

This forum is about OCCT - most users have no idea how these gmsh calls are translated to OCCT. I guess you better post your question on GMSH forum first, and maybe then roll-back with more OCCT-related scenario.

And please share code samples directly on the forum rather than through some external service.

#include <gmsh.h>
#include <stdio.h>

int main(int argc, char **argv) 
{

  gmsh::initialize();

  gmsh::model::add("occ");

  gmsh::fltk::initialize();

  gmsh::model::occ::addPoint(0.0, 0.0, 0.0, 0.01, 1);

  gmsh::model::occ::synchronize();

  const int n=10000;
  for (int i=1;i <= n; i++) {

      gmsh::model::occ::translate({{0,1}}, 0.001, 0.0, 0.0);

      if (i == n) {
        printf("i = %d\n", i);
      }

  }

  gmsh::model::occ::synchronize();
  printf("finished");

  gmsh::fltk::run();
  gmsh::finalize();

  return 0;
}

Large memory and time consumptions in Gmsh using OCC GK has been disappeared unexpectedly. Nothing special has been made in PC except latest Gmsh 4.13.1 source code installation and debugging in VS2022 and an installation of a precompiled OCC 7.7.0 (instead of 7.6.0). Also, I have traced Gmsh-OCC interface, it looked rather sofisticated (for me). Thank you for advices. Hope, I'll not meet this problem again.

The problem of huge time and memory consumption does still exist.

It exists with gmsh.exe and gmsh script:

SetFactory("OpenCASCADE");
Point(1)={0,0,0};
n=10000; 
For k In {1:n}
   Translate {0.00001, 0, 0} { Point{1}; }
   EndFor

and with c++ test linked with shared gmsh library:

#include <gmsh.h>
int main(int argc, char **argv)
{
  gmsh::initialize();
  gmsh::model::add("occ");
  gmsh::fltk::initialize();
  gmsh::model::occ::addPoint(0.0, 0.0, 0.0);
  gmsh::model::occ::synchronize();

  const int n = 10000;
  for(int i = 1; i <= n; i++) {
    gmsh::model::occ::translate({{0, 1}}, 0.001, 0.0, 0.0);
  }

  gmsh::model::occ::synchronize();
  printf("finished");
  gmsh::fltk::run();  
  gmsh::finalize();
  return 0;
}

Both gmsh.exe and c++ test were debuged in VS2022 (Windows 10). The Task Manager shows memory consumption of about 20 GB with large stack size reserved (no any memory leaking were observed). If to reduce stack size both codes meet unhandled exception : Stack Overflow (ntdll.dll). The smaller stack size - the stack overflow occurs sooner (with smaller consumption memory).

Stack Frames are as following:

ntdll.dll
ntdll.dll
ucrbase.dll
TKernel.dll
TKMath.dll
...
TKMath.dll   <exceeded max number of stack frames supported by VS>

Gmsh flow execution is as following:

  gmsh::model::occ::translate({{0, 1}}, 0.001, 0.0, 0.0);

    GModel::current()->getOCCInternals()->translate(dimTags, dx, dy, dz);

      return _transform(inDimTags, &tfo, nullptr);

          _unbindWithoutChecks(inShapes[i]);

          _bind(outShapes[i], dim, tag, true);

I do not have enough qualification to trace all memory allocations. However, I have noticed that _attributes->_all vector of OCCAttributes growths with each cycle iteration. It is provided by _attributes->insert(new OCCAttributes(0, transformed, lc)); and _attributes->insert(new OCCAttributes(0, vertex));.

Corresponding codes are

bool OCC_Internals::_transform(
  const std::vector<std::pair<int, int>> &inDimTags,
  BRepBuilderAPI_Transform *tfo, BRepBuilderAPI_GTransform *gtfo)
{
  // build a single compound shape, so that we won't duplicate internal
  // boundaries
  BRep_Builder b;
  TopoDS_Compound c;
  b.MakeCompound(c);
  for(std::size_t i = 0; i < inDimTags.size(); i++) {
    int dim = inDimTags[i].first;
    int tag = inDimTags[i].second;
    if(!_isBound(dim, tag)) {
      Msg::Error("Unknown OpenCASCADE entity of dimension %d with tag %d", dim,
                 tag);
      return false;
    }
    TopoDS_Shape shape = _find(dim, tag);
    b.Add(c, shape);
  }

  std::vector<TopoDS_Shape> inShapes;
  _addSimpleShapes(c, inShapes);

  TopoDS_Shape result;
  if(tfo) {
    tfo->Perform(c, Standard_False);
    if(!tfo->IsDone()) {
      Msg::Error("Could not apply transformation");
      return false;
    }
    result = tfo->Shape();
  }
  else if(gtfo) {
    gtfo->Perform(c, Standard_False);
    if(!gtfo->IsDone()) {
      Msg::Error("Could not apply transformation");
      return false;
    }
    result = gtfo->Shape();
  }

  // copy vertex-based meshing attributes
  TopExp_Explorer exp0;
  for(exp0.Init(c, TopAbs_VERTEX); exp0.More(); exp0.Next()) {
    TopoDS_Vertex vertex = TopoDS::Vertex(exp0.Current());
    TopoDS_Shape transformed;
    if(tfo)
      transformed = tfo->ModifiedShape(vertex);
    else if(gtfo)
      transformed = gtfo->ModifiedShape(vertex);
    if(!transformed.IsNull()) {
      double lc = _attributes->getMeshSize(0, vertex);
      if(lc > 0 && lc < MAX_LC)
        _attributes->insert(new OCCAttributes(0, transformed, lc));     <???>
    }
  }

  // try to re-bind trasnformed shapes to same tags as original shapes
  std::vector<TopoDS_Shape> outShapes;
  _addSimpleShapes(result, outShapes);

  if(inShapes.size() != inDimTags.size() ||
     inShapes.size() != outShapes.size()) {
    Msg::Error("OpenCASCADE transform changed the number of shapes");
    return false;
  }
  for(std::size_t i = 0; i < inDimTags.size(); i++) {
    int dim = inDimTags[i].first;
    int tag = inDimTags[i].second;
    if(CTX::instance()->geom.occFastUnbind) {
      // bypass safe _unbind checks by unbinding the shape and all its subshapes
      // without checking dependencies: this is a bit dangerous, as one could
      // translate e.g. the face of a cube (this is not allowed!) - which will
      // unbind the face of the cube. But the original face will actually be
      // re-bound (with a warning) at the next syncronization point, so it's not
      // too bad...
      _unbindWithoutChecks(inShapes[i]);
    }
    else {
      // safe, but slow: _unbind() has linear complexity with respect to the
      // number of entities in the model (due to the dependency checking of
      // upward adjencencies and the maximum tag update). Using this in a for
      // loop to translate copies of entities leads to quadratic complexity.
      _unbind(inShapes[i], dim, tag, true);
    }
    // TODO: it would be even better to code a rebind() function to reuse the
    // tags not only of the shape, but of all the sub-shapes as well
    _bind(outShapes[i], dim, tag, true);
  }

  return true;
}

void OCC_Internals::_bind(const TopoDS_Vertex &vertex, int tag, bool recursive)
{
  if(vertex.IsNull()) return;
  if(_vertexTag.IsBound(vertex)) {
    if(_vertexTag.Find(vertex) != tag) {
      Msg::Info("Cannot bind existing OpenCASCADE point %d to second tag %d",
                _vertexTag.Find(vertex), tag);
    }
  }
  else {
    if(_tagVertex.IsBound(tag)) {
      // this leaves the old vertex bound in _vertexTag, but we cannot remove it
      Msg::Info("Rebinding OpenCASCADE point %d", tag);
    }
    _vertexTag.Bind(vertex, tag);
    _tagVertex.Bind(tag, vertex);
    setMaxTag(0, tag);
    _changed = true;
    _attributes->insert(new OCCAttributes(0, vertex));     <???>
  }
}

and

class OCCAttributes {
private:
  int _dim;
  TopoDS_Shape _shape;
  double _meshSize;
  ExtrudeParams *_extrude;
  int _sourceDim;
  TopoDS_Shape _sourceShape;
  std::string _label;
  std::vector<double> _color;

public:
  OCCAttributes() : _dim(-1), _meshSize(MAX_LC), _extrude(0), _sourceDim(-1) {}
  OCCAttributes(int dim, TopoDS_Shape shape)
    : _dim(dim), _shape(shape), _meshSize(MAX_LC), _extrude(0), _sourceDim(-1)
  {
  }
  OCCAttributes(int dim, TopoDS_Shape shape, double size)
    : _dim(dim), _shape(shape), _meshSize(size), _extrude(0), _sourceDim(-1)
  {
  }
  OCCAttributes(int dim, TopoDS_Shape shape, ExtrudeParams *e, int sourceDim,
                TopoDS_Shape sourceShape)
    : _dim(dim), _shape(shape), _meshSize(MAX_LC), _extrude(e),
      _sourceDim(sourceDim), _sourceShape(sourceShape)
  {
  }
  OCCAttributes(int dim, TopoDS_Shape shape, const std::string &label)
    : _dim(dim), _shape(shape), _meshSize(MAX_LC), _extrude(0), _sourceDim(-1),
      _label(label)
  {
  }
  OCCAttributes(int dim, TopoDS_Shape shape, double r, double g, double b,
                double a = 1., int boundary = 0)
    : _dim(dim), _shape(shape), _meshSize(MAX_LC), _extrude(0), _sourceDim(-1)
  {
    _color.resize(boundary ? 5 : 4);
    _color[0] = r;
    _color[1] = g;
    _color[2] = b;
    _color[3] = a;
    if(boundary) _color[4] = boundary;
  }
  ~OCCAttributes() {}
  int getDim() { return _dim; }
  TopoDS_Shape getShape() { return _shape; }
  double getMeshSize() { return _meshSize; }
  ExtrudeParams *getExtrudeParams() { return _extrude; }
  int getSourceDim() { return _sourceDim; }
  TopoDS_Shape getSourceShape() { return _sourceShape; }
  const std::string &getLabel() { return _label; }
  const std::vector<double> &getColor() { return _color; }
};

I hope it will encourage Gmsh maintainers/developers with possible support of OCC developers (if needed) to fix this behaviour. Sorry for being lazy to build OCC from sources.

P.S. This minimalistic example was discovered trying to draw dynamics of a cloud of particles (what are not at all a CAD purpose). Just trying to avoid other kind of software learning. I have overcome the problem just by switching to gmsh geometrical kernel.

Below is an example of particles cloud.

A use of Debug version of OCC showed a deep (hundreds stack frames) recursion of the TopLoc_Location TopLoc_Location::Multiplied(const TopLoc_Location& Other) const in TopLoc_Location.cxx.

Why it could happened for a single geometrical point under simple translation is a deep mystery.

Stack frames picture is attached.

Now it is obvious for me that the program makes a lot of nested locations when it translates the same shape with new and new matrices.
If you don't want the locations to be nested you need to pass the option theCopyGeom of the method Perform of BRepBuilderAPI_Transform as true instead of false.

Great! I just changed in the OCC::Internals::_transform in the line tfo->Perform(c,Standard_False) the Standard_False to the Standard_True and the bad behaviours (huge time and memory consumption) have disappeared. Really Great!

Possibly, in gfto->Perform(c,Standard_False) the second argument is also have to be changed to Standard_True.

I hope, now the OCC branch of Gmsh will be less lazy as it seemed earlier.