If you want computing AABB (axes-aligned bounding box) of shape within a local coordinate system, the only thing to do is to calculate a proper transformation to this local system and apply it to shape or AABB.
Lets start from defining an auxiliary tool for printing AABB (Bnd_Box):
auto printBox = [](const Bnd_Box& theBox, const char* theName) {
gp_Pnt aMin = theBox.CornerMin(), aMax = theBox.CornerMax();
gp_XYZ aDim = aMax.XYZ() - aMin.XYZ();
std::cout << theName << "\n"
<< " MIN: " << aMin.X() << " " << aMin.Y() << " " << aMin.Z() << "\n"
<< " MAX: " << aMax.X() << " " << aMax.Y() << " " << aMax.Z() << "\n"
<< " DIM: " << aDim.X() << " " << aDim.Y() << " " << aDim.Z() << "\n";
};
and construct a trivially-oriented Cone:
TopoDS_Shape aCone0 = BRepPrim_Cone(gp::XOY(), 100, 0, 400).Shell();
gp_Trsf aTrans; // add some location for testing
aTrans.SetTranslation(gp::Origin(), gp_Pnt(50, 0, 0));
aCone0.Location(aTrans);
DBRep::Set("c0", aCone0);
Bnd_Box aBox0;
BRepBndLib::Add(aCone0, aBox0);
printBox(aBox0, "AABB of a simple Cone");
// MIN: -50.0000001 -100.0000001 -1e-07
// MAX: 150.0000001 100.0000001 400.0000001
// DIM: 200.0000002 200.0000002 400.0000002
then define a Cone having the same dimensions, but with a more complex orientation:
gp_Ax2 aConeAx1(gp_Pnt(100, 200, 300), gp_Dir(1.0, -1.0, 0.0));
TopoDS_Shape aCone1 = BRepPrim_Cone(aConeAx1, 100, 0, 400).Shell();
aCone1.Location(aTrans);
DBRep::Set("c1", aCone1);
Bnd_Box aBox1;
BRepBndLib::Add(aCone1, aBox1);
printBox(aBox1, "AABB of a complex Cone");
// MIN: 79.2893217813453 -82.842712574619 199.9999999
// MAX: 432.842712574619 270.710678218655 400.0000001
// DIM: 353.553390793274 353.553390793274 200.0000002
The bounding boxes do not match, which is expected. Now, lets define transformation gp_Trsf from one coordinate system to another, apply it to our complex-oriented Cone as location TopLoc_Location and compute AABB for it:
gp_Trsf aTrsf;
aTrsf.SetTransformation(gp::XOY(), aConeAx1);
TopoDS_Shape aCone2i = aCone1.Located(aTrsf); // override location
Bnd_Box aBox2i;
BRepBndLib::Add(aCone2i, aBox2i);
printBox(aBox2i, "AABB of a inverse-located Cone (override)");
// MIN: -100.0000001 -100.0000001 -1e-07
// MAX: 100.0000001 100.0000001 400.0000001
// DIM: 200.0000002 200.0000002 400.0000002
We see, that dimensions of this AABB now matches to AABB of trivially-oriented Cone, but min/max is shifted. This is because within TopoDS_Shape::Located() we have overridden original local transformation of the shape. To fix this, we need multiplying transformations:
TopoDS_Shape aCone2 = aCone1.Located(aCone1.Location() * aTrsf);
Bnd_Box aBox2;
BRepBndLib::Add(aCone2, aBox2);
printBox(aBox2, "AABB of a inverse-located Cone (multiplied)");
// MIN: -50.0000001 -100.0000001 -1e-07
// MAX: 150.0000001 100.0000001 400.0000001
// DIM: 200.0000002 200.0000002 400.0000002
Now we got expected AABB of a Cone within the local coordinate system (in this case, this local coordinate system is where Cone was actually defined - just to be able to compare values).
Instead of computing a new AABB on transformed (located) TopoDS_Shape, we might also apply transformation to AABB Bnd_Box itself:
Bnd_Box aBox3 = aBox1.Transformed(aTrsf);
printBox(aBox3, "inverse-located AABB of a Cone");
// MIN: -285.355339159327 -100.0000001 -64.6446610406726
// MAX: 214.644661040673 100.0000001 435.355339159327
// DIM: 500.0000002 200.0000002 500.0000002
Notice that transformed AABB is valid - it contains the entire Cone, - but it is less optimal, as it will grow up while transforming AABB as a simple box. A new AABB computed for transformed TopoDS_Shape will be more tight, as it will consider transformation on-the-fly.
Notice, that in either way BRepBndLib::Add() computes AABB that might be a little bit larger than actual geometry by design, to may calculations reasonably fast. Flag useTriangulation to BRepBndLib::Add() would ask computing AABB on triangulation instead of geometry, when it is available. There are also methods BRepBndLib::AddOptimal() calculating a more precise AABB, and BRepBndLib::AddOBB() calculating an oriented-bounding box instead of axes-aligned one (the latter should more friendly to be transformed).