Check the documentation of BRepBndLib::Add():
The resulting bounding box may be somewhat larger than the object
So that this method is expected to return larger box, not only due to topological tolerance (which you are trying to eliminate by Bnd_Box::SetGap()), but also by rough algorithms applied on analytical geometry to get bounding box faster. Normally, larger bounding box returned by BRepBndLib::Add() is not an issue, because it is expected to be larger to a small extent, and because it is expected to be used in algorithms where bounding box is supplimentary (used in optimization structures).
BRepBndLib::AddOptimal() looks like more appropriate for calculating precise bounding box. I have some doubts if it is used often, so that BRepBndLib::Add() has probably lesser number of unknown bugs. You may note from implementation details, that it adds sub-boxes in different way than BRepBndLib::Add():
aLocBox.Enlarge(P3d->Deflection() + Tol);
...
Standard_Real xmin, ymin, zmin, xmax, ymax, zmax;
aLocBox.Get(xmin, ymin, zmin, xmax, ymax, zmax);
B.Update(xmin, ymin, zmin, xmax, ymax, zmax);which practically means that Shape tolerance becomes integral part of final Bnd_Box, which cannot be eliminated by Bnd_Box::SetGap(0.0).
Both have flags to use triangulation to speed up bounding box calculations. In that case occuracy of triangulation stored inside of TopoDS_Shape dramatically affects result bounding box, and practically speaking BRepBndLib::AddOptimal() works almost the same as BRepBndLib::Add().