Archived issue #0031940

Foundation Classes - TCollection_ExtendedString::Print() corrupts UNICODE strings and does not compile with C++20

CommunityOCCT:Foundation Classesclosed9 public notes

Description

C++20 has removed operator<<(std::basic_ostream) overloads accepting char16_t (implicitly converted to integer), leading to TCollection_ExtendedString::Print() compilation error:

    Standard_ExtCharacter c = mystring[i];
      astream << "&#" << c << ";";

This code is obsolete anyway, as OCCT is now expected to print UTF-8 strings to stream.

When compiled with gcc 10.2 -std=gnu++2a

/usr/bin/c++   -I/data1/work/studio/src/ext/opencascade-7.5.0/inc -I/data1/work/studio/src/ext/opencascade-7.5.0/src/inc -I/usr/include/freetype2 -g -march=native   -fPIC -std=gnu++2a -MD -MT src/ext/opencascade-7.5.0/src/CMakeFiles/occt-objs.dir/TCollection/TCollection_ExtendedString.cxx.o -MF src/ext/opencascade-7.5.0/src/CMakeFiles/occt-objs.dir/TCollection/TCollection_ExtendedString.cxx.o.d -o src/ext/opencascade-7.5.0/src/CMakeFiles/occt-objs.dir/TCollection/TCollection_ExtendedString.cxx.o -c /data1/work/studio/src/ext/opencascade-7.5.0/src/TCollection/TCollection_ExtendedString.cxx
/data1/work/studio/src/ext/opencascade-7.5.0/src/TCollection/TCollection_ExtendedString.cxx: In member function ‘void TCollection_ExtendedString::Print(Standard_OStream&) const’:
/data1/work/studio/src/ext/opencascade-7.5.0/src/TCollection/TCollection_ExtendedString.cxx:623:26: error: use of deleted function ‘std::basic_ostream<char, _Traits>& std::operator<<(std::basic_ostream<char, _Traits>&, char16_t) [with _Traits = std::char_traits<char>]’
  623 |       astream << "&#" << c << ";";
      |                          ^
In file included from /usr/include/c++/10/iostream:39,
                 from /data1/work/studio/src/ext/opencascade-7.5.0/inc/Standard_Stream.hxx:20,
                 from /data1/work/studio/src/ext/opencascade-7.5.0/inc/Standard_OStream.hxx:19,
                 from /data1/work/studio/src/ext/opencascade-7.5.0/inc/Standard_ExtCharacter.hxx:28,
                 from /data1/work/studio/src/ext/opencascade-7.5.0/inc/Standard_PrimitiveTypes.hxx:27,
                 from /data1/work/studio/src/ext/opencascade-7.5.0/inc/Standard_Transient.hxx:20,
                 from /data1/work/studio/src/ext/opencascade-7.5.0/inc/Standard.hxx:91,
                 from /data1/work/studio/src/ext/opencascade-7.5.0/inc/TCollection_ExtendedString.hxx:20,
                 from /data1/work/studio/src/ext/opencascade-7.5.0/src/TCollection/TCollection_ExtendedString.cxx:15:
/usr/include/c++/10/ostream:549:5: note: declared here
  549 |     operator<<(basic_ostream<char, _Traits>&, char16_t) = delete;
      |     ^~~~~~~~
[2743/3480] Building CXX object src/ext/opencascade-...src/CMakeFiles/occt-objs.dir/TopTools/TopTools.cxx.o
ninja: build stopped: subcommand failed.

Steps to reproduce

The following test command:

static Standard_Integer TestExtString (Draw_Interpretor& theDI,
                                       Standard_Integer ,
                                       const char** theArgVec)
{
  TCollection_AsciiString    aStrUtf8 (theArgVec[1]);
  TCollection_ExtendedString aStrUtf16 (aStrUtf8);
  theDI     << "[DI]   TCollection_AsciiString:    '" << aStrUtf8 << "'\n";
  theDI     << "[DI]   TCollection_ExtendedString: '" << aStrUtf16 << "'\n";
  std::cout << "[cout] TCollection_AsciiString:    '" << aStrUtf8 << "'\n";
  std::cout << "[cout] TCollection_ExtendedString: '" << aStrUtf16 << "'\n";
  return 0;
}

produces the following output:

Draw[2]> TestExtString "What чёрта с два unmöglich français! 你好"
[cout] TCollection_AsciiString:    'What чёрта с два unmöglich français! 你好'
[cout] TCollection_ExtendedString: 'What чёрта с два unmglich franais!  你好'
[DI]   TCollection_AsciiString:    'What чёрта с два unmöglich français! 你好'
[DI]   TCollection_ExtendedString: 'What чёрта с два unmöglich français! 你好'

Note that on printing to cout, the symbols in Latin-1 range are lost ("unmöglich français" -> "unmglich franais"), and symbols in other code ranges are converted to hex values.

It is expected that printing TCollection_ExtendedString should produce the same output as TCollection_AsciiString.

Public activity

9 archived notes

Participants are labeled by their role within this record.

01Commenter 22020-11-18 14:56

Branch [archived branch] has been created by Commenter 3.

[revision removed]

Detailed log of new commits:

Author: Commenter 3
Date: Wed Nov 18 14:56:59 2020 +0300

    0031940: Foundation Classes - TCollection_ExtendedString::Print() corrupts UNICODE strings and does not compile with C++20

    TCollection_ExtendedString::Print() now converts string into UTF-8
    instead of printing character indexes.

02Commenter 32020-11-18 16:36

Patch is ready for review
- OCCT branch: [archived branch]

http://jenkins-test-12.nnov.opencascade.com:8080/view/CR31940-master-KGV/

03Commenter 22020-11-22 21:46

Branch [archived branch] has been updated by Participant.

[revision removed]

Detailed log of new commits:

Author: abv
Date: Sun Nov 22 21:46:52 2020 +0300

# added description of the change in upgrade.md

04Commenter 42020-11-22 21:47

Reviewed with minor amendment, please integrate

05Commenter 22020-11-23 08:18

Branch [archived branch] has been updated forcibly by Commenter 3.

[revision removed]

06Commenter 22020-11-23 08:18

07Commenter 12020-11-28 12:45

Combination -
OCCT branch : [archived branch]
master SHA - [revision removed]
[revision removed]
Products branch : [archived branch] SHA - [revision removed]
was compiled on Linux, MacOS and Windows platforms and tested in optimize mode.

Number of compiler warnings:
No new/fixed warnings

Regressions/Differences/Improvements:
No regressions/differences

CPU differences:
Debian80-64:
OCCT
Total CPU difference: 18049.94000000008 / 18024.44000000013 [+0.14%]
Products
Total CPU difference: 12325.960000000132 / 12167.95000000011 [+1.30%]
Windows-64-VC14:
OCCT
Total CPU difference: 19703.46875 / 19697.21875 [+0.03%]
Products
Total CPU difference: 13792.6875 / 13647.203125 [+1.07%]

Image differences :
No differences that require special attention

Memory differences :
No differences that require special attention

08Commenter 22020-11-28 13:05

Branch [archived branch] has been deleted by Participant.

[revision removed]

09Commenter 22020-11-28 13:05

Branch [archived branch] has been deleted by Participant.

[revision removed]

Related records