SysTutorials

Linux Manual Pages › Section 3

stdcodecvt_utf8(3)

std::codecvt_utf8 - std::codecvt_utf8

Synopsis

Defined in header<codecvt>
    template <
        class Elem, (since C++ 11)
                        unsigned long Maxcode = 0x10ffff,
        (deprecated in C++ 17)
            std::codecvt_mode Mode = (std::codecvt_mode)0
        >
    class codecvt_utf8 : public std::codecvt<Elem, char, std::mbstate_t>;

std::codecvt_utf8 is a std::codecvt facet which encapsulates conversion between a UTF-8 encoded byte string and UCS2 or UTF-32 character string (depending on the type of Elem). This codecvt facet can be used to read and write UTF-8 files, both text and binary.

Template Parameters

Elem - either char16_t, char32_t, or wchar_t Maxcode - the largest value of Elem that this facet will read or write without error Mode - a constant of type std::codecvt_mode

Member functions

              constructs a new codecvt_utf8 facet constructor (public member function)               destroys a codecvt_utf8 facet destructor (public member function)

 std::codecvt_utf8::codecvt_utf8

explicit codecvt_utf8( std::size_t refs = 0 );

Constructs a new std::codecvt_utf8 facet, passes the initial reference counter refs to the base class.

Parameters

refs - the number of references that link to the facet

 std::codecvt_utf8::~codecvt_utf8

~codecvt_utf8();

Destroys the facet. Unlike the locale-managed facets, this facet’s destructor is public.

Inherited from std::codecvt

Member types

Member type Definition intern_type internT extern_type externT state_type stateT

Member objects

Member name Type id (static) std::locale::id

Member functions

              invokes do_out out (public member function of std::codecvt<InternT,ExternT,State>)               invokes do_in in (public member function of std::codecvt<InternT,ExternT,State>)               invokes do_unshift unshift (public member function of std::codecvt<InternT,ExternT,State>)               invokes do_encoding encoding (public member function of std::codecvt<InternT,ExternT,State>)               invokes do_always_noconv always_noconv (public member function of std::codecvt<InternT,ExternT,State>)               invokes do_length length (public member function of std::codecvt<InternT,ExternT,State>)               invokes do_max_length max_length (public member function of std::codecvt<InternT,ExternT,State>)

Protected member functions

do_out converts a string from internT to externT, such as when writing to file                  (virtual protected member function of std::codecvt<InternT,ExternT,State>) [virtual]

do_in converts a string from externT to internT, such as when reading from file                  (virtual protected member function of std::codecvt<InternT,ExternT,State>) [virtual]

do_unshift generates the termination character sequence of externT characters for incomplete conversion                  (virtual protected member function of std::codecvt<InternT,ExternT,State>) [virtual]

do_encoding returns the number of externT characters necessary to produce one internT character, if constant                  (virtual protected member function of std::codecvt<InternT,ExternT,State>) [virtual]

do_always_noconv tests if the facet encodes an identity conversion for all valid argument values                  (virtual protected member function of std::codecvt<InternT,ExternT,State>) [virtual]

do_length calculates the length of the externT string that would be consumed by conversion into given internT buffer                  (virtual protected member function of std::codecvt<InternT,ExternT,State>) [virtual]

do_max_length returns the maximum number of externT characters that could be converted into a single internT character                  (virtual protected member function of std::codecvt<InternT,ExternT,State>) [virtual]

Inherited from std::codecvt_base

Member type Definition enum result { ok, partial, error, noconv }; Unscoped enumeration type

Enumeration constant Definition ok conversion was completed with no error partial not all source characters were converted error encountered an invalid character noconv no conversion required, input and output types are the same

Notes

Although the standard requires that this facet works with UCS2 when the size of Elem is 16 bits, some implementations use UTF-16 instead. The term "UCS2" was deprecated and removed from the Unicode standard.

Example

The following example demonstrates the difference between UCS2/UTF-8 and UTF-16/UTF-8 conversions: the third character in the string is not a valid UCS2 character. // Run this code

#include <iostream>
#include <string>
#include <locale>
#include <codecvt>
int main()
{
// UTF-8 data. The character U+1d10b, musical sign segno, does not fit in UCS2
std::string utf8 = u8"z\u6c34\U0001d10b";
// the UTF-8 / UTF-16 standard conversion facet
std::wstring_convert<std::codecvt_utf8_utf16<char16_t>, char16_t> utf16conv;
std::u16string utf16 = utf16conv.from_bytes(utf8);
std::cout << "UTF16 conversion produced " << utf16.size() << " code units:\n";
for (char16_t c : utf16)
std::cout << std::hex << std::showbase << c << ‘\n’;
// the UTF-8 / UCS2 standard conversion facet
std::wstring_convert<std::codecvt_utf8<char16_t>, char16_t> ucs2conv;
try {
std::u16string ucs2 = ucs2conv.from_bytes(utf8);
} catch (const std::range_error &e) {
std::u16string ucs2 = ucs2conv.from_bytes(utf8.substr(0, ucs2conv.converted()));
std::cout << "UCS2 failed after producing " << std::dec << ucs2.size() << " characters:\n";
for (char16_t c : ucs2)
std::cout << std::hex << std::showbase << c << ‘\n’;
}
}

Output:

  UTF16 conversion produced 4 code units:   0x7a   0x6c34   0xd834   0xdd0b   UCS2 failed after producing 2 characters:   0x7a   0x6c34

See also

Character locale-defined multibyte UTF-8 UTF-16 conversions (UTF-8, GB18030)                                                           codecvt<char16_t, char, mbstate_t> UTF-16 mbrtoc16 / c16rtomb(with C11’s DR488) codecvt_utf8_utf16<char16_t> N/A                                                           codecvt_utf8_utf16<char32_t>                                                           codecvt_utf8_utf16<wchar_t> UCS2 c16rtomb(without C11’s DR488) codecvt_utf8<char16_t> codecvt_utf16<char16_t>                                                           codecvt_utf8<wchar_t>(Windows) codecvt_utf16<wchar_t>(Windows)                                                           codecvt<char32_t, char, mbstate_t> codecvt_utf16<char32_t> UTF-32 mbrtoc32 / c32rtomb codecvt_utf8<char32_t> codecvt_utf16<wchar_t>(non-Windows)                                                           codecvt_utf8<wchar_t>(non-Windows) system wide: mbsrtowcs / wcsrtombs UTF-32(non-Windows) use_facet<codecvt No No UCS2(Windows) <wchar_t, char, mbstate_t>>(locale)                       converts between character encodings, including UTF-8, UTF-16, UTF-32 codecvt (class template) codecvt_mode tags to alter behavior of the standard codecvt facets                       (enum) (C++11) (deprecated in C++17) codecvt_utf16 converts between UTF-16 and UCS2/UCS4                       (class template) (C++11) (deprecated in C++17) codecvt_utf8_utf16 converts between UTF-8 and UTF-16                       (class template) (C++11) (deprecated in C++17)