Hacker News new | ask | show | jobs
by Longhanks 38 days ago
> A related issue solved along was Windows string representation of paths. std::filesystem::path stores its text in wchar_t encoded as UTF-16 (Windows native). But p.string() narrows it down to the active code page, rather than UTF-8 which is what the formatting library expects. The result was a non-ASCII path could get transcoded to gibberish. The C++26 std::formatter<std::filesystem::path> converts Windows native UTC-16 to UTF-8 using Unicode transcoding and avoiding code pages, therefore solving the problem.

...only to then convert it back to UTF-16 for WriteConsoleW(), which std::print() usually calls (unless not running in a console) (https://github.com/microsoft/STL/blob/488e7953685722d2d6666f...).

1 comments

> Ill-formed UTF-16 is replaced with U+FFFD by default, or escaped under {:?}.

Silently corrupting the path seems an odd choice in this day and age, when WTF8 has existed for many years and fixes this round trip bug / security vulnerability

Yes, but WTF-8 is not UTF-8.
Yeah, but picking utf8 over wtf8 for this in 2026 feels a lot like picking utf16 over utf8 in 1995. You know the choice is broken and you know there is a correct fix already out there.