Shader compilation fails when file path has utf8 characters

[Sinamore]

Description

Shader compilation fails when file path has utf8 characters (Cyrillic, Korean, ... in our case username is part of the path).

We use dxc as a shared library to generate shaders in multi-threaded context.
That's how our CompileShader function is structured:

#include "dxcapi.h"

bool CompileShader( CStringView Source, ... )
{
    CComPtr<IDxcLibrary> Library;
    HRESULT hr = DxcCreateInstance( CLSID_DxcLibrary, IID_PPV_ARGS( &Library ) );
    ...
    CComPtr<IDxcCompiler3> Compiler;
    hr = DxcCreateInstance( CLSID_DxcCompiler, IID_PPV_ARGS( &Compiler ) );
    ...
    DxcBuffer SourceBuffer {};
    SourceBuffer.Ptr = Source.GetData();
    SourceBuffer.Size = Source.GetSize();
    SourceBuffer.Encoding = DXC_CP_ACP;

    CComPtr<IDxcResult> Result;
    hr = Compiler->Compile( &SourceBuffer, Args.AccessData(), Args.GetSize(), nullptr, IID_PPV_ARGS( &Result ) );
    ...
}

Source contains shader source code, I've checked that it has only ASCII characters.
Args is an array that contains <source file path>/<filename>.scache -spirv -fvk-b-shift 0 all -fvk-t-shift 0 all -fvk-s-shift 0 all -fvk-u-shift 0 all -fvk-auto-shift-bindings -fvk-use-dx-layout -HV 2021 -fspv-target-env=vulkan1.1 -T cs_6_0.

Steps to Reproduce
Compile any shader with the CompileShader above.

Actual Behavior

On Windows: hr is E_FAIL and Result contains "error: error reading '/000000005dcd549100000000.scache", even though the file exists in the folder.

On Linux: hr is E_FAIL and Result contains "Internal compiler error:" (See next post for more info)

Environment

• DXC version 1.8.2407 and 1.8.2502
• Host Operating System Windows 11 and Ubuntu 24.04.2

[Sinamore]

I've also tried standalone dxc binary on Linux. It crashed then source file path contained utf8 characters. I've used changes from #6725 and removed else if (cchWideChar < cbMultiByte) check at lib/DxcSupport/Unicode.cpp:49 (it's obviously incorrect, since cchWideChar is always not less than cbMultibyte: every utf8 character is 1+ byte in multibyte, but 1 symbol in widechar). With those changes dxc works as expected.

There are 2 issues still:

• The changes are inside #ifndef _WIN32 which makes them inapplicable to Windows builds, if I understand correctly, so the issue with Windows is somewhere else.
• When using libdxcompiler.so built with the changes above in the CompileShader function, it still fails sometimes (once in 2-3 runs of full rebuild of shadercache). I observed 2 types of failures: segfault in clang::FrontendAction::Execute function, while 1 or 2 other threads are somewhere deep inside dxc; and another where it looks like Result that is produced by Compiler->Compile call is empty (the errorblob is also empty).