Unable to mix string literal shader and filesystem shader

[sbrkopac]

Describe the bug
I moved the vert_PushConstants.spv into a string literal. When loading up the vertex shader as a string literal but leaving the fragment shader loading from the filesystem, VSG seems to lose the fragment shader. I think this is due to VSG converting the fragment shader to SPIRV and not the vertex shader.

To Reproduce

std::string vert_PushConstants = R"(#version 450
#extension GL_ARB_separate_shader_objects : enable

layout(push_constant) uniform PushConstants {
    mat4 projection;
    mat4 modelview;
} pc;

layout(location = 0) in vec3 inPosition;
layout(location = 1) in vec3 inColor;
layout(location = 2) in vec2 inTexCoord;

layout(location = 0) out vec3 fragColor;
layout(location = 1) out vec2 fragTexCoord;

out gl_PerVertex {
    vec4 gl_Position;
};

void main() {
    gl_Position = (pc.projection * pc.modelview) * vec4(inPosition, 1.0);
    fragColor = inColor;
    fragTexCoord = inTexCoord;
}
)";

auto dataPath = std::filesystem::current_path() / "data";

// set up search paths and load shaders
vsg::Paths searchPaths = { dataPath.string() };

 // set up search paths and load shaders
vsg::ref_ptr<vsg::ShaderStage> vertexShader = vsg::ShaderStage::create(VK_SHADER_STAGE_VERTEX_BIT, "main", vert_PushConstants);
vsg::ref_ptr<vsg::ShaderStage> fragmentShader = vsg::ShaderStage::read(VK_SHADER_STAGE_FRAGMENT_BIT, "main", vsg::findFile("shaders/frag_PushConstants.spv", searchPaths));
if (!vertexShader || !fragmentShader)
{
    log->error("Could not create shaders.");
}

Expected behavior
I would expect that the program would compile as it does when loading both from the filesystem

Screenshots

Vertex Shader:

   1: #version 450
   2: #extension GL_ARB_separate_shader_objects : enable
   3:
   4: layout(push_constant) uniform PushConstants {
   5:     mat4 projection;
   6:     mat4 modelview;
   7: } pc;
   8:
   9: layout(location = 0) in vec3 inPosition;
  10: layout(location = 1) in vec3 inColor;
  11: layout(location = 2) in vec2 inTexCoord;
  12:
  13: layout(location = 0) out vec3 fragColor;
  14: layout(location = 1) out vec2 fragTexCoord;
  15:
  16: out gl_PerVertex {
  17:     vec4 gl_Position;
  18: };
  19:
  20: void main() {
  21:     gl_Position = (pc.projection * pc.modelview) * vec4(inPosition, 1.0);
  22:     fragColor = inColor;
  23:     fragTexCoord = inTexCoord;
  24: }


Fragment Shader:


Warning: Program failed to link.
glslang info log:
ERROR: Linking fragment stage: Missing entry point: Each stage requires one entry point
glslang debug info log:
--------

Desktop (please complete the following information):

• OS: Win10
• VSDK: 1.2.182.0
• Visual Studio 2019

Additional context
I worked around the problem by duplicating how VSG loads a shader from the filesystem:

template<typename T>
bool read(T& buffer, const std::string& input)
{
    using value_type = typename T::value_type;
    size_t valueSize = sizeof(value_type);
    size_t bufferSize = (input.size() + valueSize) / valueSize;

    buffer.resize(bufferSize);

    std::istringstream stream(input);
    stream.read(reinterpret_cast<char*>(buffer.data()), input.size());

    return true;
}

And then I was able to do the following:

vsg::ShaderModule::SPIRV spirv_vert_PushConstants; read(spirv_vert_PushConstants, vert_PushConstants);
vsg::ref_ptr<vsg::ShaderStage> vertexShader = vsg::ShaderStage::create(VK_SHADER_STAGE_VERTEX_BIT, "main", vert_PushConstants, spirv_vert_PushConstants);
vsg::ref_ptr<vsg::ShaderStage> fragmentShader = vsg::ShaderStage::read(VK_SHADER_STAGE_FRAGMENT_BIT, "main", vsg::findFile("shaders/frag_PushConstants.spv", searchPaths));

[robertosfield]

Hi Sam, Could you create a standard alone example that illustrates this problem, as I could try and cobble together an example but if it worked then I'd have no idea whether it's just not close enough to what you are doing, or that it's a Window specific issue. Also could you output the path that std::filesystem is reporting? FYI, std::filesystem may be part of the Standard but unfortunately it's not universally supported on all the compilers on actively used OS's, in particular gcc on 18.04 does't support it. Given this I've avoided using it in the VSG and don't plan to support it. If you plan on porting your software you may wish to consider this. It would be good to have some level of interoperability between the VSG file system types/functions and std::filesystem. To that end could you print out the string that std::filesystem is generating? Could this also be an issue with std::wstring vs std::string? Cheers, Robert.

[sbrkopac]

Could you create a standard alone example that illustrates this problem, as
I could try and cobble together an example but if it worked then I'd have
no idea whether it's just not close enough to what you are doing, or that
it's a Window specific issue.

example.cpp.txt

Also could you output the path that std::filesystem is reporting? FYI,
std::filesystem may be part of the Standard but unfortunately it's not
universally supported on all the compilers on actively used OS's, in
particular gcc on 18.04 does't support it. Given this I've avoided using
it in the VSG and don't plan to support it. If you plan on porting your
software you may wish to consider this.

We don't normally use the native filesystem and use an in-memory one so this won't normally be an issue. It only popped up because I was trying to rapidly prototype something with the shaders and distribute it.

It would be good to have some level of interoperability between the VSG
file system types/functions and std::filesystem. To that end could you
print out the string that std::filesystem is generating? Could this also
be an issue with std::wstring vs std::string?

My compiler is reporting that it's just a normal std::string and the output is:

dataPath: "C:\\Programming\\projects\\vsg-siege\\build\\data"
dataPath(generic): C:/Programming/projects/vsg-siege/build/data

[robertosfield]

Thanks for the example. I have built and run it on my Ubuntu 21.04 system and get an exception throwin in ShaderModule.cpp:
Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:49
49 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) where
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:49
#1 0x00007ffff77b6864 in __GI_abort () at abort.c:79
#2 0x00007ffff7b88a31 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#3 0x00007ffff7b944fc in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#4 0x00007ffff7b94567 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6
#5 0x00007ffff7b94809 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6
#6 0x00005555556c2915 in vsg::ShaderModule::read (filename="")
at /home/robert/Dev/VulkanSceneGraph/src/vsg/state/ShaderModule.cpp:92
#7 0x000055555560573c in vsg::ShaderStage::read (in_stage=VK_SHADER_STAGE_FRAGMENT_BIT, in_entryPointName="main",
filename="") at /home/robert/Dev/VulkanSceneGraph/src/vsg/state/ShaderStage.cpp:57
#8 0x00005555555ec63b in main (argc=1, argv=0x7fffffffd1b8) at /home/robert/Contributors/FileSystem/example.cpp:43

The code in question is:
ref_ptr ShaderModule::read(const std::string& filename)
{
SPIRV buffer;
if (readFile(buffer, filename))
{
return ShaderModule::create(buffer);
}
else
{
throw Exception{"Error: vsg::ShaderModule::read(..) failed to read shader file.", VK_INCOMPLETE};
}
}

The debugger has the filename variable set to "".

The filename will be empty because the caller uses:

vsg::ref_ptr<vsg::ShaderStage> fragmentShader = vsg::ShaderStage::read(VK_SHADER_STAGE_FRAGMENT_BIT, "main", vsg::findFile("shaders/frag_PushConstants.spv", searchPaths));

Without checking the return value of vsg::findFile(). The ShaderModule::read() method is deprecated, it's now better to use vsg::read_cast<>:

vsg::Paths searchPaths = { dataPath.string() };

auto options = vsg::Options::create();
options->paths = searchPaths;

vsg::ref_ptr<vsg::ShaderStage> vertexShader = vsg::ShaderStage::create(VK_SHADER_STAGE_VERTEX_BIT, "main", vert_PushConstants);
//vsg::ref_ptr<vsg::ShaderStage> fragmentShader = vsg::ShaderStage::read(VK_SHADER_STAGE_FRAGMENT_BIT, "main", vsg::findFile("shaders/frag_PushConstants.spv", searchPaths));
vsg::ref_ptr<vsg::ShaderStage> fragmentShader = vsg::read_cast<vsg::ShaderStage>("shaders/frag_PushConstants.spv", options);

This return nulls when the search and/or load and it more coherent with how other types of files are read. Perhaps it's time for me to remove this old ShaderModule::read() method that is obvious too easy to use and not realize that it throws an exception to handle not being able to load a file.

[robertosfield]

I have restructured how vsg::ShaderStage:read(..) is implemented so that it now uses the standard vsg::read(..) mechanism and no longer throws an exception #317. The API has changed a bit with an additional ref_ptr parameter, but the original still compiles and your example now returns {} so that your example runs without crashing when the file isn't found.