Harmonise Convert Naming Convention for Binaries Between Function Arguments and Metadata Blocks and Struct Members

[andrew-lunarg]

The original Vulkan binary dumps of function arguments use hand-written names in the form <global_counter>-<manual_name>.bin whereas the structs in D3D12 from here use <struct_name>.<field_name>-<instance_counter>.bin. Metadata blocks for both APIs share this naming.

Proposal for naming the binary blob files:

• Method Argument: marg-<object_name>.<method_name>.<argument_name>-<instance_counter>.bin
• Function Argument: farg-<function_name>.<argument_name>-<instance_counter>.bin
• Struct Member: smem-<struct_name>.<field_name>-<instance_counter>.bin
• Metadata Block Member: mmem-<metadata_name>.<field_name>-<instance_counter>.bin

instance_counter: fixed width numeric field with leading zeros so files sort cleanly in ls, dir, and in GUI file managers.

Proposal for Implemention

• Redirect the RepresentBinaryFile function in JsonWriter through the version in json_util.h/cpp
• Copy the code generation for the D3D12 structs case using a static thread local counter to the other places that binaries are dumped:

  static thread_local uint64_t D3D12_SHADER_BYTECODE_pShaderBytecode_counter{ 0 };
  const bool written = RepresentBinaryFile(options, jdata["pShaderBytecode"], "D3D12_SHADER_BYTECODE.pShaderBytecode", D3D12_SHADER_BYTECODE_pShaderBytecode_counter, meta_struct.pShaderBytecode);
  D3D12_SHADER_BYTECODE_pShaderBytecode_counter += written;
  

Steps above will have to be adjusted to use the suggested prefixes.

There may be users of the original naming scheme who need a heads-up, although a script just using the filenames in a json file to map across to binaries should be fine, its only a script running directly on the directory of binaries that might have been written with an assumption of the naming used.

Caveat

This is a proposal. the inverse approach to harmonisation of giving each struct member dumped to a binary a well-known manually created short name is equally valid. I do think it makes sense to suffix the counters rather than prefix them though, and to have one counter per blob type (especially because architecturally it is hard for struct convert functions (FieldToJson for a struct in current naming) to get access to the global counter inside the JsonWriter without a rearchitecture of the Json dumping and so structs at least will have to use local counters at present and it makes sense to be consistent).

Further Work

Note, it might be useful in the future, out of the scope of this issue, to name binaries as the hash of their contents so binaries and the JSON files referencing them can't get out of sync as the two are archived and moved around over time.

See Also

There is a bunch of work that could be done around the dumping binaries feature. The initial implementation for Vulkan which defined the current commandline argument dumps the exact binaries desired by the implementor and has on/off coarse control. As we add more binaries controlled by the same flag, it will no longer be optimal for them. If we move to a naming convention automatable in codegen like this issue proposes the original implementor will need to adapt their downstream tooling. If issues could be grouped, these would be considered all at once:
#1345 #1357 #1367