Skip to content
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update on "[ET-VK][Ops] aten.convolution (Bias=False)"
The final touches to get ET-VK convolution on-par with ATen-VK's convolution.

## Idea
In our shaders, we add the bias to our sum.
```
${VEC4_T[DTYPE]} sum = texelFetch(bias_in, ivec2(pos.z, 0), 0);
```
To keep our shaders as is, we implement having no bias by allocating a buffer of zeros. Then, our shader adds zero to our sum.

## Issue
If `Bias=False`, dummy buffer of zeros is not serialized with the graph. The bias ValueRef is deserialized in the runtime as `TypeTag::NONE`, not `TypeTag::TENSORREF`.

## Solution
If `TypeTag::NONE` is given, (1) create the `vTensor` using the `out_channels` value from the weights and (2) allocate a StagingBuffer of that size. The StagingBuffer will be transferred to GPU memory and initialized to zeros.

Differential Revision: [D55814589](https://our.internmc.facebook.com/intern/diff/D55814589/)

[ghstack-poisoned]
  • Loading branch information
Jorge Pineda committed Apr 8, 2024
commit bf72506cb5b3021c8ae03873f7c7020ec9759964
4 changes: 3 additions & 1 deletion backends/vulkan/runtime/graph/ops/PrepackNode.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,13 @@ PrepackNode::PrepackNode(
api::StorageBuffer PrepackNode::create_staging_buffer(ComputeGraph* graph) {
vTensor& packed = graph->get_val(packed_).toTensor();

// If no TensorRef is provided, create a zeroed staging buffer according to
// If no TensorRef is provided, create a staging buffer of zeros according to
// the vTensor metadata.
if (graph->get_val(tref_).isNone()) {
size_t numel = api::utils::multiply_integers(packed.sizes());
api::StorageBuffer staging(graph->context(), packed.dtype(), numel);
size_t nbytes = numel * api::element_size(packed.dtype());
copy_zeros_to_staging(staging, nbytes);
return staging;
}

Expand Down
7 changes: 7 additions & 0 deletions backends/vulkan/runtime/graph/ops/utils/StagingUtils.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,13 @@ void copy_staging_to_ptr(
memcpy_from_mapping(mapping, dst, nbytes, staging.dtype());
}

void copy_zeros_to_staging(api::StorageBuffer& staging, const size_t nbytes) {
void* data = malloc(nbytes);
memset(data, 0, nbytes);
copy_ptr_to_staging(data, staging, nbytes);
free(data);
}

api::ShaderInfo get_nchw_to_image_shader(const vTensor& v_dst) {
if (v_dst.is_quantized()) {
VK_THROW("Quantized Tensors are currently not supported!");
Expand Down
2 changes: 2 additions & 0 deletions backends/vulkan/runtime/graph/ops/utils/StagingUtils.h
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ void copy_staging_to_ptr(
void* dst,
const size_t nbytes);

void copy_zeros_to_staging(api::StorageBuffer& staging, const size_t nbytes);

//
// Functions to get shaders
//
Expand Down