Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 15 additions & 18 deletions recipes_source/recipes/reasoning_about_shapes.py
Original file line number Diff line number Diff line change
@@ -1,18 +1,15 @@
"""
Reasoning about Shapes in PyTorch
PyTorch의 Shape들에 대한 추론
=================================
번역: `이영섭 <https://github.com/0seob>`_

When writing models with PyTorch, it is commonly the case that the parameters
to a given layer depend on the shape of the output of the previous layer. For
example, the ``in_features`` of an ``nn.Linear`` layer must match the
``size(-1)`` of the input. For some layers, the shape computation involves
complex equations, for example convolution operations.
일반적으로 PyTorch로 모델을 작성할 때 특정 계층의 매개변수는 이전 계층의 출력 shape에 따라 달라집니다.
예를 들어, ``nn.Linear`` 계층의 ``in_features`` 는 입력의 ``size(-1)`` 와 일치해야 합니다.
몇몇 계층의 경우, shape 계산은 합성곱 연산과 같은 복잡한 방정식을 포함합니다.

One way around this is to run the forward pass with random inputs, but this is
wasteful in terms of memory and compute.
이를 랜덤한 입력으로 순전파(forward pass)를 실행하여 해결할 수 있지만, 이는 메모리와 컴퓨팅 파워를 낭비합니다.

Instead, we can make use of the ``meta`` device to determine the output shapes
of a layer without materializing any data.
대신에 ``meta`` 디바이스를 활용한다면 데이터를 구체화하지 않고도 계층의 출력 shape을 결정할 수 있습니다.
"""

import torch
Expand All @@ -29,8 +26,8 @@


##########################################################################
# Observe that since data is not materialized, passing arbitrarily large
# inputs will not significantly alter the time taken for shape computation.
# 데이터가 구체화되지 않기 때문에 임의로 큰 입력을 전달해도 shape 계산에 소요되는 시간이
# 크게 변경되지는 않습니다.

t_large = torch.rand(2**10, 3, 2**16, 2**16, device="meta")
start = timeit.default_timer()
Expand All @@ -42,7 +39,7 @@


######################################################
# Consider an arbitrary network such as the following:
# 다음과 같은 임의의 네트워크를 가정합니다:

import torch.nn as nn
import torch.nn.functional as F
Expand All @@ -61,23 +58,23 @@ def __init__(self):
def forward(self, x):
x = self.pool(F.relu(self.conv1(x)))
x = self.pool(F.relu(self.conv2(x)))
x = torch.flatten(x, 1) # flatten all dimensions except batch
x = torch.flatten(x, 1) # 배치를 제외한 모든 차원을 평탄화 합니다.
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
x = self.fc3(x)
return x


###############################################################################
# We can view the intermediate shapes within an entire network by registering a
# forward hook to each layer that prints the shape of the output.
# 각각의 계층에 출력의 shape을 인쇄하는 forward hook을 등록하여 네트워크의
# 중간 shape을 확인할 수 있습니다.

def fw_hook(module, input, output):
print(f"Shape of output to {module} is {output.shape}.")


# Any tensor created within this torch.device context manager will be
# on the meta device.
# torch.device context manager(with 구문) 내부에서 생성된 모든 tensor는
# meta 디바이스 내부에 존재합니다.
with torch.device("meta"):
net = Net()
inp = torch.randn((1024, 3, 32, 32))
Expand Down
4 changes: 2 additions & 2 deletions recipes_source/recipes_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -122,8 +122,8 @@ Recipes are bite-sized bite-sized, actionable examples of how to use specific Py
:tags: Basics

.. customcarditem::
:header: Reasoning about Shapes in PyTorch
:card_description: Learn how to use the meta device to reason about shapes in your model.
:header: PyTorch의 Shape에 대한 추론
:card_description: meta 디바이스를 사용하여 모델의 shape을 추론하는 방법을 알아봅니다.
:image: ../_static/img/thumbnails/cropped/generic-pytorch-logo.png
:link: ../recipes/recipes/reasoning_about_shapes.html
:tags: Basics
Expand Down