Skip to content

Update autograd_saved_tensors_hooks_tutorial.py - #598

Merged
hyoyoung merged 4 commits into
PyTorchKR:masterfrom
stella-oh:stella-oh-patch-1
Sep 15, 2022
Merged

Update autograd_saved_tensors_hooks_tutorial.py#598
hyoyoung merged 4 commits into
PyTorchKR:masterfrom
stella-oh:stella-oh-patch-1

Conversation

@stella-oh

@stella-oh stella-oh commented Sep 6, 2022

Copy link
Copy Markdown
Contributor

๋ผ์ด์„ ์Šค ๋™์˜

๋ณ€๊ฒฝํ•ด์ฃผ์‹œ๋Š” ๋‚ด์šฉ์— BSD 3ํ•ญ ๋ผ์ด์„ ์Šค๊ฐ€ ์ ์šฉ๋จ์„ ๋™์˜ํ•ด์ฃผ์…”์•ผ ํ•ฉ๋‹ˆ๋‹ค.

๋” ์ž์„ธํ•œ ๋‚ด์šฉ์€ ๊ธฐ์—ฌํ•˜๊ธฐ ๋ฌธ์„œ๋ฅผ ์ฐธ๊ณ ํ•ด์ฃผ์„ธ์š”.

๋™์˜ํ•˜์‹œ๋ฉด ์•„๋ž˜ [ ]๋ฅผ [x]๋กœ ๋งŒ๋“ค์–ด์ฃผ์„ธ์š”.

  • ๊ธฐ์—ฌํ•˜๊ธฐ ๋ฌธ์„œ๋ฅผ ํ™•์ธํ•˜์˜€์œผ๋ฉฐ, ๋ณธ PR ๋‚ด์šฉ์— BSD 3ํ•ญ ๋ผ์ด์„ ์Šค๊ฐ€ ์ ์šฉ๋จ์— ๋™์˜ํ•ฉ๋‹ˆ๋‹ค.

๊ด€๋ จ ์ด์Šˆ ๋ฒˆํ˜ธ

์ด Pull Request์™€ ๊ด€๋ จ์žˆ๋Š” ์ด์Šˆ ๋ฒˆํ˜ธ๋ฅผ ์ ์–ด์ฃผ์„ธ์š”.

์ด์Šˆ ๋˜๋Š” PR ๋ฒˆํ˜ธ ์•ž์— #์„ ๋ถ™์ด์‹œ๋ฉด ์ œ๋ชฉ์„ ๋ฐ”๋กœ ํ™•์ธํ•˜์‹ค ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. (์˜ˆ. #999 )

PR ์ข…๋ฅ˜

์ด PR์— ํ•ด๋‹น๋˜๋Š” ์ข…๋ฅ˜ ์•ž์˜ [ ]์„ [x]๋กœ ๋ณ€๊ฒฝํ•ด์ฃผ์„ธ์š”.

  • ์˜คํƒˆ์ž๋ฅผ ์ˆ˜์ •ํ•˜๊ฑฐ๋‚˜ ๋ฒˆ์—ญ์„ ๊ฐœ์„ ํ•˜๋Š” ๊ธฐ์—ฌ
  • ๋ฒˆ์—ญ๋˜์ง€ ์•Š์€ ํŠœํ† ๋ฆฌ์–ผ์„ ๋ฒˆ์—ญํ•˜๋Š” ๊ธฐ์—ฌ
  • ๊ณต์‹ ํŠœํ† ๋ฆฌ์–ผ ๋‚ด์šฉ์„ ๋ฐ˜์˜ํ•˜๋Š” ๊ธฐ์—ฌ
  • ์œ„ ์ข…๋ฅ˜์— ํฌํ•จ๋˜์ง€ ์•Š๋Š” ๊ธฐ์—ฌ

PR ์„ค๋ช…

์ด PR๋กœ ๋ฌด์—‡์ด ๋‹ฌ๋ผ์ง€๋Š”์ง€ ๋Œ€๋žต์ ์œผ๋กœ ์•Œ๋ ค์ฃผ์„ธ์š”.

@bub3690 bub3690 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

๋ฒˆ์—ญ๋“ค์ด ์•„๋ž˜ ์ค„๋กœ ์ ํ˜€์žˆ์Šต๋‹ˆ๋‹ค.

์˜๋ฌธ์„ ์ง€์šฐ๊ณ  ๋‹ค์‹œ ์ ์–ด์ฃผ์…”์•ผ ํ•ฉ๋‹ˆ๋‹ค.

@bub3690 bub3690 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

๋ฌธ๋ฒ•์ด ๊นจ์ง„ ๋ถ€๋ถ„์ด ๋งŽ์•„ ๋ณด์ž…๋‹ˆ๋‹ค.

๋นŒ๋“œ ๋ถ€ํƒ๋“œ๋ฆฝ๋‹ˆ๋‹ค.

# :width: 500
# :align: center

# .. ๊ทธ๋ฆผ :: https://user-images.githubusercontent.com/8019486/130124570-f1074098-1bb3-459e-bf5a-03bf6f65b403.png

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.. figure
:width

์ด ๋ถ€๋ถ„๋“ค์€ ๋ฌธ๋ฒ•์ด๋ผ ๋ฒˆ์—ญ์„ ํ•˜๋ฉด ์•ˆ๋  ๊ฒƒ์œผ๋กœ ๋ณด์ด๋„ค์š”.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

๋„ต.. ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค. ๋ฐ˜์˜ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.

# 1000) and how to combine this feature with ``nn.DataParallel``.
#
# ์ด ๋งˆ์ง€๋ง‰ ์˜ˆ์—์„œ๋Š”
# ๋ฐ˜๋“œ์‹œ ์ €์žฅํ•ด์•ผํ•˜๋ฉฐ ์ด ๊ธฐ๋Šฅ์„ ``nn.DataParallel``์™€ ์–ด๋–ป๊ฒŒ ์กฐํ•ฉํ•˜๋Š” ํ…์„œ๋“ค์— ๋Œ€ํ•ด ์–ด๋–ป๊ฒŒ ํ•„ํ„ฐํ•˜๋Š”์ง€์— ๋Œ€ํ•œ ์ฆ๋ช…์„ ํ•ด๋ณด์•˜์Šต๋‹ˆ๋‹ค.

@bub3690 bub3690 Sep 6, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

`` ์•ž๋’ค๋กœ ๋›ฐ์–ด์“ฐ๊ธฐ๊ฐ€ ์žˆ์–ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
๋นŒ๋“œํ•ด์„œ ํ™•์ธ ๋ถ€ํƒ๋“œ๋ฆฝ๋‹ˆ๋‹ค.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

๋„ต.. ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค. ๋ฐ˜์˜ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.

@stella-oh
stella-oh requested a review from bub3690 September 6, 2022 11:56
@hyoyoung hyoyoung added the ์ปจํŠธ๋ฆฌ๋ทฐํ†ค ์˜คํ”ˆ์†Œ์Šค ์ปจํŠธ๋ฆฌ๋ทฐํ†ค ๊ด€๋ จ ์ด์Šˆ/PR label Sep 9, 2022

@hyoyoung hyoyoung left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

๋ฒˆ์—ญ์ด ์™„๋ฃŒ๋œ ์˜์–ด ๋ถ€๋ถ„์€ ์ œ๊ฑฐ ๋ถ€ํƒ๋“œ๋ฆฝ๋‹ˆ๋‹ค.

@@ -1,5 +1,6 @@
"""
Hooks for autograd saved tensors
๋ฏธ๋ถ„์ž๋™ํ™”(autograd) ์ €์žฅ๋œ ํ…์„œ๋ฅผ ์œ„ํ•œ HOOK

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hooks๋ฅผ ๋ฒˆ์—ญ ์•ˆํ•˜๊ธฐ๋กœ ๊ฒฐ์ •ํ•˜์…จ๋‹ค๋ฉด
HOOK์ด ์•„๋‹ˆ๋ผ Hooks๋ฅผ ๊ทธ๋Œ€๋กœ ๋‘์‹œ๋Š”๊ฒŒ ๋” ๋‚˜์„๋“ฏํ•ฉ๋‹ˆ๋‹ค

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

๋„ต.. ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค. ๋ฐ˜์˜ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.

@gorae17 gorae17 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

๊ณ ์ƒํ•˜์…จ์Šต๋‹ˆ๋‹ค. ์ด๋ฏธ ๋ฆฌ๋ทฐ๋ฐ›์œผ์‹  ๊ฒƒ์ฒ˜๋Ÿผ, ์˜๋ฌธ์€ ์ง€์šฐ๊ณ  ์˜๋ฌธ ์œ„์น˜์— ๋ฒˆ์—ญ ๋ฌธ์žฅ์„ ์ ์€ ํ›„ make html-noplot์œผ๋กœ ๋นŒ๋“œ ํ›„ html ํŒŒ์ผ์„ ํ™•์ธํ•ด๋ณด์‹œ๋ฉด ๊ฒฐ๊ณผ ์‚ดํŽด๋ณด์‹œ๋Š” ๋ฐ์— ๋„์›€์ด ๋  ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค.

######################################################################
# In this example, PyTorch saves intermediary values :math:`a` and
# :math:`b` in order to compute the gradient during the backward.
# ์ด ์˜ˆ์ œ์—์„œ ํŒŒ์ดํ† ์น˜๋Š” ์ค‘๊ฐ„ ๊ฐ’ :math:`a` ๋ฐ :math:`b`๋ฅผ ์ €์žฅํ•˜์—ฌ ์—ญ๋ฐฉํ–ฅ ๋™์•ˆ ๊ธฐ์šธ๊ธฐ๋ฅผ ๊ณ„์‚ฐํ•ฉ๋‹ˆ๋‹ค.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

` ๊ฐ€ ํ•˜๋‚˜ ์žˆ์„ ๋•Œ์—๋„ ์•ž๋’ค๋กœ ๋„์–ด์“ฐ๊ธฐ๊ฐ€ ์žˆ์–ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

๋„ต.. ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค. ๋ฐ˜์˜ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.

@stella-oh

Copy link
Copy Markdown
Contributor Author
  1. ์˜๋ฌธ์‚ญ์ œ
  2. ๋ฌธ๋ฒ• ๊นจ์ง„๋ถ€๋ถ„ ์žฌ๋นŒ๋“œ
  3. ์–ดํผ์ŠคํŠธ๋กœํ”ผ ์‚ฌ์šฉ์— ๋Œ€ํ•œ ์•ž๋’ค ๊ณต๋ฐฑ
  4. HOOK-> hooks(original)
    ์œ„์— ๋Œ€ํ•œ ์ˆ˜์ •๋‚ด์šฉ์„ ๋ฐ˜์šฉํ•˜์—ฌ prํ–ˆ์Šต๋‹ˆ๋‹ค.. ํ™•์ธ ๋ถ€ํƒ๋“œ๋ฆฝ๋‹ˆ๋‹ค.

# ๊ทธ๋Ÿฌ๋‚˜ ํŠน์ • ์ž‘์—…์—์„œ๋Š” ์—ญ์ „ํŒŒ๋ฅผ ์ˆ˜ํ–‰ํ•˜๊ธฐ ์œ„ํ•œ ์ค‘๊ฐ„๊ฒฐ๊ณผ๋ฅผ ์ €์žฅํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
# ์ด๋ฒˆ ํŠœํ† ๋ฆฌ์–ผ์—์„œ๋Š” ์ด๋Ÿฌํ•œ ํ…์„œ๋ฅผ ์ €์žฅ/๊ฒ€์ƒ‰ํ•˜๋Š” ๋ฐฉ๋ฒ•๊ณผ ํŒจํ‚น/์–ธํŒจํ‚น ํ”„๋กœ์„ธ์Šค๋ฅผ ์ œ์–ดํ•˜๊ธฐ์œ„ํ•œ hooks์„ ์ •ํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์•ˆ๋‚ดํ•ฉ๋‹ˆ๋‹ค.

# ์ด ํŠœํ† ๋ฆฌ์–ผ์—์„œ๋Š” ์‚ฌ์šฉ์ž๊ฐ€ ์—ญ์ „ํŒŒ๊ฐ€ ์ด๋ก ์ƒ์—์„œ ์ž‘๋™ํ•˜๋Š” ๋ฐฉ์‹์— ์ต์ˆ™ํ•˜๋‹ค๊ณ  ๊ฐ€์ •ํ•ฉ๋‹ˆ๋‹ค. ์•„๋‹ˆ๋ผ๋ฉด, ์•„๋ž˜์˜ ์ž๋ฃŒ๋ฅผ ๋จผ์ € ์ฝ์–ด๋ณด์„ธ์š”.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

์›๋ฌธ์— ์ถฉ์‹คํ•˜๊ฒŒ ๋ฒˆ์—ญ์„ ํ•ด์ฃผ์…จ๋Š”๋ฐ์š”. ๊ฐœ์ธ์ ์ธ ์˜๊ฒฌ์œผ๋กœ๋Š” ์•„๋ž˜์™€ ๊ฐ™์ด ์ˆ˜์ •ํ•˜๋ฉด ์กฐ๊ธˆ ๋” ์ž์—ฐ์Šค๋Ÿฝ์ง€ ์•Š์„๊นŒํ•˜์—ฌ ์ œ์•ˆ๋“œ๋ ค๋ด…๋‹ˆ๋‹ค.
'์ด ํŠœํ† ๋ฆฌ์–ผ์—์„œ๋Š” ์—ญ์ „ํŒŒ๊ฐ€ ์ด๋ฃจ์–ด์ง€๋Š” ๋ฐฉ์‹์— ๋Œ€ํ•ด ๋…์ž๊ฐ€ ์ด๋ก ์ ์œผ๋กœ ์ž˜ ์•Œ๊ณ  ์žˆ๋‹ค๊ณ  ๊ฐ€์ •ํ•ฉ๋‹ˆ๋‹ค.'

# as it never copies any tensors. However, the graph can keep *references*
# to tensors that would otherwise have gone out of scope: those are
# referred to as **saved tensors**.
# ๋ชจ๋ธ์˜ ํ›ˆ๋ จ์€ ์ผ๋ฐ˜์ ์œผ๋กœ ๊ฐ€์„ค์˜ ์ถ”๋ก ์„ ์œ„ํ•ด์„œ ์‹คํ–‰ํ•˜๋Š” ๊ฒƒ๋ณด๋‹ค ๋” ๋งŽ์€ ๋ฉ”๋ชจ๋ฆฌ๋ฅผ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

๊ฐ€์„ค์˜ ์ถ”๋ก ์ด๋ผ๋Š” ๋œป๋ณด๋‹ค๋Š” inference (classification์ด๋‚˜ regression ๊ฐ™์€ prediction) ์ธก๋ฉด์—์„œ ์˜๋ฏธ๊ฐ€ ์ „๋‹ฌ๋˜๋Š” ๊ฒƒ์ด ๋” ์ ํ•ฉํ•  ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค. (ํ•œ๊ตญ์–ด๋กœ ํ‘œํ˜„ํ•˜๋ฉด '๋ชจ๋ธ์„ ์ธํผ๋Ÿฐ์Šคํ•˜๋Š” ๊ฒƒ๋ณด๋‹ค ํ•™์Šตํ•˜๋Š” ๊ณผ์ •์—์„œ ๋ฉ”๋ชจ๋ฆฌ๊ฐ€ ๋” ๋งŽ์ด ์‚ฌ์šฉ๋œ๋‹ค' ์ด๋Ÿฐ ๋œป์œผ๋กœ ์›Œ๋”ฉ์ด ์กฐ๊ธˆ ์ˆ˜์ •๋˜๋ฉด ์ข‹์„ ๊ฒƒ ๊ฐ™์•„์š”!

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

์ด ๋ถ€๋ถ„์€ @garam24 ๋‹˜ ๋ง์”€๋Œ€๋กœ ์ˆ˜์ •ํ•˜๋Š”๊ฒŒ ์ข‹์„๋“ฏํ•ฉ๋‹ˆ๋‹ค.
๋ฌธ๋งฅ์ƒ ์•Œ๊ธฐ๊ฐ€ ์–ด๋ ต์Šต๋‹ˆ๋‹ค


######################################################################
# Why does training a model (typically) requires more memory than evaluating it?
# (์ผ๋ฐ˜์ ์œผ๋กœ)๋ชจ๋ธ์„ ํ›ˆ๋ จํ•˜๋Š”๋ฐ ํ‰๊ฐ€๋ณด๋‹ค ๋” ๋งŽ์€ ๋ฉ”๋ชจ๋ฆฌ๋ฅผ ์†Œ์š”ํ•˜๊ฒŒ ๋˜๋Š” ์ด์œ ๋Š” ๋ฌด์—‡์ผ๊นŒ์š”?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'๋ฉ”๋ชจ๋ฆฌ๋ฅผ ์†Œ์š”ํ•˜๊ฒŒ'๋ณด๋‹ค '์†Œ๋ชจํ•˜๋Š”', 'ํ•„์š”๋กœ ํ•˜๋Š”' ๋“ฑ์œผ๋กœ ์ˆ˜์ •ํ•˜๋ฉด ๋” ์ž์—ฐ์Šค๋Ÿฌ์šธ ๊ฒƒ ๊ฐ™์•„ ์ œ์•ˆ ์˜๊ฒฌ ๋“œ๋ฆฝ๋‹ˆ๋‹ค.

@hyoyoung hyoyoung left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

๋ช‡๊ฐ€์ง€ ์ˆ˜์ •์‚ฌํ•ญ์„ ๋” ๋ถ€ํƒ๋“œ๋ฆฝ๋‹ˆ๋‹ค.

"""
Hooks for autograd saved tensors
๋ฏธ๋ถ„์ž๋™ํ™”(autograd) ์ €์žฅ๋œ ํ…์„œ๋ฅผ ์œ„ํ•œ Hooks
=======================

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

๊ธธ๊ฒŒ๋œ ์ œ๋ชฉ๋งŒํผ =๋„ ๋Š˜๋ ค์ฃผ์„ธ์š”

# as it never copies any tensors. However, the graph can keep *references*
# to tensors that would otherwise have gone out of scope: those are
# referred to as **saved tensors**.
# ๋ชจ๋ธ์˜ ํ›ˆ๋ จ์€ ์ผ๋ฐ˜์ ์œผ๋กœ ๊ฐ€์„ค์˜ ์ถ”๋ก ์„ ์œ„ํ•ด์„œ ์‹คํ–‰ํ•˜๋Š” ๊ฒƒ๋ณด๋‹ค ๋” ๋งŽ์€ ๋ฉ”๋ชจ๋ฆฌ๋ฅผ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

์ด ๋ถ€๋ถ„์€ @garam24 ๋‹˜ ๋ง์”€๋Œ€๋กœ ์ˆ˜์ •ํ•˜๋Š”๊ฒŒ ์ข‹์„๋“ฏํ•ฉ๋‹ˆ๋‹ค.
๋ฌธ๋งฅ์ƒ ์•Œ๊ธฐ๊ฐ€ ์–ด๋ ต์Šต๋‹ˆ๋‹ค

# to tensors that would otherwise have gone out of scope: those are
# referred to as **saved tensors**.
# ๋ชจ๋ธ์˜ ํ›ˆ๋ จ์€ ์ผ๋ฐ˜์ ์œผ๋กœ ๊ฐ€์„ค์˜ ์ถ”๋ก ์„ ์œ„ํ•ด์„œ ์‹คํ–‰ํ•˜๋Š” ๊ฒƒ๋ณด๋‹ค ๋” ๋งŽ์€ ๋ฉ”๋ชจ๋ฆฌ๋ฅผ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค.
# ๋Œ€๋žต์ ์œผ๋กœ ๋งํ•˜๋ฉด ํŒŒ์ดํ† ์น˜๋Š” ์—ญ์ „ํŒŒ๋ฅผ ํ˜ธ์ถœํ•˜๋Š”๋ฐ ํ•„์š”ํ•œ ๊ณ„์‚ฐ ๊ทธ๋ž˜ํ”„๋ฅผ ์ €์žฅํ•ด์•ผํ•˜๋ฏ€๋กœ ์ถ”๊ฐ€ ๋ฉ”๋ชจ๋ฆฌ ์‚ฌ์šฉ๋Ÿ‰์ด ์žˆ๊ธฐ ๋•Œ๋ฌธ์ด๋‹ค ๋ผ๊ณ  ๋งํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

๋Œ€๋žต์ ์œผ๋กœ ๋งํ•˜์ž๋ฉด ~ ๋งํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค
๋ผ๊ณ  ๋๋‚ฉ๋‹ˆ๋‹ค ์ด ๋ฌธ์žฅ์€

๋Œ€๋žต์ ์œผ๋กœ ๋งํ•˜๋ฉด ํŒŒ์ดํ† ์น˜๋Š” ์—ญ์ „ํŒŒ๋ฅผ ํ˜ธ์ถœํ•˜๋Š”๋ฐ ํ•„์š”ํ•œ ๊ณ„์‚ฐ ๊ทธ๋ž˜ํ”„๋ฅผ ์ €์žฅํ•ด์•ผํ•˜๋ฏ€๋กœ ์ถ”๊ฐ€ ๋ฉ”๋ชจ๋ฆฌ ์‚ฌ์šฉ๋Ÿ‰์ด ์žˆ๊ธฐ ๋•Œ๋ฌธ์ž…๋‹ˆ๋‹ค.

์ •๋„๋กœ ์ˆ˜์ •ํ•˜๋Š”๊ฒŒ ์ข‹์„๋“ฏ ํ•ฉ๋‹ˆ๋‹ค

# In fact, PyTorch provides an API to conveniently use those hooks (as
# well as the ability to use pinned memory).
#
# ์‹ค์ œ๋กœ ํŒŒ์ดํ† ์น˜๋Š” ์ด๋Ÿฌํ•œ ํ›„ํฌ๋ฅผ ํŽธ๋ฆฌํ•˜๊ฒŒ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋Š” API๋ฅผ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค. (๊ณ ์ •๋œ ๋ฉ”๋ชจ๋ฆฌ๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ๊ธฐ๋Šฅ๋„ ํฌํ•จ.)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ํ›„ํฌ -> hooks

# module and saves its tensors to CPU.
# ๋ฌผ๋ก  ๋„คํŠธ์›Œํฌ ํŠน์ •๋ถ€๋ถ„๋งŒ CPU์— ์ €์žฅํ•˜์—ฌ ์ ˆ์ถฉ์•ˆ์„ ์กฐ์ •ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
#
# ์˜ˆ๋ฅผ ๋“ค์–ด, ๋ชจ๋“  ๋ชจ๋“ˆ์„ ๋ž˜ํ•‘ํ•˜๊ณ  ํ•ด๋‹น ํ…์„œ๋ฅผ CPU์— ์ €์žฅํ•˜๋Š” ํŠน๋ณ„ํ•œ

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

๋ž˜ํ•‘ -> ๊ฐ์‹ธ๋‘๊ณ 

# The reason the above code is bad is that we are leaking files on the
# disk and they are never cleared. Fixing this is not as trivial as it
# seems.
# ์œ„์˜ ์ฝ”๋“œ๊ฐ€ ๋‚˜์œ ์ด์œ ๋Š” ๋””์Šคํฌ๋Š” ์ง€์›Œ์ง€์ง€ ์•Š์Šต๋‹ˆ๋‹ค.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

์œ„์˜ ์ฝ”๋“œ๊ฐ€ ๋‚˜์œ ์ด์œ ๋Š” ๋””์Šคํฌ์— ์ €์žฅ๋œ ํŒŒ์ผ์ด ๋ˆ„์ถœ๋˜๊ณ  ํ•ด๋‹น ํŒŒ์ผ์„ ์ง€์šธ์ˆ˜๋„ ์—†๊ธฐ ๋•Œ๋ฌธ์ž…๋‹ˆ๋‹ค.

# saved tensor hooks and how they can be useful in a few scenarios to
# tradeoff memory for compute.
#
# ๋งŒ์ผ ์—ฌ๊ธฐ๊นŒ์ง€ ๋”ฐ๋ผ์˜ค์…จ๋‹ค๋ฉด, ์ถ•ํ•˜ํ•ฉ๋‹ˆ๋‹ค! ๋‹น์‹ ์€ ์ €์žฅ๋œ ํ…์„œ hooks์„ ์–ด๋–ป๊ฒŒ ์‚ฌ์šฉํ•˜๋Š”์ง€ ๊ทธ๋ฆฌ๊ณ  ๊ณ„์‚ฐ ๋ฉ”๋ชจ๋ฆฌ์˜ ํŠธ๋ ˆ์ด๋“œ ์˜คํ”„์— ์žˆ์–ด์„œ ๋ช‡๊ฐ€์ง€ ์‹œ๋‚˜๋ฆฌ์˜ค๋ฅผ ์œ ์šฉํ•˜๊ฒŒ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค!

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

๊ณ„์‚ฐ ๋ฉ”๋ชจ๋ฆฌ์˜ ํŠธ๋ ˆ์ด๋“œ ์˜คํ”„์— ์žˆ์–ด์„œ ๋ช‡๊ฐ€์ง€ ์‹œ๋‚˜๋ฆฌ์˜ค๋ฅผ ์œ ์šฉํ•˜๊ฒŒ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค!

->

๊ณ„์‚ฐ ๋ฉ”๋ชจ๋ฆฌ์˜ ์ ˆ์ถฉ์•ˆ์—์„œ ๋ช‡๊ฐ€์ง€ ์‹œ๋‚˜๋ฆฌ์˜ค๋ฅผ ์œ ์šฉํ•˜๊ฒŒ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค!

@hyoyoung hyoyoung left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good

@hyoyoung
hyoyoung merged commit 04bfd08 into PyTorchKR:master Sep 15, 2022
hyoyoung added a commit that referenced this pull request Sep 15, 2022
hyoyoung added a commit that referenced this pull request Sep 15, 2022
hyoyoung added a commit that referenced this pull request Sep 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

์ปจํŠธ๋ฆฌ๋ทฐํ†ค ์˜คํ”ˆ์†Œ์Šค ์ปจํŠธ๋ฆฌ๋ทฐํ†ค ๊ด€๋ จ ์ด์Šˆ/PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants