์•ˆ๋…•ํ•˜์„ธ์š”. 

 

Plain Network(๋‹จ์ˆœํžˆ Layer์„ ๊นŠ๊ฒŒ ์Œ“์Œ)์—์„œ ๋ฐœ์ƒํ•˜๋Š” Vanishing Gradient(๊ธฐ์šธ๊ธฐ ์†Œ์‹ค), Overfitting(๊ณผ์ ํ•ฉ) ๋“ฑ์˜ ๋ฌธ์ œ๋ฅผ ํ•ด๊ฒฐํ•˜๊ธฐ ์œ„ํ•ด ReLU, Batch Nomalization ๋“ฑ ๋งŽ์€ ๊ธฐ๋ฒ•์ด ์žˆ์Šต๋‹ˆ๋‹ค. 

 

ILSVRC Challenge
2021๋…„ 3์›” 24์ผ ๊ธฐ์ค€ ์ธ์šฉ

ILSVRC ๋Œ€ํšŒ์—์„œ 2015๋…„, ์ฒ˜์Œ์œผ๋กœ Human Recognition๋ณด๋‹ค ๋†’์€ ์„ฑ๋Šฅ์„ ๋ณด์ธ ๊ฒƒ์ด ResNet์ž…๋‹ˆ๋‹ค.

๊ทธ ์œ„์šฉ์€ ๋ฌด์ง€๋ง‰์ง€ํ•œ ๋…ผ๋ฌธ ์ธ์šฉ ์ˆ˜๋กœ ํ™•์ธํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. 

 

๊ทธ๋ ‡๊ธฐ ๋•Œ๋ฌธ์— ResNet์€ ๋”ฅ๋Ÿฌ๋‹ ์ด๋ฏธ์ง€ ๋ถ„์•ผ์—์„œ ๋ฐ”์ด๋ธ”๋กœ ํ†ตํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. 

 

Plain Netwrok Vs ResNet

Plain Network๋Š” ๋‹จ์ˆœํžˆ Convolution ์—ฐ์‚ฐ์„ ๋‹จ์ˆœํžˆ ์Œ“๋Š”๋‹ค๋ฉด, ResNet์€ Block๋‹จ์œ„๋กœ Parameter์„ ์ „๋‹ฌํ•˜๊ธฐ ์ „์— ์ด์ „์˜ ๊ฐ’์„ ๋”ํ•˜๋Š” ๋ฐฉ์‹์ž…๋‹ˆ๋‹ค. 

 

Residual Block

F(x) : weight layer => relu => weight layer 

x : identity

 

weight layer๋“ค์„ ํ†ต๊ณผํ•œ F(x)์™€ weight layer๋“ค์„ ํ†ต๊ณผํ•˜์ง€ ์•Š์€ x์˜ ํ•ฉ์„ ๋…ผ๋ฌธ์—์„œ๋Š” Residual Mapping ์ด๋ผ ํ•˜๊ณ , ๊ทธ๋ฆผ์˜ ๊ตฌ์กฐ๋ฅผ Residual Block์ด๋ผ ํ•˜๊ณ , Residual Block์ด ์Œ“์ด๋ฉด Residual  Network(ResNet)์ด๋ผ๊ณ  ํ•ฉ๋‹ˆ๋‹ค.

Residual Mapping์€ ๊ฐ„๋‹จํ•˜์ง€๋งŒ, Overfitting, Vanishing Gradient ๋ฌธ์ œ๊ฐ€ ํ•ด๊ฒฐ๋˜์–ด ์„ฑ๋Šฅ์ด ํ–ฅ์ƒ๋์Šต๋‹ˆ๋‹ค.

๊ทธ๋ฆฌ๊ณ  ๋‹ค์–‘ํ•œ ๋„คํŠธ์›Œํฌ ๊ตฌ์กฐ์—์„œ ์‚ฌ์šฉ๋˜๋ฉฐ, 2017๋…„ ILSVRC์„ ์šฐ์Šนํ•œ SeNet์—์„œ ์‚ฌ์šฉ๋ฉ๋‹ˆ๋‹ค. ( ์ด ๊ธ€์„ ์“ด ์ด์œ ์ด๊ธฐ๋„ ํ•ฉ๋‹ˆ๋‹ค. ) 

 

Plain Network VS ResNet (Error)

 

Residual Block

class Residual_Block(nn.Module):
    def __init__(self, in_dim, mid_dim, out_dim):
        super(Residual_Block,self).__init__()
        # Residual Block
        self.residual_block = nn.Sequential(
                nn.Conv2d(in_dim, mid_dim, kernel_size=3, padding=1),
                nn.ReLU,
                nn.Conv2d(mid_dim, out_dim, kernel_size=3, padding=1),
            )            
        self.relu = nn.ReLU()
                  
    def forward(self, x):
       out = self. residual_block(x)  # F(x)
        out = out + x  # F(x) + x
        out = self.relu(out)
        return out
    

 

๊ทธ๋ฆฌ๊ณ  Residual Block ์†Œ๊ฐœ ํ›„ BottleNeck์ด ๋‚˜์˜ต๋‹ˆ๋‹ค. ์•„๋ž˜ ๊ธ€์„ ์ฐธ๊ณ ํ•˜์‹œ๋ฉด ์ข‹์„ ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค. 

coding-yoon.tistory.com/116?category=825914

 

[๋”ฅ๋Ÿฌ๋‹] DeepLearning CNN BottleNeck ์›๋ฆฌ(Pytorch ๊ตฌํ˜„)

์•ˆ๋…•ํ•˜์„ธ์š”. ์˜ค๋Š˜์€ Deep Learning ๋ถ„์•ผ์—์„œ CNN์˜ BottleNeck๊ตฌ์กฐ์— ๋Œ€ํ•ด ์•Œ์•„๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค. ๋Œ€ํ‘œ์ ์œผ๋กœ ResNet์—์„œ BottleNeck์„ ์‚ฌ์šฉํ–ˆ์Šต๋‹ˆ๋‹ค. ResNet์—์„œ ์™ผ์ชฝ์€ BottleNeck ๊ตฌ์กฐ๋ฅผ ์‚ฌ์šฉํ•˜์ง€ ์•Š์•˜๊ณ , ์˜ค๋ฅธ์ชฝ์€..

coding-yoon.tistory.com

ResNet ์›๋ฌธ

arxiv.org/abs/1512.03385

 

Deep Residual Learning for Image Recognition

Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with

arxiv.org

 

728x90
๋ฐ˜์‘ํ˜•
18์ง„์ˆ˜