์๋ ํ์ธ์.
Xception ๋ ผ๋ฌธ ๋ฆฌ๋ทฐ 2ํ์ฐจ์ ๋๋ค.
1ํ์ฐจ ๋ ผ๋ฌธ๋ฆฌ๋ทฐ๋ก An Extreme version of Inception module์ด Depthwise Separable Convolution๊น์ง ์๊ฐํ์ต๋๋ค.
์๋ ์ ํธ์ ๊ธ์ ์ฝ๋ ๊ฒ์ ์ถ์ฒ๋๋ฆฝ๋๋ค.
https://coding-yoon.tistory.com/78?category=825914
An Extreme version of Inception module๊ณผ Depthwise Separable Convolution๋ ๊ต์ฅํ ๋น์ทํ ํํ๋ฅผ ๊ฐ์ง๊ณ ์์ต๋๋ค.
๋ ผ๋ฌธ์์ ์๊ฐํ๋ ๋ ๊ฐ์ ์ฐจ์ด์ ์ ์ค๋ช ํ๊ณ ์์ต๋๋ค.
Two minor differences between and “extreme” version of an Inception module and a depthwise separable
convolution would be:
“extreme” version of an Inception module๊ณผ depthwise separable
convolution์ ๋ ๊ฐ์ง ์ฌ์ํ ์ฐจ์ด์ ์ ๋ค์๊ณผ ๊ฐ์ต๋๋ค.
1. Order(์์)
• The order of the operations:
depthwise separable convolutions as usually implemented (e.g. in TensorFlow)perform first channel-wise
spatial convolution and then perform 1x1 convolution, whereas Inception performs
the 1x1 convolution first.
Depthwise Separable Convolution : Depthwise Convolution(3x3, 5x5 ...), Pointwise Convolution
An Extreme version of Inception module : Pointwise Convolution , Depthwise Convolution(3x3, 5x5 ...)
Convolution์ ์์๊ฐ ๋ค๋ฅด๋ค๊ณ ํฉ๋๋ค.
We argue that the first difference is unimportant, in particular because these operations are meant
to be used in a stacked setting.
์คํ ์ค์ ์์ ์ฌ์ฉ๋๊ธฐ ๋๋ฌธ์ ์์์ ์ฐจ์ด์ ์ ์ค์ํ์ง ์๋ค๊ณ ํฉ๋๋ค.
2. Non-Linearity(๋น์ ํ์ฑ)
• The presence or absence of a non-linearity after the first operation.
In Inception, both operations are followed by a ReLU non-linearity, however depthwise separable
convolutions are usually implemented without non-linearities.
ReLU(๋น์ ํ)์ ์ ๋ฌด.
Inception์ Convolution ์ํ์ด ํ ReLU๊ฐ ๋ถ๋ ๋ฐ๋ฉด,
์ผ๋ฐ์ ์ผ๋ก Depthwise Separable Convolution๋ ReLU๊ฐ ์์ด ๊ตฌํ๋ฉ๋๋ค.
The second difference might matter, and we investigate it in the experimental section
(in particular see figure 10).
ReLU์ ์ ๋ฌด๋ ์ค์ํ๋ฉฐ, Figure 10์์ ์คํ ๊ฒฐ๊ณผ๋ฅผ ๋ณด์ฌ์ค๋๋ค.
Depthwise Separable Convolution์์ ReLU๋ฅผ ์ฌ์ฉํ์ง ์์์ ๋ ๋ ๋์ Accruacy๋ฅผ ๋ณด์ฌ์ค๋๋ค.
It may be that the depth of the intermediate feature spaces on which spatial convolutions are applied
is critical to the usefulness of the non-linearity:
for deep feature spaces (e.g. those found in Inception modules) the non-linearity is helpful,
but for shallow ones (e.g. the 1-channel deep feature spaces of depthwise separable convolutions) it becomes harmful,
possibly due to a loss of information.
๊น์ ํน์ง ๊ณต๊ฐ์ ๊ฒฝ์ฐ, ReLU๊ฐ ๋์์ด ๋์ง๋ง ์์ ๊ณต๊ฐ( 1x1 Convolution )์์๋ ReLU์ ์ํด ์ ๋ณด ์์ค์ด ์๊ธธ ์ ์๊ธฐ ๋๋ฌธ์ ์ฌ์ฉํ์ง ์๋ ๊ฒ์ด ์ข๋ค๋ ๊ฒฐ๊ณผ์ ๋๋ค.
๋ค์ ๊ธ์ Xception์ Architecture์ ๋ํด ์ค๋ช ํ๊ณ , Pytorch๋ก ๊ตฌํํจ์ผ๋ก ๊ธ์ Xception ๋ ผ๋ฌธ ๋ฆฌ๋ทฐ๋ฅผ ๋ง์น๋๋ก ํ๊ฒ ์ต๋๋ค. ํผ๋๋ฐฑ์ฃผ์๋ฉด ๊ฐ์ฌํ๊ฒ ์ต๋๋ค.