We know each channel in Convolution block represents some high level feature say in human, each channel would map a body part(just for example!). Then this idea of depthwise conv proposes that in process of identifying one body part don't process info from other body part info, it would disturb the signal in processing.