ResNet50 on multi-spectral image segmentation

Question

Is there a way to use any pretrain network (not necessarily Resnet) to segment multispectral images in MATLAB?     deeplabv3plusLayers only allows [height width 3] or [height width] input images. While I tried bypassing the error deeplabv3plusLayers returns, when I used trainNetwork I get an error referring to the wrong input data 224x224xN.   Can the first convolutional layer of the pretrained network be replaced to process more than 3 channels?

Prashant Kumar · Accepted Answer

You can copy the layerGraph of the pretrained network and change the imageInputLayer, the first convolutionLayer to match the input image channel dimension & convolution filter dimensions. Then you can freeze/unfreeze the existing pretrained weights during training the new network accordingly.
 
You can do something like below:(N=50)
 
imageSize = [224 224 3];

% Specify the number of classes.
numClasses = 10;

N = 50;
% Create DeepLab v3+.
lgraph = deeplabv3plusLayers(imageSize, numClasses, "resnet50");
analyzeNetwork(lgraph)
layers = lgraph.Layers

%%
newlgraph = replaceLayer(lgraph,'input_1',imageInputLayer([224 224 N],'Name','input'));
newlgraph = replaceLayer(newlgraph,'conv1',convolution2dLayer(7,64,'stride',[2 2],'padding',[3 3 3 3],'Name','conv1'))
analyzeNetwork(newlgraph)

ResNet50 on multi-spectral image segmentation

Get a Free Consultation or a Sample Assignment Review!