PetterS asked . 2021-08-05

GPU training of neural network with parallel computing toolbox unreasonably slow, what am I missing?

%CPU training
T = num2cell(sin(1:0.01:10));
net = narnet( 1:2, 10 ); 
[ Xs, Xsi, Asi, Ts] = preparets( net, {}, {}, T );
rng(0)
net.trainFcn = 'trainscg';
tic
net = train(net,Xs,Ts,'showResources','yes' );
toc %2.77

The training takes less than 3 seconds. But when doing the exact same thing on a CUDA supported GTX 760 GPU:

%GPU training
T = num2cell(sin(1:0.01:10));
net = narnet( 1:2, 10 ); 
[ Xs, Xsi, Asi, Ts] = preparets( net, {}, {}, T );
rng(0)
net.trainFcn = 'trainscg';
tic
net = train(net,Xs,Ts,'useGPU','yes','showResources','yes' );
toc % 1247.6

%Improved GPU training
T = num2cell(sin(1:0.01:10));
net = narnet( 1:2, 10 ); 
[ Xs, Xsi, Asi, Ts ] = preparets( net, {}, {}, T );
rng(0)

net = configure(net,Xs,Ts); 
Xs = nndata2gpu(Xs);
Ts = nndata2gpu(Ts);
Xsi = nndata2gpu(Xsi);

for i=1:net.numLayers
  if strcmp(net.layers{i}.transferFcn,'tansig')
    net.layers{i}.transferFcn = 'elliotsig';
  end
end

net.trainFcn = 'trainscg';
tic
net = train(net,Xs,Ts,'showResources','yes' );
toc  %70.79

gpu , narnet , parallel computing , neural network