How to get numbers as answers? Values show NaN

Because you chose bad variable names, you didn't see what was causing the error. Look at how i goes with m -- they're rows. And j goes with n -- they're columns. But when you come to index the array, you swap the rows and columns:

A(j,:)=A(i,:);

See how j is the first index, which should be the rows? But you associated j and n with columns, meaning they should come in the second place, not the first index place. This code is closer but it's still messed up because of the other problem : no comments. So because of that I can't figure out what you intend, but at least with proper variable names you should be able to figure it out. So dump your code and start repairing this:

function my_rref(A)
[rows, columns]=size(A);
for col=1:columns-1
	for row=col+1:rows
		if abs(A(row,col))>abs(A(col,col))
			t=A(row,:);
			A(row,:)=A(col,:);
			A(col,:)=t;
		end
	end
end
for col=1:columns-1
	for row=col+1:rows
		x=A(row,col)/A(row,row);
		for k=col:rows+1
			A(row,k)=A(row,k)-x*A(col,k);
		end
	end
end

for row=rows:-1:2
	for col=row-1:-1:1
		A(col,:)=A(col,:)-A(row,:)*(A(col,row)/A(row,row));
	end
end
for col=1:columns
	A(col,:)=A(col,:)/A(col,col);
end
disp('Rref of A is')
A