core: don't force a socket if the dst af is different