You are sticking with Lockeanism: we cannot exist without consciousness, and hence we were never foetuses; and we go where our consciousness goes, and hence Barry and Gary swap bodies.
Consider now a variant of the original 'body swapping' case, brought to prominence by Derek Parfit. This time not only is Barry's consciousness transplanted into Gary, it is also transplanted into Larry and Carrie as well. So what has happened to Barry in this case? It seems that Barry cannot be the Gary-body person, the Larry-body person, or the Carrie-body person, since they are distinct from each other - we have three people here, not one! And it seems arbitrary to say that Barry is one of the three - what could decide which of these is really Barry?
This case seems to show that you can have psychological successors who are not you - the Gary-body person, the Larry-body person, and the Carrie-body person are all psychologically continuous with Barry, but none of them are Barry. Still, you think that psychology is relevant to your existence, since you were not a foetus, and also because Barry and Gary swap bodies.