Your explanation of the origin of centrifugal force is correct, but there is more subtlety behind it, as gnargenox explains. An xkcd comic also said it well.
Centrifugal forces appear when you formulate the laws of motion in certain accelerated or non-inertial frames of reference. What you really mean when you say "there's no such thing" is that they are "fictitious" -- but this has a very specific meaning in physics. It means the forces can be made to vanish by adopting an appropriate frame of reference, where the accelerated motion of the body is produced by a force which is actually moderated by an interaction -- like a normal force, or tension.
Usually when people are learning about laws of motion and forces, they try to emphasize physics in inertial frames of reference, and any other non-inertial forces are just "illusions" which can be readily explained by inertia + 'real' forces. This is common teaching and it often leads people to dismiss things like centrifugal force. But I think that is too narrow. Physics with fictitious forces is just as valid as physics without them. In fact, we analyze problems in noninertial frames and call upon fictitious forces as if they are real all the time. For example, we do it when modelling atmospheric motions for weather forecasting (Coriolis force is a fictitious force, but we use it in the equations! Why? Because it's way easier!)
There's a philosophical side to this as well. Suppose you are in a sealed crate which is being swung around in a large slow circle. If you have no knowledge of the world exterior to your crate, how do you know you are being pressed against the side because it is moving in a circle and your body is trying to follow a straight line? How do you know the crate isn't simply sitting on its side on the surface of the Earth? What experiment could you perform inside the crate that would unambiguously distinguish between these two situations?
This very logic (it's actually the essence of the Equivalence Principle) leads us to conclude that gravitation is a fictitious force, too! What you "feel" as gravity only appears because you are in an accelerated frame -- the surface of the Earth with respect to coordinates which are in freefall. And freefall is the most inertial frame of reference there is! If you are in a frame of freefall, the force due to gravity disappears as far as any localized experiment can tell!
So every time you do physics and say there is a gravitational force F=mg acting on something, you are actually invoking a fictitious force. Calling upon fictitious forces can be a very useful way of describing nature. It just depends on your perspective. =)
-edited (hopefully) for clarity