If this occurs on every road, check your tire pressures. On some sections of grooved roads, my Focus follows the grooves cut in the concrete and it's a bit annoying.
If the tire pressures are OK and it's not the grooves in the road, have the alignment checked. If that's within spec, there may be a mechanical problem or something loose. If there aren't any problems found, you can have the caster adjusted so that it isn't as sensitive.