There are actually studies about sitting in a regular chair vs. an exercise ball. Here and here for example. Neither seems to show net benefits. The limitations of both studies are small sample size and short time period of observation.
Anecdotally, I know at least two people who both have done it and they seem happy with it (but this is recent, so I can't speak for long term use).
A few considerations:
Height - With an exercise ball, you will not be able to vary its height. So if it's fairly low relative to your desk, you might be forced to straighten your back. If it's too high you might end up curving your spine.
Arm rests - Just from experience, I find that in chairs without arm rests, my shoulders are less slouchy, and my back is a bit straighter. I don't know to what extent this makes a difference for you or others, but keep it in mind.
Soft vs. hard seat - I wonder how this influences blood flow to your legs, other parts of the body, and if it also affects posture.