A more complex case is given in Peirce’s account of Kepler’s work on planetary orbits. “Abduction and induction” in Philosophical Writings of Peirce, Justus Buchler (ed.), 2001. (archive.org) He chides J.S. Mill for (allegedly) “den[ying] that there was any reasoning in Kepler’s procedure. He says it is merely a description of the facts. He seems to imagine that Kepler had all the places of Mars in space given him by Tycho’s observations; and that all he did was to generalize and so obtain a general expression for them.” – that is, Mills claimed a pure inductive procedure not different from concluding that all swans are white. Peirce claims it required a great deal of non-inductive reasoning.
The problem (as I understand it) was this:
The Ptolemaic (earth-centered) theory “agrees with the appearances, although there were various difficulties in making it fit exactly.” Those difficulties required kludges: epicycles and equants.
The Copernican theory originally inherited the kludges of Ptolemy. For example, it did not actually put the sun at the center of the universe with everything orbiting around it. Instead, if I understand correctly, it orbited around a central point, as did all planets. (That is, they all orbited around the same point, not a different center for each planet’s orbit. I think.)
At that point, Copernican theory could be seen as a mathematical transformation of Ptolemaic theory: one that is purely instrumental, not necessarily true, and not allowing any new predictions. Kepler wanted better evidence for the heliocentric universe than that. He also knew (as Ptolemy did not) that the Sun is much bigger than the Earth: at least 15 times bigger. So it seemed sensible for the Sun to have a stronger causal role in planetary motion – that it wasn’t just one planet of many.
How did Kepler proceed?
Because of epicycles (a planet’s orbit is not it moving around a circle, but around a circle that’s itself moving in a circle), each planet has a point where it’s furthest from the center and a point where it’s nearest. Draw a line between the two (“the line of apsis”). Kepler looked looked at the apsides (plural of “apsis”) for the Earth and Mars. “[He] utilized various observations [drink!] most ingeniously to infer that they probably intersected in the sun” (not the center of the sun’s supposed orbit).
From this, Kepler thought it reasonable just to put the sun, immobile, at the center. That has consequences for when you take observations to support calculation of an orbit. “Thence it followed that the proper times at which to take the observations of Mars for determining its orbit were when it appeared just opposite the sun-the true sun-instead of when it was opposite the mean sun, as had been the practice.” “Carrying out this idea, [Kepler] obtained a theory of Mars which satisfied the longitudes at all the oppositions observed by Tycho and himself, thirteen in number, to perfection.”
“But unfortunately; it did not satisfy the latitudes at all and was totally irreconcilable with observations [drink!] of Mars when far from opposition.” However, it was progress that allowed him to make a leap from the earlier notion that planets sweep out a constant angle per unit time and replace it with the idea that they sweep out a constant area.
At this point, Kepler was still assuming circular orbits, so I don’t know how angle vs. makes an observable difference if the sun is at the center. Perhaps this is another shift (like from Ptolemaic to heliocentric cosmology) that produces no new predictions but readies you to look in a new direction. A quote from Richard Feynmann comes to mind: Richard Feynman, The Character of Physical Law, 1967, p. 53, emphasis mine.
“Mathematically each of the three different formulations [of the law of gravitation], Newton’s law, the local field theory and the minimum principle, gives exactly the same consequences. What do we do then? You will read in all the books that we cannot decide scientifically on one or the other. That is true. They are equivalent scientifically. It is impossible to make a decision, because there is no experimental way to distinguish between them if all the consequences are the same. But psychologically they are very different in two ways. First, philosophically you like them or do not like them; and training is the only way to beat that disease. Second, psychologically they are very different because they are completely unequivalent when you are trying to guess new laws.”
… because:
“Subsequently, finding [drink!] that the planet moves faster at ninety degrees from its apsides than it ought to do, the question is whether this is owing to an error in the law of areas or to a compression of the orbit. He ingeniously proves that the latter is the case.”
That is, the orbit isn’t a perfect circle. But what is it? An egg shape? Or what?
“He accomplished [finding that planets move in ellipses] by his incomparable energy and courage, blundering along in the most inconceivable way (to us), from one irrational hypothesis to another, until, after trying twenty-two of these, he fell, by the mere exhaustion of his invention, upon the orbit which a mind well furnished with the weapons of modern logic would have tried almost at the outset.“ “The Fixation of Belief”, in …. p. 6
It’s instructive to contrast this description of the interaction between observation and theorizing with Lakatos' account of Newton’s development of his theory of gravitation. In that case, In the Beginning Were the Words: specifically, Newton’s three laws of dynamics and his inverse-square law for gravitation. Lakatos describes Newton as then applying those laws to a succession of more and more realistic thought experiments. Observation doesn’t play any role in Lakatos' story. It only occurs when Newton is correcting the Astronomer Royal’s observations [spit!] by schooling him on a new theory of refraction. “Thus Newton constantly criticized and corrected Flamsteed’s observational theories. Newton taught Flamsteed, for instance, a better theory of the refractive power of the atmosphere; Flamsteed accepted this and corrected his original ‘data’. One can understand the constant humiliation and slowly increasing fury of this great observer, having his data criticized and improved by a man who, on his own confession, made no observations himself.” I’m suspicious of this account of Newton, given that his Optics was an experimental work full of observation, and given Kuhn claims that Lakatos grossly undervalues experiment in other “rational reconstructions” of science.
Peirce, from my point of view, rings more true.