[Piotrsko]

At those altitudes, the stall speed will not be significantly below the VNE, perhaps only a knot or two. tthere may nort be many air molecules, but they are moving awful fast. The major structural failure point is flutter of the control surfaces leading to flutter of the other surfaces. The F104 didn't have that problem because it was capable of mach + speeds. It had other issues especially near or at stall. The survivor sitting on a pole outside the test pilot school at Edwards AFB has a bunch of reaction rockets scattered around the fuselage to control the nasty stall/ tumble habits at apogee.

Col Payne has stories of his altitude records in a diamante of both almost freezing and controlling the glider at I believe 60,000 ft perhaps much higher.