I have been concerned about some points of Ptolemy's solution to the side of the pentagon lately and the book, Science Awakening I by B. L. van der Waerden, has provided some answers. Ptolemy considers it sufficient that the lengths in his construction are in "extreme and mean ratio" in order to show that they yeild the chords of the decagon and pentagon. On page 101 of van der Waerden's book the author states that the Pythagoreans knew that lines of the five pointed star divided themselves in extreme and mean ratio and that they knew how to solve the resulting quadratic equation. Van der Waerden later goes further into the procedures found in the works of Euclid, in the Elements and the Data, and shows that they provide solutions to standard problems (see pages 118-24). The methods are primarily geometrical. He also states that there seems to have been a work in ancient times known as "The Tradition of Pythagoras" which contained their knowledge of geometry and possibly drew on Babylonian sources.
But Ptolemy is rather emphatic with his, "I say that ZD is the side of the [regular] decagon, and BZ the side of the [regular] pentagon." on p. 48 of Toomer's Ptolemy's Almagest.