Ptolemy shows how to construct the length of the side of a pentagon in Fig. 1-1 on page 48 of Toomer's Ptolemy's Almagest. It is BE in the figure below but Ptolemy does not explain how this was arrived at. There seems to be another missing key needed to understand his methods.
One possibility is that he used two procedures relating the chords of a pentagon to solve the equivalent of two equations with two unknowns. Ptolemy's theorem can be used to relate the sides of an inscribed quadrilateral and for the pentagon there are only two unknown chords. Applying Ptolemy's theorem to three neighboring sides of a pentagon and the larger chords needed to complete the quadrilateral results in a quadratic equation involving the two unknown chord lengths. The quadratic equation can be solved for the ratio of the lengths of the two sides which turns out to be the golden ratio.
One can then use the formula for finding the chord of half an angle to obtain another relation between the two chords. Substituting the golden ratio into this formula simplifies it and reduces it to an equation involving one unknown which can be easily solved for the length of the side of a pentagon.
The figure above is a copy of the one in Ptolemy's Almagest. Ptolemy assumed that the diameter of the circle had 120 parts but in what follows it is assumed that the radius of the circle is 1.
Following the construction through one step at a time and converting the lengths involved into numbers we end up with the same formula obtained by the algebraic solution above. Ptolemy follows the steps of the construction computing the lengths and sides of the triangles in the figure and thus obtains the chord of 72°.
Ptolemy's method for finding the chord of half the angle given the chord of an angle is algebraically equivalent to the formula used above. He also uses the proportion of the golden ratio in his proof. So it is not unlikely that the solution of these equations was involved in finding the formula for the side of the pentagon.