David, of course, was buried in his city on the eastern hill (1 Kgs.2:10). In the Byzantine period, however, he and James, the Jewish and Christian founders of Jerusalem, were the focus of a liturgical celebration in the Church of Mount Sion. Eventually this gave rise to the popular belief that the two were buried on Mount Sion. David’s tomb was located here and that of James in the Armenian Cathedral. The Franciscans built a monastery here in 1335 when they returned to assume the guardianship of the holy places. In the C15 the legend of treasures buried with the king (Antiquities, 16:179-82) gripped the imagination of fanatical Muslims who made it their objective to gain control of the site. They succeeded in 1524, but in turn they lost out to religious Jews in 1948. Beneath the present floor of the Tomb of David are Crusader, Byzantine, and Roman floors, so the foundations of the building go back at least to the C2 AD. It is not impossible that it should have been the ‘little church of God’ mentioned by Epiphanius of Salamis (315-403) as having been in existence on Mount Sion in AD 130. Danger and difficulty of access exclude Christian invention of a new holy place in the C2 AD. If they continued to frequent the site, it must have been of great importance in the previous century. At that stage this was an affluent quarter and a wealthy follower of Jesus may have turned his house into a place of assembly (Acts 2:44-5).
Source: The Holy Land by Jerome Murphy- O’Connor