The Forum was located in the City of the Gods, on Kobol. After the Cataclysm on Kobol and the establishment of the Twelve Colonies of Man a second Forum was built on Caprica, either in Delphi or Caprica City, in tribute of the original. This may imply that Caprica was the first of the Twelve Worlds to be colonized and be a reason for its importance above the other Colonies.
It is not clear if the Forum was religious in nature (perhaps a temple to the Lords of Kobol), governmental, simply a civic gathering place. Laura Roslin describes it as having a colonnade of pillars surounding it in Kobol's Last Gleaming, Part I.
Other structures on Kobol that might be mirrored on Caprica include the Opera House and the Temple. It is possible the Forum and the rumored Temple are one in the same.