From Battlestar Wiki, the free, open content Battlestar Galactica encyclopedia and episode guide
Neutral Space is an area of space outside of Colonial controlled space. The border between this and Colonial space is demarcated by marker buoys.
Galactica ventures into this space in search of Julian DiMarco's lost Expeditionary Fleet ship. It is possibly considered hostile due to the fact that Galactica prepared for condition escalation as it crosses the border (Battlestar Galactica: Season Zero 0).