The ever-evolving and intricate nature of cyber environments, coupled with the escalating risk of cyber-attacks, necessitates robust solutions in the realm of cybersecurity. Knowledge graphs have emerged as a promising avenue for consolidating, representing, managing, and reasoning over cyber threat intelligence. However, applying knowledge graphs to tackle real-world challenges in cyber-attack and defense scenarios remains an area requiring further exploration. This paper aims to address this gap by providing a comprehensive overview of the fundamental concepts, schema design, and construction methodologies for the cybersecurity knowledge graph. To facilitate future research endeavors, we have carefully curated datasets and open-source libraries tailored for knowledge construction and information extraction tasks. Furthermore, we present a detailed comparative review of recent advancements in the application scenarios of cybersecurity knowledge graphs. To provide clarity and organization, we introduce a novel classification framework that categorizes interconnected works into distinct primary categories and subcategories. The paper concludes by outlining potential research directions in the cybersecurity knowledge graph domain, paving the way for further advancements and innovations in the field.