Developing network intrusion detection systems (IDS) presents significant challenges due to the evolving nature of threats and the diverse range of network applications. Existing IDSs often struggle to detect dynamic attack patterns and covert attacks, leading to misidentified network vulnerabilities and degraded system performance. These requirements must be met via dependable, scalable, effective, and adaptable IDS designs. Our IDS can recognise and classify complex network threats by combining the Deep Q-Network (DQN) algorithm with distributed agents and attention techniques.. Our proposed distributed multi-agent IDS architecture has many advantages for guiding an all-encompassing security approach, including scalability, fault tolerance, and multi-view analysis. We conducted experiments using industry-standard datasets including NSL-KDD and CICIDS2017 to determine how well our model performed. The results show that our IDS outperforms others in terms of accuracy, precision, recall, F1-score, and false-positive rate. Additionally, we evaluated our model s resistance to black-box adversarial attacks, which are commonly used to take advantage of flaws in machine learning. Under these difficult circumstances, our model performed quite well.We used a denoising autoencoder (DAE) for further model strengthening to improve the IDS s robustness. Lastly, we evaluated the effectiveness of our zero-day defenses, which are designed to mitigate attacks exploiting unknown vulnerabilities. Through our research, we have developed an advanced IDS solution that addresses the limitations of traditional approaches. Our model demonstrates superior performance, robustness against adversarial attacks, and effective zero-day defenses. By combining deep reinforcement learning, distributed agents, attention techniques, and other enhancements, we provide a reliable and comprehensive solution for network security.