With the development of network technologies, network intrusion has become increasing complex which makes the intrusion detection challenging. Traditional intrusion detection algorithms detect intrusion traffic through intrusion traffic characteristics or machine learning. These methods are inefficient due to the dependence of manual work. Therefore, in order to improve the efficiency and the accuracy, we propose an intrusion detection method based on deep learning. We integrate the Transformer and LSTM module with intrusion detection model to automatically detect network intrusion. The Transformer and LSTM can capture the temporal information of the traffic data which benefits to distinguish the abnormal data from normal data. We conduct experiments on the publicly available NSL-KDD dataset to evaluate the performance of our proposed model. The experimental results show that the proposed model outperforms other deep learning based models.