The enhancement of big data security in cloud computing has become inevitable dues to factors such as the volume, velocity, veracity, Value, and velocity of the big data. These enhancements of big data and cloud technologies have computing enabled a wide range of vulnerabilities in applications in organizational business environments leading to various attacks such as denial-of-service attacks, injection attacks, and Phishing among others. Deploying big data in cloud computing environments is a rapidly growing technology that significantly impacts organizations and provides benefits such as demand-driven access to computational services, a distorted version of infinite computing capacity, and assistance with demand-driven scaling up, scaling down, and scaling out. To secure cloud computing for big data processing, a variety of encryption techniques such as RSA, and AES can be applied. However, there are several vulnerabilities during processing. The paper aims to explore the enhancement of big data security in cloud computing using the RSA algorithm to improve the deployment and processing of the variety, volume, veracity, velocity and value of the data utilizing RSA encryptions. The novelty contribution of the paper is threefold: First, explore the current challenges and vulnerabilities in securing big data in cloud computing and how the RSA algorithm can be used to address them. Secondly, we implement the RSA algorithm in a cloud computing environment using the AWS cloud platform to secure big data to improve the performance and scalability of the RSA algorithm for big data security in cloud computing. We compare the RSA algorithm to other cryptographic algorithms in terms of its ability to enhance big data security in cloud computing. Finally, we recommend control mechanisms to improve security in the cloud platform. The results show that the RSA algorithm can be used to improve Cloud Security in a network environment.