Modern software makes extensive use of third-party open source packages and libraries, referred to as dependencies ^[12]. However, the use of open source has opened up new attack vectors, as malicious or vulnerable code can sneak into a software through these third-party dependencies ^[11]. Further, practitioners recommend keeping dependencies up to date with the latest version ^[1], resulting in developers to keep pulling in new code through frequent dependency updates ^[4], often automatically and without a security review ^[10].

Recent times have seen popular open source packages to be compromised and push malicious updates, such as the attack through ua-parser-js, an npm library used by major software corporations including Facebook ^[6]. Attacks through open source dependencies, such as the preceding example, are referred to as supply chain attack. While the existence of malicious packages in the package ecosystems is well-known by now ^[11], popular packages from reliable sources can also be compromised through (i) stolen credential of maintainer account(s), (ii) maintainer going rogue or account handover through social engineering, (iii) build system compromise etc. ^[11]. Therefore, practitioners now recommend reviewing dependency updates before merging into the codebase, to encourage shift-left supply chain security ^[2].

However, manually reviewing all the code changes in each dependency update may not be a practical solution, as projects often have dependencies in the magnitude of hundreds, through directly and transitively ^{[8, 12]}. Further, actively maintained packages get frequent updates, over-burdening any project that would employ such strict security measures. In the context where automatic merging of dependency updates exposes a project to supply chain attacks and manually reviewing all the updates is an impractical solution, we propose that dependency updates should go through automated security and quality checks before merging, which can act as a first line of defense and can aid developers in securely accepting dependency updates.

Recently, the software industry has proposed multiple security frameworks to define the compliance standards for maintaining, distributing, and using open source packages, such as the SCIM ^[7] and the SLSA ^[3] framework. Specifically, SLSA provides a checklist of standards and controls for secure use of open source packages in four levels, from easy to adopt to the highest security assurances. Among others, SLSA requires an open source package to employ two-person code review to be used as a dependency as a level 4 requirement ^[5]. However, while projects like OSSF scorecard 1 exists to perform heuristic-based checks of a package’s adherence to various security standards to aid in dependency selection, little research has been done yet to examine the feasibility and usefulness of employing automated security and quality checks during dependency updates ^[13].

In this paper, we aim to measure if all the code changes in a dependency update have passed through a code review process. As noted before, code review is a SLSA level 4 requirement to accept an open source dependency. Code review requirement can guard against the threat scenarios where only one maintainer’s account has been compromised, or the build system has been compromised without the knowledge of the maintainers, while also helping higher code quality ^[9]. However, the feasibility of employing an automated check for code review requirement during dependency updates has not been studied yet. Further, no empirical study exists that measures the code review practices among top open source packages to understand how practical it is to employ this SLSA requirement currently in practice. The goal of this study is to aid developers in securely accepting dependency updates by measuring if the code changes in an update have passed through a code review process. In this study, we investigate the below research questions:

RQ1: Can we effectively measure the code review coverage in a dependency update?

RQ2: What is the code review coverage in the version updates of the most downloaded open source packages?

Nasif Imtiaz is a Ph.D. student at North Carolina State University. His current research interest lies in the area of software engineering, software security, and software supply chain security. Contact him at simtiaz@ncsu.edu.

REFERENCES

[1] 2019. 9 Reasons for keeping software dependencies up to date. https://nullbeans. com/9-reasons-for-keeping-software-dependencies-up-to-date/.

[2] 2020. Shifting supply chain security left with dependency review. https://github.blog/ 2020-12-17-shifting-supply-chain-security-left-with-dependency-review/.

[3] 2021. 1https://github.com/ossf/scorecard HotSoS, Hot Topics in the Science of Security Symposium, Nasif Imtiaz and Laurie Williams

[4] 2021. Push and pull: when and why to update your dependencies. https:// pythonspeed.com/articles/when-update-dependencies/.

[5] 2021. SLSA Requirements. https://slsa.dev/spec/v0.1/requirements.

[6] 2021. SUPPLY CHAIN ATTACK: NPM LIBRARY USED BY FACEBOOK AND OTHERS WAS COMPROMISED. https://hackaday.com/2021/10/22/ supply-chain-attack-npm-library-used-by-facebook-and-others-was-compromised/.

[7] 2021. The Supply Chain Integrity Model (SCIM). https://github.com/microsoft/ scim.

[8] Nasif Imtiaz, Seaver Thorn, and Laurie Williams. 2021. A comparative study of vulnerability reporting by software composition analysis tools. In Proceedings of the 15th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). 1–11.

[9] Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E Hassan. 2016. An empirical study of the impact of modern code review practices on software quality. Empirical Software Engineering 21, 5 (2016), 2146–2189.

[10] Samim Mirhosseini and Chris Parnin. 2017. Can automated pull requests encourage software developers to upgrade out-of-date dependencies?. In 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 84–94.

[11] Marc Ohm, Henrik Plate, Arnold Sykosch, and Michael Meier. 2020. Backstabber’s knife collection: A review of open source software supply chain attacks. In International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. Springer, 23–43.

[12] Synopsys. 2021. 2021 Open source security and risk analysis report. https://www.synopsys.com/software-integrity/resources/analyst-reports/ open-source-security-risk-analysis.html. [13] Duc-Ly Vu, Fabio Massacci, Ivan Pashchenko, Henrik Plate, and Antonino Sabetta. 2021. LastPyMile: identifying the discrepancy between sources and packages. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 780–792.