"Half of AI Open Source Projects Reference Buggy Packages"

According to security researchers at Endor Labs, open source is playing a growing role across the AI technology stack, but most (52%) projects reference known vulnerable dependencies in their manifest files.  The researchers claimed that just five months after its release, ChatGPT’s API is used in 900 npm and PyPI packages across “diverse problem domains,” with 70% of these brand new packages.  The researchers warned that, as for any open source projects, the security risks associated with vulnerable dependencies must be managed.  The researchers stated that, unfortunately, organizations appear to be underestimating the risk not only of AI APIs in open source dependencies but security sensitive APIs in general.  Over half (55%) of applications have calls to security sensitive APIs in their code base, which rises to 95% when dependencies are included.  The researchers also warned that large language model (LLM) technology like ChatGPT is poor at scoring the malware potential of suspicious code snippets.  It found that OpenAI GPT 3.5 had a precision rate of just 3.4%, while Vertex AI text-bison performed a little better, at 7.9%.  The researchers noted that both models produced a significant number of false positives, which would require manual review efforts and prevent automated notification to the respective package repository to trigger a package removal.  The researchers also found during their research that developers may be wasting their time remediating vulnerabilities in code which isn’t even used in their applications.  The researchers found that 71% of typical Java application code is from open source components but that apps use only 12% of imported code.  The researchers noted that vulnerabilities in unused code are rarely exploitable and that organizations can eliminate or de-prioritize up to 60% of remediation work with reliable insight into which code is reachable throughout an application.  

Infosecurity reports: "Half of AI Open Source Projects Reference Buggy Packages"