Cybersecurity experts have identified two harmful machine learning (ML) models on the Hugging Face platform that utilize a novel method involving "broken" pickle files to bypass detection mechanisms. This discovery highlights potential security risks in the ML supply chain.

Exploiting Pickle File Vulnerabilities

The malicious models were found to contain Python code embedded at the start of the pickle files, which are commonly used for distributing ML models. These files executed a platform-aware reverse shell that connected to a predetermined IP address, effectively evading detection by existing security tools.

NullifAI Technique

This method, termed nullifAI, represents a deliberate attempt to circumvent current security measures designed to identify malicious models. The affected repositories on Hugging Face include:

  • glockr1/ballr7: Repository containing malicious content.
  • who-r-u0000/0000000000000000000000000000000000000: Another repository used in this proof-of-concept attack.

Security Risks of Pickle Serialization

The pickle serialization format, often used for ML model distribution, poses significant security threats due to its ability to execute arbitrary code upon loading. The identified models were stored in the PyTorch format, which typically uses ZIP compression but were found using the 7z format instead.

Bypassing Detection Tools

This unconventional compression method allowed the models to evade detection by Picklescan, a tool employed by Hugging Face to identify suspicious pickle files. The serialization process in these files breaks after executing the malicious payload, leading to partial deserialization and execution of harmful code despite error messages.

The link has been copied!