What do we want to do in 2025?

[TheFoxAtWork]

• Information exchange still nice to be done here
• Building on the Model Signing project’s work.
• from Jay - if we could iron out what security efforts look like that are developing
  • Smaller language models, “open models” being used.
  • Thinking about supply chain security efforts - how developers pull off datasets, developing and using smaller language models, developing open model systems. What are those security efforts from an openssf perspective.
  • How this can be used for other organizations? What is the open source elements those organizations can take advantage of?
  • Things have improved we’re in a position we can do this.
  • Best Practices, pipeline security, supply chain transparency
  • Vulnerabilities in ML - here or joint with the vuln wg
  • What is the overlap of AIML security with the other WGs in OpenSSF, what should we be engaging with them on.

Please add your ideas here so the group can create distinct issues for items we choose to pursue.

“definition of done” is when individual issues have been created and prioritized.

Please have any additional items added here by January 19th 2025

[mihaimaruseac]

On the supply chain security/transparency track, I'm thinking of how we can adapt SLSA for ML. We can build on top of model signing, sign datasets, create SLSA-aware ML training pipelines that practitioners can use with minimal changes to their workflows

[sevansdell]

Andrey Shorov, Elif Soykan and I want to produce the following (with the rest of the WG help fi you'd all like):

• Q1: White paper: MLSecOps Map AIML WG outputs to MLSecOps diagram #16
  MLSecOps - Google Drive. Using the Ericsson reference architecture, how OWASP ML top ten are prevented, where current tools are generated/leveraged (e.g. OpenSSF tools like SLSA, Model card signing, generate an AIBOM, and where there are gaps)
  -Q2: White paper: LLMSecOps: Using a combination of OPEA reference architecture and https://air-governance-framework.finos.org/ Build out how OWASP LLM top ten are prevented, where current tools are generated/leveraged (e.g. .g. OpenSSF tools like SLSA, Model card signing, generate an AIBOM, and where there are gaps?

[sevansdell]

Something that came up in the Model Signing SIG, that is out of scope of the SIG, but could be an output of the AIML WG: provide input on model card metadata. Potential sync with Open Oasis Data Provenance standard, LF Model Openness Framework, and CosAI RFC Supply Chain work group

[sevansdell]

Do we/should we have any advice for developers leveraging deepseek.ai to apply OpenSSF tools/concepts for security?

[mihaimaruseac]

I like both of these ideas!

[sevansdell]

What does a security model of an agentic architecture look like? Much of this is OSS in the tool chain: langchain, langgraph/knowledge graphs, APIs. Where are their architecture choices developers and enterprises may need to understand that introduce more risk/opportunity for compromise?