base_model.md

Configure the Input Base Model (S3 or OCI)

The input model to be fine-tuned must either be in either an S3-Compliant object store or OCI repository. How the InstructLab pipeline downloads the input model varies depending on the method used.

OCI Input Model

To utilize the OCI input, ensure the following:

• The input model is located in an oci compliant repository (e.g: quay.io, registry.redhat.io, etc.)
• The input model follows the model car format
• The host OCP cluster is configured to allow pods to pull this model care image
  • One way to do this is to configure the cluster global pull secrets (if you are using a private registry)
• You have the full oci path to the model for example: oci://registry.redhat.io/rhelai1/modelcar-granite-7b-starter:1.4

S3-Compliant Object store

For utilizing an S3-Compliant store for your input base model, you will need to upload this model to your bucket.

We will be working with the granite-7b-starter.

$ mkdir -p s3-data/

Download ilab model repository in s3-data model directory

# You can also use Oras or Skopeo cli tools to download the model
# If using other tools besides ilab, ensure that filenames are mapped
# appropriately
$ ilab model download --repository docker://registry.redhat.io/rhelai1/granite-7b-starter --release 1.4
$ mkdir s3-data
$ cp -r <path-to-model-downloaded-dir>/rhelai1/granite-7b-starter s3-data/granite-7b-starter

Generate tar archive

$ cd s3-data

Upload the model to your object store.

# The model should be copied in such a way that the *.safetensors are found in s3://your-bucket-name/teach-model/*.safetensors
s3cmd sync s3-data/granite-7b-starter s3://<your-bucket-name>/granite-7b-starter