Building off pre existing yolov8n-pose.pt

[StatisticalHorse]

As the title says, I need to build another model that adds more classes to the nano pose model (and keeps the human pose recognition that it already has). I am trying to see if there is a way to do so, as when I did have a custom human recognition in my dataset it was projecting all the key points onto one side. I would prefer not to have to run 2 separate models for what I am working on. Thanks!

[UltralyticsAssistant]

👋 Hello @StatisticalHorse, thank you for your interest in Ultralytics 🚀! We recommend a visit to the Docs for new users, where you can find many Python and CLI usage examples and where many common questions may already be answered.

For this custom pose-training ❓ question, please provide as much information as possible, including your dataset format, class and keypoint definitions, sample images with annotations, model configuration, training command, and training logs. Please also verify that your workflow follows our Tips for Best Training Results. If you are encountering unexpected keypoint behavior, please include a 🐛 minimum reproducible example so we can investigate it effectively.

Join the Ultralytics community where it suits you best. For real-time chat, head to Discord 🎧. Prefer in-depth discussions? Check out Discourse. Or dive into threads on our Subreddit to share knowledge with the community.

This is an automated response 🤖, and an Ultralytics engineer will also assist you soon.

Upgrade

Upgrade to the latest ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8 to verify your issue is not already resolved in the latest version:

pip install -U ultralytics

Environments

YOLO may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLO Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

Fine-tuning yolov8n-pose.pt only on custom data will not reliably preserve COCO human pose; train on a combined dataset containing the original 17-keypoint person annotations plus your new classes. Pose models use one global kpt_shape, so all classes must share that keypoint schema; otherwise use separate models. Keypoints collapsing to one side usually indicates incorrect normalized coordinates, keypoint ordering, or flip_idx; verify these against the pose dataset format, then delete any *.cache files under labels/ before retraining.

[fadyabirached]

You can do this in one model, don't need two. The thing to understand is YOLOv8-pose has one keypoint skeleton (kpt_shape) for the whole model, it's not per class.

So the fix isn't training separate pose heads per class. Build your dataset so non-human classes have their keypoints marked invisible.

Keep kpt_shape: [17, 3] in your data.yaml, don't touch it. For your new non-human classes, still include 17 keypoint slots per label, but set visibility to 0 for all of them, x/y can just be 0 too. That tells the loss function to skip keypoint loss for those instances, they just get bbox + class loss.

Fine-tune starting from yolov8n-pose.pt, not from scratch, so you keep the pretrained pose head for the human class and just extend nc for your new classes.

That "projecting keypoints onto one side" thing you saw before sounds like exactly this problem. If your non-human instances had visibility not set to 0, or had garbage keypoint values instead of properly zeroed and invisible ones, the model tries to learn keypoint positions for objects with no consistent keypoint geometry, and it drags everything toward some averaged, degenerate solution.

So: single model, single kpt_shape, mixed dataset where only the person class has real keypoint annotations and everything else has them zeroed and marked invisible. No need for two models.

[glenn-jocher]

Yes—one model can handle this, but kpt_shape is global rather than per class. Keep kpt_shape: [17, 3], annotate people with the 17 COCO keypoints, and provide 17 (0, 0, 0) keypoints for each non-person instance so their coordinate loss is masked; fine-tune from yolov8n-pose.pt while retaining representative person-pose training data to avoid forgetting. See the pose dataset format, delete any *.cache files under labels/ after correcting annotations, and filter non-person keypoints during inference.

[rishig47-dev]

If you want to keep the existing human-pose estimation and add more information, I would first clarify whether the new outputs are object classes or additional keypoints.

If they are additional body/keypoint locations, you should extend the pose dataset's kpt_shape and annotations rather than adding them as detection classes. For example, YOLOv8-pose uses kpt_shape: [17, 3] for 17 keypoints with (x, y, visibility), and Ultralytics reads the kpt_shape directly from the dataset configuration.

You can still start from the pretrained yolov8n-pose.pt weights. Ultralytics transfers compatible pretrained weights when fine-tuning on a custom pose dataset, while layers whose shapes no longer match are reinitialized.

So I would recommend:

1. Keep the existing pose keypoints you want to retain.
2. Add the new keypoints to kpt_shape and annotate them consistently in every relevant image.
3. Update flip_idx if you have symmetric keypoints.
4. Train using the pretrained yolov8n-pose.pt as the starting point.
5. Validate the new keypoints separately to make sure the additional outputs are actually learned.

If by "more classes" you mean completely different objects rather than additional keypoints, then that is a different setup: you would change the dataset names/nc and train the appropriate detection/pose configuration.

[pderrenger]

Yes, but yolov8n-pose.pt is only the initialization: train on a combined dataset containing the original person class with COCO’s exact 17-keypoint order plus your new classes, keeping one global kpt_shape: [17, 3]; use visibility 0 for unlabeled keypoints on classes without poses. Keypoints collapsing to one side usually indicates incorrect normalization, ordering, or flip_idx; compare labels with the pose dataset format, inspect the generated training plots, then delete labels/*.cache after correcting them.