Skip to content

Few-shot tests on GLM-10B #203

@Vispstar-V

Description

@Vispstar-V

Hi, I'm trying to conduct a multiple-choice test on GLM-10B using few-shot prompting. I have tried to start a service using the following command:

bash scripts/generate_block.sh \
     config_tasks/model_blocklm_10B.sh

Then I input my few-shot prompts, such as:

Please answer this multiple-choice question.\nQuestion: Which instrument measures atmospheric pressure?\nOptions: A: Thermometer. B: Anemometer. C: Barometer. D: Hydrometer.\nAnswer: C.\n\nQuestion: Which gas is most commonly used in light bulbs?\nOptions: A: Nitrogen. B: Oxygen. C: Helium. D: Argon.\nAnswer: D.\n\nQuestion: What is the largest organ in the human body?\nOptions: A: Heart. B: Liver. C: Skin. D: Brain.\nAnswer: C.\n\nQuestion: What is the pH level of pure water?\nOptions: A: 5. B: 7. C: 9. D: 11.\nAnswer:[gMask]

But on most tests, the model fails to provide a correct option. I wonder if there is a flaw in my prompt design, or it is a common issue.
Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions