How can I fine-tune GPT-3 to generate quotes without using a prompt?

Below is some guidance you might consider for your use case:

Add a Consistent Hint

Rather than leaving the prompt empty in every training example, you can include a very brief instruction or hint that specifies the expected output. For example, every training example might start with a constant preamble such as:

JSONCopied
{
  "prompt": "Quote:",
  "completion": "The curated quote text goes here."
}

This approach helps the model understand that it should generate a quote in the style of your subject.

Benefits of Using a Hint

• Consistency: Including a constant instruction gives the model a clearer signal about your desired output format.
• Guidance: Even when not using prompts at inference time, the model may internalize that it should generate text following that structure.

Considerations

• If you include the hint during training, be mindful that the model might bake it in. This means that even if you change or remove it at inference time, the model may still behave as if it were present.
• As your training data (110 examples) is relatively small, consistent hints can help the model quickly learn the desired style and structure.

Experimentation

Feel free to experiment with different levels of instruction detail. For instance, you might try a very minimal hint ("Quote:") versus something slightly more descriptive ("Generate a wise quote in the style of [Person's Name]:") to see which produces output closer to your expectations. Always test the outputs in the playground environment before deploying the fine-tuned model.

Using a small, consistent prompt in your training data is a recommended approach, especially when working with a limited number of examples, to help steer the model towards generating the desired quotations.