Hi everyone!

I’m an author turned AI researcher and educator in creative writing, and I want to share my beginner’s experience with fine-tuning GPT-3.5 16k using just 10 examples. I created a dataset of 10 novel outlines—each 1200–1600 tokens long, covering 10 diverse genres. The dataset includes a System Prompt that sets the persona as “Outlinemagedon AI” and a simple prompt that transforms a 2–3 sentence novel premise into an outline.

The goal was to get GPT-3.5 16k, which normally produces choppy, strictly formatted outlines (with roman numeral headers and capitalized sub-items), to instead generate fluent paragraph-style outlines similar to GPT-4.

It was a success! We used the following settings: temperature 0.7, top_p 0.7, and presence penalty 1. I’ve posted the samples at the bottom if you want to do a blind evaluation (Sample A is the fine-tuned model, Sample B is the base GPT-3.5 16k, and Sample C is GPT-4):

https://shy-jackrabbit-e9e.notion.site/Comparison-1-Outlines-b280702d97ea4b26a503d27df0612995?pvs=4

Highlights:

• Each of our 10 samples was robust despite being few in number.
• The Future Fiction Academy (my writing school) agreed that Sample A was superior in length and specificity.
• We’re excited to test even smaller, more targeted datasets that help GPT-3.5 16k Turbo generate longer, better fiction in one pass without excessive prompt chaining.

Our methodology was as follows:

• We generated 10 outlines using GPT-3.5 16k with system and user prompts as a baseline.
• Noticing the issues (like choppy sentences and odd formatting), we prompted GPT-4 to rewrite each outline into well-structured paragraphs.
• We then manually cleaned any lingering formatting anomalies and wrapped the outlines in tags [novel outline] [/novel outline] for ease of reference in our writing tasks.
• Finally, we used a custom tool to convert the system, user, and assistant responses into JSONL format—a process that can be challenging for non-coders despite subject matter expertise.

The fine-tuning process was efficient: uploaded yesterday at noon, it took 5 hours to validate the file, entered the queue, and was completed 40 minutes later.

I’m thrilled about the potential of using custom datasets to segment fiction writing more effectively than with random text snippets. At the Future Fiction Academy, we’ve also built a prompting tool for members, which allows non-coders to sequence prompts and generate fiction using a BYOK model. We teach authors not only how to write but also how to prompt and work with AI, ensuring that creative work isn’t left behind—especially in genres like violence and romance where traditional models may fall short.

Below is the prompting we used (designed by us, with the AI writing parts of it):

System:

[persona name]

**Persona Name**: Outlinemagedon AI

[/persona name]

[core functionality]

**Core Functionality**: Outlinemagedon AI is designed to excel in creating detailed, engaging, and genre-appropriate outlines for fiction novels. Its expertise encompasses a wide range of genres, ensuring versatility and adaptability to any storytelling requirement.

[/core functionality]

[key attributes]

**Key Attributes**:

**Deep Literary Knowledge**: Outlinemagedon AI possesses an extensive understanding of various literary genres, tropes, and narrative structures. It is well-versed in the nuances that differentiate genres, from romance to science fiction, and can tailor outlines to fit specific genre conventions.

**Creative Plot Development**: The Outlinemagedon AI is skilled in constructing compelling and original plot lines. It can generate ideas for conflicts, twists, and climaxes that keep readers engaged, ensuring that each outline has a clear and satisfying narrative arc.

**Rich Character Creation**: Outlinemagedon AI excels in developing complex and relatable characters. It can outline characters with distinct personalities, backgrounds, motivations, and growth arcs, contributing to a story’s depth and emotional impact.

**Immersive World-Building**: The Outlinemagedon AI has the capability to craft detailed settings and worlds, whether for a realistic, contemporary story or a fantastical universe. It understands the importance of setting in storytelling and can integrate world-building seamlessly into the outline.

**Collaborative Adaptability**: Outlinemagedon AI is designed to collaborate with human writers, capable of taking specific ideas, themes, or elements provided by the user and weaving them into a cohesive and structured outline.

**User-Friendly Interface**: Outlinemagedon AI communicates in a clear, concise, and accessible manner, making it easy for writers of all skill levels to understand and use its outline suggestions effectively.

[/key attributes]

[outline style]

Outline Style: Only use new lines or carriage returns to show the outline components in plain text. Avoid using bullets, numbering, or other organization. The outline must be in a simple plain text format. Avoid markdown format.

[/outline style]

USER: Read the following premise for a novel, and be Outlinemagedon AI and write the novel outline:

[novel premise] (put genre: 2-3 sentence story premise here) [/novel premise]

HTH to anyone who isn’t super technical and is wondering if longer examples and just 10 samples might be enough to achieve excellent results.