Midjourney, known for its AI image generation, is venturing into text-based large language models (LLMs) and has collaborated with New York University to release new research.

The research introduces two innovative techniques, Diversified Direct Preference Optimization (DDPO) and Diversified Odds Ratio Preference Optimization (DORPO), designed to enhance the creativity of LLMs while ensuring coherence and readability.

These new methods promote diversity during the training phase by utilizing deviation scores to prioritize unique responses over repetitive outputs.

Current LLMs often produce homogenous outputs in creative writing due to a reliance on user preferences and instruction tuning that favors common responses.

Key findings from the research indicate that models trained using DDPO significantly outperformed traditional methods in terms of output diversity while maintaining quality, even surpassing GPT-4o.

The study emphasizes the importance of tuning LLMs during the training stage to minimize the need for post-processing adjustments, which enhances human-like interactions and storytelling capabilities.

The implications for enterprises utilizing AI for creative tasks include increased diversity in conversational AI, content marketing, and narrative design, resulting in more engaging outputs.

Future applications of these techniques could extend to poetry, screenwriting, and other generative tasks, potentially revolutionizing AI-driven creative projects.

The research utilized models such as Meta’s Llama-3.1-8B and Mistral-7B-v0.3, training them on creative writing tasks sourced from the subreddit r/writingPrompts.

The researchers plan to make their code publicly available, providing a valuable resource for those interested in implementing these diversity-enhancing techniques in their LLMs.