Promptly Figma Plugin

Problem Space

Text-to-image AI has witnessed significant growth across industries, including design. It helps designers brainstorm design concepts, rapid prototype, create content, and more, but utilizing it effectively involves crafting the right prompts, which can present challenges.

The problem becomes, how might we overcome prompt writing challenges for accurate UI generations on text-to-image AI?

Solution Highlights

Promptly is a Figma plugin that eliminates prompt vagueness and allows for designers to efficiently and effectively generate accurate UI creations. We use NLP techniques to automatically identify key entities in prompts, highlight areas for improvement, and offer suggestions based on best practices for clearer, more effective prompts for images. Promptly also is an educational tool to show users how to structure effective prompts, making generative AI more accessible to beginners.

Design Process

Given the time and objectives, we prioritized idea exploration and prototyping our solution to approach this project.

1. Initial Discovery
   1. Hackathon Prompt Dissection
   2. Secondary Research
2. Problem Definition
   1. Further Research
   2. Goal Definition
   3. Narrowing our Scope
3. Prototyping
   1. User Flows
   2. Low and Mid Fidelity Prototyping
   3. User Testing
4. Final Design
   1. Final Presentation

Initial Discovery

This project is completed as a part of CMU's annual design hackathon, with the theme was "Design with AI" where teams are tasked to innovate a new AI solution or improve upon an existing one for designers. To kickoff, we coffee chatted with a CMU professor with 20+ years of experience in design, where he told us: "Maybe someday in the far future, UI designers will all be prompt writers." This raised our interest in text-to-image AI.

Problem Definition

When doing further research, we read "Is this the end for UX designers", where the author shares his experience of using text-to-image AI to generate an app, demonstrating both its capabilities and limitations. It took 6 iterations to reach the desired results. Can we make the steps go from 6 to 1?

Prototyping

The limited time of this hackathon didn't allow for a full exploration of ideation, so we thought about developing a solution that directly targeted the problem space in a straightforward way. At first, we categorized prompts into theme, dimension, and tasks, taking it step-by-step to input key terms and achieve desired results. However, after receiving feedback from HCI students, we reconsidered our prototype. This got me thinking: How can we solve the problem not only effectively but also efficiently?

To address the above issues, I thought more about a method to enhance user prompts without restricting creativity. This approach uses NLP techniques to automatically offer suggestions based on best practices for clearer, more effective prompts for images.

Final Design

After some further design, we arrived at Promptly, a Figma plugin that eliminates prompt vagueness and allows for designers to efficiently and effectively generate accurate UI creations. It is also an educational tool by showing you how to structure effective prompts. We can help make AI more accessible to beginners by familiarizing them with how to craft effective prompts.

Project Takeaways

There are a lot of things that I would've liked to explore but couldn't due to time. Some additional features that might've been beneficial to include having a history, to view past inputs, as well as including a reference image for style. Furthermore, Promptly raises ethical considerations surrounding gen AI. I wonder about how we can ensure that the images generated by Promptly and Midjourney is original and qualifies under fair use, and how we can promote a healthy use of AI.