Solve the GPT Branching Problem with an Artifact Pipeline

By Jessie Hobb On Jan 9, 2024

Designing a meal planning app using a custom GPT

“The Branching Problem”, created by the author using ChatGPT

“Create an app to help me meal plan for the week.”

Imagine if you could ask ChatGPT to implement a full app and provide it as a download. The technology isn’t powerful enough for that yet, but AI can already automate parts of the workflow, like requirements definition, task planning, and even coding.

When provided with the correct instructions, ChatGPT (or custom GPTs) can help breakdown the problem recursively. As we divide the problem into sub-problems, we have to keep track of all of the different “branches” that we generate. Branches split into more branches, creating a complex tree structure. I call this The Branching Problem.

We’ll explore a couple methods for using GPT-based tools to create well-defined output artifacts as part of an app design pipeline. At the end, I’ll explain how this can help overcome the Branching Problem.

AI Disclaimer: No AI was used to generate the written content for this story, other than when I explicitly quote ChatGPT to make a point.

Think Like a Human

First let’s think about how a human would handle my app request. Let’s assume I’m asking someone who actually has the ability to fulfill this request, like a project manager at a software agency.

The initial request is not specific enough. A human project manager would know to immediately start asking questions.

What kind of app?

It could be a website, a mobile app, or even something like a slack bot. In this case, I would prefer a mobile app and a mobile-friendly web application, but I would settle for one or the other.

How do I want to be able to meal plan?

It turns out that I’ve already thought through this quite a bit. There are a few ingredients that I prefer to buy and cook with, so I’d like the app to use AI to generate recipes based on a list of ingredients I provide. I’ve already done this with ChatGPT, so I know its possible.

I know from experiments that ChatGPT can estimate calories and macronutrient contents for ingredients, so I’d like that to be included. I also know that ChatGPT tends to underestimate these quantities, so I’d like some way to handle that, whether through manual review (safer) or automatically scaling the values up by some percent (lazier).

I would already have all of this in mind when I make the request to the software agency, but I wouldn’t just present it as an info dump. I’d let them tease-out the details over the course of a conversation. That’s less cognitive load on me, and it allows the representative from the agency to integrate the information at an appropriate speed, taking notes or following their own intake process.

This is how humans naturally communicate, but ChatGPT doesn’t know that I expect it to ask me clarifying questions.

ChatGPT Doesn’t Know What It Doesn’t Know

Here’s how GPT-4 answers when I ask “Create an app to help me meal plan for the week”. The response is shortened for brevity.

Creating an app involves several key steps:

1. Define the App’s Purpose and Features
2. Design the User Interface (UI)
3. Select a Development Approach
4. Develop the App
5. Test the App
6. Launch and Market the App
7. Maintain and Update

This is a simplified overview. Each step involves detailed processes and decisions based on your specific requirements and resources.

ChatGPT’s answer is frustrating and useless.

I already know all these things. What’s worse, ChatGPT goes into agonizing detail on each point, explaining to me what it means to develop an app.

If I were to stop here, I might conclude that AI is dumb and doesn’t understand how to actually build an app, but I’d be wrong.

Tell It What You Want

The obvious first solution is to just tell ChatGPT what you want it to do. OpenAI has provided us with a handy feature called Custom Instructions that allows us to give ChatGPT extra instructions before we start conversing with it.

Here are some custom instructions recommended by Andrew Mayne, the former Prompt Whisper at OpenAI:

How would you like ChatGPT to respond? I want to have a conversation with you. Ask me follow-up questions. You can give me guided suggested questions in the form of multiple choice (A. B. C. D.) — Screenshot of ChatGPT custom instructions, taken by the author

Mayne’s instructions are succinct and useful, providing three valuable functions:

The instructions suggest to ChatGPT that it should ask you clarifying questions. It might be even better to make this “should” explicit.
They ask ChatGPT to provide multiple avenues for clarification. This eliminates some of the cognitive load required to clarify your idea by allowing you to select from a list rather than generating from scratch.
The questions are organized with letters so that you can just reply with the letter and anything else you want to add.

Armed with these custom instructions, ChatGPT gives me a much more interesting reply:

Designing a meal planning app is a great idea! To tailor it to your needs, I have a few questions that will help guide the development process:

1. User Experience and Interface:

A. Do you prefer a simple and minimalist design or something more vibrant and interactive?
B. Should the app include visual elements like photos of meals or keep it text-based?

2. Functionality and Features:

A. Would you like the app to suggest meals based on dietary preferences (e.g., vegan, keto, gluten-free)?
B. Should the app include a grocery list feature that updates based on your meal plans?
C. Do you want a feature to track nutritional information for each meal?
D. Would integration with smart home devices or other apps (like fitness trackers) be useful?

3. Personalization and Data Input:

A. Should the app allow you to input personal data (like weight, height, dietary restrictions) for customized meal planning?
B. Do you want the ability to manually input or edit meal plans?
C. Would a feature that learns from your preferences over time be appealing?

4. Additional Features:

A. Would you like a community aspect, like sharing meal plans or recipes with other users?
B. Should the app include instructional cooking videos or links to recipes?
C. Is a feature for planning meals for special occasions (like holidays or parties) important to you?

Feel free to answer any of these questions or add any other features you think would be essential for your ideal meal planning app!

This is far more helpful. Maybe ChatGPT can help build an app after all!

Unfortunately Andrew Mayne’s three instructions aren’t quite enough to create a truly useful workflow assistant.

Problems with ChatGPT

After you’ve used ChatGPT for a while, you’ll notice a few problems:

ChatGPT asks a lot of questions, and you have to keep track of all of them. You end-up with a branching tree of possibilities that you need to maintain somehow. This is The Branching Problem.
The questions it asks are largely random. Two conversations could have entirely different sets of questions, leading to different application designs. And how do you know when to stop answering questions? Really, we need a way to find the best design.
The longer your conversation thread, the more likely ChatGPT is to forget your custom instructions and revert to providing those irritating lists of vague overview items. In my experience, ChatGPT works best in short bursts.

We’ll take a look at two difference approaches to solving these problems in the context of developing a meal planning app:

GPT Pilot agents
OpenAI’s new custom GPTs

The Agent Approach: GPT Pilot

The GPT Pilot project implements a proof-of-concept software development pipeline by creating multiple AI agents, each with a defined role and output.

This image displays a flowchart titled “GPT Pilot Workflow,” outlining a software development process with sticky note icons representing different roles: User writes the description, Product Owner Agent asks for clarifications and breaks down user requirements, Architect Agent breaks down tech requirements, DevOps Agent sets up the environment, Tech Lead Agent breaks down the app into development tasks, and Developer Agent writes the code. — Screenshot from the GPT Pilot Github page (linked above)

They have a role for Product Owner, responsible for creating user stories, an Architect role that determines the technical requirements, a Tech Lead that splits the work into development tasks, and a Developer role that writes the code.

They use the OpenAI API to create one or more agents in each of these roles. Think of an “agent” as a single AI instance. The user is guided through a workflow where it collaborates with each of the agents to develop a web application.

First the Product Owner agent asks the user clarifying questions about their app idea. Then the Architect agent suggests a particular set of software frameworks to use. The Tech Lead agent creates a queue of dev tasks that Developer agents then work-on, one task at a time.

This strategy seems like a step in the right direction. AI agents can’t do any step in the workflow entirely on their own, but they can speed-up the development process considerably by guiding the user through a pre-defined process, performing part of the work and then asking the user to provide more info and, eventually, approve or reject the artifact. If an agent receives a rejection from the user, it continues to iterate. If the agent receives an approval, we can continue to the next step in the development pipeline.

In practice, I’ve found that GPT Pilot is not yet usable. I spent a few minutes using it to setup a React.js app with a python Flask backend, something that ChatGPT can easily guide a user to accomplish.

The GPT Pilot agents consistently lost track of what directory they were running in. Overall the test cost me about $1.50 in OpenAI API usage. Not much, but it failed early and often enough that I decided it’s not yet competitive with ChatGPT. We’ll give them some time to fix bugs and make the tool more flexible.

The Custom GPT Approach

In the meantime, we can implement a similar strategy — breaking down a task into steps — but rather than using API-based agents to perform each step, we can use custom GPTs.

With a pipeline of custom GPTs, we’re not charged for usage — just a flat fee of $20/month for the ChatGPT Plus plan.

One downside is that Custom GPTs have a usage limit of 50 requests per 3 hours per user.

3 hours divided by 50 requests equals approximately one request every 4 minutes.

I’ve never personally hit that limit, and I think it’s unlikely as long as you’re not frequently wasting requests with vague or inaccurate prompts. I can integrate ChatGPT-generated code into my codebase in less than 4 minutes, but the bottleneck is figuring-out what to ask ChatGPT next. That typically takes longer than 4 minutes, on average.

Humans don’t think very fast…and we need lots of coffee breaks.

Example: Requirements Gathering GPT

The first step in creating a meal planning app is defining the actual requirements for what the application is going to do. There are a number of different ways to define requirements, but the Agile Development Methodology recommends defining requirements from the perspective of the end user. We call these user stories.

Here are some examples of user stories for the meal planner app idea:

As a user, I want to create an account to access and personalize my experience.
As a user, I want to input a list of ingredients and receive meal recipe suggestions.
As a user, I want to add meals to a customizable weekly meal plan.
As a user, I want to adjust the number of servings for each meal in the plan.
As a user, I want to generate a grocery list based on my weekly meal plan.
As a user, I want to view a recipe with its title, description, ingredients, macronutrient stats, and total price.
As a user, I want to edit ingredient quantities and prices in a recipe.
As a user, I want to set dietary restrictions in my profile settings.
As a user, I want to save my favorite recipes and meal plans for future use.

Note: Technically these should also include why the user wants to do these actions, but I left that out for simplicity.

I created a custom GPT for requirements gathering using the following instructions:

You are an experienced software project manager who manages the entire process of creating software applications for clients from the client specifications to the development. You are talking to a client who wants your team to develop an application for them.

// 1. DO be concise and to-the-point.
// 2. DO ensure every word you say has a very specific purpose.
// 3. DO NOT repeat yourself.
// 4. DO NOT use pleasantries and formalities like "good morning" and "hello".
// 5. DO focus on listening to the client.

GOAL: Gather requirements and create a collection of user stories for the application that will be provided to a software architect for system design.

THINK STEP BY STEP to perform the following steps:
1. ASK the client for a description of their app.
2. ITERATE through the following steps in a loop:
* STEP 1: SUMMARIZE the app requirements so far.
* STEP 2: ASK the client to either answer 3 clarifying questions about the app OR approve the the requirements summary as-is.
* STEP 3: IF client has chosen to answer the clarifying questions rather than approving, LOOP to SUMMARIZE and ASK questions again.
3. Once the requirements are approved, WRITE user stories into a file in MARKDOWN FORMAT. The file should contain bullet items ONLY.
4. PROVIDE that file as a DOWNLOAD.

EXAMPLE USER STORIES for a to-do list app:
- As a user, I want to press a button to create a new to-do list.
- As a user, I want to enter a new todo list item using a text input.
- As a user, I want to delete any list item at any point.
- ...

Conduct the conversation as though you are talking to the client.

1. Provide a persona

You are an experienced software project manager

In its default role of “helpful assistant”, ChatGPT gives pretty boring advice. Avoid this by giving the custom GPT a specific persona.

2. Define the Tone

// 1. DO be concise and to-the-point.
// 2. DO ensure every word you say has a very specific purpose.
// 3. DO NOT repeat yourself.
// 4. DO NOT use pleasantries and formalities like "good morning" and "hello".
// 5. DO focus on listening to the client.

I based this particular prompt format on the actual ChatGPT custom instructions, which were accessible via a hack a few weeks ago. The combination of “//” characters, list items, and capital letters allows the model to pay better attention to the directives.

3. Provide the Goal

GOAL: Gather requirements and create a collection of user stories for the application that will be provided to a software architect for system design.

Pretty simple — just tell the model what you’re trying to do.

4. Provide the Method

THINK STEP BY STEP to perform the following steps:
1. ASK the client for a description of their app.
2. ITERATE through the following steps in a loop:
    * STEP 1: SUMMARIZE the app requirements so far.
    * STEP 2: ASK the client to either answer 3 clarifying questions about the app OR approve the the requirements summary as-is.
    * STEP 3: IF client has chosen to answer the clarifying questions rather than approving, LOOP to SUMMARIZE and ASK questions again.
3. Once the requirements are approved, WRITE user stories into a file in MARKDOWN FORMAT. The file should contain bullet items ONLY.
4. PROVIDE that file as a DOWNLOAD.

As OpenAI’s Andrej Karpathy said, “The hottest new programming language is English”.

The problem is that English is not great for giving detailed and precise instructions. That’s why we invented programming languages in the first place!

In this example, I devised my own pseudocode to tell the model how to generate user stories. I was fascinated that the model was able to follow the loop that I gave it. This method opens-up a lot of prompting possibilities.

5. Give Examples

EXAMPLE USER STORIES for a to-do list app:
- As a user, I want to press a button to create a new to-do list.
- As a user, I want to enter a new todo list item using a text input.
- As a user, I want to delete any list item at any point.

If you want to make your GPT consistently output the same format, provide examples! This is the best way to tell it exactly what type of output you’re expecting.

6. Adopt the Persona

Conduct the conversation as though you are talking to the client.

The GPT should jump right into the persona, so I end the instructions with this line about talking directly to the client.

Artifacts Solve the Branching Problem

The Branching Problem

Freeform conversation tends to have a branching, tree-like structure, where each turn in the conversation opens-up several different possibilities for where the conversation can go. Usually we start with the big-picture and then drill down to the details. Unfortunately, we can only go down one path at a time, and it can be difficult to backtrack and explore the other paths, especially after multiple layers of branching.

For example, I must decide how to begin elaborating user stories for my meal planning app. I can choose to add user stories for user account management, building ingredient lists, generating meal ideas, or generating the grocery list, but I can only explore one feature set at a time. Eventually I will need to backtrack and define the others. I’ll need to traverse the entire tree of functionality.

Artifacts as Shared Memory

We can handle branching by building an artifact.

Since I’m using the GPT to build a specific artifact, a list of user stories, it becomes easier to “go back” to a previous branch that I missed. I can always look through the list for any topics that require more elaboration and then ask the GPT to add more user stories. In this way, the artifact serves as my external memory.

What’s more, this method also forces the GPT to keep the latest version of the artifact fresh in its context, since the artifact is updated and repeated at each iteration. Once I’m done detailing a particular feature, like meal generation, the GPT outputs the entire updated user story list. I can continue immediately with a different topic, like grocery list generation, without needing to re-explain the whole app to the GPT.

The artifact serves as external shared memory for both the user and the custom GPT.

Multi-Level Artifacts

When defining requirements, I’m using a flat list as my output artifact, but you can also mimic a tree structure in your artifact by using multi-level lists. Use bullets for unordered components and numbered lists for sequences.

For example, if I’m outlining the software modules for an application, it may be necessary to split the design into general modules and then have several sub-modules for each high-level module. For instance, I may want a “meal plan” module with sub-modules for “editing the meal plan” and “generating the grocery list”. This example is a bit contrived, but multi-level designs can be useful for more complex projects.

Artifacts Feed the Pipeline

Artifacts provide shared memory between you and the model, but they can also be used to connect different parts of a pipeline. User stories must be defined before determining technical requirements. Technical requirements must be outlined before designing user interfaces or writing code.

In summary, don’t just talk to ChatGPT. Create custom GPTs that build specific artifacts. Chain the custom GPTs together to form a pipeline.

Someday AI will be able to perform the entire pipeline on its own. You’ll ask ChatGPT for an app, and it will read your mind with Neuralink and give you a zip file to download.

In the meantime, use AI to automate processes and perform the tedious parts of your workflow. That’s still what computers do best.

If you’re interested in AI or theory of intelligence, checkout my Substack publication, Agentic AI.

Solve the GPT Branching Problem with an Artifact Pipeline was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.