I may have found a template for GenAI

It's been a few weeks since I worked on Sharpee. Tonight, I think I found a workable template for using GenAI (Claude Sonnet 3.5) to help with the coding.

I've set up a set of basic rules for the project as such:

"You are an expert in parser-based Interactive Fiction, C# 8, and design patterns, including Domain-Driven Design. You have a thorough understanding of in-memory graph data stores. You understand how a parser and world model work together. You will favor fluent interfaces and using LINQ lambda queries for data structures."

I started designing the system from the "bottom", so data storage first. Then Grammar Definitions, a rudimentary Standard Library, and then a translation layer between the Standard Library and the Data Store.

In my past attempts, I would just keep hammering at these elements and eventually the system would begin to forget what I'm doing, even with included code in the project. This is the first recommendation for Anthropic. Have a setting for "Check the attached code before each prompt." and if this is true, do the thing.

The next step is to break up your design into very small bites.

So, I wrote this prompt:

Please review the attached code.

We're going to implement a set of classes that act as a translation layer between the StandardLibrary and the DataStore. We're calling this the IFWorldModel.

The code in IFWorldModel should be usable in a fluent manner.

We should define Thing as a base Room is a Thing

Animal is a Thing that has a "animate" property

Person is an Animal that has a "sentient" property

Rule is a Thing that represents an event

Before Rule is something that can happen before any action within the story

After Rule is something that can happen after any action within the story

Rules can be ordered and there can be more than one of each

Scenery is a Thing that is "in" a Room, can be examined, but not moved.

Door is a Thing that can connect two Rooms, can be open or closed, locked or unlocked, transparent or opaque.

Each of these should be constructed in their own class files.

Let's start with Thing. I'll ask for each class as we go.

****

This results from this have been excellent. But if I don't like the results, I can go into design mode with Claude, and we can work through issues until the target class is completed. When I feel like the code is mostly the way I want it, I add it to my Visual Studio project, add unit tests, compile, run tests, and then upload the file to the Anthropic Project.

I change the prompt to say we've completed certain classes, remove those classes from the tasks, and then change the last statement to the next class.

Rinse and repeat.

Note I absolutely needed to have that first line so Claude would explicitly review the attached code and I needed to explicitly alter the prompt to the "current state", otherwise Claude gets "lost".

****

You can see an operative "template" here that could be integrated into a design and development process. Claude could integrate with dev tools like Visual Studio, allow for a list of tasks that are associated with code, track "blessed" code, and allow the developer to specify the list of tasks (code) to work on in a "one shot at time" manner.

1. Setup project prerequisites.
2. Define prompt that encompasses all future work, but only request one thing from the list. This should be an explicit "to do list".
3. Turn on "always check code first".
4. Allow Claude to examine warnings and errors and have a separate window for "suggestions".
5. Don't add code to the VS project until the user directs Claude to do that and where it belongs (I might want a new project in a solution or to name the class a certain way).
6. Always expect to write unit tests before or after the code is designed. (Claude should adhere to TDD if directed)
7. Once code is added, allow the user to tick off the task on a list.

****

Given this "template", I might see how GenAI could be transformative in a traditional dev team, but:

This only works if the developer already knows what they're building. This implies a product manager has likely helped define the prerequisites and the list of design and coding tasks, and it implies the developer can iterate through each task until it "works as designed".

And that's another layer that can't be automated by GenAI (yet). I could see how a product manager could iterate through some business requirements and come up with a set of tasks, but that would still need to integrate with existing code. Is that possible? I think so. Not out of the box and the GenAI tool would need to be able to traverse a code repository for the relevant code to compare to these new features. That's probably a future-state GenAI thing and is probably tricky to incorporate.

****

I still think GenAI overall won't match the hype around "AI". This is essentially a dumb-smart code generation 4GL. By dumb I mean you have to break your planning up a lot (and know what you're breaking up). By smart I mean in small chunks, GenAI can be extremely helpful, but someone still has to validate that what is being created and that it matches the product manager's and stakeholder's interests.

If anyone knows how to create Visual Studio integrations, hit me up. Maybe we could actually build what I envision.