Testing o1-preview

I got access to o1-preview today.
Recently in my videogame, I made it so users can generate images from text.
Specifically I made so they can generate vector icons with transparent backgrounds.

Prompt: Pets, dogs, care
Output image:

I found that feature very useful, and I wanted to make a website with just that.

Let's go

Since I don't have a premium ChatGPT account, I cannot use o1-preview directly on their webisite.
However, I do have an API key with money on it. So I decided to make a quick nodejs project to communicate with o1-preview.

Then I added a script to package.json to run o1-preview

I created the script, plan.mjs

This takes a prompt as input and generates a file, plan.md, which is the response from o1-preview

I feed it with this massive prompt, describing the entire application:

and it gave me this long, beautiful output:

Opening plan.md

To open plan.md and view it with proper formatting, I used the Markdown Preview Enhanced extension.

...though I found out now that VSCode actually has really good Markdown support out of the box, and you can just press CTRL + Shift + V to toggle preview, but whatever.

Following the plan

So o1-preview seemingly made me a plan, that would take me from nowhere to a finished project.
That would be the case, until I hit my first bottleneck

The environment variables wasn't picked up by NestJS.
I had to manually patch it in, so my developer skills became useful.

Then I also had to patch the API calls.
The input type wasn't specified properly, so when generating the backend to use with the front-end, the input wouldn't get specified correctly.
Luckily also an easy patch, as I just had to add two lines of type specification.

The generated API was also missing a way to associate an endpoint with it - it seemingly tried to request http://localhost/ but my back-end was running on port 3000 and not 80.
I had to manually patch it as well - was very easy to do.

The auth was also broken.
Despite the plan including an Auth guard which is seemingly integrated into the NestJS constroller, I was met by an error I was too lazy to resolve.
Have I had a little more determination, I would have resolved this too, but decided as per the example just to remove auth.
I wasn't going to deploy this anyways.

And so it kept going, for a long time, until I eventually had something running:

It actually managed to generate an image with Together AI, upload it to Wasabi and create a Node in the Neo4j database!
That is just incredible... but it wasn't exactly according to the prompt

What it missed

Stripe integration
I was hoping to get a skeleton for payment integration, as I used a lot of the prompt to describe it.

User page implementation
It only got the very basics right, and seemingly forgot about the ability to fuel and see your profile page.

Authentication flow
There is a stub for the login, but it isn't implemented. I expected the model to deliver a more complete plan.

IP Restriction
I asked the model to restrict the anonymous user after generating 5 images, but it forgot about this completely

Reality check

While I think o1-preview is an amazing model that has strong capabilities, it still misses the input from the developer to really deliver anything.
And while I know that isn't the point of the model, but more to plan out your project, I think it serves as a proof of concept of what probably is to come.
One thing is for sure, and that is the future of this is exciting.

The goods

The way the plan created the NestJS services was very clean and modular.
I like how it created an S3 service, a Together AI service, and then used the two of those in an Image service.
It's a better approach than cluttering it all in one Image service, as you are able to re-use the S3 service and the Together service across multiple services.

I think as of now, this model feels like an extension of ChatGPT that is able to reason in a broader sense than the previous, but doesn't work perfectly well without human intervention.
It's still really good!