After initial disbelief, failed trials, and frustration, I finally started to appreciate having a little assistant on my side in the form of AI. While one-off simple questions were usually fine when using AI chat, I had never attempted a bigger project before.

Now, I am working on a new programming language, partially written by AI. I feel more like an architect and tester, sometimes taking on the role of a senior engineer, while the AI is like a restless and enthusiastic young developer. For this, I am using ChatGPT Plus.

I learned that you should not give complex tasks to GPT-4, but rather break them down into smaller, progressive chunks. I’ve been surprised by the deep reasoning of the O3 model, which had no trouble analyzing dozens of files and making sense of them. These files include the lexer, parser, AST, type checker, code generator, and tests for them. So, it’s not just about reading prose, but analyzing a tightly connected network.

For this reason, models like GPT-4 sometimes struggle with context, and I had to repaste files. But even then, it was often too much for it, and the quality would quickly degrade.

If I were to start another bigger project, I would first generate a detailed outline using O3 or 04-mini. After that, I would proceed to the main model like GPT-4 or 4o-mini and let it take small steps.

Overall, I am enjoying the process. I haven’t tried using the API yet. I think it could be better for my workflow (especially for reading and sending context), as I often had to paste code samples manually. However, the API would be much more expensive.