Actually Using LLMs for Programming

Justin Nearing

LLM’s like ChatGPT is literally an invaluable tool for developers.

If you use it correctly.

Most of the whinging on the internet has been focused on having ChatGPT/Copilot/whatever-google-is-calling-their-chatgpt-today write the code for you.

I tried this as an experiment nearly a year ago:

This is an example of using the tool incorrectly.

It spits out real code, functional as well, but without long term context windows the AI eventually fell apart and started producing more and more errors.

This is especially true as the project increases in complexity.

Context does not scale linearly, it scales exponentially with complexity of the project.

This is why I don’t fear Devin.

The code is rarely the problem.

The problem is Susan from the product team isn’t satisfied the feature produced satisfies the requirements for which were fuzzy in the first place, with all communication going through the TPM, who only has 5 minutes to talk during the daily standup.

That is complexity.

Look at the amount of context you need just to update the color of the button to periwinkle blue!

Using It Correctly

The code is rarely the problem.

Which means most problems are easy to