string index out of range in `replace_part_with_missing_leading_whitespace` about aider

[paul-gauthier]

Good catch, thanks for reporting. I just pushed a fix. You'll need to install from github to pick it up.

The root of the issue is that GPT-4 won't consistently return properly formatted diffs. The <<ORIGINAL/>>UPDATED format aider uses is by far the most reliable format I've discovered. But GPT-4 won't always conform and output even this very simple format.

I've tried asking it to output diff -c, diff -u, sed, and many other formats. GPT-4 does much, much worse on them. Even the new functions api is not looking like a panacea here.

One key issue about the "output format" is that it's easy to "distract" the LLM if you ask it to use too structured or technical of a format. Especially when you're asking it to write code. The coding itself is a challenging task, and asking it to render actual diff or vim or sed editing outputs is going to confuse it. I have absolutely seen the quality of the coding task degrade when the output format is too complex.

Imagine asking a junior developer to "implement this cool feature, but I want you to hand type the diffs required to change the current code".

I'm hoping to write up something more quantitative and concrete about this aspect of AI coding soon.

[paul-gauthier]

Closing this for now. Feel free to re-open if you continue to have issues.