What about logs. I think to automate writing code you need. AI + Tests + Access to Logs. Thats my experience at least. Works great with tricky hard problems.