Topics
Notes on evaluation
- I spent days tuning the prompt, and what actually moved accuracy was changing the model
I went from 30.7 % to 100 % of days correct reading shift rosters. Almost none of that gain came from the prompt, and the metric that mattered wasn't accuracy.