Except they're not chilli plants at all! My Dad identified them as fat-hen, a plant that often grows in disturbed soil, for example on new road construction sites. The chilli seeds must have not germinated, but the fat-hen did. It could have been laying dormant in the compost I bought, maybe it hadn't been sterilised.
The interesting news is these plants are edible, cooked like spinach, and used a lot in Indian food.
Luckily some of my real chilli seeds DID germinate, here they are. They're not particularly big, and not flowering yet, but they're certainly chilli plants!