I think I might not be able to keep all my SPI devices reset.
- I have a SSD1322 oled driver with a reset pin wired to a gpio. I'll add a physical pulldown resistor to make sure it stays reset. Not a problem.
- I also have 2x MCP3008 which do not have reset pins but use the CS both as chip select and as shutdown. As I have one of the CS pins connected to P9_17 (spi0_cs0) and this pin is used when initializing from SPI0. I can add pullup resistors so they stay shutdown. It still might be possible to create a similar situation as you described, where sometimes the SPI boot might 'succeed' and thus not proceed to SD card boot. I can probably verify if this happens without ordering new PCBs.
As for an alternative boot order, I looked at the boot config table and I found one options that might work:
0100 0000 0010 1000 EMAC --> MMC0 --> XIP (MUX2) --> NANDI2C
If the EMAC / ethernet boot reliably fails this could be an option. I'm not sure if this is a good idea security wise and it seems to add 5 second to the boot time according to the EMAC boot initialization procedure.
To get this boot order I'd have to pull down both P8_43 and P8_41 which I can do by connecting both pins through their own resistor to one side of the switch, and the other side to ground. If I find out later that I don't have the SPI boot issue I can just not populate the P8_41 resistor.
I think I'm going to try if just shorting P8_43 to ground is enough before making further changes to the PCB.