that's not what I was aiming for really ... 3v3 has a smaller current capacity than 5V, so your amp will distort at low power and/or the board will become unstable when it reaches the maximum current on 3v3.
3v3 is indeed cleaner than 5V on Bela, which could explain the improvement. However, the power-supply rejection rate (PSRR, i.e.: how much the IC can attenuate noise at the power supply) is 83dB for the MAX98306, which means that any noise on the power supply rail should be pretty quiet, so it's still a bit of a mystery.
What 5V were you using on BelaMini? If I remember correctly, P1.05 is the voltage straight as it comes through the USB port, so it's probably the best one to use, as it will be limited only by the USB bus's current limit and not by the on-board current limiter (which I believe is below 500mA).
Another thing I wanted to know is : when the amp is used like this:
when using my laptop's headphone out directly into the amp,
where is the Adadfruit board powered from?
Also, where do you get the ground pin that goes from Bela to the Adafruit board?