Many a true word is spoken in jest
Some years ago when I was designing an amplifer for very small signals and was having problems with noise (what a surprise) I (temporarily) powered the first op-amp stage off a couple of those 12V 'car fob' batteries just to elimate the power supply as a noise source (that was not the problem but I did want to make sure).