I can recommend the Raspberry Pi. You can have fun connecting LEDs, sensors and switches to the GPIO pins, which you can't really do on a laptop or desktop PC.
CPC do a kit with a Raspberry Pi, a solderless breadboard and a bag of components for some quick, simple projects.