I suppose my first experiment was with my late Grandfather's portable wind-up gramophone that I was given to play with after dad had mended its broken spring. I took the deck out of the case to investigate and saw its centrifugal flying balls speed governor. I though that disconnecting it would make it run faster. In fact, with nothing to control the spring force there was a puff of smoke and small pieces of flying brass as the steel worm gear disintegrated the brass gear it meshed with, so that was the end of that.
I had a similar experience as others with mains and a neon bulb. It must have been a small mains voltage "Osglim" bulb that had been used in a BC socket wired in parallel with the 15A kitchen power point. It had become detached from its base and I tried connecting the two lead-in wires emerging from the glass directly to the mains (via some twin flex and a BC adaptor in the light socket: no wall sockets in the bedrooms), not appreciating the fact that there would have been a dropper resistor in the lamp base. Flash and the lights went out. Dad didn't seemed concerned at what I had done, just replaced the fuse. As I was showing an interest in electricity, he built a box to house an old mains transformer that I think had come from an early 1950's TV. It had lots of secondary windings, several 6.3V ones with different current ratings, a 4V winding , and a 400-0-400 HT winding. I did try to see it I could weld tinplate using the full 800V of the HT windings, using a 4mm banana plug with an exposed clamping screw mid-way along the insulating sleeve, but it was only good for writing on metal ( I was very careful!).
Of more practical use, I did use what I suppose must have been a small copper oxide rectifier stack (about ¾" high by ¾" diameter) salvaged from an old radio or TV in my Dublo train set, connected across an isolating rail at the end of a hidden siding so that the engine would stop automatically when it passed the isolating rail, but could be reversed out again. Not my invention, it was a hint in the "Railway Modeller".
Like boys of my generation, we went through the explosives stage at school. In the early 1960's any schoolboy who was studying chemistry worth his salt knew about explosives, and you could buy Flowers of Sulphur and Sodium Chlorate from the local Oil shop, and grind up the rods from old U2s for the Carbon. Our enthusiasm came to an end when one lad made up some of that Iodide stuff and found it fun to put a small amount on the sliding brass inkwell covers of our desks and hit it with a small toffee hammer. It made a lovely atom bomb-like mushroom cloud of smoke. Unfortunately the small tube of it that he had in his hand exploded unexpectedly and he was lucky not to lose his hand. I can understand how the SAS stun grenades work: I have no recollection of hearing an explosion, I just suddenly went completely deaf with loud whistling in my ears, and I was diagonally opposite him about as far away as you could be. Very disorienting, and it took an hour before I could hear again. It was just treated as a schoolboy prank in those days.
Much more dramatic experiments were possible in Edwardian days. Via a review of it I found some years ago in a facsimile edition of "The Model Engineer" for 1904, (see attached PDF) I tracked down an on-line copy of "Radium and other Radioactive elements" , a text book jointly written by Leonard Levy when only 18. Interestingly, a Google search no longer finds it, but it is still available as a free PDF download from the University of California on-line library here:
https://archive.org/details/radiumotherradio00levyrich