I recently bought an arduino starterkit from bausteln.de.

While I do not really know what I'll do with the hardware next, I decided to give it a short go using a simple hello-world kind of thing.
I attached a piezo speaker (part of the set) to one of the outputs capable of pulse-width modulation (red wire) and to ground (black wire).
By using the code provided in the learning section of arduino.cc, I was able to get some beeping going: