First, an acknowledgement: I got this idea from woodgears.ca. If you haven't visited that site, you should.
This is a very short video, because I own exactly four Lego balls, but the device could count 31 balls (five bits) if only I had that many.
A ball locked in place by a right-leaning rocker represents a binary "1", while an empty, left-leaning rocker represents a binary "0". Dropping balls in columns other than the rightmost one adds powers of two. You could even drop balls in several columns at once in a binary pattern to add arbitrary numbers together.
