Widely known poisons like mercury don't take much to kill you. One of the more dangerous versions, methylmercury, only takes 200 milligrams before death sets in. But just 1 gram of vaporized Polonium can kill 50 million people.
However, all of that is rather rare. Or at least, hopefully, you wouldn't intentionally consume (or put yourself in contact with) any of those things. So, how much would it take of more common items to cause your demise?
Well, it would take just about 70 cups of coffee to kill an average adult human. That amount of caffeine has a pretty severe impact on your heart, leading to heart palpitations and (eventually) cardiac arrest. Fortunately, that's quite a lot of drinking you'd need to do, so you really don't need to worry about it too much.
Stick your head under water? You've got about 4 minutes until you suffer severe brain injury and about 6 minutes until death (assuming that you don't attempt to breath in and fill your lungs with water...don't do that. Bad choice.). That said, with practice, some free drivers have been able to hold their breath for a staggering 20 minutes (maybe obvious, but absolutely do not try that either).
Want to know how much chocolate, cherry pits, or even height will kill you? Check out this video from AsapSCIENCE to get the answers.