They definitely appear on all lists of poisonous plants that I've seen! Here's what supposedly can happen if you eat them: "Upon ingestion, causes permanent paralysis, slow and weak pulse,shallow breathing, pain, weakness,tremors, excitement, and convulsions."
Sounds unpleasant indeed! And all parts of the plant are supposed to be poisonous if ingested, not just the seeds. I gather that you need to eat enough of them for it to reach this sort of state, so if you're still walking around, you should be OK but it might be a good idea to stop eating them!