Assuming it is sulfur (likely, but I'm not addressing other compounds in this answer) you should be fine as long as you keep it away from heat (melts at a low temperature and is somewhat flammable).
I've had a chunk of sulfur in my house for about 40 years and never had an issue with it. I'm pretty sure most of the cleaning products in the house are more dangerous.
From the sulfur MSDS (PDF):
Sulphur is relatively non-toxic to humans, causing only mild local irritation to the eyes, nose, throat and upper airways. However, under certain circumstances it may release toxic hydrogen sulphide and/or sulphur dioxide gas. Sulphur is not listed as a carcinogen by OSHA, NTP, IARC or ACGIH.
Both molten and solid forms are combustible and will ignite at high temperatures (>200°C), burning with a pale blue flame that may be difficult to see in daylight. Sulphur dust suspended in air ignites easily and can cause explosions in confined spaces. Sulphur dust clouds can be ignited by friction, static electricity, heat, sparks or flames. Traces of hydrogen sulphide and sulphur vapor may present an explosion hazard if evolved into a confined space or enclosed space, particularly from molten sulphur. The LEL of hydrogen sulphide (4.3% by volume in air) may be exceeded in enclosed spaces above molten sulphur.
See also the precautions section on Wikipedia.
This article states that volcanos can deposit heavy metals along with water vapor and sulfur compounds:
When volcanoes erupt they exhale a cocktail of gases – mostly steam, carbon dioxide > and sulphur dioxide – laced with evaporated heavy metals, including lead and arsenic. To the communities living alongside volcanoes, these gases are often a considerable source of air pollution and the volatile metals they carry may have long-lasting impacts on both health and environment.
It's probably best to secure them from being picked up by little hands as they tend to put items (especially their dirty fingers) back in their mouths.