It seems to me when reading about minerals, groups and series are spoken about interchangably in similar types of contexts. Is there a difference? If so, what?
Yes, there is a difference.
A mineral series represents mineral which have a well defined solid solution. One of the most known examples is olivine: olivine is not a strictly defined chemical composition for a mineral, but rather a series between the forsterite end-member (Mg2SiO4) and fayalite end-member (Fe2SiO4). Note that if you find a pure end-member in nature, then you have a mineral species. Quite rare for olivine though. There's also the olivine group, which consists of other minerals that do not form a solid solution with the forsterite-fayalite series (for example monticellite CaMgSiO4) or minerals that do form a solid solution, except that it's extremely rare in nature (for example tephroite Mn2SiO4). Here's a comprehensive list of the minerals in the olivine group: http://www.mindat.org/min-29264.html
Another example is the pyroxene group. So for simple Ca-Mg-Fe pyroxenes, you have the clinopyroxene and orthopyroxene series, which consist of the end-members (species, if you find them pure) diopside (CaMgSiO6) - hedenbergite (CaFeSiO6) and enstatite (MgSiO3) - ferrosilite (FeSiO3), respectively. Intermediate clinopyroxenes are also called augite.
The distinction is not always clear. For example, the feldspar group traditionally has the plagioclase series (albite-anorthite, NaAlSi3O8-CaAl2Si2O8) and the K-feldspar species (KAlSi3O8, either microcline, orthoclase or sanidine, depending on structure). But in some cases there is a solid solution between albite and K-feldspar, forming the alkali-feldspar series. Nature doesn't always behave according to our rules, and it's important to remember that our classification systems work well with ideal systems but break down often.