I have heard the claim by creationists that geologists date rocks by the fossils they contain and date fossils by the rocks in which they are found. This supposedly invalidates radiometric ages because they are a result of circular reasoning. Is this statement purely wrong, or is there some truth to it?
Yes, there is some amount of circular reasoning in the statement: "geologist date rocks by the fossils they contain and date fossils by the rocks in which they are found".
However, that statement does not fully describe how geologists date rocks or fossils. Dating rocks by fossils is a branch of geology called "biostratigraphy". This is a non-absolute and a relative method of dating. You know that one fossils is younger than another, so you can be sure that one rock is younger than another rock given that they have those fossils, respectively. How old are they exactly? Based on biostratigraphy alone, this is an unknown.
The problem is solved using geochronology. Using radioactive decay to accurately and precisely date rocks. This is independent of fossils, but requires that the rock have to be dateable. This means that you need some kind of volcanic material (among other things) interbedded with the fossil bearing rocks. When you do that - you can be sure that a certain fossil formed at some exact point in time. Let's say 100 million years ago. Then you can use that knowledge to assign an age to rocks containing similar fossils where you do not have any rocks that you can date.
That said, new technological advances allow us to directly date fossils. One of the methods is using the Lu-Hf or U-Th-Pb systems on apatite - a mineral that forms teeth and bones. In cases where this method has been applied it showed that biostratigraphy actually works (unsurprisingly and to the dismay of creationists).
In addition to what @Michael said, one can add that the claim that "geologist date rocks by the fossils they contain and date fossils by the rocks in which they are found" is misleading: are some rocks being dated using fossils? Yes. Are some fossils being dated by the rock they are contained in? Yes. But they are not the same fossils or the same rocks in both cases!
An example using marine micropaleontology (both because this is what I know best, but also because this is the discipline in which biostratigraphy is the most commonly used): some marine sections (not all) have unique sedimentary properties that allow the use of geochronological tools, or the use of magnetostratigraphy. But those sections are in practice a minority, so what we do is that we use the "good" sections as references: on those, we look at abundant microfossils having a clearly marked first and/or last occurrence, as a bio-datum. And then we use those datums on the other sections to estimate their age. Eventually other microfossils are found on those sections and therefore their age are estimated this way. But those microfossils won't be used to date anything.