It's not easily answered, but I think, given the criteria you've laid out, the effects are generally pretty small.
I am not interested on events that can in fact interact with the
timescales of other climatic components. For example when land masses
ocupied positions near the poles and the effect of the resulting
increase in glaciation can be on the same time scale as other
non-lithospheric physical phenomena. Nor am I interested in singular
volcanic eruptions or activity of that sort. I am interested in how
the processes such as continental drift, mountain forming, erosion,
affects climate change.
Can I assume that you want to ignore events like the closing of the Isthmus of Panama which may have had a significant climate effect?
It's hard to say with any precision or absolute certainty but I don't think the climate cares much if mountains are growing or shrinking, it cares where the mountains are and how high they are, but the actual movement of plates and continents in and of itself shouldn't have a big effect on climate.
I say that is a few reasons. 1) we've had two significant tectonic events recently, one just off Japan, where Japan moved 3 feet closer to California and there was the huge, devastating tsunami, and that happened at a time when we've been monitoring climate change very closely and I've not seen a single article that suggested that the Japanese earthquake played a key role in 2011's weather, much less long term. Same with the large earthquake off Indonesia and Thailand a few years before, but the 1991 volcanic eruption (Pinatubo) did have a measured climate effect. Here's an article that suggests the same:
http://blogs.ei.columbia.edu/2011/04/28/can-big-earthquakes-disrupt-world-weather/
The atmosphere and oceans get energy every minute of every day from the inside of the earth, but even during the largest upheavals, there's no clear change in climate for the single year, much less long term. Volcanoes which release a lot of heat into the atmosphere have a cooling effect, not a warming effect, because of particulates and sulfates that block sunlight following the Volcano - so I have a hard time seeing that heat or movement from the earth directly effects climate.
It's worth noting that particular events can impact climate - like the Isthmus of Panama, mentioned in the other answer, or this one, below. which was far more sudden than the formation of the Isthmus of Panama, though the effect was also smaller and perhaps temporary.
https://climate.gov/news-features/features/day-yesterday-when-abrupt-climate-change-came-chesapeake-bay
and while the Isthmus of Panama is often credited for the beginning of the ice age cycle, it's worth noting that the Himalayas are quite young. I couldn't find an exact date, but they began growing about 10 million years ago and may have had a significant effect on climate over the last 5 million years as well. So, specific events, like the drifting of Antarctica over the south pole (30 million years ago), the crashing of India into Asia (10 million years ago) and the separation of the Atlantic and Pacific by the land bridge in Panama (3 million years ago), each may have had significant climate effects, but the general movement of continental plates and gradual rising of mountains in and of itself, I don't see why that would affect climate much.
The fastest and most significant recent changes to the shape and altitude of land isn't the growth or erosion of mountains at all, but it's glaciers. Half of North America being covered in 1.5 miles of ice likely had a significant effect on climate, not just because of albedo but also, that much change in altitude over so much land effects weather patterns and probobly global climate too. (see article below).
http://www.bristol.ac.uk/news/2014/november/heinrich-events-and-climate-change.html
Finally, my skepticism that gradual changes in plate movement and mountain growth or erosion effects climate is because there's no good evidence that it does, at least not short term. Changes over millions of years - yes. But on a smaller scale, I don't see it. Certainly changes can take much less than millions of years if there's a specific bottleneck that's either formed or undone, like the Isthmus of Panama, or, the black sea deluge / expansion (possibly the flood of Noah and Gilgamesh), which was likely due to the breaking of a natural dam, though I'm not sure if that one truly affected global climate. An earthquake will make a few very big waves but it won't change ocean currents, unless it creates a blockade or opens up a major spillway of some kind.
Changes in climate over the last 2.5 million years have been linked to orbital cycles, changes in CO2, the occasional very big volcano (more temporary), and, to a smaller extent, oceanic cycles and solar minimums, and there's some geological evidence of the occasional, fairly quick climate change without clear cause, but as far as I know, there's no good evidence that changes in tectonic forces drive climate over the short term. Over millions of years - yes, over the shorter term, probobly not.
Mega our outburst floods might have some effect on climate, but those are pretty significant events, easily comparable to a very large volcano and no outburst floods have happened in the last 7,000 years or so. It's also worth pointing out that while there's no likeliness that one will happen soon, we could see an outburst flood at some point in the next century or millennia, as a result of man made climate change.
http://en.wikipedia.org/wiki/Outburst_flood
I also thought this was interesting, though it's not clear what effect it had on climate: http://phys.org/news179598629.html
And, granted, my answer is speculative, but I'm not sure there is an exact answer to this that isn't somewhat speculative. Hope that wasn't too long and confusing. :-)