In The Encyclopedia of Paleoclimatology and Ancient Environments by Vivien Gornitz (2008, Springer) it is found that 50 My ago the sea temperature was about 13ºC above the present value.
.
AFAIK the several proxies gave consistent answers.
Why is the temperature always decreasing ?
I recommend you read the original paper Lear et al. (2000). In it they explain that the results are controlled by processes in polar regions and likely associated with the polar surface temperature. The surface temperature will vary considerable depending on the proximity and extend of the deep water formation areas. As these areas are fairly local, relatively small changes in the location and extend of deep water formation can result in deep water temperature being altered. The uncertainty associated with using single cores to estimate global temperatures is thus quite large. What the authors suggest is a predominantly ice-free world during most of the Paleocene and Cretaceous periods and several ice-sheet expansion events during the Oligocene, Miocene and Pleistocene. If there is no ice, the polar and sub-polar deep-water formation is limited and the water temperatures in the deeper part of the ocean is much warmer.
The temperature record shows a general cooling trend for the past 50 My, but its resolution is too low to define abrupt temperature extremes. Assuming that the process of deep water formation at high latitudes prevailed throughout the past 50 My, these deep-ocean temperatures can be inferred to represent polar surface temperatures (1). The temperature trend (Fig. 1C) appears to comprise four main cooling phases separated by periods of more stable temperatures. These cooling phases occur during the early Middle Eocene, the Late Eocene through Early Oligocene, the late Middle Miocene, and the Plio-Pleistocene.
Alternative evidence supports the existence of these principal cooling phases. A sharp cooling in the early Middle Eocene has been invoked to explain the abundance of siliceous sediments of this age (20). The Early Eocene warm climate is thought to have promoted in- tense chemical weathering of siliceous rocks at high latitudes, which resulted in high concentrations of dissolved silica in the oceans. The subsequent cooling stimulated upwelling of nutrient-rich water and intense biosilicification. The cooling over the Late Eocene through Ear- ly Oligocene is coincident with the transition from “Paleogene” to “transitional” benthic foraminiferal faunas and high extinction rates in planktonic and terrestrial faunas. The initiation of this deep water cooling also coincides with a reduction in the extent of tropical rainforests on land (21, 22). The cooling in the late Middle Miocene is marked by a narrowing of tropical and warm subtropical biotic provinces in terrestrial and marine environments (23). Warmwater diatom assemblages were replaced by cold-water species at this time (24). The Plio-Pleistocene cooling is not represented by an extinction phase in benthic foraminiferal faunas; instead, faunal groups adapted to the changing bottom water conditions by migration (21).
When you use data from other regions, you get a much different result. For instance, Elderfield & Ganssen (2000) showed how different the temperature changes can be for different species and locations. 
Their data are much more recent and suggest an increase in temperature over the last 20,000 years. As always, everything depends on the temporal scales. One issue that tends to be true is that estimates for periods long time ago tend to have larger uncertainties.
The reference in the figure (where the plot was cropped from - the period 70M-50M years is missing) is helpful.
The resolution on that plot is quite low but it can be seen that, for the past 50M years, the temperature is not always decreasing. There appear to be four main cooling trends flanked by periods of constant or increasing temperatures.
According to the authors, the long-term cooling trend is a result of several processes that work together in a nonlinear way to modify the distribution of heat on the surface of the Earth. However, we require many more global records of temperature before we are able to quantify their respective contributions.
From the paper:
The long-term cooling over this period is thought to have resulted from a combination of factors that altered the amount and distribution of heat over Earth’s surface. Variations in atmospheric greenhouse gases (principally CO2), Earth’s albedo, and linked changes in ocean-atmosphere circulation patterns, which are also influenced by the opening and closing of ocean gateways, could contribute to this long-term cooling.
Lear, Caroline Helen, H. Elderfield, and P. A. Wilson. "Cenozoic deep-sea temperatures and global ice volumes from Mg/Ca in benthic foraminiferal calcite." science 287.5451 (2000): 269-272.
Actual figure from the original paper:
