Erratic Climate
Lets take Broome, Australia as our example. Broome gets 615mm of rain a year (24 inches), including 58/182/180/102mm in Dec through March. Coupled with average highs in the low 90s, this would be fine for the sort of agriculture you see in South Texas. However, Broome's rainfall is very erratic. For example, in 2016, the rainy season started January 15th, with 23 mm of rain over three days. In 2017 (really 2016, again) the rainy season started December 4th. December 2016 also featured 222mm of rain (remember the average is only 58mm) including 113mm on Dec 23rd and 24th. 2016-2017 featured 222/230/249/114mm of rain, with significant rainfall later into April. 2015-2016 featured 0/119/26/28mm of rain. One year saw too little rainfall to grow even millet, while the next year had enough rain to support paddy rice.
Laterite
Old tropical soils in rainy or monsoonal rainy regions tend to have their soils leached of useful nutrients. The resulting soil is called laterite, and it is heavily enriched in aluminum and iron ore. On the one hand, this explains the mineral wealth of tropical Australia. On the other, laterite (alternately called ferralsols, especially by the UN Food and Agriculture Organization (FAO) ) is hard to farm, with few minerals for the plants, poor moisture retention, and the tendency to form an iron-hardened crust in the dry season.
Not enough people
This is sort of a meta-reason, but Northern Australia is a long way from anywhere. Without very many people, it is hard to build a stable agricultural economy. If a bunch of people where sent there and forced to survive, they'd probably eke something out (for example, like the people who first populated Sydney, or South Africa). But since that never happened...population of the reason never happened. Sort of circular reasoning, but without ever having a reason for a lot of people to show up in Northern Australia (prison colony, religious persecution, etc) no one has ever made the infrastructure that successful farming operations could be based off of.