Cable Loads

Currrent in the cable heats it up. Air is less thermally conductive than earth, and doesn't carry the heat away as fast. Hence less allowable current. Similarly, adjacent or closely spaced conductors generate higher local temperatures, so multiple conductors in a conduit are derated, and underground conductors have minimum space requirements between each other.

In addition, air temperature tends to fluctuate more than earth temperature, which is relatively constant. In other words when they rate cable "for 40 degC in free air" that is assuming the ambient air temperature will vary from 20 to 60 C, so the rating has to cover the spread and therefore must deal with the likelihood of having to carry the rated current when it is 60 C ambient. In ground, the average temp is something like 12 C in temperate climates, and it stays that way fairly consistently year round.