Can't cut down trees in Survival Mode?!?

The second I loaded a survival game, I tried to cut down a tree and it wouldn't let me. Same in hard core mode. I must have clicked away for a full minute before quitting out and restarting the game. Still did not load. What's going on?