A large part of it is hierarchy. If you're a grunt, and have completed your tasks, you don't have anything to do. Your boss knows this, and probably doesn't care. But his boss sees you on your phone, and immediately starts asking awkward questions.
Excess capacity. Some companies employ surge capacity and try to wrangle some level of productivity out of their employees during non-surge moments. Sometimes there are legitimate tasks (cleaning) which need to be done; sometimes an employee is not effective unless cross trained, Get a salaried remote job in a specialty field
Some jobs don't have the same level of activity throughout the work day. Managers need to make sure people are on task. If you're done with your tasking they'll ask why you aren't doing some other work/wasting time.
In this case a clipboard is your best friend. Just putz around with a clipboard and look at shit for a while. No one questions the guy with the clipboard.