Yep, I already use what you suggest, but that is not the point:
In List Management even if a field is selected not to be displayed on profile, it can still be used for sorting and filtering. However it can not be selected for display in the list.
Hovever, if the list itself is limited to be viewable only for users with back end access, they do actually have access to see (and edit) those fields. They can only NOT see them in the administrator defined lists.
This is, as I see it, an unnecessary limitation.
The limitations for which fields can be selected for display in a defined list could better be governed in accordance with the user access level for the list itself.