Setting field access privileges as noted in Assigning groups to fields on forms will also be applied to columns in browses that reference the field. Fields that the user has not got viewing rights to are shown in browse lists as ****. But what about browse columns that are calculated rather than referencing a file field? You can control these too. Just right-click in the column heading of a browse and select the Set field/button access option, as shown below:

The procedure is then the same as Assigning groups to fields on forms.