On the sidebar, the groups are sorted A-Z while in the Groups browser, the Groups are sorted by # of contacts in each group. Could you also change this to A-Z (or provide both options)?
For nested groups, in my view it would be be best to show the groups in the same way as when the groups are expanded on the side bar (i.e. first the parent group starting with "A" and then all subgroups, then the parent group starting with "B",...)