Hi, and yes, we have done the same. We also like to get out there and meet others but sometimes it just does not work that way when traveling with others. First they have to belike minded. When I say that, they need to be out going and willing to meet others. IF they do not, then you get the feeling that you are ignoring them and end up spending all your time with them. ALL the others in the group HAVE to understand that we will socialize but not hang with them all day. That you have other people that you would like to meet and that if we were to hang together, we could have done that a lot cheaper at home.
We make sure of this and go forth.
It is also very difficult to get seating for dinner for a group. A couple is fine but anymore than that and it gets cumbersome.
IF the others in the group understand that you plan is to meet others and have some fun then it would be OK.
We have found some folks that understand and we now have fun if we go with them.
Hope this helps. Above all, have a blast!!