We've all seen the family gathering where the brothers and sisters or grandchildren are lined up together with their backs against the wall and are stiffly standing shoulder to shoulder facing the camera with zombie faces. AUGH! AUGH! AUGH!
Many people don't like their picture taken in these situations to begin with so getting them to pose more naturally is next to impossible. The least you can do is to find an interesting background and to have them facing an available light source such as a large window so they're not back-lit.
One thing you can do, if there's time as these can be impromptu, is to talk to the people to make them feel more at ease and pose more naturally. Try to have them turn their bodies a bit so they're not facing the camera. If you can get them to interact, have them lean on each other, put arms around another, etc. If you can have some sitting or arrange the bodies so they're not all in a straight line, even better.
If you can get the group to snuggle up together you can zoom in for a tight shot. You can't do this with people who are more acquaintances than family/friends but for goodness sakes, try to zoom in tightly so you get people from the waist up; toes don't need to be in the image.
If the people are familiar with each other try to get them to interact with each other. It can be very spontaneous and difficult but you will be rewarded if you are patient and work quickly at the same time. Children sticking out their tongues and doing the rabbit ears with each other, those are more fun and lively than the sit-and-grimace poses. That's what the boring school portraits are for.
For a "Rock Band" type portrait, this is a situation where the people may not be that close but will be accommodating for a more artful type image. Here you may want full body toe-to-head captures which breaks the rules in the paragraph above. Utilize the venue you're in, generally the people are more willing to take the time for a good shot. You've all seen good rock band portraits; arrange the bodies in interesting positions. If there are props around you may want to incorporate those in the scene.
Family portraits, it's rare to have the people all the same height in the frame. Some are higher, others lower. You can go broke buying books on formal portraits so all the information you need on that subject is out there and available.
For a formal group of people, they will be unwilling to interact with each other which is fine, usually that is inappropriate. Again, utilize available light and the venue. If there is a leader of the group, try to emphasize that individual. Put the leader in the middle and have the others on the side with their bodies angled to face the leader. The type of situation and group many times dictates the proper way to photograph.
Lastly, the face itself. If you're doing something moody or serious, it's not that hard to get the subjects to produce that kind of look. The one look that is always a challenge is the SMILE. Go look in the mirror (Yes, do this! Go look in the mirror!) and smile. Practice smiling until you have a natural, nice smile. Be aware of how your face feels and the muscles you are using. This is so YOU know how to smile when you have your picture taken. Because... we all know what it's like to take a picture of someone who stiffly bares their teeth with an unnatural smile. When people do this, show them your smile. Laugh with them, talk to them, whatever it takes to get them to lighten up and be more natural.
So... many times the venue and group will dictate what you can do. The secretary will not do rabbit ears to the boss and the boss will not put his arm around the secretary. In other words, be aware of the limits of the group. If it's a formal studio shot, use traditional posing and lighting. If it's an impromptu situation, think quickly about background and available light. Interact with the people to produce the desired look on the face. Props can do wonders, even if it's something simple like a hat. Magic can happen when people interact with each other. Mood and artful composition can be achieved when people are spaced apart.
Nikon D800, D100, Canon G15