We provide intuitive, objective and reliable metrics for evaluating the participants of a discussion based on a basic set of roles. This method can be applied to any temporally ordered discussion transcript. We show that it is possible to partially automate the evaluation process by defining numerical attributes which can be extracted automatically from the content of discussion transcripts. We demonstrate its application to chat groups using the microblogging application Twitter.