In no order: - tracking - payout rate - payout reliability - program relevance - website quality - quality of creative - payment terms And many many more!
Offer conversions - test a few networks out on the same offer to see if one under reports over the other. There will be slight differences in tracking
Carrying On from Some of the ones mentioned above: How saturated the marketing is for the product that you want to promote Does the program offer bonuses if you meet certain sales targets Sites customer service and what they do to make their customers happy How long does the affiliate programs cookies last. The longer the better. Etc...