I have advertised with Yahoo and Fox Network and they are using Ad.Yieldmanager.com for ad serving and we got referral for that link, I have searched in Google that it is considering as spam in Google Analytics. I am not sure that it will effect to SERPs or not. After received the traffic about 14,000 clicks in one day that after one week, the Pagerank is dropped from 5 to 4.
I don't think ad networks effect rankings but yes their are few those create problems to open webpage with good speed .
What do you think that if we use Google Analytics and Google will use those information because it shows the traffic came peak in few hours and they might think that we are spamming but actually it is traffic from the advertising from ad network.