Malware warning = decrease rank?

I recently got my site off the malware warning on google. It had the warning for about 2 weeks.
Since it's come off (the warning) it seems that the incoming hits from google have dropped, even though my main keywords (that i rank highest in) haven't changed.
Have the less common keywords that my site used to rank for dropped or what?

 

If you had loyal visitors , they may be scared and pass your site.

 

I'm talking unique hits though.

 

i think it may decrease your serp.

you can start visiting http://stopbadware.org/home/review to solve that problem