Solve the problem regarding single and double quotes

I have encountered with the problem, that the single quotes and double quotes of my website automatically convert to left and right single quotation marks and left and right double quotation marks respectively. Can somebody help me to solve the problem.

My website is http://rubensargsyan.com.

Thank you.

 

Can you set up a test article where you only write some " and ' ???

 

Thank you for your reply CSM. For example you can see the last post of my website: http://rubensargsyan.com/wordpress-plugin-useful-banner-manager/. Look at the quotes of title and comments.

 

If nobody has the same problem, please write how do you do so?