I found an add-on that did this up to 3.05 but it stopped working. I created a button that appears in the HTML tab of the Add New Post interface. The button is for WP_LaTeX but you can modify it however you want. I dn't see any bugs, but forgive me if there are any. Also, please let me know if there is an easier way to do this, especially if there is a way to create a button in the Visual tab of the Add New Post interface. I might poke around a bit and figure it out, but I have other things I want to do first. Step 1: Go to the wp-includes/js folder, and download quicktags.dev.js and quicktags.js. Make a copy of both files in case you flub things up. Step 2: Open up quicktags.dev.js. Around line 32 you should see: edButtons[edButtons.length] = new edButton('ed_strong' ,'b' ,'<strong>' ,'</strong>' ,'b' ); Code (markup): Copy and paste that code: edButtons[edButtons.length] = new edButton('ed_strong' ,'b' ,'<strong>' ,'</strong>' ,'b' ); edButtons[edButtons.length] = new edButton('ed_strong' ,'b' ,'<strong>' ,'</strong>' ,'b' ); Code (markup): Then edit it as follows: edButtons[edButtons.length] = new edButton('ed_strong' ,'b' ,'<strong>' ,'</strong>' ,'b' ); edButtons[edButtons.length] = new edButton('ed_latex' // I think something to name the button ,'latex' //The visible name of the button ,'$latex ' //The beginning tag ,' &s=1$' //The ending tag ,'e' //The back-end name of the button? I chose the letter 'e' to avoid repetition. ); Code (markup): Step 3:Open up quicktags.js. This file is a one-liner, but look for edButtons[edButtons.length]=new edButton("ed_strong","b","<strong>","</strong>","b"); Code (markup): Duplicate that: edButtons[edButtons.length]=new edButton("ed_strong","b","<strong>","</strong>","b"); edButtons[edButtons.length]=new edButton("ed_strong","b","<strong>","</strong>","b"); Code (markup): Then in the second copy, modify it as follows: edButtons[edButtons.length]=new edButton("ed_strong","b","<strong>","</strong>","b"); edButtons[edButtons.length]=new edButton("ed_latex","latex","$latex "," &s=1$","e"); Code (markup): Notice the terms in the second function match the terms in step 2. Step 4: Upload the files I hope this works for you as well as it did for me! Post improvements here! If you can figure out how to create a button in the Visual tab of the Add New Post interface, please post here as well!
The reason it stopped working after the upgrade was that the core file was overwrote by the process. You should avoid editing core files. Have you tried the LaTeX plugin? http://wordpress.org/extend/plugins/wp-latex/
It looks like you are using the Post Editor Buttons plugin and it is not working in 3.1, so it appears that something has changed in the methods used to add new buttons to the editor. Another plugin, AddQuickTag, is also not functioning. BTW, you started a new post at WordPress Forums when one was already going concerning this problem. I do not think the author of that plugin has updated since 2.9, so you may have better luck with the other plugin to add the tags and see if that author will release a new update to it. Somewhere in the coding of those plugins you will find the filters and actions required to add new buttons to the editor. You may want to look at the code closer to see how it is done.