The New Google Crawler is Definitely Here!

Google definitely has a new crawler, and I can verify that the rumors about CSS are true - it crawls style sheets.

I have articles on my site that repeated the headline at the bottom of the page enclosed with H4 tags that ranked very highly with Google. Well, I'd recently reviewed my site & saw that for some long headlines the text in the H4 tags wrapped and looked unsightly ('unsightly' - get it? Web site? Unsightly? Aw, never mind...).

As a quick fix, I edited the linked style sheet to take H4 text down to the size of 2pt text. Ouch - suddenly, these articles, many of which were ranked in the Google Top 5 for their targeted keywords, were disappearing from Google - didn't show up in the cache, didn't show up in a Google site search.

Later, when one page that ranked #2 for it's targeted keywords showed up as #80 (yes, I counted), I removed the size info for the H4 tag. Within five days, everything returned to normal.

Thinking about this from Google's perspective, I'd say that, with their resources, I'm not surprised that they could invent a crawler that can determine what the browser view of a page is.

 

Finally it has to be done

 

Wow, this is great news, thanx

 

Hope they dont start mistaking css hiding for good purposes with malicious ones. I personally have a few items set to "display: none;" just because they are later called by javascript to be displayed.

 

That's good to hear. Thanks for the heads up, Corwin.

 

Interesting stuff. I had no idea about the H4 method.

 

Maybe I need to rethink my H tags, I dont even believe I use H4 on any of my pages, I use mostly H1s and H2s. Do you think H4s help? Also are you suggesting that because you reduced you font size for the H tag that Google saw that as bad? Sorry if this sounds noobish.

 

I checked my logs and Googlebot 2 has never looked at my css file. They must have another bot using an assumed name that reads style sheets.

 

It doesn't matter what the tag it is, the fact you manipulated the tag via CSS in an attempt to influence the search engines rankings in googlebots eyes.

No the h4 tag is not more important than h1, h2 , that is the reason it is #4

Because one person is not able to see the bot... does not mean it does not exist.

 

There is more than one Googlebot, and if I am not mistaken googlebot2 is for adwords usage.

 

interesting stuff... thanks for the heads up.

 

This is exactly one of the points I was trying to make on another thread about Googlebot reading CSS; because CSS is sometimes relative and not absolute, I find it hard to believe that an automated system will know what's what 100% of the time.

 

An automated system will not know 100% of the time.

Again live by Google die by Google

Your marketing efforts should concentrate on the fact that Google may decide to toss your site out of it's index or so far down satan wouldn't find you.

The onus is not on Google to build your business....

Use Google to help you but don't let them be your crutch.

 

Maybe not 100% of the time, but an automated system does display CSS and html, and knows exactly how it looks to the human eye. Your browser does this. Google should be able to know exactly what is what, and how it is displayed, it's complicated, but it's not much different than say.. a scanner converting what you write to text.

It should be able to see the code, form a picture of how it's layed out and know what is what.. i guess if something is displayed later with JS, it might not, unless it can read JS now? Anyway, if you hide something, that's not necessarily bad, if it can understand CSS, and sees the way the page is layed out, and see that it's hidden, it should just ignore it. I think ideally, google wants to see the same page the user sees.. it's like old school blue/black text on dark bg, google should just ignore it?

 

There is no maybe about it.

If the machine could be accurate 100% of the time then there would be no spamming.

It also cannot be 100% effective as a human since the scripts cannot determine 'intent'.

Pages are scored based on known parameters and therefore have no fault tolerance as well.

---------------------------------------------------------------------

Google uses a text browser to look at pages. Lynx is the one they tell developers to use as well. Again this goes to the accessibility issues Google has always stood behind such, for example image alt text for eye sight impaired. Using the alt image tags was always a good thing till it became spammer heaven now the use of the tags probably has no score.

---------------------------------------------------------------------

It does go back to the old days where color of text and background were used to determine intent....the thing with the scripts is an all or nothing usage... since they cannot determine intent.... so when looking at same text background colors if 72% of the time it was determined to be used by spammers then under script programming anyone who uses it would be found in violation...all or nothing....so the 28% who were not using same colored text on background are label spammers due to the 72%...

Therefore search engines toss everything out the window,,,,even the good.

The font characteristics were made a certain way...manipulating them looks to the search engine, like you are trying to influence the rankings.

Catch22

 

I wrote about this on adsensebits before I sold it. It made the front page of digg, so I ended up receiving several replies from other webmasters confirming the same type of behavior as well. But I suppose old news is better than no news.

http://www.adsensebits.com/node/24

 

Greg-J

Not bad... a bit old (Googles been using lynx for before the current millenium). indeed but you're catching on.


Add in a dose of LSI and then you'll begin to understand the Google perspective.

You see as much as people have shouted organic results drive clicks to websites that is really BS.

Here is why.....people used yellow pages for years ...now the yellow pages is a big book to haul out and do nothing with...(Great child booster seat) so the reason for dragging out old Yellow was actionable...

Fastforward with me to 2006 and Joe Blow wants to buy a new 2007 lexus...he puts this into the old search box and whats he going to find...in the paid ads it will tell him clearly whos selling a car whos selling accessories, etc

in the organic results who knows what he'll get when he clicks a link???


Hot girls wrapped on a 2007 lexus...while the surfer might like it for a few seconds not what he was looking for and back button to check the next result..

how much of this hunt and check bullshit will the average consumer put up with.?? when it is much easier just to click the clear ad that sells what I want!!

Remember marketing folks,,,,,the end consumer is no longer your hostage ..they dont like your ad....THEY DO NOT HAVE TO SEE IT!!!!!

This isnt your Daddys television!!!

Google cares much more about the quality of the paid ads than the organic SERPs...

The serps are a needed evil to display Googles Ad inventory. If you remember this it will serve you well.


My 1.5 cents

 

Sem-Advance,

You talk a lot, but you're not really saying anything. You're not educating anyone with your babble, and you're not impressing anyone with your condescension.

Just a piece of advice, but comments like yours are a dime a dozen around here. And as I tell my children; You won't learn anything if you already know everything.

 

Ok so your point in telling me that I offer no value (Which is your jealousy peeking out)

And that you are the arbiter of all that is wisdom and sage advice??

I guess next youre going to tell me you didn't say that either...

Next you imply that I said somewhere that I know everything...I don't... not... yet... but alot of what I say in regards to online marketing and building a business online is dead on accurate,... or you feel I know quite a bit and you are again just being critical as a way of making yourself feel better.

My tact....wish it was better ......we all have flaws.

I can offer you some advice which you may or may not find constructive but I have found works well.

If you do not like what I have to say...do not read what I post!!!

Have a better day

Peace

 

PS Greg

In your desperation to market... think you could stop blinding people with your rude red font glaring link message which I am sure the Google employees who surf this board dont see...

cough