Oddly enough I've been seeing more and more whacky things with Google and with various hosting companies out there.

Here are some of my favorites:

I implemented a canonical url fix on an apache server using mod_rewrite. In the rewriterule flag I declared the redirect to be a 301 ([R=301]). After checking the headers it redirected but the header returned only a 200 ok... Nice...


Here is a good one from Google:

A clients home page listing is showing up in Google as https://www.somedomain.com/

In Google's Help center http://www.google.com/support/webmasters/bin/answer.py?answer=35302 they recommend you deny the bots from the ssl by using the robots.txt file.

Since some hosts map the non ssl and ssl protocols to the same folder on the server you could literally tell Goolge to deny both....Not good....

Here is my favorite:

I built a site a while back and had 2 forms of navigation. The first for the user, it was a drop down form for ease of use, it would send the user to the correct page and the url was dynamic. The second form of navigation was text links and static urls via mod_rewrite to the exact same pages as the form.

This would allow search engines to completely see everything a user would see. No links went to the dynamic versions. All linking was done to the static mod rewrite version.

After doing a site:www.domain.com for the url I discovered Google had a cache on both dynamic and version.

Since when did Google put together the variables in a form and crawl the urls??? Go figure...

We've seen a recent PageRank up date this week. One thing that people have been noticing is that some of the home page PageRank is lower that their internal pages.

There is a discussion underway here at search engine watch on the topic. View Thread >>>

I personally brought up 3 points that could be the reason. They are as follows:

• Advertising links not using the rel="nofollow" tag, and being off topic
• Better Link Popularity on the internal pages
• Just another one of those wacky glitches

What ever it is, it hasn't seemed to be messing with rankings on any of my personal sites. Webforgers.net being one of them.

In the course of doing business it is a given that you are going to piss some people off. Hey we've all had that one client that you just can't satisfy, right? Consumers, now more than ever, have access to blogs, forums and other means to complain about services rendered.

With that being said, if you do a search for your company name and you see consumer complaints showing up, then this is the blog post for you.

Here are some Do's and Don't when it comes to repairing your brand.

Don't try and defend your brand by going in and declaring that you represent the company. participate in any forums or blogs where they are hammering your brand. Search engines love content and adding to the thread or comments is like adding wood to the fire. You'll end up cooking your own goose and making it a lot harder to repair the damage.

I was approached by a company that had participated in a consumer complaint forum. Not only was there 10 pages in the thread all about the said company but their user name when posting was their company name. Let's put it like this, the forum ranks higher than their company home page.

Don't try and sneek in a few posts that defend the company. You're not fooling anyone when you do that. You can pretty much gaurantee another page of flammage if you do this.

Don't start a thread and link to any bad publicity asking for advice on how to counter it. Raising awarness may fuel the fire as well. Forums are open and word spreads fast. You may not link to it, but a reader looking for a topic to rant about may not be so shy to drop a link or two to the complaint.

So now we know a few of the most important Don'ts lets move on to the Do's.

Do get your name out there. Submit as many good press releases as you can. Open the flood gates on your company in a positive way. Mediawire & PRweb will outrank the complaints since they have tremendous page scores that they pass to their press releases.

Do take the time to internally optimize your site. You should beef up the about us sections in your site (if you have one). Make your company name promenent in the title tag, and use it conservatively in the content. Add a link into the footer of your site using your company name as the link text. That link should go to the optimized page about your company.

Hey there are 10 rankings on page 1 so lets keep going...

Do write testimonials en masse! If you have any business to business partners write the best testimonial and get your business partners to put them up on their sites. You can even add some extra power by linking from your site to the testimonial your wrote for you business friends! Make sure you put your company name in your signature and in the testimonial itself.

Ok so it's all done, plan on it taking a few months, the more you aggressively push your company name, the more you keep the complaints from even reaching that first page!

Now if you are rendering crappy service then uhh April fools I switched the do's and don'ts around.

Cedric Beust, an engineer at Google, said: "I think all the pieces are there on the server side. But I think we'll get a whole new ecosystem around AJAX [Asynchronous JavaScript and XML]. It's like a bad cold. More and more people are coming to this thing and we have a whole new excitement for Web sites we never had before, with things like mashups." Source >>

Like flash however AJAX is invisible to the search engine's eyes. MSN's Live uses AJAX to display search results. GMAIL uses it to give you real time email updates while you are logged into the system.

Hopefully Blogs and CMS developers won't use AJAX as a way to feed content to their pages! They'll look cool but they won't rank for squat!

Thursday afternoon the Zunch crew gathered around in the conference room, beamed Eric Ward's Linking Webinar up on the Screen, dimmed the lights and popped open the soda. An hour and a half later we shut down and chatted about what we knew already, what we didn't know and what if anything was wrong.

Let me state at first that we thought this would be an advanced linking strategy webinar and that it was marketed to other firms/professionals in the SEO/SEM industry as such.

The webinar covered all the basics and gave useful information for those that are just starting out and trying to understand what linking and link popularity is all about. For the seasoned SEO however you may pick up a one or two things there that you may not of seen in the popular SEO forums.

Here are the things I found helpful:

• Sources for free one way links from authoritative sites
• Email subject and body copy suggestions that keep you out of the deleted folder
  
• Link Software Screen shots and personal analysis
  
• How to use blogs to further empower your core site's links

Here are a few things that I thought that may have been confusing to the attendees:

• It was stated that buying links could penalize your site, then later in the webinar strategy for buying links was discussed. Eric stated that he had clients that he had bought links for and no clarification was giving between buying links & and how they may penalize a site so it sounded a like a contradiction.
• Someone asked what an RSS feed was. I thought the answer was drawn out and confusing and probably should have been addressed after the Webinar in the Q&A time.

Here are some things that I know were stated and are actually wrong:

• It was stated that Spiders (specifically Googlebot) will not crawl deep folder structures. Therefore don't waist time getting links to a file burried in your site. His analogy was if the page is in a hundred foot deep lake and the link is only 50 feet of rope then you can't reach the page. This is wrong. IF the PageRank of the page that is linking has high enough PageRank to justify a crawl to a file that is being linked to then the spider WILL index the page and it CAN be reached.
• It was stated that "Undiscovered Links" or pages behind forms are unreachable by Spiders. In Googles case this is wrong. I recently built a site that had a simple 1 variable form containing 50 values. Google used the url in the action="" and then for each of the 50 values added the forms name to form a query"?state_id=" to the action url thus completing the url then crawled all 50 variations of the landing page. I posted this over in spider-food.net.
  
  Currently you cannot see the results of this in Google but I think the forum post and the responces to the post indicate proof that Google did infact index those pages and that there were NO links to those variable pages because the mod rewrite implemented in the bottom links were for the bots/users that wanted to click and link to a static version of the form page. The form was there for convenience/usability and was NOT suppost to be crawled YET it was.

I've tried to give a fair evaluation for both new and experienced webmasters. If you are just getting into links I recommend this seminar. If you are experienced with linking the price of the webinar for the little tidbits of new information may not be worth your time.

Eric has had a lot of success with his current clients I think he did a good job explaining linking and showed great examples of success and how powerful links can be to an SEO campaign. He is one that thinks outside of the box and has done his linking home work I would definitely heed any free advice you may find on his website.

Eric Ward's Website >>

While going through ranking reports today I noticed something that was just flat out wrong! I manage a client that has had previous SEO done to their site. As I went through the site I noticed some old spam in the page. About half the coding on the home page was spam.

I instructed the client to move it immediately! In the last two bi-monthly ranking reports I've noticed that the Google Rankings have steadily increased... Now here is the part that is just flat out wrong the MSN rankings have started to decrease...

Now I know that MSN backlinks are easily gamed, anyone can plop a site wide link up for a term and in 2 weeks start to rank for the link text. Now I'm starting to see that the on page content is very prone to spam as well.

Here's a good example:

A search for "Dallas Movers" pulls up a good example of spam in both search.msn.com and live.com

The site h2movers is about as bad as it gets... MSN has a very long way to go if they want to beat out Google. Google is what it is because they've held the #1 search engine position for so long. They have actively had people try and game their system because of that #1 spot. Everytime someone catches on to a loop hole in the Google system, the Google engineers have plugged it.

In conclussion, I can't put the spam back nor will I for the client. A drop in MSN rankings is a loss I'm willing to take for a gain in Google. As per MSN you've got a long way to go before you can over take anyone, I'm sure all the spammers out there are rooting for ya :)

To add to Google Earth and Google Moon, Google has launched the Google Mars page to show you a detailed view of the Marsian surface.

Google Mars >>

I see it everyday. Someone proposing a url structure in the forums and asking for advice on whether it'll get crawled or if there are any visual flaws. So let this be an end-all post on the subject.

Out of all the experimenting with urls, I've found that
.com/ to
.com/folder/file.htm
works the best at being crawled regulary and deeply. PERIOD.

Now let me go over the problems with the other structures that I see everyday as I cruise the forums.

The Directory Structure & PR Dither Rewrite
This is like the Camero Mullet (see 2nd image) of mod_rewrites, olds school and looked cool back in the day.

.com/ (PR5)
/folder/file.hml (PR4)
/folder/folder/file.html (PR3)
/folder/folder/folder/file.html (PR0)

In the directory tree structure the root links to the first folder, the file in the first folder links to the file in the 2nd folder and so on. Once you get to 3 folders deep you lose your PR in most cases. This is because a puny PR 3 isn't strong enought to warrant a crawl that deep into a site. You could even see this in some of the deeper sections of the Yahoo! directory. The further you went the lower the PR and some sections were just so deep off the root they didn't even warrant a cache.

Of course the solution to the above would be to link to every page in the site on every teir or use the high PR from a root sitemap to feed spiders deeper. This also dissolves if you have a large site. A good example would be a country: USA (1 page) >> State (50) >> County (~3250) >> City (17,500*)

Obviously no single page could hold 20,000+ links and be crawled. Plus browsers would strain to render that coding. Then deciphering all the navigation. Its just not logical.

Junk Rewrites
Like The Tron Guy, It Should be avoided at all costs.
Nice Moose Knuckle by the way Jay!

These are the cases where the developer stuffs every variable thats not needed into the url. So the effect is a nonsensical jump from Root to a file 3-5 folders deep.

.com/ (PR 5) | (PR 3)
/folder/folder/folder/folder/folder/file.html (PR 2-3) | (PR 0)

The problem is that a site has to gain a significant amount of PR on the home page just to push the spiders into the rest of the site. This is why there are many complaints when a developer that has switched to mod rewrite static urls and complains, "I can't get my new urls to get indexed". You could be waiting months or years depending on how fast you can get inbound links.

It's not that they can't get indexed it just that your home/root pages aren't powerful enought to warrant a deep crawl. I see this alot with shopping cart/cms add-ons for mambo & oscommerce. For windows servers I would suggest using ISAPI Rewrite. ISAPI Rewrite gives you the same functionality and control as the mod_rewrite application for Apache.

The Tried and True Solution

Short and simple and 1 step away from the root at all times. I've come to this because I did all of the above and learned the hard way.

For example the site ~www.sbdpro.com was patterned after the Yahoo! Directory. It has consitantly for the last 2 years had a PR 4 home page. But with that structure it could never get the spiders deeper than 2 categories or 2 folders deep. Since the deepest depths in that directory is 8 tiers down there really was no solution or point of keeping this url structure.

It was changed about a year ago. All links from the root go to /directory/file-name.htm It didn't matter how many tiers down you went, all categories were now 1 tier away from the root, all subcategories were one tier from the root. The crosslinking all stayed the site had no index problems as all.

The effect was astonishing the PR still drops out in the 3rd tier with a PR 2 yet the spiders still followed the links through 5 more tiers of crosslinking & PR 0 pages to reach the 8th tier down.

My advice to all new mod rewrites.

1. Don't get married to your first try at mod rewritten urls. Change them because you're in it for the long run (hopefully). The above site ranked for "small business directory" page 1 consistantly during the change in Yahoo and MSN. It even jumped to page 2 in Google for that term as well and did a stretch for a while.

2. Keep your urls simple and close to the root. You will see more spider activity and have less headaches.

3. Make sure you rewriterule syntax is optimal and you are not bogging down your server. See this thread here >>

Other Resources

Webforgers.net - Mod Rewrite Tutorials
Ilovejackdaniels.com - Mod Rewrite Cheat Sheets

~ = 3rd party database of counties I bought.
* = From spidering Yahoo directory for city names under the state sections.

Funny when you try and sign up for a service and you have to jump through a few technical hoops just to get it done. Like when Tyson (SEO Specialist) this morning was trying to create an MSN Adcenter account using IE 7. Adcenter kept spitting out an incompatible web browser error.

Looks like you have to revert back to IE 6 if you want to get anything done in MSN Adcenter. I would suggest not to switch your user agent header to IE 6 using the FF UA switcher either, the styles on all the forms break and you get a lot of form layer overlap issues.

Short list of browsers that Adcenter doesn't support:

• Anything != IE 6

I thought the days of "you must conform" were past us...

I think the best way to describe this service is Froogle combined with Yahoo Store & Paypal. It's not opened to the public yet, but there are some vendors beta testing.

We're starting with a very small number of sellers and we expect to include more over the next several months. If you're a seller and you're interested in getting an announcement when this feature is generally available, let us know. And if you want to know how this functionality relates to Google's broader work in payments, read this update. We hope this feature will make it even easier for people to use Google Base to post and distribute a wide range of content, whether information for sharing or goods for sale.

You can read the official blog post here.

Google SiteMaps Update

(1) Your average position by your top search queries. This is basically a site ranking report for the your top keywords, the keywords people search to find you on.

(2) If you have mobile enabled site, you will also now see your top search queries on mobile devices.

(3) You can now download "details, stats, and errors" to a SV file that you can then do what you like with it.

More details at Google Sitemaps Blog. Source SEW Blog >>

Barry Schwartz posts an update about the Google Sitemaps program over on the SEW Blog.

Webmasters should note that this service is free and easy to use. Support on the other hand is still the well known Google Blackhole. If you can find the email address to be able to send a question don't be surprised if you get anything back :)

In Michael Martinez's latest post over at Spider-food.net he runs over a few factors related to link theory. As Big Daddy pours out and rankings shift around keep these factors in mind.

• Document Inception Date (see sections 0034-0044)
• Content Updates/Changes (see sections 0046-0056)
• Query analysis (see sections 0058-0065)
• Link-based Criteria (see sections 0067-0080)
• Anchor text (see sections 0082-0086)
• Traffic (see sections 0088-0091)
• User behavior (see sections 0093-0095)
• Domain-related information (see sections 0097-0102)
• Ranking history (see sections 0104-0112)
• User maintained/generated data (see sections 0114-0117)
• Unique words, bigrams, phrases in anchor text (see sections 0119-0121)
• Linkage of independent peers (see sections 0123-0125)
• Document topics (see sections 0126-0129)

Source: forums.spider-food.net