We’re finally nearing the end of our series about Google’s Webmaster Tools. We’ve covered almost every aspect of the service, everything from diagnostics to statistics, links to sitemaps. All that’s left to examine are a few handy functions buried within the Tools Section. We’ll discuss a few of theses items today, and the rest in a post later this week. Let’s get going.

Analyze Robots.txt

I mentioned the robots.txt file before, but only in passing. Today, I’ll provide a more complete explanation.

The robots.txt file is a text file that tells the Google robot, when it crawls your web site and adds pages to the Google index, which files and directories are open and available and which aren’t. And it’s not just for the Google robot; all search engine crawlers should read and follow its directives.

While robot.txt files can certainly be complicated, their basic format is very simple. Here’s an example of an exceedingly simple robot.txt file:

User-agent: *
Disallow: /family/

Suppose you have a directory on your web site that contains some pages for and about your family that you don’t want others to find via a random Google search. Well, the above robots.txt file would tell all robots to stay out of that family directory.

The Analyze Robots.txt function you test your robots.txt file, so you can be sure it’s telling Google exactly what you want it to. Simply put a URL in the box at the bottom of the page, then click the button. Note: You need to input in the complete URL if you want to check a blocked page.

Set Geographic Target

Did you know that you can actually target your site geographically, and that if you do, Google will provide the code (more…)

In Part 3 of our series about Google’s Webmaster Tools, we took a close look at the Statistics Section. This time, we’ll discuss Links and Sitemaps Sections.

The Links Section

Pages with External Links. The pages listed here are those that have been linked to by others. Once again, this is important information that affects each page’s PageRank. There’s a lot of detail here because you don’t just see the links that lead to your home page, you see all the individual pages that are linked to from the outside. You can also view the outside pages if you click on the External Links number. You can check to see if they are real links, or just links from your test server to your real server.

Pages with Internal Links. The pages on your site that link to other pages on your site are listed here. Honestly, I’m not sure why this is useful information. I have a feeling Google provides these statistics simply because they’re available. If anyone has a better explanation, I’d like to hear it.

Sitelinks. You may have noticed that when you search with Google, the primary results often come with their own set of links. These are site links, and they’re created by Google when it indexes your site.

Let’s say you’re searching for information about Celiac Disease. Your initial search may return the home page of a site about the disease, but the info you really want is buried deeper within that site. (more…)

Last week, we discussed the Diagnostics section of Google’s webmaster toolset. This week, it’s Statistics, where you’ll find lots of information about how your site is indexed and how it appears in Google’s search results.

The various categories and sub-categories of data are discussed below.

Top Search Queries

You can get some of this information no matter what package you use for web statistics, but there are are a few special tidbits available via Google’s tools.

You can see the top 20 searches that returned your site and compare those to the top 20 searches where the user actually clicked through to one of your pages. Are there differences? You can also see how your site ranked on those searches. If you’re seeing numbers higher than 10, you may want to spend some time tuning your site to those keywords.

You can also sort searches by time period, type (what engine was used - Google Directory, Google News, Google Images or good old fashioned Google Web Search), and location (are the search terms different in the US and Canada?). It can be helpful to see how many people found you through the Google Directory as opposed to the main search page.

Unfortunately, none of this tells you what terms return sites similar to yours but not your own.

What the Googlebot Sees

Getting into Google is all about how the Googlebot indexes your site. This page provides some insight that process.

Phrases: The Googlebot looks at the phrases associated with links to your site. This can affect your search results because Google assumes that the links will correctly define the content of your site. Unfortunately, you don’t have much control in this situation, since the links come from outside sites.

Keywords: The Googlebot also looks at the text on the pages, and you can see what search terms will produce high-ranking results. You have some control over these keywords, so make sure the ones you want associated with your site appear in the list. For example, (more…)

So, after reading Part 1 of this series, you signed up for Google’s webmaster tools and added and verified some of your sites. Now what are you going to do? Well, why not start using the tools to understand and improve your Google page rank, and squash some bugs in the process?

Here’s what’s available via Google’s webmaster tools:

Overview –This is Google’s summary view of your site.

Diagnostics – These are the errors the Google robot discovered while crawling the site.

Statistics – This is how your site appears in the Google index, and it tells you which search terms will return your site.

Links – You can see the pages that link to yours and the sitelinks generate by Google.

Sitemaps – A sitemap is a list of all the pages on your site, and it’s useful if you have a deep site that cannot be easily indexed by Google’s robot.

Tools - You can use these to target your site geographically. You can also determine the crawl rate of your site and do other good things.

Using the Tools: The Overview and Diagnostics Sections

Here you can see when the home page was last crawled and check the site’s status in Google. This is helpful when the client asks, “Is my site in Google?”. You can also find out what problems Google has when crawling the site; this is the meat of the Diagnostics section.

The Google robot crawls every link on your site and will let you know if it encounters any problems. The most common error messages and their meanings are discussed below.

HTTP Errors - Have you ever tried to visit a web page and instead received a message about an HTTP error? Well, that’s the outcome the Google robot encountered here. You’ll see (more…)

So, you’ve created a site for a client (or yourself), and the site is live on the internet. What do you do now? Well, if you want people to find your site, you submit it to Google. Then, you wait. Wouldn’t it be nice if there was a way to find out if your site was in Google and how it was showing up (other than running google searches obsessively)?

There is a way: Google’s webmaster tools. I learned about them when a client was having problems with Google; her site had been blocked for spreading malware (a long story for another post). While researching that problem, I discovered Google’s webmaster tools, a free service from Google that lets you track how Google sees and ranks your site.

In this post, I’ll show you how to sign up for Google’s webmaster tools and how to add sites to your dashboard. In future posts, we’ll discuss what you can do with the various tools.

Sign Up

To sign up for Google’s webmaster tools, go to this page, and log in using your Google Account name and password.

Add a Site

When you log in, you’ll see your dashboard. Once you’ve added sites, they’ll all be listed here. Let’s add a site now:

Type a url into the text box, and click the Add Site button. You should get some interesting information right away. (more…)

In Part 1 of this post, I extolled the virtues of Google Analytics and walked you through the set-up process. Today, I’ll show you how to view some simple statistics.

Viewing Your Data

Since you’ve already copied Google’s code to your pages, you’re ready to start tracking your sites. Go to the Google Analytics home page, and make sure that it’s receiving data from all your sites. That way you’ll know that Google Analytics is getting the information it needs.

Once the data is flowing, view the reports for any site, and you’ll see the relevant data. Google doesn’t just dump a bunch of numbers into a web page. Instead, it presents the information in graphs and even maps, so it’s very easy to understand.

The default dashboard is a great place to start. It graphs your traffic over time and tells you how long visitors are staying on your site. The default dashboard also shows you where your visitors are coming from, and what pages are most popular.

Google has a great help center, so if you have a question (for example, what is a bounce rate?), go ahead and click on the Help link in the upper right-hand corner. (more…)

Google is a well-known and well-respected search engine. So much so in fact, that its name is synonymous with web search. But that’s not all that Google does. The company also offers a host of free services that every webmaster and web designer should know about.

One of the best of these offerings is Google Analytics, a free service that both records visitor data and helps you make sense of the information. Collectively, these processes are called web analytics.

But what do that really mean? Basically, it’s about analyzing where your users go and what they do on your website. It’s essentially the marketing side of web design. Many books on the subject will dive right into issues such as how to set up campaign specific landing pages or conduct a-b testing, but even a small site stands to gain from something as simple as the tracking of visitors.

With Google Analytics, (more…)

AJAX, Web 2.0, Ruby on Rails, and a host of other amusing acronyms and phrases are the current buzzwords in web design. Today’s cutting edge sites use these server platforms and programming languages to make web sites jump through hoops for users. What does that mean for Jane Web Designer, who’s serving Jane Business Owner, down in the trenches?

It’s good news actually, because the technology that powers the high-profile dynamic sites (think Amazon.com) is tried and true, and it’s available to everyone with an $8 a month web hosting account.

If coding your own content management system or online shopping cart sounds like a daunting task, I suggest getting your feet wet with PHP includes. We’ll deal with mySQL in a separate post.

Though they’re not the most glamorous bits of code in the world, PHP includes do make life a lot easier. Suppose I have a site all coded and just about ready to launch and the client says, ‘Hey, we’re not ready with the information for page x, can we pull it from the site?’ Do I tear my hair out because I have to change the HTML for the navigation on every page of the site? No. I make one change in one file, upload that changed file, and I’m done.

That file I changed contains just the code for the navigation, and I use PHP to include that code on every page of my site where I want the navigation to go.

So, how can you harness the power of includes? (more…)

Search engine optimization is big business these days. Type SEO into Google and you’ll find hundreds of individuals and organizations promising to place your page near the top of the major indexes, for a fee of course.

Unfortunately, it’s rarely that simple, for each search engine has its own secret formula for generating results. Meta tags, links and who knows what else influence these calculations, so there is just no one right answer. However, at the most basic level, (more…)

I am an engineer by training, a programmer by inclination and a web designer by chance. So, I love organizing information, I can spend hours working on a css or php problem, and I founder when it comes to the colors and the pictures. I’ve read several books about using CSS to make beautiful sites (Zen Garden, transcending css). Wonderful books, but they assume you know how to design the beautiful part of the beautiful sites. What I needed was a book for programmers that explained the basics of designing a good looking site. And, behold, perhaps a month after I thought that, The Principles of Beautiful Web Design by Jason Beaird was published. Talk about timing. Needless to say, I rushed out, bought it, and read it.

Leading You Through the Design Process
The book is organized around the design process of the author, so not only does he tell you how to choose (more…)