There's More Than Google on the Interweb
Up until today I used to use an adhoc method of determining what's going on with my merchandise sites. I would look at my Adwords reports to see how much I was spending. I'd look at my CafePress reports to see how much I was selling, and my server logs to get a general idea of how people were finding my site. Today I completed the first part of a database that I've been working on using Access 2003. Rather than getting a general, and sometimes fuzzy, picture of what's going on, I decided I needed a tool that would give me a dashboard view of what's up. Sort of like the difference between data and information. Before I was looking at data. Now I'm looking at information. Building this, I've found, is great exercise in thinking about what information I really need to see in order to make my sites more profitable.

The first part of this application shows me the terms people searched for, what search engine they used, and whether they clicked on a search result or an ad on a search results page. This information tells me a few things, but most importantly it lets me know what people are looking for when they come to my sites (and if I don't have it, I know I should offer it), and it lets me know whether my search engine optimization (SEO) is working or not. In fact, I learned something already. I thought my SEO efforts hadn't been as successful as I hoped, but that was because I was using Google to see my rankings for different relevant terms. What I learned was that my optimization efforts have worked pretty well on Yahoo and MSN. For many search terms my sites are in the first 5 listings. Search for "texas tshirt" on Yahoo right now and you'll find I'm listed as #1. That rocks!

Hopefully I can get this application to a point which will allow me to spend minimal time on managing the sites, while making the best decisions for the sites. As for the next stage of the application, I need to be able to quickly see how my advertising budget influences sales. Using the view of the data from my various reports, it seemed that it didn't make a difference in sales if I spent $10 a day or $2 a day. But perhaps I was missing a sales trend that lags behind the advertising by a few days. I'll have to think about what information is important to making decisions and avoiding displaying too much information. When its all done I'll post some of the screens (and maybe the application itself for those who could use it for their own sites).
- posted by Drew @ 11:45 PM  

 
Google Adwords - Some Lessons
I've been using Google Adwords for my Texas merchandise sites since the beginning of the year and started using it on my hurricane t-shirt site last week. The ads have been the primary way that customers have found my sites. Without this service, or one like it, I would have to rely on customers finding my sites through search engines or links from other sites, and that wouldn't work so well. I can't name a site that I would even approach to list one of my commercial sites as a link. And my sites haven't always had the highest rankings on search engines, although hopefully some recent changes will fix that. A couple years ago you needed a big multi-thousand dollar advertising budget to advertise online. Adwords, Overture and others now offer advertising on big name sites at a cost of 5 cents and up per click. This is all very good for small sites. Since starting to use these service I've learned a couple lessons and thought I would share them here.

- About 20% of the clicks I pay for are clicks that the advertising systems accidentally counted twice. This sucks, but there's not much I can do about.
- Increasing my advertising budget (on a per day basis) hasn't lead to an increase in sales. I tried $5 a day (at about 10 cents a click), and got the same number of sales as when I set my budget to $2 a day.
- For regionally focused products, like my "I survived hurricane Charley" t-shirts, I set my ads to only display in Florida.
- These advertising services allow your ads to be displayed on search engines as well as on many other type of sites that have content that is relevant to your ads. I've found it best to disable showing of your ads anywhere but on search engines. I only want to display ads to someone searching for "Texas tshirts", rather than someone reading an article about the beers of Texas. I don't think a $18 t-shirt (plus shipping) is going to work as an impulse buy.
- Use a lot of negative keywords. I look through my server logs primarily because it shows me what page someone was on when they clicked on a link to one of my sites. This referring page can often tell me what search terms they were looking for when they found my site (or saw the ad for my site). I use this information to either confirm that my ads are targeting the right people, or to determine that my ads are displaying to someone who's looking for something that I don't have, yet my ad shows on their search. For example, someone searching for "Texas boots" might see my ad on their search results and click on it. Yet I don't sell boots. In this case I would add a negative keyword for "boots". This will prevent my ads from showing when someone includes the word "boots" in their search.
- posted by Drew @ 4:35 AM  

 
Killer Robot Lawyers from Outerspace
I was looking through my server logs and noticed that someone had been going through every page on hurricane-tshirts.com. In fact, they went through the entire site in about 3 minutes. I'll usually see that sort of activity when a search engine is crawling a site, but this was different. Whoever, or whatever, had gone to my site was showing that it was Internet Explorer on some pages, Netscape on some other pages and Safari on others. Also, some of the IP numbers (e.g. 255.255.255.002) were slightly different, but still on the same subnet. So this definitely raised my suspicion about what was going on. At first I thought it might be a spambot. A bot, or robot, is an automated program than goes through an collects information from websites. Some bots are good, and some are bad. Spambots are bad. They go through site and look for email address which the spammers then use to send junk mail. The good bots are catagloging sites for search engines. This obviously wasn't a good bot since it was trying to disguise itself. It also wasn't any spambot that I've seen before, because they at least don't try to disguise themselves as regular web browsers. There's really no need for the spambots to try to cloak what they really are. So after searching the IP numbers I found that the bot that was on my site was from an intellectual property watch service. In essence they crawl through sites and look for copyright and trademark infringements on their clients behalf. In addition, they sometimes provide competitor research for their clients. Well, I don't like this. Some might say that if you're not doing anything wrong then you don't need to worry. I look at it in the same way I look at warrants. Even though I'm not doing anything wrong, I don't want the police to freely search my home. Same goes with these lawbots. I don't need them rummaging through my sites looking for anything they might be able to use to their advantage. So after some more research I've implemented a way to block them from my sites.

Here's how I'm doing it: First I need to find the IP number ranges that this company uses. To do this I performed a WHOIS with the IP numbers in my server log. Then I modified my htacess file on my server. If you're on an Apache webserver you've got this, or you can create one by naming it ".htaccess". This is what I added to the htaccess file:

RewriteEngine on
Options +FollowSymlinks
RewriteBase /

RewriteCond %{REMOTE_ADDR} ^65\.102\.23\. [OR]
RewriteCond %{REMOTE_ADDR} ^65\.102\.12\. [OR]
RewriteCond %{REMOTE_ADDR} ^63\.227\.217\. [OR]
RewriteCond %{REMOTE_ADDR} ^12\.148\.209\.19 [OR]
RewriteCond %{REMOTE_ADDR} ^12\.148\.209\.2
RewriteRule ^.*$ [G]

This sends anyone from the IP subnets of 65.102.23.xxx, 63.227.217.xxx and 12.148.209.190 - 12.148.209.255 to a server page that says the site is gone. I would have preferred to send them to a page with my own customized text, but I couldn't get that to work. Another downside is that I can only prevent lawbots, or anyone else for that matter, when I know their IP addresses. But this will do for now.
- posted by Drew @ 7:37 AM