Why isn't The Oil Drum indexed by Google? UPDATE: We are!
Update [2005-9-30 10:20:36 by Super G]: Progress has been made! Now a search of site:theoildrum.com brings us up. We are also indexed by Google Blog search. If someone out there helped us out, thanks!
My website, www.theoildrum.com , has been live since August 20 but it does not come up in Google searches.
We have plently of incoming links. According to our logs, the site gets crawled by Googlebot daily. Further, I am participating in the Google Sitemaps program.
I do have a robots.txt file to exclude certain admin pages from being crawled, but I believe it stills permit the crawling of all of the content pages, because those content pages are getting hits from Google's bot.
Here is the contents of my robots.txt file:
This was their reply:
Thank you for your note. We understand your concern about your site's not appearing in our index. Your page has been blocked from our index because it does not meet the quality standards necessary to assign accurate PageRank. We cannot comment on the individual reasons your page was removed. However, certain actions such as cloaking, writing text in such a way that it can be seen by search engines but not by users, or setting up pages/links with the sole purpose of fooling search engines may result in permanent removal from our index. Please read our Webmaster Guidelines at http://www.google.com/webmasters/guidelines.html for more information.
The Google Team
So I came back with:
To whom it may concern:
I assure you that our site complies with Google's webmaster guidelines. We do not use any SEO techniques. I invite you to verify this by inspecting our website at http://www.theoildrum.com .
Our site is a weblog that uses the Scoop platform. Several major websites, such as Kuro5hin and DailyKos, use this platform. They have no trouble being indexed by Google.
We lauched our domain in August, and redirected traffic from our old webpage on Blogger at that time. We, like many sites, participate in a blogroll provided by Blogrolling.com. This program provides ~50 inbound links. An acquaintance warned me that an influx of links to a new domain could raise a red flag with Google, but I saw no mention of this policy in the Google documentation. Is this a possibility? If so, what can be done to rectify the situation?
I look forward to your response. Thank you.
Their response this time:
Thank you for your reply. We appreciate your efforts to comply with Google's quality standards. We have sent your request on to our engineering team who will review your site to determine if it is eligible for re-inclusion. If the website is reinstated, it will appear in Google's search results sometime in the next few months. We appreciate your patience during this process.
The Google Team
So, they say we will appear in Google's search results sometime in the next few months! I think we can do better than that. Are there any Google insiders among our readers that can help us out?