tales of an seo detective by christine smith

#SMX #XXa @SpeakerName #SMX #21A Christine Smith @websmithc

Christine Smith IBM Search Marketing Technical Lead

TALES OF AN SEO DETECTIVE


Case #1: Thousands of Missing Pages

Why were thousands of support pages

missing from Google SERPs?


§ >1M technical documents

§ Self-support for server and software admins

§ Searchable by error codes, etc.

IBM’s self-support site

http://www-01.ibm.com/support/docview.wss?uid=swg21363866


§ Normal traffic § 1.77M Google referrals per month

November 2013

December 2013 § Traffic dropping § 1.35M Google referrals per month

§  Down 24% in one month!


Google monthly referrals dropped 28% by April 2014


Thousands of pages missing from SERPs!


§ 30 example URLs missing from SERPs § The following were ok: ü Pages displayed correctly ü Redirects were working normally ü Canonical URLs were correct ü Robots.txt was not blocking anything

Assigned to the “case” in April 2014


§ Only 10% of Sitemap URLs were indexed § Regenerated sitemaps

ü Corrected URLs to match canonical URLs ü Improved to 60% of Sitemap URLs indexed

ü  Today, 88% are indexed

§ But, still no Google referral improvement

Sitemaps?


§ Opened Google Site Search support ticket § Google findings: § A sampling of the 30 URLs were:

1.  Marked as a duplicates of the Support Registration page and

2.  Last crawled Nov 2013

Engaged Google: April-May 2014


§ According to Google engineers: §  No way to get a list of all URLs marked as

duplicates of the support registration page

§ Panda 4.0 and Data Refresh – May 19

May 2014 – Google


§ Submitted a Manual Actions Reconsideration Request through GWT §  Some technical docs were incorrectly flagged

§ Requested increased crawl rate for the domain §  Hoping the pages would be revisited

May 2014 – IBM actions


Google referrals increased 22% May to June 2014


§ Traffic bounced back.. Almost overnight. § GSS ticket was closed, but not resolved § Was it.. Ø Increase crawl rate? Ø Better sitemaps? Ø Panda and data refresh? Ø Normal re-crawl of the pages?

What was the fix?


§ Likely culprit: §  Faulty redirect or §  A bad site maintenance redirect

§ Typical “outage” responses (not recommended): §  500 or 504 HTTP response §  Or 302 redirect to a maintenance page

Postmortem: What was the cause?


§ Right way to indicate site maintenance: ü Give a 503 Service Unavailable HTTP

response! ü Retry-after time in the header helpful §  Tells Google to come back later.

§ Do not set all 5xx responses to 503 §  Google will ignore and assume site is down

Lesson Learned


§ Some web platforms automatically give 503 HTTP response during an upgrade §  e.g. Wordpress

§ Others require workarounds, like §  Apache, IHS (IBM), IIS (Microsoft) rewrite rules §  Akamai logic §  Details in backup slides for server admins!

How to indicate site maintenance


Case #2: Missing

Link Credit

Why was there no internal link credit from my landing page?


§ http://www.ibm.com/smarterplanet/us/en/

§ Started a new “card” system

§ But, the links on the cards gave no internal link credit

No credit for landing page links?


§ Robots.txt disallowed the JavaScript directory that rendered the cards

ü Once unblocked, Google gave internal link credit

§ Caution: Some search engines (e.g. Baidu and Yandex) do not support JS content!

The culprit?


Case #3: Mistaken

URL Identity Why were my articles not

showing up in search?


§ Smarter Risk Journal site moved existing content to a new dynamic platform

§ Articles were recreated and republished §  https://www.ibm.com/web/portal/iris/smarter-risk/

Smarter-Risk-Journal

Smarter Risk Journal moved


§ Problem: § The logic that created the canonical

URLs was flawed: §  Same URL as landing page

§ Effectively indicated that all articles were duplicates of the Smarter Risk Journal landing page… and deindexed by Google.

Journal articles were missing from SERPs


§ The Fix: ü Correct the logic of the canonical URL

code to match the specific page URL ü  Take special care with encoded

characters §  Test canonical URLs by pasting into a browser §  Encoded characters could give errors §  e.g. replace a “%20” with a “+” for spaces

Journal articles were missing from SERPs


Advice


ü Always double check ü  Canonical URLs ü  Robots.txt ü  Redirects

ü Verify sitemaps ü Use 503 Service Unavailable HTTP

responses during site maintenance

Lessons Learned


THANK YOU!

Join us @SMX East NEW YORK CITY

Sept. 29th – Oct. 1st 2015


Configuring 503 HTTP response

Backup slides


§  Apache: §  https://gist.github.com/jjulian/1889874

§  IBM HTTP Server (IHS): §  http://www.ibm.com/support/docview.wss?

uid=swg21397422 §  Microsoft IIS:

§  http://serverfault.com/questions/483145/how-to-add-a-site-wide-downtime-error-message-in-iis-with-a-custom-503-error-co

Web Server Rewrite Rules for 503 Response


Akamai flow for 503 HTTP response

tales of an seo detective by christine smith

Internet