account performance and optimization

Google Confidential and Proprietary

AdWords API Workshops 2013

Account Performance & OptimizationFinding the needle in your haystack


Overview1. How to Optimize

○ The Optimization Cycle

2. Developing a solid reporting backend○ Reporting Workflow○ Strategies for scaling, processing and storage○ Introduction to sample applications

3. Keyword Optimizations○ What are TIS and TES○ Discovering good ideas with TIS and TES○ Automated keyword optimizations strategy

4. Discovering optimization opportunities with Kratu○ Introduction to Kratu○ AdWords Optimization Report in Kratu○ Kratu backend example


How to Optimize?


Profile

Optimize

Measure

Optimization Cycle


Performance dataStructural data

ServicesAWQL AdHocReports

Profile

Optimize

Measure


Profile

KratuCustom (eg. Epsilon)

Predictive Analysis

Optimize

Measure


AutomaticManualSemi Automatic

Profile

Optimize

Measure


Developing a solid reporting backend


Reporting Workflow

ManagedCustomer

Service

MCC Info

List<ManagedCustomer>

Report Downloader

n-Threads

Report Definition

ReportType

DateRange

Fields

Data StorageMySql

MongoDB...

ReportProcessor


Report Downloader

n-Threads

Data StorageMySql

MongoDB...

ReportProcessor

Process local temp files

Parses rows using a CSVReader

Write calls to Storage must be in batches

Batch size depends on technology/memory

Shard data based on account, subMCCs, dates or common attributes

Keys must guarantee Uniqueness per ReportRow(Date/Day, Segments)

Rows Updates are frequent

Create indexes for:● AccountId● Date/Day● CampaignId● Segments...

Based onMultipleClientReportDownloaderexample

Each Thread runs a ReportDownloder with:

● AdWordsSession● ClientCustomerId● ReportDefinition● CSV & GZip


Implementing a MySql ReportProcessorconnection.setAutoCommit( false);statement.execute(" SET UNIQUE_CHECKS=0; ALTER TABLE ... DISABLE KEYS;");

for (File file : localFiles) { preparedStatement = "INSERT INTO .. (.) VALUES (.) ON DUPLICATE KEY UPDATE (.)";

CSVReader csvReader = new CSVReader(new InputStreamReader (new FileInputStream(

file.getAbsolutePath() + ".gunzip"), " UTF-8"), ',', '\"', 1);

reportRows = new ModifiedCsvToBean <Report>().parseReport(mapping, csvReader);

for (Report reportItem : reportRows) { addInsertToBatch(reportItem, preparedStatement);

// Executing the Batch every 500 inserts to reduce memory usage if (batchSize++ >= 500) {

preparedStatement.executeBatch(); connection.commit(); batchSize = 0; }

}...}...statement.execute(" SET UNIQUE_CHECKS=1; ALTER TABLE ... ENABLE KEYS;");


Implementing a MongoDB ReportProcessor

for (File file : localFiles) {

CSVReader csvReader = new CSVReader(new InputStreamReader(new FileInputStream(file),"UTF-8"), ',', '\"', 1);

List<Report> reportObjectList =ModifiedCsvToBean<Report>().parseReport(mapping, csvReader);

csvReader.close();

// Save in MongoDB (there is also an AppEngine implementation) // Iterates using Google Gson: gsonBuilder.create().toJson(Object) // and DBObject: com.mongodb.util.JSON.parse(jsonObject) noSqlStorage.save(reportObjectList);}


KeywordOptimizations


Targeting Idea Service

SearchParameter[ ]

TargetingIdeaSelectorKeyword / Placement

Ideas / Stats

AttributeType[ ] (fields)

TargetingIdea[ ]Type_AttributeMapEntry[ ]

(field,value)

TIS


TIS - Search Parameters & Fields

SearchParameter[ ] AttributeType[ ] (fields)

● AdSpecList● CategoryProductsAndServices● Competition● ExcludedKeyword● IdeaTextFilter● IncludeAdultContent● Language● Location● Network NEW● PlacementType● RelatedToQuery● RelatedToUrl● SearchVolume● SeedAdGroupId

● Average Cpc● Average Targeted Monthly Searches● Category Products and Services● Competition● Criterion● Extracted from Webpage● Global Monthly Searches● Idea Type● Keyword Category● Keyword Text● Negative Keywords● Search Share● Search Volume● Targeted Monthly Searches

...


TIS EXAMPLELanguageSearchParameter = englishRelatedToUrlSearchParameter = http://www.someplumbers.com/LocationSearchParameter = uk

RequestType.IDEAS, IdeaType.KEYWORD

{AttributeType.KEYWORD_TEXT, AttributeType.SEARCH_VOLUME,AttributeType.AVERAGE_CPC ,AttributeType.COMPETITION}

Targeting Idea Service

KEYWORD_TEXT searchVol avgCPC Competition'london plumbing service' 16 5.13 0.858'london plumbing services' 46 9.41 0.765'emergency plumbing service' 28 6.71 0.912 ... new


TES

criterionId, Min & Max StatsEstimates

campaignId

Traffic Estimator Service

CampaignEstimateRequest[ ]

TrafficEstimatorSelector

CampaignEstimate[ ]AdGroupEstimate[ ]

AdGroupEstimateRequest[ ]

KeywordEstimateRequest[ ]

KeywordEstimate[ ]

Criterion[ ], campaignId

adGroupId, maxCPC

keyword, maxCpc, isNegative

adGroupId

averageCpc, averagePosition, clicksPerDay, impressionsPerDay, clickThroughRate, totalCost


campaignEstimateRequest.setCriteria(new Criterion[] { uk, english });

adGroupEstimateRequest.setMaxCpc(new Money(null, 1000000)); // 1€

adGroupEstimateRequest.setKeywordEstimateRequests( ...'london plumbing service''emergency plumbing service london''emergency plumbing services london'... )...

Keyword avgCPC avgPosition dayClicks totalCost CTR dayImpressions

'london plumbing service', 0.87, 2.00, 0.02, 0.02, 0.04, 0.58'plumbing services london', 0.71, 8.56, 0.54, 0.39, 0.01, 49.33 ...

new new

Traffic Estimator Service


Example Optimization - Epsilon Greedy Inspiredhttp://en.wikipedia.org/wiki/Multi-armed_bandit

windshield repair

vehicle glass repair

car window repair

fix car glass

auto glass repair

car glass replacement

car glass

glass window

glasses repair

window repair

KW ResearchUsing TIS+TES

Automatic● Regular● Set and forget● Needs close monitoring

Semi-automatic● Manual lookup● Suggest KWs for exchange● KW Research can be

manual

KW


Discovering optimization opportunities with Kratu


Kratu - Open Source Issue and Opportunity Discovery


Kratu - Open Source Issue and Opportunity Discovery

● Open Source, Google driven○ http://google.github.com/kratu/

● Client-side, runs in the browser. Implemented in JavaScript

● Includes prebuilt working example for AdWords○ Starting point created by experienced Google Account Managers○ Used internally at Google to display data for millions of accounts○ Provided as-is. Use with caution. Google accepts no liabilities.

● BYOD - Bring Your Own Data○ Anything that can be converted to a JS Array with key/value objects○ Includes simple JSON and CSV loaders

● We offer a Google built sample backend

● Easily integratable with other, non-AdWords data


Demo Timehttp://google.github.com/kratu/examples/adwordshealthcheck/


Kratu - Implementation - Overview

Web server

Reporting backend

FetchProcess

Store

/kratu/

/rest/accounts/

Data StorageMySql

MongoDB...


Kratu - Implementation

● Easy to add other signals○ Add any key/value pair to your data, define a weight and a threshold

● Flexible model○ Custom and built-in calculation of opportunity/issue○ Compose new signals from multiple data points○ Custom and built-in event handlers (integrate with other systems)○ Custom and built-in formatting

● Paginations for larger number of accounts

● Re-use data and signals to produce new reports○ Targeted reports, eg. upselling opportunities, mobile adoption○ Personalized reports, eg. individual for Account managers

● End-to-end tutorial available here:○ http://google.github.com/kratu/tutorial/


Quick example using Restlet and Gson:router.attach("/accountreports/{accountId}", ReportRest.class);

String accountId = (String) getRequestAttributes().get("accountId");

@Getpublic JsonRepresentation getAccountPerformanceReportHandler() {

// Gets Reports from MongoDB or MySql as Java Objects List<Report> listAccountReports = Report.getReportByAccountIdMonth( accountId, date, "AccountReport");

String resultJsonList = gson.toJson(listAccountReports);

JsonRepresentation jsonRepresentation = new JsonRepresentation(resultJsonList); jsonRepresentation.setMediaType(MediaType.APPLICATION_JSON); return jsonRepresentation;}

Kratu - backend example


Json response example:[{

"lostISBudget": 0,"lostISRank": 90,"key": "7767761884-Search Network" ,"mccId": 2742928629,"timestamp": "Mar 6, 2013 8:51:08 AM" ,"accountId": 7767761884,"accountDescriptiveName": "Julian 1$-day","cost": 0,"clicks": 0,"impressions": 0,"conversions": 0,"ctr": 0,"avgCpm": 0,"avgCpc": 0,"avgPosition": 0,"currencyCode": "USD","adNetwork": "Search Network"

}]

Kratu - backend example

account performance and optimization

Technology

keys google confidential

value google confidential

haystack google confidential

totalcostgoogle confidential

newgoogle confidential

google gson

ideaservice keyword

csvreader csvreader