You are on page 1of 31

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Reporting Types

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Welcome

Welcome to Report Types! This reference document guides


you through the different areas of Maxymiser Reporting. View
the highlighted areas of the following reports to learn how to
gain insights for your campaigns.

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Table Columns
Click to include additional metrics in the Experiences table
Rank: How the experience ranks compared to other experiences. For example, a
conversion rate of 53% will be ranked higher than a conversion rate of 49%.
Generations: The number of generations (usually visitors) for the campaign.
Elements: The elements being tested (e.g. a landing page) and the list of variants for
the element.

Conversion
Action Count: The number of converted generations for the experience.
Conversion Rate: The percentage of generations (usually visitors) who have
converted out of all the generations who saw the experience.

Revenue per Generation

Multiple Action Count: The number of actions performed for the experience.

Revenue: The monetary value assigned to an action for the given experience.

Revenue per Generation: The revenue divided by generations (usually visitors) for each experience.

Average Order Value

Multiple Action Count: The number of actions performed for the experience.

Revenue: The monetary value assigned to an action for the given experience.

AOV: The revenue divided by the multiple action count (i.e. the total number of all actions by visitors who saw the experience).

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Table Columns - Continued

Revenue per Converter

Multiple Action Count: The number of actions performed for the experience.

Revenue: The monetary value assigned to an action for the given experience.

Revenue per Converter: The revenue per unique visitor for the experience. For example, for purchase actions, Revenue per Converter is
the revenue per visitor who has completed at least one purchase through the campaign.

Engagement

Multiple Action Count: The number of actions performed for the experience.

Revenue: The monetary value assigned to an action for the given experience.

Actions per Generation: The average number of conversions per visitor who enters the campaign. It is calculated as the multiple action
count (i.e. the total number of actions by visitors who saw the experience) divided by generations.
Margin of Error: The standard deviation for the conversion rate (or other metric e.g. Revenue per Converter) of the experience multiplied by a
factor that depends on the confidence level.
Error: The Margin of Error for conversion rate (or other metric, e.g. Revenue per Converter) as a percentage of the Conversion Rate (or Revenue
per Converter, etc). It is calculated as the Margin of Error divided by the Conversion Rate (or Revenue per Converter, etc).
Uplift: The relative difference between the experience and the control. Uplift is calculated using the following formula: Uplift (%) = (Conversion rate
of experience - Conversion rate of control experience) / (Conversion rate of control experience) x 100%.
To calculate uplift for other metrics, replace the value for 'conversion rate' with the value for the metric, e.g. Revenue per Converter.
Confidence Index: Measures the observed test results and is used to test a statistical hypothesis on the equality between an experience and the
control metric, e.g. conversion rates. The confidence index is the probability of obtaining a result equal to or 'less extreme' than what was actually
observed, assuming that the experience and the control have equal metric values, e.g. equal conversion rates.
Illustrations: Show the confidence intervals for Conversion Rate (or other metric, e.g. Revenue per Converter). Green to the right of the control is
a positive uplift and red to the left is a negative uplift. A gray bar means there is an overlap with the control experience. When there is no overlap
with the control, it means the confidence index is greater than the selected confidence level.
Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Campaign Performance Report


The Campaign Performance report shows your campaign's key metrics at a glance to help you understand the
experience that gets you maximum conversion, engagement and revenue. You can use this report with other reports to
make key business decisions for your organization.
Filters and Page Options

Campaign Status

Current duration: The number of days from


the start of the campaign to the current date, or
the date the campaign finished.

e.g. Live, Paused

You can filter the results displayed in the report by


clicking the filter or Page Options buttons of the
report. Page Options allows you to selct the funnel
view.
Note: See Report Filtering video for information on
how to use the filters.

Total generations: The number of generations


(usually visitors) for the campaign.

Total generations: The number of generations


(usually visitors) for the campaign.

Oracle Marketing Cloud Academy

Experiences: The number of experiences in the test. A combination of


variants make up an experience. For example, a welcome page
without an image, medium size banner, and call-to-action button
saying View Our Best Offers.

2015, Oracle and/or its affiliates.

Top 3 Ranked Experiences


Indicator that the
experience has a higher
conversion rate than the
control experience

Indicator that there is no


difference between the
average revenue for the
experience and the control

Experience ID

Default experience
(Control)

How the
experience
ranks, e.g. by
conversion rate

Conversion Rate

Uplift, e.g. in
conversion

Confidence Index

51.54% positive uplift in


revenue

Revenue per generation


Actions per generation

66.48 positive uplift in actions per


generation

Oracle Marketing Cloud Academy

Statistical error for the metric, e.g.,


conversion rate error

2015, Oracle and/or its affiliates.

Top 3 Ranked Experiences

Which Experience are displayed in the funnel

Percentage of conversion
rate for displayed
experiences

Actions that are included in your graph

Widget allows you to control


which actions you want to
display in the graph as well as
selecting the experience you
want to compare

Percentage of drop off


from one step to another

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Report Types
Element Level reports compare each variant to the default. These reports are more indicative of future
behavior, because each test element is exposed to more traffic than each test experience.
Conversion By Elements
Use the Conversion Report to see how many visitors completed the selected
action (single action count).
Revenue By Elements
Use the Revenue Report to see how much revenue is generated based on the selected
action. Please note, this is not limited to revenue; it can also be used to see sales quantity.
Note: A default value of 1 is assigned to each conversion event unless an Action is
configured to capture a particular source of revenue. Multiple Action Count is displayed by
default to capture all revenue generated from the Campaign.

Engagement By Elements
Use the Engagement Report to assess the frequency at which visitors complete
an action (multiple actions).

Metric Breakdown By Elements


Use the Metric Breakdown Report to understand what percentage of total visitors
completed the selected action and the distribution of actions by the action attribute.

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Report Types
Experience level reports compare combinations of variants to the default experience. These reports should supplement the element
level results in an MVT campaign, because each experience is exposed to less traffic than each element. Note: When assessing an
A/B campaign, experience and element reports are the same.

Conversion
Use the Conversion by Experiences Report to evaluate the performance of test
variants in terms of conversion rates based on campaign experiences.

Revenue
Use the Revenue by Experiences Report to evaluate the performance of test
variants in terms of revenue generated from campaign experiences.
Engagement
Use the Engagement by Experiences Report to evaluate the performance of test
variants in terms of visitor engagement with campaign experiences (i.e. visitor
conversion and frequency of conversions). Both single action and multiple action
counts are displayed in this report.
Example: Visitor A clicks a button 3 times; their multiple action count would be 3.
Whereas, in a Conversion Report, in which you look at single actions, Visitor A's
action would count as 1.

Metric Breakdown
Use the Metric Breakdown Report to assess the conversion split between
specific Action Attributes, such as currency of purchases or geographical location
of the conversion events. By default, Single Action Counts are displayed.

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Report Types

MaxPREDICT 2.0
Use the MaxPREDICT Campaign Report to
evaluate data obtained for MaxPREDICT
Campaigns.
Site Reports

Rule Builder

Site Reports show data for MaxRECOMMEND campaigns. The reports


provide the following data:
MaxRECOMMEND Summary
Product Trends
Product Categories
Best Sellers

This feature is available for


MaxSEGMENT, MaxPREDICT and
MaxPREDICT 2.0 accounts only.
Contact Maxymiser for details.

Scheduled Reports
You can schedule reports to deliver aggregated reporting data to your email with a
set time interval (e.g., daily, weekly or monthly). Scheduled Reports can be
configured for any Campaign Report. Within a scheduled report, you can select
filtering criteria, report type, etc., and all these settings are saved for scheduled
generations (cumulative/daily).

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Summary Report
Date Range
Use the Date Range filter to select a different date
range, for example, to select a different date range if
the campaign is modified. By default, the entire date
range of the campaign is displayed.

Metric
A metric is an action a visitor takes that will measure campaign
success. This drop down defaults to the primary metric but can be
altered depending on what data you are looking for.

Campaign Iterations
Use this drop down to specify the iteration for which
you wish to view data. An iteration identifies periods
when changes were published. Please note, if you
select an iteration, you only see the data from that
iteration till the next listed iteration.
Filter By
Use this filter to look at your data by action or generation date. Selecting
Action Date shows you all actions that occurred within the specified time
period, regardless of when the visitor generated into the campaign.
Alternatively, selecting Generation Date only shows actions that occurred from
visitors who generated into the campaign within the specified time period.
Note: This filter should be used when you alter the date range to ensure you
are only looking at generations that happened within the specified range.

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Summary Report
Winning Experience by Conversion Rate
The Winning Experience by Conversion Rate shows details
about the current winning experience (by conversion). The
current winning experience ID is displayed under a winners cup
icon. The combinations of variants that make up this experience
are shown within the right column of this box.

Scheduled & Saved Reports


Lists all Saved and Scheduled
reports for the selected campaign.
See Scheduled Reports for more
information.

Best 3 Experiences by
Conversion Rate
This graph shows the
Conversion Rate for the top
three experiences and the
default experience. For
example, you can clearly
identify if the current winner is
a winner for the whole period
of the campaign by checking
for deviations in the graph.

Oracle Marketing Cloud Academy

View Full Conversion Report


Click to open the detailed Conversion by
Experiences Report.

2015, Oracle and/or its affiliates.

Table Elements
Generations:
The total number of visitors who
have been served that experience

Action Count:
The number of actions
performed

Element:
An identifiable part
of the page or site
template

Variant:
A possible configuration
of that particular element

Oracle Marketing Cloud Academy

Conversion Rate:
A measure of success
for a Content Variant or
experience

Error:
The margin or error as a
percentage of the
conversion rate

Margin or Error:
Indicates the range of potential
values for the conversion rate

Confidence Index:
Probability that there is a
significant difference between the
default and test variant

Uplift:
Percentage change seen in the
test variant against the default

2015, Oracle and/or its affiliates.

Summary Report

Total Visitors
The total number of visitors who generated
into the campaign.
Current Duration
The number of days from the start of the campaign to
the current date or the date the campaign finished.
Total Days Live
The total number of days that the campaign is in a live
state. For example, if a campaign that shows a Current
Duration of 100 days is paused for 10 days, then the
Days Live is shown as 90 days.

Segments Discovered
MaxPREDICT 2.0 Customer Segmentation and Discovery identifies
meaningful segment profiles based on their best performing experiences.
This field shows the number of MaxPREDICT 2.0 segments discovered and
the number of segment profiles. Click Segments Discovered to open the
Guided Process - Review page on which you can view details for all
discovered segments for this campaign.

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Summary Report

Winning Experience by Engagement


Displays details about the current winning experience by
engagement (multiple actions).

Winning Experience by Revenue


Displays details about the current winning
experience by revenue/sales quantity.

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Conversion By Elements Report

Report Filters
You can filter the results
displayed in the report
by clicking the filter bar
at the top of the graph.

Report Graph
You can view any report on a
daily basis (Daily report) or
cumulatively (Cumulative
report). The Daily report remains
very volatile for any period of
time, since changes are recorded
daily, while the Cumulative report
gets smoother with time.

Note: See
Report Filtering video for
information on how to
use the filters.

Alternate between the two views


using the Report type drop down
at the bottom left of the report.

Element
An area of the page which is
tested.

Count
Total number of unique visitors
who completed the selected
action.

Variant
The alternative content
that is tested against the
default.
Generations

Oracle Marketing Cloud Academy

The total number of visitors


who entered the campaign.

2015, Oracle and/or its affiliates.

Conversion By Elements Report

Conv Rate
The Conversion Rate shows the percentage of visitors who completed the selected action. The
range of values shown indicate the Conversion Rate +/-, the MOE (Margin of Error). This is known
as the range of acceptable values. Conversion rate (%) = (number of actions / number of
generations) x 100

Confidence Index
The probability that there is a
significant difference between the
default and test variant.

Example: Suppose 1000 visitors have entered a campaign, and 200 of them have converted (e.g.,
clicked on the content to be optimized), then the conversion rate for that particular content is 20%.

Conv Rate Error

Uplift

The Margin of Error (MOE) as a


percentage of the conversion rate.

The percentage change between the conversion rate


of the new variants and the default conversion rate.

Conv Rate Illustrations


This is an illustration of the range of acceptable values (see Conversion Rate) for each experience.

Oracle Marketing Cloud Academy

When the range of acceptable values for a new variant/experience goes above the default's range of
acceptable values, the bar turns green. When it falls below the default range, the bar turns red.

2015, Oracle and/or its affiliates.

Revenue By Elements Report


Report Filters
You can filter the results
displayed in the report
by clicking the filter bar
at the top of the graph.

Report Graph
You can view any report on a
daily basis (Daily report) or
cumulatively (Cumulative
report). The Daily report remains
very volatile for any period of
time, since changes are recorded
daily, while the Cumulative report
gets smoother with time.

Note: See
Report Filtering video for
information on how to
use the filters.

Alternate between the two views


using the Report type drop down
at the bottom left of the report.
Element
An area of the page which is
tested.

Count
Variant
The alternative content
that is tested against the
default.

Total number of unique visitors who completed


the selected action.
Generations
The total number of visitors
who entered the campaign.

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Revenue By Elements Report

Revenue Per Conversion


The average amount of
revenue generated per visitor
who converted. Revenue per
Conversion = Total Revenue /
Single Actions.

Revenue per Generation Error

Confidence Index

The Margin of Error (MOE) as a percentage of the


Revenue per Generation

The probability that there is a


significant difference between the
default and test variant
Uplift in Revenue per Generation

Revenue per Generation


The average amount of revenue generated per
visitor who entered the campaign. The range of
values shown indicates the Revenue Per
Generation +/- the MOE (Margin of Error) which is
known as the range of acceptable values. Revenue
per Generation = Total Revenue / Generations.

The percentage change between the


conversion rate of the new variants and
the default conversion rate
Revenue per Generation Illustration
This is an illustration of the range of acceptable values (see Revenue Per
Generation) for each experience.
When the range of acceptable values for a new variant/experience goes
above the default's range of acceptable values, the bar turns green. When
it falls below the default range, the bar turns red.

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Engagement by Elements Report


Report Filters
You can filter the results
displayed in the report
by clicking the filter bar
at the top of the graph.

Report Graph
You can view any report on a
daily basis (Daily report) or
cumulatively (Cumulative
report). The Daily report remains
very volatile for any period of
time, since changes are recorded
daily, while the Cumulative report
gets smoother with time.

Note: See
Report Filtering video for
information on how to
use the filters.

Alternate between the two views


using the Report type drop down
at the bottom left of the report.
Element
An area of the page which is
tested

Variant

Multiple Action Count

The alternative content


that is tested against the
default

Multiple Action Count measures the frequency


at which an action is completed. It tracks each
time a visitor completes the selected action.
Generations
The total number of visitors
who entered the campaign

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Engagement by Elements Report

Actions per Converter


The average number of actions
recorded per visitor who completes the
selected action. Multiple Actions /
Single Actions.

Actions per Generation Error


The Margin of Error (MOE) as a percentage of the
Actions Per Generation

Confidence Index
The probability that there is a
significant difference between the
default and test variant.

Actions per Generation


The average number of actions recorded per
visitor who generated into the campaign. Multiple
Actions / Generations

Uplift in Actions per Generation


The percentage change between the conversion rate of
the new variants and the default conversion rate.

Actions per Generation Illustration


This is an illustration of the range of acceptable values (see Actions per Generation) for each experience.
When the range of acceptable values for a new variant/experience goes above the default's range of acceptable
values, the bar turns green. When it falls below the default range, the bar turns red.

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Metric Breakdown By Elements Report


Report Filters
You can filter the results
displayed in the report
by clicking the filter bar
at the top of the graph.

Report Graph
You can view any report on a
daily basis (Daily report) or
cumulatively (Cumulative
report). The Daily report remains
very volatile for any period of
time, since changes are recorded
daily, while the Cumulative report
gets smoother with time.

Note: See
Report Filtering video for
information on how to
use the filters.

Alternate between the two views


using the Report type drop down
at the bottom left of the report.
Element
An area of the page which is
tested
% of Total
The percentage contribution of each attribute to the total number of actions
Variant
Single Action Count

The alternative content


that is tested against the
default

The first action recorded for each unique visitor who


completes the selected action
Generations
The total number of visitors
who entered the campaign

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Conversion By Experience Report


Report Filters
You can filter the results
displayed in the report
by clicking the filter bar
at the top of the graph.

Report Graph
You can view any report on a
daily basis (Daily report) or
cumulatively (Cumulative
report). The Daily report remains
very volatile for any period of
time, since changes are recorded
daily, while the Cumulative report
gets smoother with time.

Note: See
Report Filtering video for
information on how to
use the filters.

Alternate between the two views


using the Report type drop down
at the bottom left of the report.

Exp ID
Each experience is given an ID.
To see which combination of
variants each experience
contains, hover over the Exp ID.

Count
Total number of unique visitors
who completed the selected
action.

Rank
Each experience is ranked on how
well it is performing compared to the
default experience.

Oracle Marketing Cloud Academy

Generations
The total number of visitors
who entered the campaign.

2015, Oracle and/or its affiliates.

Conversion By Experience Report

Conv Rate
The Conversion Rate shows the percentage of visitors who completed the selected action. The
range of values shown indicate the Conversion Rate +/-, the MOE (Margin of Error). This is known
as the range of acceptable values. Conversion rate (%) = (number of actions / number of
generations) x 100
Example: Suppose 1000 visitors have entered a campaign, and 200 of them have converted (e.g.,
clicked on the content to be optimized), then the conversion rate for that particular content is 20%.

Confidence Index
The probability that there is a significant
difference between the default and test variant

Conv Rate Error

Uplift

The Margin of Error (MOE) as a


percentage of the conversion rate

The percentage change between the conversion rate of


the new variants and the default conversion rate

Conv Rate Illustration


This is an illustration of the range of acceptable values (see Conversion Rate) for each experience.

Oracle Marketing Cloud Academy

When the range of acceptable values for a new variant/experience goes above the default's range of
acceptable values, the bar turns green. When it falls below the default range, the bar turns red.

2015, Oracle and/or its affiliates.

Revenue By Experience Report


Report Filters
You can filter the results
displayed in the report
by clicking the filter bar
at the top of the graph.

Report Graph
You can view any report on a
daily basis (Daily report) or
cumulatively (Cumulative
report). The Daily report remains
very volatile for any period of
time, since changes are recorded
daily, while the Cumulative report
gets smoother with time.

Note: See
Report Filtering video for
information on how to
use the filters.

Alternate between the two views


using the Report type drop down
at the bottom left of the report.

Exp ID
Each experience is given an ID. To see which
combination of variants each experience
contains, hover over the Exp ID.

Count
Total number of unique visitors
who completed the selected
action

Rank
Each experience is ranked on
how well it is performing
compared to the default
experience.

Generations
The total number of visitors
who entered the campaign

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

Revenue By Experience Report

Revenue Per Conversion


The average amount of revenue generated
per visitor who converted. Revenue per
Conversion = Total Revenue / Single
Actions.

Revenue per Generation Error


The Margin of Error (MOE) as a percentage of the
Revenue per Generation.

Confidence Index
The probability that there is a
significant difference between the
default and test variant

Revenue per Generation

Uplift in Revenue per Generation

The average amount of revenue generated per


visitor who entered the campaign. The range of
values shown indicated the Revenue Per
Generation +/- the MOE (Margin of Error) which
is known as the range of acceptable values.
Revenue per Generation = Total Revenue /
Generations.

The percentage change between the


conversion rate of the new variants and
the default conversion rate

Oracle Marketing Cloud Academy

Revenue per Generation Illustration


This is an illustration of the range of acceptable values (see Revenue Per
Generation) for each experience.
When the range of acceptable values for a new variant/experience goes
above the default's range of acceptable values, the bar turns green. When
it falls below the default range, the bar turns red.

2015, Oracle and/or its affiliates.

Engagement By Experience Report


Report Filters
You can filter the results
displayed in the report
by clicking the filter bar
at the top of the graph.

Report Graph
You can view any report on a
daily basis (Daily report) or
cumulatively (Cumulative
report). The Daily report remains
very volatile for any period of
time, since changes are recorded
daily, while the Cumulative report
gets smoother with time.

Note: See
Report Filtering video for
information on how to
use the filters.

Alternate between the two views


using the Report type drop down
at the bottom left of the report.

Exp ID
Each experience is given an ID. To see which
combination of variants each experience contains,
hover over the Exp ID.

Multiple Action Count


Measures the frequency at which an
action is completed tracking each time a
visitor completes the selected action.
Rank
Each experience is ranked on how
well it is performing compared to the
default experience.

Oracle Marketing Cloud Academy

Generations
The total number of visitors
who entered the campaign

2015, Oracle and/or its affiliates.

Engagement By Experience Report

Actions per Converter


The average number of actions
recorded per visitor who completes
the selected action. Multiple
Actions / Single Actions

Actions per Generation Error


The Margin of Error (MOE) as a percentage of the
Actions Per Generation

Confidence Index
The probability that there is a
significant difference between the
default and test variant

Actions per Generation

Uplift in Actions per Generation

The average number of actions recorded per


visitor who generated into the campaign. Multiple
Actions / Generations.

The percentage change between the


conversion rate of the new variants and
the default conversion rate

Actions per Generation Illustration


This is an illustration of the range of acceptable values (see Actions per Generation) for each experience.
When the range of acceptable values for a new variant/experience goes above the default's range of
Oracle Marketing Cloud Academy acceptable values, the bar turns green. When it falls below the default range, the bar turns red.

2015, Oracle and/or its affiliates.

Metric Breakdown By Experience Report


Report Filters
You can filter the results
displayed in the report
by clicking the filter bar
at the top of the graph.

Report Graph

Note: See
Report Filtering video for
information on how to
use the filters.

You can view any report on a daily basis


(Daily report) or cumulatively (Cumulative
report). The Daily report remains very
volatile for any period of time, since
changes are recorded daily, while the
Cumulative report gets smoother with time.
Alternate between the two views using the
Report type drop down at the bottom left of
the report.

Exp ID
Each experience is given an ID. To see
which combination of variants each
experience contains, hover over the Exp ID.
% of Total
The percentage contribution of each
attribute to the total number of actions

Rank
Each experience is
ranked on how well it is
performing compared to
the default experience.

Single Action Count


The first action recorded for each unique visitor who
completes the selected action
Generations

Oracle Marketing Cloud Academy

The total number of visitors


who entered the campaign

2015, Oracle and/or its affiliates.

Oracle Marketing Cloud Academy

2015, Oracle and/or its affiliates.

You might also like