How To Use the t Test To Tell If Your New Marketing Worked

How To Use the t Test To Tell If Your New Marketing Is Working

If you have just set up a new marketing or advertising campaign, you want to know as soon as possible whether it's working. You can apply a simply statistical test known as the t test to quickly find this out. This article will show in easy-to-follow, step-by-step instructions how to run a t test in Excel.

The t Test - Simplified and Done in Excel

If you had recently launched a new marketing campaign, you would want to know as soon as possible whether the campaign was working. If you are able to take a large sample of before and after measurements (for example, in all of the sales a data analysis tool called%uF0E0territories), Excel has the perfect tool for you the two-sample paired t-test for means. It is very simple to use and the output is straight-forward and easy to interpret.

t Test - General Description

This test will tell you whether the difference between the before and after numbers is genuine or whether this difference could merely have been the result of chance. Overall a t-test compares two means and determines within a specified degree of certainty whether the two means really are different, or whether the difference might have occurred by chance.

Two-Sample, Paired t Test

The two-sample paired t-test for means evaluates whether the average difference between the before and after measurements is greater than zero or not. In other words, this test evaluates within a specified degree of certainty whether the average measured difference between before and after is real or could have occurred merely by chance.

Before we start discussing this specific test in detail, The t-test needs to be generally explained. The basic question to be answered is:

The t Test - What Is It?

The t test is a statistics test generally used to test whether means of populations are different. In the t test, a t value is calculated based upon the difference in the means and variances of the two populations. The greater the t value, the more certain it is that the means are different.

The t value can be generally described as follows:

t value = (Difference between the group means) / (Variability of the groups)

There are many variations of the t test. Each has its own specific formula for calculating a t value for the sampled data sets. All of the t value formulas can be described by the above formula.

The Higher the t Value - The More Likely the Groups Are Different

The higher the t value is, the more likely it is that the two means are different. If the two groups being compared have a high degree of variance (t value has a high denominator), it is much harder to tell them apart. On the other hand, if the two groups being compared have a low degree of variance (the t value has a low denominator), it is much easier to tell the two groups apart.

The Lower the Combined Variance, the Higher the t Value

The illustrations below should clarify how the degree of variance in the two groups determines how easy or difficult it is to state that the means of the two groups are really different. The t test quantifies this relationship and provides a way to determine whether the measured difference between two means can be considered real or not based upon the amount of variance in both groups. Here are illustrations that should clarify this relationship.

Graphical Representation of the t Test

The Increased Variance in the 2 Groups On the Right Make It Harder To Tell That These Are 2 Separate Groups

We can see that pair of data sets on the right are much easier to differentiate because they have much less overlap than the pair of data sets on the right. The overlap represents the overall variability between the two data sets in each pair. The higher the total variability within the pair of data sets, the higher will be the denominator in the t value formula. The higher the denominator, the lower the t value for the pair of data sets. The lower the t value, the less likely it is that the two data sets are separate data sets with different means.

T-Test Paired Two Sample for Means

A paired t test or paired difference t test is use to determine whether the average of the "before" and "after" measurements taken of a single set of objects is the same. The Null Hypothesis being tested states that there is no difference between the average "before" and "after" measurements. Specifically, the Null Hypothesis states that the mean of all "after" measurements minus the mean of all "before" measurements taken of the same objects equals 0.

We are going to use the paired t test to determine within 95% certainty whether the average sales from a group of sales territories increased after a new marketing program was implemented. We will simply measure the before and after sales from each territory and apply this t test using Excel to get our result.

A Little Bit More About This t Test

The t Test in general is a special case of one-way (sometimes called "single factor") ANOVA. This paired two-sample student's t test is applied when there is a natural pairing of samples. It is most often used to determine whether "before" and "after" means of a sample of the same objects have changed during an experiment. One really great thing about this t test is that the paired two-sample t test does not require that the variances of both populations to be the same.

To sum up the paired two-sample student's t test, a single t value is calculated from data from both samples. Here is the formula to calculate the t value for a paired two-sample student's t test if you are testing to determine whether the difference between two means is greater than zero:

t Value

t value = Average Difference Between Each Pair /
[ Stan. Dev. Of Average Differences / SQRT(n) ]

You can see that this follows the general formula for calculating the t value in a t test, which is:

t value = (Difference between the group means) / (Variability of the groups)

The t value is a specific point on the x-axis in the t distribution (student's t distribution). If this t value falls outside the region of required certainty, it can be stated that the two means are probably different. If this t value falls within the region of required certainty, it cannot be stated that the two means are probably different.

The required region of certainty depends upon the degree of certainty required in the test. If 95% certainty is required, then the required region of certainty consists of 95% of the area under the student's t distribution. The outer 5% is the region of uncertainty. This is also referred to as alpha or the degree of significance. If the t value is large enough to be located all the way out on the x-axis in the 5% region of uncertainty, it can be stated within 95% certainty that the two means are different.

One or Two-Tailed Test

A t test can be a one-tailed test or a two-tailed test. A one-tailed test determines whether the means are different in one specific direction. For example, a one-tailed test could be used to determine only if the mean of the "after" measurements is greater than the mean of the "before" measurements. A two-tailed test determines whether the two means are merely different.

Two-Tailed t Test Is More Stringent

The two-tailed test is more stringent because the area in the outer tails outside of the region of required degree of certainty is split into two tails. For example, if the required degree of certainty is 95% on a two-tailed test, the calculated t value must be all the way out in the outer 2.5% of either tail for the t test to conclude within 95% certainty that the means are different.

One-Tailed t Test Is Less Stringent

A one-tailed test is less stringent. If the required degree of certainty is 95% on a one-tailed test, the calculated t value only has to be within the outer 5% of whatever tail is being tested to be able to state the two means are probably different.

Doing The Paired Two-Sample t Test in Excel

The Data Needs to Be Arranged In Excel As Follows:

We are testing to determine whether a new marketing campaign has increased sales in a group of six sales territories. In this case the sample size (n) equals 6. For this type of t test, the degrees of freedom = n - 1 = 5.

Access the t Test In Excel

t Test Dialogue Box

Now, access this Excel t Test as follows (this is Excel 2003):

Tools / Data Analysis / t-Test: Paired Two Sample for Means

The above dialogue box will appear.

Input the Data As Followings:

Variable 1 Range: Select everything that is highlighted light blue, including the label "Sales After New Ads." If you are trying to determine whether the "after" measurements have gone up, the "after" data is Input Variable 1. If you are trying to determine whether the "after" measurements have gone down, the "after" data is Input Variable 2.

Variable 2 Range: Select everything that is highlighted in yellow, including the label "Sales Before New Ads."

Hypothesized Mean Difference: 0

Labels: Check the box because you included the labels for Variables 1 and 2.

Alpha: This depends on your desired degree of certainty. 0.05, if you desired 95% certainty. 0.20 if you desire 80% certainty.

Output Range: Select the cell that you want the upper left corner of the output to appear in.

Hit "OK" to run the analysis and the following Excel output appears:

Excel Output

t Value = 2.511

This output can be interpreted as follows:

The t value is 2.511.

One-Tailed Test

This t value is greater than the critical t value for a one-tailed test (2.015). We can therefore state with 95% certainty that the mean sales has increased as a result of the new marketing campaign.

The above conclusion can also be reached because the p Value for the one-tailed test (highlighted in light blue on the Excel output) is 0.027. This is less than alpha (0.05). The p Value being less than alpha is an equivalent result to the t value being greater than the t critical value.

Two-Tailed Test

A different result is arrived at for the two-tailed test. The two-tailed test is more stringent because the alpha region of uncertainty (5% of the area under the student's t distribution curve) is now divided between both outer tails. The t value needs to be larger for the two-tailed test to wind up in the outer 2.5% area of either outer tail.

In this case, the t value was not large enough to be positioned in the outer 2.5% of either outer tail. The t value (2.511) is smaller than the critical t value for the two-tailed test (2.571). This indicates that it cannot be stated with 95% certainty that there has been a change in the mean from before to after.

The p value calculated for the two-tailed test (0.054) is larger than alpha (0.05). This is an equivalent result to the above.

Verifying Excel's Calculation of t Value and p Value

Original Sample Data

Let's calculate the t value and p values for the one and two-tailed tests by hand to make sure that Excel has done a correct job. The t value is stated as the t statistic.

Hand Calculation of t Value and p Value

Here are the Hand Calculations

Hand Calculations of t Value and p Value

Here is the hand calculation of the t value and p values for the one and two-tailed tests for this Paired Two-Sample t Test. The hand calculation agrees with the Excel outputs. There are very slight differences due to rounding differences:

Conclusion

The Paired Two-Sample t Test is a very simple test to run and can be applied to nearly any aspect of your marketing program to see if a single change affected a large number of elements whose before and after measurements can be taken. One note: the before and after measurements must be continuous and using the same scale.

By Far, The Clearest Graduate-Level Statistics Lessons - All In Excel! ---> Click On Image To See Reviews !

The Excel Statistical Master eBook - Over 400 Pages and Videos of Easy-To-Understand, Step-By-Step Business Statistics in Excel - Download It Now !

Excel Statistical Master - Over 400 Pages and Videos of Easy-To-Understand Statistics

1) It's LOADED With Completed Problems ALL in Excel.

2) All Solved Problems Are REAL-WORLD, BUSINESS Problems.

3) Nothing But SIMPLE Explanations.

4) More Than Half of the Lessons Are Supplemented With VIDEOS To Help You Understand Quicker.

5) Lessons Are All in BITE-SIZE Chunks.

6) The Statistics Are MBA-LEVEL - Over 400 Pages and Videos.

7) All Lessons Are In CLEAR, EASY-TO-FOLLOW FRAMEWORKS.

8) NO USELESS THEORY Is Taught.

9) NO PRIOR STATISTICAL KNOWLEDGE Is Needed.

10) Every Lesson Is Entirely In Excel. NO LOOKING UP ON CHARTS.

11) You Already Know Excel. NO NEW SOFTWARE NECESSARY.

Wanna Be a Master of Business Statistics FAST ? Check Out My Other Lenses !

My Other Lens That Will Make You a Master of Business Statistics - FAST !

http://www.squidoo.com/the-22-most-common-mistakes-made-during-statistical-analysis: If you make any of these mistakes, you'll either get a totally wrong answer or you will spend way more time than necessary getting to the right answer.
Excel's Easiest and Most Robust Normality Test - The Chi-Square Goodness-of-Fit Test: This Normality test is very easy to implement in Excel. This lens explains in easy-to-follow instructions exactly how to perform the test and also how the Chi-Square Goodness-Of-Fit Test actually works.
The 7 Biggest Reasons That Your Data Is Not Normally Distributed: Here is a list of the main (but correctable) reasons that your data does not appear to be normally distributed. If your data is normally distributed, you can perofrm most statistical tests on that data. This lens tells you why your data doesn't appear to be normally distributed and what you can do to correct that.
How To Use Dummy Variable Regression in Excel To Do Conjoint Analysis: Conjoint Analysis is a great marketing technique that will tell which products attributes your customers like the best and also quantifies the customers' degree of liking of each attribute. This lense shows exactly how to do it in Excel. This lens has a video to make it easier to understand.
Taguchi Testing - Is It a Good Tool For Landing Page Optimization?: This lens explains in simple terms exactly what Taguchi testing is and why it is or is not a good tool for landing page optimzation.
How To Create a User-Interactive Graph Of the Normal Distribution in Excel: You will see in step-by-step, easy-to-follow lessons, which includes a video, exactly to create a user-interactive graph of the normal distribution in Excel. The user will be able to vary the mean and standard deviation and watch those changes reflected in the Excel graph that you will be able to make after viewing this lens.
How To Use the t Test in Excel To Tell If Your New Marketing Is Working: The t Test is the easiest tool in the world to tell if your new marketing or advertising campaign really worked. Here's how to do it in Excel.
How To Find Out What Makes Your Customers Buy WIth the Chi-Square Independence Test In Excel: The Chi-Square Indepedence Test is a versatile test for determining if one thing is related to the other.l Here's how to do it in Excel, This lens has a video in it to make it easy to understand.
How To Quickly Read the Output of Regression in Excel: The output of regression in Excel in complicated. Here is how to quickly look at this output and identify the 4 main, important parts and what they mean. This lens has a video in it for easier understanding.
Download the 400+ Page eManual About How To Do Business Statistics in Excel: The 400+ page eManual, The Excel Statistical Master, simplfies business statistics by presenting everything in frameworks that are easy-to-follow and easy-to-duplicate. Loaded with worked-out problems, this eManual focuses on solving real-world business statistics problems. Everything is conveniently solved with Microsoft Excel.

Links To Learn Business Statistics At a More Basic Level - These Lessons Are Easy-To-Follow !

Business Statistics Made SIMPLE - For Those Who Don't Speak Math !

If you are interested in learning business statistics quickly and being able to solve real-world statistics problems, these links are for you.

Confidence Intervals For the Business Manager: Confidence Intervals made easy-to-understand for the business manager. Lots of solved problems. Each problems solved with a step-by-step framework that you will be able to duplicate and use quickly. No wasting time with academic theory that is not useful for solving problems.
Prediction Using Regression For the Business Manager: Regression analysis has a lot of important uses in business. This lesson show you in detail how to perform regression without confusing you. Loaded with solved problems, this lesson teaches regression in an easy-to-follow, step-by-step method that you will pick up right away.
How To Solve ALL Hypothesis Tests in Only 4 Steps in Excel: This lesson presents a framework that shows you show to solve ALL of the many kinds of hypothesis tests in only 4 steps in Excel. Extremely useful !
The Normal Distribution For the Business Manager - With Lots of Worked-Out Problems !: How to use the Normal Distribution to solve real-world business problems. Lots of worked-out problems solved with and easy-to-follow- step-by-step framework. The emphisis is entirely on solving problems with no time wasting on complicated acedemic theory that is not useful for problem solving. Loaded with solved problems !
The t Distribution For the Business Manager: The t Distribution is often used for analyzing small samples. It is very close to the normal distribution but very useful when sample size is less than 30.
Combinations and Permutations for the Business Manager - With Lots of Worked-Out Problems !: Combinations and Permutations are extremely useful statistical tools. Anyone who does statistical analysis should know them.
Correlation and Covariance For the Business Manager - With Lots of Worked-Out Problems !: Correlation and Covariance describe linear relationships between different variables. It allows you to quickly determine to what degree the movement of one object is related to the movement over another object.
The Binomial Distribution For the Business Manager - With Lots of Worked-Out Problems !: The Binomial Distribution is one of the most valuable and commonly used statistical tools. The Binomial Distribution is used whenever a process has only two possible outcomes. This is a very common occurrence.
ANOVA for the Business Manager - With Lots of Worked-Out Problems !: ANOVA, Analysis of Variance, is a test to determine whether the means of several groups are equal. As such, ANOVA is often used to determine if three or more different methods or treatments have the same effect on a population. For example, ANOVA testing might be used to determine if three different training methods produce the same sales results with a group of salespeople.
Other Useful Distributions For the Business Manager - With Lots of Worked-Out Problems !: Learn how many other distributions, such as the Weibull and the Poisson Distributions can be used in business.
Download the 400+ Page eManual About How To Do Business Statistics in Excel: The 400+ page eManual, The Excel Statistical Master, simplfies business statistics by presenting everything in frameworks that are easy-to-follow and easy-to-duplicate. Loaded with worked-out problems, this eManual focuses on solving real-world business statistics problems. Everything is conveniently solved with Microsoft Excel.

Squidoo