Jonathan Tomlin and Robert Zeithammer, experts in economic damages and pricing and quantitative assessment, highlight weaknesses of both the demand measurement and the subsequent price premium modeling in several recent labeling class action cases.
In product labeling class actions, restitution is frequently measured by the “price premium” for the product at issue, defined as “the difference between the market price actually paid by consumers and the true market price that reflects the impact of the unlawful, unfair, or fraudulent business practices” (Werdebaugh v. Blue Diamond Growers, 2014 BL 146743).
Plaintiff experts often propose using “conjoint analysis” surveys to help measure this price premium. Conjoint analysis is a method for measuring consumer demand for a product. To predict the counterfactual “true market price” needed for estimating the price premium for a product, demand estimates need to be combined with supply-side information by using additional models and assumptions.
In several recent product labeling class actions, the class certification decision hinged on two questions:
- whether the court accepted the proposed conjoint analysis as a valid measure of consumer preferences, and
- whether conjoint survey results were correctly combined with supply-side information to produce a valid price premium estimate.
In this article, we highlight weaknesses of both the demand measurement and the subsequent price premium modeling in several recent labeling class action cases.
Three failures in the survey design emerge:
- failure to properly characterize label claims,
- failure to allow for what experts call the “outside choice,” and
- failure to include important “attributes.”
Each of these failures results from a disconnect between the way products are presented to respondents in a survey and the actual shopping environment for the product at issue. We also describe a common, and critical, failure in interpreting results from conjoint studies in product labeling lawsuits—failing to consider supply factors in the market.
In order to implement reliable conjoint studies in future labeling suits, experts will need to create a survey design that reflects actual shopping choices. They will also need to develop models of the supply-side if conjoint survey results are to be used to measure a price premium.
The Basics of Conjoint Analysis
Conjoint analysis is a survey-based statistical tool for measuring consumer demand for products that can be readily described to survey respondents as a collection of “attributes” (i.e., product characteristics) that jointly drive purchase decisions. For example, a smartphone may be readily described by attributes of color, brand, screen size and quality, data capacity, camera quality, ruggedness, and price (e.g., Silver iPhone with 5.8 inch OLED screen, 256 GB capacity, dual 12 MP camera, and water resistance up to 2 meters depth, sold for $400). A fundamental assumption of conjoint analysis is that survey respondents can imagine a product based on an attribute profile that is presented to them and make a simulated shopping choice among a set of products with differing attributes.
Once simulated choice data is collected from a survey of consumers, analysts apply “choice modeling” statistical techniques to estimate what each survey respondent is willing to pay for each attribute level relative to some baseline level. For example, an analyst may estimate that “respondent number 78” is indifferent between a phone with 256 GB capacity and an otherwise identical phone with only 64 GB capacity but a price lower by $62. This price difference of $62 is the “willingness to pay” for this consumer for the extra capacity. Responses from many survey respondents are typically aggregated into estimates of average willingness to pay for a particular product feature in the target population of consumers.
Conjoint surveys do not require the existence of an actual product with a specific set of attributes. They only require survey respondents to be able to imagine a product with a specified set of attributes and make a simulated shopping choice. Because conjoint surveys can be applied to hypothetical products, they are often used for measuring counterfactual scenarios. For example, in the field of marketing, the potential success of introducing new features for a product is often evaluated using conjoint surveys. In the litigation context, experts have often attempted to employ conjoint surveys to measure the willingness to pay of consumers of an allegedly false product claim such as “natural” or “GMO-free.”
Three Conjoint Survey Design Failings
Defendants have frequently challenged the validity of the simulated shopping choice in conjoint surveys performed by the plaintiffs’ expert in recent product labeling class actions. Below, we identify three deficiencies in the design of conjoint surveys performed in recent product labeling class actions. Each of these deficiencies is due to a divergence between the way products are presented to respondents in a survey and the way consumers would actually encounter these products in a retail setting.
1) Failure to Properly Characterize Label Claims. In Townsend v. Monster Beverage Co., the plaintiffs’ expert performed a survey which altered the actual language contained on the product label of the beverages at issue in the case ( 2018 BL 166134). Perhaps attempting to balance the length of five contested claims, the plaintiffs’ expert embellished some of the label claims (e.g., he used “Re-hydrate to bring you back” instead of merely “Re-hydrate” which was the language on the label) and shortened other label claims. The court excluded the plaintiffs’ expert from testifying on all the label claims whose language was altered, leaving only one of the five originally contested claims—“Hydrates like a Sports Drink,” which was used verbatim in the expert’s survey.
While the court did not exclude the plaintiffs’ expert from testifying on his conjoint-analytic measurement of the last remaining claim, the court nevertheless denied class certification. The plaintiffs alleged that “Hydrates like a Sports Drink” was misleading because it implied that the beverage included electrolytes when the product did not actually contain electrolytes. However, survey results from one of the plaintiffs’ experts showed that only 5 percent of those surveyed interpreted “Hydrates like a Sports Drink” as meaning the product “contains electrolytes.” In addition, the court concluded that the survey results of this expert did not show this statement to be a “driving factor in the purchasing decision” and, therefore, was not material.
2) Failure to Allow for the “Outside Choice”. A second deficiency in survey design in some recent conjoint surveys in product labeling cases is a failure to give respondents the choice not to purchase any product (the “outside option”) when presenting them with product choices. For example, in Morales v. Kraft Foods Grp. Inc., the plaintiffs’ damages expert attempted to measure consumer “willingness to pay” for the label claim “natural cheese” through a conjoint survey by presenting survey respondents with choices between different versions of Kraft Fat Free Shredded Cheddar Cheese (2017 BL 216539). The conjoint survey he performed did not allow respondents a choice of “none of the above.” In contrast, the plaintiffs’ survey expert in In re Dial Complete Mktg. included the outside option in his survey design (2017 BL 96999).
The impact of the presence of the outside option on survey results has been hotly debated in academic circles (See, e.g., Jeffrey R. Parker and Rom Y. Schrift, The Rejectable Choice-Set: How Seemingly Irrelevant No-Choice Options Affect Consumer Decisions, Journal of Marketing Research, Vol. 48, 2011). However, the presence of this choice can be a requirement for a reliable conjoint survey in some labeling class actions. This is because some experts attempt to measure the willingness to pay of the “marginal consumer,” which is the consumer who is indifferent between purchasing the product and not purchasing the product at prevailing market prices. Aside from being insufficient for estimating a price premium (a point we argue later in this article), the marginal customer measurement cannot be reliably accomplished unless survey respondents have the “outside option” of choosing not to purchase any product at the prices presented to them.
3) Failure to Include Important Attributes. The conjoint survey performed in Morales contained an additional deficiency—a failure to include important purchase-drivers as product attributes in the survey. The product profiles presented to the survey respondents varied only in terms of price and whether the label included “natural cheese” or not. The defendant argued that this caused respondents to place undue attention on the “natural cheese” attribute to the exclusion of other attributes. More appropriate conjoint surveys tend to include more attributes than just price and the focal claim. For example, in Dial the plaintiffs’ expert measured the consumer willingness to pay for a challenged claim that the product at issue “Kills 99.99% of Germs.” His survey included not only price and the presence of the claim as attributes, but also added three other binary attributes known to drive consumer choice of liquid soaps: “antibacterial,” “foaming,” and “moisturizing.” The court certified the class.
A survey that causes respondents to draw undue attention to a particular attribute can lead these respondents to infer the “correct answer” expected by the surveyor (e.g., in Morales that “natural cheese” is both an important and a desirable attribute). This results in a “demand effect” whereby survey respondents say what the interviewer wants to hear instead of responding truthfully. Even if such an effect does not occur, drawing undue attention to one attribute can make respondents forget about other attributes that also influence their real-life purchase decisions—a bias called “focalism.”
Whether courts will accept conjoint surveys plagued by focalism bias is unclear. In both Townsend and Morales, the defendant argued that the survey performed by the plaintiffs’ expert suffered from focalism bias and moved to exclude the testimony of the plaintiffs’ survey expert. The court denied the defendant’s motion in both cases. However, the court in Townsend denied class certification after concluding that the survey design of the conjoint analysis performed by the plaintiffs’ expert “suffers from focalism bias, rendering it useless for determining price premiums …”
The Insufficiency of ‘Willingness to Pay’ Alone to Measure a Price Premium
Although the court did not exclude the testimony of the plaintiffs’ survey expert in Morales, it decertified the class because it concluded that the consumer value measured by the plaintiffs’ expert “did not measure the market value of the Product either with the ‘natural cheese’ label or without it.” In any competitive market, market value arises from competition among sellers, each acting to maximize its profits given its costs. Therefore, a successful estimation of the market value of a labeling claim (and, thus a successful estimation of a price premium) must convincingly account for supply-side factors—i.e., costs and competitive conditions. A properly constructed conjoint survey measures consumer demand for a benefit implied by a labeling claim, such as “natural,” “re-hydrates,” or “made with real ginger.” Since conjoint survey results only measure demand and do not account for the supply-side, their results cannot, standing in isolation, measure a price premium.
Like the court in Morales, courts in at least two additional recent product labeling cases recognized the inability of a conjoint survey alone to estimate a price premium and denied class certification. In Saveedra v. Eli Lilly & Co., the plaintiffs’ expert proposed using a conjoint survey to measure willingness to pay and, thereby, “benefit of the bargain damages” related to allegedly false claims regarding the probability of side effects occurring for an antidepressant (2014 BL 368152). The court noted that it had “found no case holding that a consumer may recover based on consumers’ willingness to pay irrespective of what would happen in a functioning market (i.e. what could be called sellers’ willingness to sell).” Based on this, and despite what appears may have been a reasonably executed conjoint analysis, the court declined to certify the putative class for failing to provide a method of damages tied into the theory of liability as required by Comcast v. Behrend (2013 BL 80435). In In re NJOY Inc. Consumer Class Action Litig., the plaintiffs’ expert proposed calculating price premium damages by performing a conjoint study to determine consumers’ willingness to pay for a claim regarding the safety of e-cigarettes (2016 BL 58999). The court concluded that the methodology focused on demand and “completely ignores the price for which NJOY is willing to sell its products, what other e-cigarette manufacturers say about their products, and the prices at which those entities are willing to sell their products.” It also denied class certification.
In several other cases, plaintiff experts using a conjoint survey have used erroneous arguments to overcome the hurdle of needing to examine supply conditions. We identify three such erroneous arguments that plaintiff experts have recently employed.
1. The first argument was used by plaintiff experts in Dialand Fitzhenry-Russell v. Dr. Pepper Snapple Grp. Inc. (2018 BL 227311). The plaintiffs in these cases suggested that the conjoint survey conducted measured the willingness to pay of the “marginal consumer,” and this made willingness to pay equivalent to a price premium. The courts in both Dial and Fitzhenry-Russell cited to an article on conjoint analysis (Lisa Cameron, Michael Cragg, and Daniel McFadden, The Role of Conjoint Surveys in Reasonable Royalty Cases, Legal Times, October 16, 2013) and interpreted a sentence in the article as suggesting that determining the willingness to pay of the marginal consumer is the same as calculating a price premium.
The marginal consumer’s willingness to pay is generally not equal to a price premium, even in a monopoly setting. A profit-maximizing monopolist would typically charge a premium for a product with a challenged claim that was less than the willingness to pay for this claim by the marginal consumer. In more realistic competitive settings, marginal consumer analysis does not even provide a bound on the correct price premium. Why did the courts accept such a patently erroneous method? It appears that one reason is that the article cited was, in fact, misinterpreted. One of the article’s authors described in a more recently published article that a conjoint analysis alone could not properly measure a price premium (Greg Allenby, Peter Rossi, Lisa Cameron, and Yikang Li, Computing Damages in Product Mislabeling Cases: Plaintiffs’ Mistaken Approach in Briseno v. ConAgra, 45 PSLR 208, 2/27/17).
2. A second erroneous argument that the courts in Dial, Fitzhenry-Russell, and Schneider v. Chipotle Mexican Grill, Inc. (2018 BL 361012), noted was the assertion of the plaintiffs’ expert that he had considered market factors or “real world” prices in conducting his analysis and that this therefore accounted for supply conditions. For example, the court in Fitzhenry-Russellstated that “importantly” the plaintiffs’ expert “declared he considered ‘supply-side factors and real-world market transaction information from consumer transaction data for the Defendants’ products and for its competitors.’ ” The court in Schneider concluded that the expert’s use of “actual prices” was sufficient to establish that willingness to pay results from his survey were tied “to the real-world marketplace” and therefore “demonstrate that a price premium model is ‘consistent with’ Plaintiff’s liability case.”
In order to meaningfully account for supply factors and calculate a price premium, an expert must conduct an empirical analysis that actually adjusts for these factors and their impact on a price premium. Mere lip service to market conditions should not supplant actual analysis of market conditions. The use of “real world” prices and transactions in a conjoint survey does not overcome the need to model supply conditions. The price that would have existed “but-for” the allegedly wrongful conduct at issue determines the price premium. This “but-for” price is not a “real world” price that occurred but, instead, is the price that would have occurred in the absence of allegedly wrongful conduct. It follows that a valid estimate of such counterfactual pricing needs to rely on a model of the supply-side, not just on mere observations of what actually transpired in the marketplace.
3. The third erroneous argument that has supported use of willingness to pay from a conjoint survey as a measure of a price premium is the plaintiffs’ argument that their expert assumed supply was fixed, and this assumption amounted to considering supply in the analysis (See, e.g., Dialand Fitzhenry-Russell). Assuming that supply is fixed means assuming that suppliers do not respond to changes in demand in setting the quantity that they sell. Because of this, this assumption does not meaningfully consider supply and leads to a measure of a price premium that is unlikely to comport with any actual impact on pricing of the product labeling at issue.
In order to calculate a price premium based on conjoint analysis, an analyst needs to develop a model of the supply-side in conjunction with survey results. In one fairly recent academic research article, Allenby, et al. studied the question of estimating the value of a patent and proposed such a framework (Greg M. Allenby, Jeff Brazell, John R. Howell, and Peter E. Rossi, Valuation of Patented Product Features, Journal of Law and Economics 2014). Their supply-side simulations are applicable to estimating a counterfactual price without a disputed claim, but such simulations require assumptions about the competitor set, their competitive interaction, and their costs. These can be difficult hurdles to clear in the litigation context because additional assumptions always invite additional avenues of challenge.
Conclusion
Conjoint surveys can play a role in product labeling class actions when they are well designed, and their results are properly combined with a consideration of supply-side factors to yield a valid estimate of the price premium. Good survey design requires simulating actual products and shopping experiences. Even when a conjoint survey is appropriately designed, the results from a conjoint study alone cannot measure a price premium.
Unfortunately, several conjoint studies performed in product labeling class actions have been poorly designed, have purported to measure a price premium when they were incapable of measuring such a price premium, or both. Hopefully, courts will be able to identify the “con” in conjoint studies that contain these deficiencies.
Reproduced with permission. Published November 01, 2018. Copyright © 2018 The Bureau of National Affairs, Inc. Article originally published by Bloomberg BNA.
The views expressed herein are those of the author(s) and not necessarily the views of Ankura Consulting Group, LLC., its management, its subsidiaries, its affiliates, or its other professionals. Ankura is not a law firm and cannot provide legal advice.
