NielsenIQ

The Consumer Panel Data comprise a representative panel of households that continually provide information about their purchases in a longitudinal study in which panelists stay on as long as they continue to meet NielsenIQ's criteria. NielsenIQ consumer panelists use in-home scanners to record all of their purchases (from any outlet) intended for personal, in-home use. Consumers provide information about their households and what products they buy, as well as when and where they make purchases.

Years Available: Starting with 2004 and including annual updates.

Panel Size: 40,000–60,000 active panelists (varies by year), projectable to the total United States using household projection factors.

Panelists: Household demographic, geographic, and product ownership variables are included, as well as select demographics for the heads of household and other members.

• Demographic variables include household income range, size, composition, presence and age of children, marital status, type of residence, race, and Hispanic origin. Male and female heads of household also report age range, birth date, hours employed, education, and occupation. For other family members, birth date, employment, and relationship/sex are reported.
• Geographic variables include panelist zip code, FIPS state and county codes, region (East, Central, South, West), and Scantrack Market code (assigned by NielsenIQ, and called Syndicated Major Markets (SMM) starting in 2021).
• Product Ownership variables include kitchen appliances, TV items, and internet connection.

Products: From 2004 to 2020, all 10 NielsenIQ food and nonfood Departments (~1.4 million UPC codes). These Departments are dry grocery, frozen foods, dairy, deli, packaged meat, fresh produce, nonfood grocery, alcohol, general merchandise, and health and beauty aids.

Starting in 2021 data, NielsenIQ implemented a new Product Hierarchy structure. The original Product Hierarchy of Departments, Product Groups, and Product Modules has been phased out. For now, Product Module Codes will still be available to make the transition smoother.

Product Characteristics: All products include UPC code and description, brand, multipack, and size, as well as NielsenIQ codes for Department, Product Group, and Product Module. Some products contain additional characteristics (e.g., flavor). Starting in 2021 data, NielsenIQ expanded the number of additional product characteristics.

Purchases: Each shopping trip contains the date, retail chain code, retail channel, first three digits of store zip code, and total amount spent. For each product purchased, the UPC code, quantity, price, and any deals/coupons are recorded. Note that retailer names are not available.

Geographies: Entire United States, divided into 62 major markets.

Retail Channels: All retail channels—grocery, drug, mass merchandise, superstores, club stores, convenience, health, and others.

Retail Scanner Data consist of weekly pricing, volume, and store environment information generated by point-of-sale systems from more than 90 participating retail chains across all US markets.

Years Available: Starting with 2006 and including annual updates.

Store Demographics: Includes store chain code, channel type, and area location. Retailer names are masked to protect identity.

Weekly Product Data: For each UPC code, participating stores report units, price, price multiplier, baseline units, baseline price, feature indicator, and display indicator.

Products: Weekly product data for 2.6-4.5* million UPCs including food, nonfood grocery items, health and beauty aids, and select general merchandise aggregated into 1,100 product categories store environment variables (i.e., feature and display indicators) from a subset of stores. The 1,100 product categories are categorized into 125 product groups and 10 departments. The structure matches that of the consumer panel data. All private-label goods have a masked UPC to protect the identity of the retailers.

Starting in 2021 data, NielsenIQ implemented a new Product Hierarchy structure. The original Product Hierarchy of Departments, Product Groups, and Product Modules has been phased out. For now, Product Module Codes will still be available to make the transition smoother.

Product Characteristics: All products include UPC code and description, brand, multipack, and size, as well as NielsenIQ codes for department, product group, and product module. Some products contain additional characteristics (e.g., flavor).

Geographies: Scanner Data from 35,000-50,000* participating grocery, drug, mass merchandiser, and other stores, covering more than half the total sales volume of US grocery and drug stores and more than 30 percent of all US mass merchandiser sales volume. Data cover the entire United States, divided into 62 major markets, and include the same codes as those used in the consumer panel data.

Retail Channels: Food, drug, mass merchandise, convenience, and liquor.

*As of 2018, approximately 15,000 more stores have been added.

Complementary to the Consumer Panel data, this annual survey contains information about panelists’ ailments and shopping they experience during the time surveyed. Researchers must be subscribed to the Consumer Panel dataset to have access to these data, however, there are no additional fees required.

Years Available: Beginning in 2011 and including annual updates.

Panel Size: 40,000 - 62,000 active panelists (varies by year).

Types of Data: Survey questions vary by year, however the following are several categories of information included in this dataset.

Ailments and Diagnoses: Panelists report for themselves and/or other household members whether they have experienced different ailments and illnesses. The list includes diabetes, cancer, heart disease, ADHD, obesity, cholesterol problems, and more. Beginning in 2021, Covid-19 is also included in the list. For a full list of ailments, see below.

Special Diets and Exercise: Respondents provide information about whether they or other household members follow special diet(s). In addition, the survey asks for information about individual exercise habits. For a full list of special diets, see below.

Health Treatments: Related to household member health ailments, the survey collects data on how individuals manage their ailments via medical treatments, exercise, dietary changes, surgery, prescription medication and more.

Shopping Preferences: The survey asks panelists for information about how ailments and health concerns affect their shopping behaviors. For example, they ask if shoppers consider product attributes (see list below) when making a purchase decision. In addition, respondents rate their perceptions of health food selections at different retailers.

Ailments and Conditions Measured
Acid Reflux / GERD / Heartburn
Celiac Disease
Gluten Sensitivity (but not diagnosed with Celiac Disease)
Allergies – food related only (not including gluten sensitivity or Celiac Disease)
Allergies (outdoor, hay fever, indoor, seasonal, etc.)
Lactose Intolerance
Alzheimer’s Disease / Dementia
Arthritis
Joint / Neck / Back Pain (not Arthritis)
Asthma
Attention Deficit Disorder/Hyperactivity
Autoimmune (Lupus, Graves, etc.)
Blood Conditions (e.g., Anemia, Bleeding disorders)
Chronic Bronchitis/Pulmonary Disease/COPD/Emphysema
Cancer
Cholesterol Problems (i.e., High Cholesterol, Triglycerides)
Upper Respiratory Illness (RSV, Coronavirus/COVID-19, Flu, Cold, etc.)
Pre-Diabetes           
Diabetes – Type I
Diabetes – Type II
Gestational Diabetes
Depression/Anxiety
Dry Eye (loss or reduction of ability to produce tears)
Eye Disease (Glaucoma, Cataracts, Macular Degeneration, etc.)
Imperfect Vision (i.e., Near or far-sightedness, Astigmatism, etc.)
Gum Disease (Gingivitis, Inflammation, Periodontis, etc.)
Hair Loss
Headache – chronic/tension
Headache – migraine
Heart Disease / Heart Attack / Angina / Heart Failure
High Blood Pressure / Hypertension
Insomnia / Sleeplessness
Chron’s Disease or Ulcerative Colitis
Irritable Bowel Syndrome (IBS)
Constipation – chronic
Menopause/Perimenopause
Muscle Pain / Spasms / Musculoskeletal Pain (e.g. Fibromyalgia)
Muscular Dystrophy / Multiple Sclerosis
Obesity / Overweight
Osteoporosis / Bone Loss
Restless Leg Syndrome (RLS)
Acne
Psoriasis
Eczema
Other Skin Condition (Skin Rash, Irritation, Dermatitis, etc.)
Urinary Incontinence / Overactive Bladder

Special Diets Measured
Vegan
Vegetarian
Mediterranean
WW (Weight Watchers)
Atkins
Jenny Craig
South Beach
Dash
Whole30
Keto
Low sugar
Low calorie
Low fat
Low sodium
High fiber
Low/no gluten
High protein
Low carb
Paleo
Kosher
Diabetic diet
Intermittent fasting
Other

Food Attributes
Heart Healthy
Superfoods
High protein
Natural
Organic
Non-GMO
Grain Free / Gluten Free
Nut free
Low sugar
Low sodium
Low fat
Kosher
Soy presence
Vitamins / Minerals presence
Free of Artificial ingredients
Hormone / Antibiotic Free
Contains Probiotics / Kefir
Low Glycemic
Lactose / Dairy Free
Free of high fructose corn syrup
High fiber

As a supplement to the Consumer Panel data, NielsenIQ sends monthly surveys to households in the panel. Since 2015 and ongoing, Michael Weber et al. have contracted with NielsenIQ to customize and run a number of additional surveys with the full panel. The first collection of those custom surveys is the Chicago Booth Expectations and Attitudes Survey (CBEAS), and the second collection is the Chicago Booth Expectations and Communications Survey (CBECS). Weber et al. will provide additional waves of data in the future. 

Subscribers to the Consumer Panel data may request access to these data at no additional cost.

Chicago Booth Expectations and Attitudes Survey (CBEAS)

This survey was two waves: one fielded in 2015 and the other in 2016. The following are several categories of information included in the surveys.

Demographic: college major, employment status, occupation, income expectations, rent, mortgage, medical expenses, and primary household shopper.

Financial Habits: Panelists report how their money is spent, as well as how they save and invest.

Inflation and Price Expectations: Respondents provide information about their beliefs and expectations about the state of the economy over the 12 months prior to the date of the survey and the future 12 months following the date of the survey. They are asked multiple questions about their perception of price inflation.

Financial Literacy and Risk Tolerance: Panelists are asked a series of questions to assess their financial literacy related to savings growth and inflation. They are also asked to rate their own willingness to take risks regarding financial matters as well as in daily activities.

CBEAS Wave 1
View Survey Questions
Dates: June/July 2015
Sample: 92,511 households
Respondents: 49,383 respondents from 39,809 households

CBEAS Wave 2
View Survey Questions
Dates: June/July 2016
Sample: 95,300 households
Respondents: 43,036 respondents from 35,694 households

Reference Articles (users must cite these in their work): 

D’Acunto, F., Malmendier, U., Ospina, J., & Weber, M. (2020). Exposure to grocery prices and inflation expectations. Journal of Political Economy, 129(5), 1615–1639. https://doi.org/10.1086/713192   

D’Acunto, F., Malmendier, U., & Weber, M. (2021). Gender roles produce divergent economic expectations. Proceedings of the National Academy of Sciences, 118(21). https://doi.org/10.1073/pnas.2008534118  

Chicago Booth Expectations and Communications Survey (CBECS)

This survey was seven waves: four in 2018 and three in 2019.

This survey topics are similar to those in the CBEAS, but researchers added various economic information treatment groups (plus one (1) control group). Some examples of the information the survey included are a consumer price index (CPI) inflation rate over the last 12 months, the inflation target of the Federal Reserve, the actual population growth over the prior three (3) years (placebo treatment), the most recent FOMC policy statement, or current unemployment numbers. The following topics describe the types of questions the panelists answered across waves. 

Demographic: The survey collects demographic information such as who the primary household shopper is, employment status, current occupation, and recent spending behavior in various categories.

Financial Habits: Panelists report how they save their money and how they invest it. They report the fractions of income that they dedicate to these categories. They also answer questions about financial constraints, savings, portfolio choice, and debt.

Inflation and Price Expectations: Respondents provide information about their beliefs and expectations about the state of the economy in the recent past and the near future. They are asked about their perception of price inflation, changes in housing prices, changes in interest rates, and the general state of the economy.  

Credit Card Usage and Debt: The survey asks respondents about their credit card ownership, including the amount of credit available to the respondent. It also asks about the household's total amount owed in debt, combining mortgage debt, auto loans, and student loans.

Financial Literacy and Risk Tolerance: Panelists answer a series of questions to assess their financial literacy and understanding of statistical information.

2018 Wave 1
View Survey Questions
Dates: April/May 2018
Sample: 94,272 households
Respondents: 10,781 unique respondents*

2018 Wave 1a
View Survey Questions
Dates: May/June 2018
Sample: 82,062 households
Respondents: 13,729 unique respondents*

2018 Wave 2
View Survey Questions
Dates: September/October 2018
Sample: 82,062 households
Respondents: 47,128 individuals from 32,658 households

2018 Wave 3
View Survey Questions
Dates: December 2018
Sample: 92,390 households
Respondents: 32,669 individuals from 29,348 households

2019 Wave 1
View Survey Questions
Dates: March 2019
Sample: 92,982 households
Respondents: 26,929 individuals from 24,886 households

2019 Wave 2
View Survey Questions
Dates: June/July 2019
Sample: 91,239 households
Respondents: 28,580 individuals from 26,661 households

2019 Wave 3
View Survey Questions
Dates: September/October 2019
Sample: 57,825 households
Respondents: 15,915 individuals from 15,029 households

*All respondents in these waves are from unique households.

Reference Articles (users must cite these in their work): 

Journal articles for 2018 surveys
Coibion, O., Gorodnichenko, Y., & Weber, M. (2022). Monetary policy communications and their effects on household inflation expectations. Journal of Political Economy, 130(6), 1537–1584. https://doi.org/10.1086/718982  

Journal articles for 2019 surveys
Coibion, O., Georgarakos, D., Gorodnichenko, Y., & Weber, M. (2023). Forward guidance and household expectations. Journal of the European Economic Association, 21(5), 2131–2171. https://doi.org/10.1093/jeea/jvad003  

Weber, M., Candia, B., Afrouzi, H., Ropele, T., Lluberas, R., Frache, S., Meyer, B., Kumar, S., Gorodnichenko, Y., Georgarakos, D., Coibion, O., Kenny, G., & Ponce, J. (2025). Tell Me Something I Don’t Already Know: Learning in Low‐ and High‐Inflation settings. Econometrica, 93(1), 229–264. https://doi.org/10.3982/ecta22764

Complementary to the Consumer Panel data, the Custom PanelView: Tax Rebate Survey (called “PanelView Surveys” in the Kilts portal) contains panelist reports of how they spent the 2008 federal tax rebate.

Years Available: This was a one-time custom survey from 2008.

Panel Size: The survey was fielded in waves. The number of responses for each wave varies between 900 and 34,000.

Complementary to the Consumer Panel data, the Custom PanelView: Tell Us More About You Surveys (called “PanelView Surveys” in the Kilts portal) contain additional demographic information about the panelists.

These surveys contain data about where household members were born, where they live now, additional information about education (e.g. college major), occupation, and purchase behavior in certain categories.

Years Available: These custom surveys are from 2008 and 2011. There are no plans for future updates.

Panel Size: The 2008 survey contains responses from 48,951 households. The 2011 survey includes responses from 56,258 households.