party2d/docs/DATA_CODING_PRINCIPLES.md

Data Coding Principles for 4D Latent Trait Model

V4 Implementation (Current Version)

As of V4 (2025-11-18), manifesto items implement the bipolar bridge structure described in this document.

Key Changes from V3.x:

• ✅ Manifesto items now load on TWO dimensions (bipolar bridges)
• ✅ Data format: type_high and type_low columns replace single type
• ✅ Stan model: Unified Gamma_man matrix replaces per-dimension arrays
• ✅ Measurement consistency: Manifesto items match expert data structure

Why V4?

• Better identification (each observation informs two dimensions)
• Estimated correlations (not imposed by construction)
• No double-counting (each quasi-sentence counted once)

See CHANGELOG.md for full V4 migration details.

---

Model Structure Overview

The model estimates four unipolar latent dimensions:

• pro_market: Support for market liberalization
• pro_welfare: Support for welfare state expansion
• cosmopolitan: Support for internationalism, diversity, openness
• traditional: Support for nationalism, security, traditional values

These are separate dimensions, not two bipolar scales. Correlations between dimensions (e.g., cosmopolitan-traditional) are estimated empirically, not imposed by construction.

---

Item Types and Loading Structure

1. Bipolar Bridge Items

Definition: Items where the sample includes mentions of BOTH sides of an issue, and "positive" counts mentions favoring one pole.

Structure:

• sample = mentions of issue (any direction)
• positive = mentions favoring one pole
• positive/sample ratio varies from 0 to 1

Loading: Should load on ONE dimension only

Examples:

Manifesto Data:

var: "Multiculturalism"
type: "cosmopolitan"
sample: per607 (pro-multiculturalism) + per608 (anti-multiculturalism)
positive: per607 (pro-multiculturalism)

• High ratio → high cosmopolitan (party favors multiculturalism)
• Low ratio → low cosmopolitan (party opposes multiculturalism)
• Anti-multiculturalism is implicitly measured as (sample - positive)

PolDem Data:

var: "Immigration (Media)"
type: "cosmopolitan"
sample: all immigration mentions (direction != 0)
positive: pro-immigration mentions (direction > 0)

• High ratio → high cosmopolitan (media coverage shows party supporting immigration)
• Low ratio → low cosmopolitan (media coverage shows party opposing immigration)

2. Why One Loading Suffices for Bipolar Items

Question: Shouldn't anti-immigration also load on traditional?

Answer: No, because:

1. Both poles are already captured: The bipolar structure means low cosmopolitan (anti-immigration) is automatically measured
2. Avoids double-counting: Each mention/quasi-sentence contributes to exactly ONE item
3. Empirical correlations emerge naturally: If anti-immigration parties also score high on nationalism/law-and-order, the posterior correlation between cosmopolitan and traditional will reflect this
4. More flexible model: Cosmopolitan-traditional relationship is estimated, not imposed

Imposed vs. Estimated Correlation:

• If we double-load immigration on both cosmopolitan (negative) and traditional (positive), we force them to be opposites
• By loading only on cosmopolitan, we let the data reveal whether anti-immigration parties are also nationalist (empirical question)

---

Coding Decision Rules

Rule 1: Each Manifesto Code Appears in ONE Item Only

Good (current structure):

"Multiculturalism" (cosmopolitan):
  - per607 (Positive), per608 (Negative)

"National Identity" (traditional):
  - per601 (Positive), per107 (Negative)

• per607/per608 only in cosmopolitan
• per601/per107 only in traditional
• Correlation between dimensions is empirical

Bad (double-loading):

"Multiculturalism" (cosmopolitan):
  - per607 (Positive), per601 (Negative)

"National Identity" (traditional):
  - per601 (Positive), per607 (Negative)

• per601 and per607 counted twice
• Imposes perfect negative correlation between cosmopolitan/traditional

Rule 2: Stance Assignment Within Items

Within each item (var), codes are assigned stance based on:

• Positive: Codes indicating support for the item's construct
• Negative: Codes indicating opposition to the item's construct

Example - "Internationalism" (cosmopolitan):

• per107 (Internationalism positive): stance = "Positive"
• per109 (Internationalism negative): stance = "Negative"
• Result: High per107 / low per109 → high cosmopolitan score

Rule 3: PolDem Direction Mapping

PolDem uses direction variable (-1, 0, +1):

• direction > 0: Support for the issue as coded
• direction < 0: Opposition to the issue
• direction == 0: Ambivalent (exclude from analysis)

Aggregation:

poldem_processed %>%
  filter(direction != 0) %>%  # exclude neutral
  group_by(party, year, country, issue_cat) %>%
  summarise(
    sample = n(),  # all non-neutral mentions
    positive = sum(direction > 0)  # supportive mentions only
  )

---

Special Cases

Immigration (Direction Ambiguity)

Codebook says: "Opposition to restrictive immigration"

Interpretation needed: Does direction = +1 mean:

• A) Support for "opposition to restrictions" → pro-immigration (cosmopolitan)
• B) Support for "restrictions" → anti-immigration (traditional)

Resolution: Must manually inspect sample sentences before finalizing coding.

If interpretation A is correct:

issue_cat == "immig" & direction > 0  → positive for cosmopolitan
issue_cat == "immig" & direction < 0  → negative for cosmopolitan

If interpretation B is correct:

issue_cat == "immig" & direction > 0  → negative for cosmopolitan
issue_cat == "immig" & direction < 0  → positive for cosmopolitan
# (REVERSED)

Europe/Euro Items

EU integration naturally maps to cosmopolitan-traditional dimension:

Manifesto Data:

• Add new items using per108 (EU integration positive) and per106 (EU integration negative)
• Create separate vars: "EU Integration Support" (cosmopolitan), "Euroskepticism" (traditional)

PolDem Data:

"EU Integration Support (Media)" (cosmopolitan):
  issue_cat = "europe" or "euro"
  sample = all mentions
  positive = direction > 0 (pro-EU)

"Euroskepticism (Media)" (traditional):
  issue_cat = "europe" or "euro"
  sample = all mentions
  positive = direction < 0 (anti-EU)

Note: Same sentences contribute to BOTH items, but counting opposite directions. This creates natural negative correlation between cosmopolitan/traditional.

Alternative approach (cleaner, recommended): Load only on cosmopolitan:

"EU Position (Media)" (cosmopolitan):
  issue_cat = "europe" or "euro"
  sample = all mentions
  positive = direction > 0

This is sufficient if we treat EU as a bipolar cosmopolitan item.

---

Data Structure Requirements

Manifesto Data Format (party-year-var level)

Each row represents one item for one party-year:

party	country	year	var	type	sample	positive	project
211	DE	2013	Multiculturalism	cosmopolitan	45	23	Manifesto
211	DE	2013	National Identity	traditional	67	58	Manifesto
211	DE	2013	Economic Intervention	pro_welfare	102	78	Manifesto

• var: Item name (e.g., "Multiculturalism", "Economic Intervention")
• type: Dimension it loads on (pro_market, pro_welfare, cosmopolitan, traditional)
• sample: Total quasi-sentences mentioning this issue
• positive: Quasi-sentences with positive stance toward this item

PolDem Data Format (same structure)

party	country	year	var	type	sample	positive	project
211	DE	2013	Immigration (Media)	cosmopolitan	23	8	PolDem
211	DE	2013	Nationalism (Media)	traditional	15	12	PolDem

Combined using bind_rows() to create unified dataset.

---

Summary

1. Bipolar items load on one dimension only - the ratio captures both poles
2. Each manifesto code appears in exactly one item - no double-counting
3. Correlations between dimensions are estimated, not imposed - more flexible model
4. Direction reversals are handled within items - via stance assignment (Manifesto) or direction sign (PolDem)
5. All items must allow varying positive/sample ratios - mix of positive and negative stances required

This structure preserves the conceptual independence of the four dimensions while allowing the data to reveal their empirical relationships.