Institutions, Politics, Democracy, and Arrow's Impossibility Theorem

Institutions are the fundamental cause of economic growth — Douglass North

I have long wanted to document the profound impact of my first encounter with “New Political Economy” in a public finance course. I decided to organize my thoughts for the following reasons:

• After studying measure theory, I gained some foundational knowledge in functional analysis (though not strictly necessary)
• During a group meeting, the professor discussed the current state of policy implementation in China, mentioning that political economy indeed provides a crucial perspective. I strongly agree. Institutional incentives represent a fascinating research domain.
• The 2024 Nobel Prize was awarded for contributions to institutional economics and political economy
• The dramatic twists in the US presidential election
• Coming across related videos on Bilibili
• With busy times ahead, if I don’t write now, I won’t have time later

From Institutions to Political Economy

Two questions:

1. What is the origin of the state (what is the essence of public finance)?
2. What determines the development path of a state?

I often use this question as an example in my blog¹ — what is the origin of government? One school of thought is classical political economy, represented by social contract theory, public goods theory², and social welfare theory. The establishment of power collectives stems from the need for collective optimization.

Of course, how to measure social welfare itself is a philosophical question: minimum (veil of ignorance thought experiment³)? maximum? mean? absolute value? marginal value? designing weighted convex functions?

Beyond this, conceptions of justice are also philosophical issues, such as Amartya Sen’s concept of freedom of choice; outcome equality in planned economies; endowment equality in initial distributions…

Tax design resembles the trolley problem: one track has the poor, another the middle class, and another the wealthy — even considering that some might transform into Super Saiyans and overturn the table before being hit.

Before introducing another school of thought, let’s look at the next question—what determines the development of a state.

One major perspective is the resource endowment theory in international trade. Country A has 4 units of resource a and 2 units of resource b, but industry a is more profitable, so it naturally chooses industry a. Country B has only 0.5 units of resource a and 1 unit of resource b, making industry b more profitable, thus it develops industry b.

Therefore, even though Country A is superior in all resources compared to Country B, since it internally focuses on developing the most profitable industry A, it chooses industry A; similarly, Country B chooses industry B. This model (H-O comparative advantage) vividly explains why weaker countries with absolute disadvantages can still export goods to stronger countries. Professor Justin Yifu Lin proposed “New Structural Economics” based on this idea.

In other words, this school of thought assumes that national development is determined by regional resources.

The following distinctions about institutional economics are based on “Institutions and Economic Growth” (Yao Yang), a book about the dialogue between Professor Yao Yang and institutional economics master North. (Haven’t finished reading it yet, will supplement later.)

Institutional economics disagrees. For instance, North argues that factors like investment, education, and capital accumulation are not causes of economic growth but manifestations of growth. If merely accumulating production factors could achieve development, theoretically all countries could imitate this, but reality proves otherwise. In fact, efficient organizations are the primary drivers of development.

Meanwhile, resource endowment theory always assumes people rationally maximize their resource advantages. A simple counterexample is the partial emergence of deglobalization trends globally.

Example 1. Coase used transaction costs to explain the origin of firms — explaining corporate emergence from an institutional perspective. This demonstrates that institutional emergence shares commonalities rather than being uniquely generated by resource endowments.

Example 2. I prefer to explain institutional economics using an anthropological perspective — our cultural behaviors can be viewed as excessive rituals⁴. For example, cured meat: historically, cured meat served as preserved food for the New Year due to resource constraints, but now with abundant resources, its significance has transformed into an embodiment of New Year delicacies.

Example 3. Partial deglobalization trends. Trump desires manufacturing repatriation; China emphasizes national security, considering strategic deployment under extreme scenarios, aiming for comprehensive industrial chain development.

Example 4. Path dependence. For instance, railway gauge width and VHS format for videotapes were determined by technological limitations of their time, neither being optimal choices today, yet they persist. This institutional inertia is what North calls “history matters.”

Example 5. Formal meetings require suits; marriage involves bride price; visiting patients shouldn’t come empty-handed… These informal rules (a concept proposed by North) constantly constrain everyone. These institutions actually possess considerable enforcement flexibility; we cannot simply understand informal rules from a resource maximization perspective.

In other words, institutional evolution is mixed with too many irrational and non-material factors, especially cultural and religious ones. How to understand this non-optimal, non-fully rational institutional evolution? Institutional economics is divided into the new institutional economics school based on the active use of neoclassical economics (e.g., Dron) and the old institutional economics school that abandons neoclassical economics (e.g., Samuelson). (This is just one of North’s classifications.)

Some distinguishing criteria: the definition of institutions; the boundary between rational and irrational assumptions; whether institutional evolution is spontaneous, relatively exogenous or endogenous to the economy; the driving force of institutional change; the role of humans in institutional change; explaining institutions through economics or explaining economics through institutions…

In other words, rather than treating the collective as a unit, institutional economics focuses on the interaction and dynamic evolution of “individual—collective—institution—development.”

The distinction between old and new institutional economics is actually quite vague. Institutional economics and development economics have historically become a big concept that can encompass anything.

By the way, Yang Xiaokai⁵ was also influenced by Buchanan’s social choice theory, and his inframarginal analysis also focuses on the general equilibrium transmission path under the basic unit of individuals.

For reference, see “Development Economics: Marginal Analysis and Inframarginal Analysis (Introduction)”

Classical political economy views public finance as a spontaneous institution for collective welfare, but from the perspective of institutional economics, reality is not like that. Public finance can actually be seen as a form of collective decision-making. Collective policy is a form of institutional design.

It’s still debated in public finance what public finance actually is. Isn’t that fascinating?

Think about it, doesn’t the process of collective decision-making resemble a discipline—politics?

New institutional economics, in its later stages, evolved into new political economy.

One of the research points of new political economy—the deviation from individual preferences to collective decision-making.

What shocked me was this mathematical description from essence to means:

• What is fairness? We can’t clearly define it, so we first study the tool to promote fairness—public finance.
• What is democracy? We can’t clearly define it, so we first study the tool to practice democracy—collective decision-making.

Thus, social choice theory was born, such as principal-agent theory⁶, official promotion tournaments⁷, and inclusive growth. Today, I want to introduce the analysis related to voting in social choice theory. All collective decision-making can be seen as a form of voting.

Positioning the research scope to “making choices” is also one of the reasons for economic imperialism.

Internationally, many big names work on auction design. These micro-theory topics are almost entirely game theory and mathematical analysis.

Social Choice Theory

This section is based on Chapter 18 of “Advanced Microeconomics (Second Edition)” (Jiang Chundian). I basically just selectively copied and pasted.

For more accessible explanations of Arrow’s impossibility theorem, refer to this video:

Social Welfare Functional

In probability theory, we used sets to represent event outcome ranges. The term “functional” is “general” because it studies mappings from set spaces to set spaces; we study a class of mapping functions, or in other words, “general” refers to studying functions of functions; nonlinear functionals represent path optimization, refer to another blog post “Calculus of Variations Study Notes.”

Some advanced microeconomic reasoning relies on sets as the space for decision elements:

Let the set of social individuals be $I=\{ 1,2,\ldots,n \}, n\geqslant2$.

$x,y \in X$ are resource sets, allocated to individuals $a_i$, i.e., $\sum_i^n a_i=X_i$.

Preferences are as follows:

• $x\succeq_{i}y$: individual $i$ considers $x$ no worse than $y$.
• $x\sim_{i}y$: individual $i$ considers $x$ and $y$ indifferent.
• $x\succ_{i}y$: individual $i$ considers $x$ better than $y$.

As mentioned earlier, a functional is a mapping from space to space. A social welfare functional is a mapping from the space of individual joint preferences to the space of social preferences, denoted as

$$ \succeq^{F}=F(\succeq_{1},\succeq_{2},\ldots,\succeq_{n}) $$

Note, this is a mapping from the preference space of all individuals to social preferences, not necessarily a one-dimensional space ($R^1$). The joint preference space of individuals can be understood as the set of all individuals’ preference outcomes, and the social preference space can be understood as the final overall preference outcome space we should acknowledge.

Next, let’s illustrate with examples:

For a dictatorial society, social preferences are forced to conform to the preferences of one individual.

$$ x\succ_{h}y\quad\Rightarrow\quad x\succ y $$

Majority voting is a mapping where the minority follows the majority, comparing based on the aggregation of similar individual preferences.

$$ N(x\succ_{i}y)>N(y\succ_{i}x)\Rightarrow\quad x\succ y $$

$$ N(x\succ_{i}y)=N(y\succ_{i}x)\Rightarrow\quad x\sim y $$

May’s Theorem: The necessary and sufficient conditions for a social welfare functional to be majority voting are these three properties:

• Anonymity of majority voting means sequence identifiers $i$ within the same preference can be interchanged;
• Neutrality manifests as equal weight for each person’s vote;
• Positive responsiveness manifests as social preference changing direction aligning with some individuals’ preference direction changes.

Majority voting seems ideal, but real-world voting encounters various problems; details can be found in the “voting paradox” (also called Condorcet paradox).

Since preferences are based on ordinal utility theory (comparative), infinite comparison cycles might occur. Real voting is more complex; for example, swing states are crucial in US elections, and bandwagon effects or retaliatory mindsets also influence voting outcomes.

Therefore, we need to establish axiomatic conditions for social welfare functionals:

• Unrestricted domain: Every individual preference yields social preferences (social preferences must at least satisfy transitivity and completeness)
• Non-dictatorship: Supplement: anonymity imposes stronger constraints than non-dictatorship.
• Pareto principle: $\forall i\in A, x\succ_{i}y \rightarrow x\succ y$
• Independence: $\forall x, y\in X$ when $x\succ y$ holds, any $i\in I$’s preferences outside $\{x, y\}$ do not affect the social preference outcome $x\succ y$. Independence actually simplifies the model, limiting voters’ considered information to current choices; this constraint is weaker than neutrality.

Now we can introduce a voting scheme — Borda count.

The core is ranking: each person ranks every option from 1 to n $c_i(x)$, with better options ranked higher,

then summing each option’s ranks. $$ c(x)=\sum_{i=1}^{n}c_{i}(x) $$ Smaller totals indicate stronger social preference.

$$ c(x)<c(y)\quad\Leftrightarrow\quad x\succ y $$

Seems reliable? But Borda count violates independence:

To discuss independence, we introduce a third choice $z$

Assume two individuals with these preferences:

$$x\succ_1z\succ_1y,\quad y\succ_2x\succ_2z$$ Pairwise ranking comparison for counting:

$$ c_{1}(x)=1, c_{2}(x)=2\quad\Rightarrow\quad c(x)=c_{1}(x)+c_{2}(x)=3$$

$$c_{1}(y)=3, c_{2}(y)=1\quad\Rightarrow\quad c(y)=c_{1}(y)+c_{2}(y)=4 $$

Thus social preference $x\succ y$.

But suppose we change $z$’s position:

$$ x\succ_{1}y\succ_{1}z,\quad y\succ_{2}z\succ_{2}x $$ Resulting counts:

$$ c_1(x)=1, c_2(x)=3\quad\Rightarrow\quad c(x)=c_1(x)+c_2(x)=4 $$ $$ c_1(y)=2, c_2(y)=1\quad\Rightarrow\quad c(y)=c_1(y)+c_2(y)=3 $$

Social preference reverses to $y\succ x$!

Intuitively, I feel this can be understood as: Borda count violates independence because it’s based on ranking scores; part of this ranking score actually measures x and y’s preference scores relative to z.

Arrow’s Impossibility Theorem

Decisive Coalition

We have many expectations for collective decision-making, so we wonder if there is a voting method that can satisfy all our demands. Arrow’s Impossibility Theorem proves that—when there are three or more options, this expectation is impossible.

Decisive preference refers to situations where a specific group has the power to influence social preferences.

Coalition $K \in I$

$$ \forall i\in K, x\succ_iy;\quad\forall j\in I\cap K^c, y\succ_jx\quad\Rightarrow x\succ y $$ That is, $I$ is divided into two preference positions, $K$ and $I\cap K^c$, and even if their views are opposite, the final social preference may reflect the preference of $K$.

• Swing states in U.S. elections.
• The moment when votes are tied, the last vote becomes the tipping point.
• Celebrities taking a stance and their fans following.

When the preference of coalition $K$ directly determines the social preference for options $(x,y) \subset X$, then $K$ is a decisive coalition under the social welfare functional, denoted as $D_K(x,y)$.

Again, note that the functional here is a mapping from space to space. Since $K$ determines social preferences, the preference of this coalition is the mapping from the joint preference space of individuals to social preferences. Naturally, when $K$ consists of only one person, it is a dictatorship.

When unrestricted domain, Pareto principle, and independence are satisfied, the decisive coalition has the following properties:

1. $(x,y) \subset X$, there is a decisive coalition $D_K(x,y)$, then for $(u,v) \subset X$, there is also $D_K(u,v)$.
2. $K\subseteq I,J\subseteq I$ (the symbol means subset or equal) are both decisive coalitions, then $K\cap J$ is also a decisive coalition.
3. For any $K\subset I$, either $K$ or $I\cap K^c$ is a decisive coalition⁸.
4. $K\subset I$ is decisive, $J\subset I$ is any coalition containing $K$, $J\subset K$, then $J$ is also decisive.
5. $K\subseteq I$ is decisive, and the number of members is greater than 1, then there exists a proper subset $J\subset K$ that is decisive.

The proofs of these properties are not recorded here.

Arrow’s Impossibility Theorem

This is Arrow’s doctoral thesis……

Definition: When the number of choices is no less than 3 (e.g., the U.S. election is essentially a two-party system, so it doesn’t apply), there is no social welfare functional that simultaneously satisfies unrestricted domain, non-dictatorship, Pareto principle, and independence.

The proof essentially uses the following properties:

1. $(x,y) \subset X$, there is a decisive coalition $D_K(x,y)$, then for $(u,v) \subset X$, there is also $D_K(u,v)$.
2. $K\subseteq I,J\subseteq I$ (the symbol means subset or equal) are both decisive coalitions, then $K\cap J$ is also a decisive coalition.
3. For any $K\subset I$, either $K$ or $I\cap K^c$ is a decisive coalition⁸.
4. $K\subset I$ is decisive, $J\subset I$ is any coalition containing $K$, $J\subset K$, then $J$ is also decisive.
5. $K\subseteq I$ is decisive, and the number of members is greater than 1, then there exists a proper subset $J\subset K$ that is decisive.

For example, proving that a social welfare functional satisfying unrestricted domain, Pareto principle, and independence must be dictatorial:

The entire society $I$ can be seen as a decisive coalition, and based on property 5, it can be infinitely decomposed into one. Then, using property 2, preferences are classified, verifying that this individual is indeed a dictator relative to other options.

Recommended references:

• When Democracy Meets Logic—Arrow’s Impossibility Theorem - A Distant Star’s Article - Zhihu
• Why is Arrow’s Impossibility Theorem Counterintuitive?
• Given Arrow’s Impossibility Theorem, Why Does Microeconomics Still Study Social Welfare Functions?

Personal understanding:

You can think of the dictatorship proof in Arrow’s Impossibility Theorem as the “last vote” that tips the scale. But the understanding needs to go deeper:

Remember: Arrow’s Impossibility Theorem studies the mapping from individual preference space to social preference space.

Because Arrow wants to solve the best mapping from individual preference space to social preference space, the most ideal democracy is one where there is no “tipping vote.”

Absolute equality, where participation is through ranking rather than cardinal utility aggregation. Of course, there are already theorems proving the existence of transformations between ordinal and cardinal preferences.

Assuming everyone votes once under unknown circumstances, there will always be one person whose preference determines (even if they don’t know they are the dictator of this game) the entire society’s preference, making the vote dictatorial. In other words, the mapping is not ideal and does not truly reflect the preferences of the majority.

Assuming multi-stage games, people may change their votes based on retaliation, altering their preferences, which violates the independence assumption. If everyone changes their initial preferences to win, the mapping no longer reflects the best outcome.

Ultimately, Arrow’s Impossibility Theorem aims to prove this point—mapping individual preferences to social preferences is fraught with difficulties.

Relaxing Assumptions

One proof is that when the number of participants approaches infinity, Arrow’s Impossibility Theorem has a solution, but it doesn’t seem to allow for asymptotic properties like in econometrics.

Relaxing the unrestricted domain assumption, limiting preferences to single-peaked, without multi-peaked preferences.

• Single-peaked preferences: If voters deviate from their most preferred outcome, regardless of the direction, their utility decreases.
• Double-peaked preferences: If voters deviate from their most preferred outcome, their utility first decreases and then increases.

Essentially, this excludes the voting paradox (Condorcet paradox) scenario. Single-peaked preferences need to be convex in a closed continuous interval.

Terence Tao’s Proof

Terence Tao also provided his own proof:

Terence Tao’s Proof

Analysis can be referenced:

Proof of Arrow’s Impossibility Theorem (via Terence Tao)

A professor of advanced probability theory once commented—he felt sorry that Terence Tao spent part of his energy on non-world-class problems😂.