Group Actions and Some Applications

Proposition 1. Suppose that $G$ acts on $X$ through an action $\star$. Let $g \in G$. Define a map $f_g : X \to X$ by
$$
f_g(x) = g \star x .
$$
Then $f_g \in \operatorname{Sym}(X)$.

Proof: It is easy to verify.

Definition 1. Suppose that $G$ acts on $X$ through an action $\star$. Define a map $\rho : G \to \operatorname{Sym}(X)$ by $\rho(g) = f_g$. It is easy to verify that $\rho$ is a homomorphism from $G$ to $\operatorname{Sym}(X)$. Such a homomorphism is called a permutation representation of $G$ on $X$.

Remark. Given an action $\star$ of $G$ on $X$, we have a representation of $G$ on $X$. Conversely, given any representation $\rho$ of $G$ on $X$, we have an action $\star$ of $G$ on $X$ defined by
$$
g \star x = \rho(g)(x)
$$
such that the corresponding representation is exactly $\rho$. It is easy to verify this.

Thus there is a faithful way to view an action as a representation, and a representation as an action, and they are related by
$$
g \star x = \rho(g)(x) .
$$

Definition 2. Let $\star$ be an action of $G$ on $X$ with the corresponding representation $\rho$. Then the kernel of $\rho$ is
$$
\ker \rho
= \{g \in G \mid \rho(g) = 1_G\}
= \{g \in G \mid \rho(g)(x) = x,\forall x \in X\}
= \{g \in G \mid g \star x = x,\forall x \in X\} .
$$
Therefore, $\{g \in G \mid g \star x = x,\forall x \in X\}$ is a normal subgroup of $G$. This subgroup is called the isotropy group or the stabilizer of the action $\star$ of $G$ on $X$, and it is denoted by $\operatorname{Stab}(G,X)$.

By the fundamental theorem of homomorphisms, we have the following proposition.

Proposition 2. $G/\operatorname{Stab}(G,X)$ is isomorphic to the subgroup $\rho(G)$ of $\operatorname{Sym}(X)$, where $\rho$ is the corresponding representation.

Remark. Suppose that $G$ acts on $X$ through an action $\star$. Then we have
$$
\operatorname{Stab}(G,X) = \bigcap_{x \in X} \operatorname{Stab}(x) ,
$$
where
$$
\operatorname{Stab}(x) = \{g \in G \mid g \star x = x\}
$$
is the stabilizer of $x \in X$.

Definition 3. Let $G$ be a group which acts on $X$ through an action $\star$. An element $x \in X$ is called a fixed point if $g \star x = x$ for all $g \in G$. Thus, $x$ is a fixed point if and only if $\operatorname{Stab}(x) = G$, or equivalently, $\operatorname{Orb}(x) = \{x\}$. The set of all fixed points of the action $\star$ is denoted by $X^G$, and it is called the fixed point set of the action. Thus,
$$
X^G = \{x \in X \mid g \star x = x,\forall g \in G\} .
$$

Remark. An element $x \in X$ is a fixed point if and only if the orbit $\operatorname{Orb}(x)$ is a singleton. Thus $X^G$ is the union of all those orbits which are singletons.

Proposition 3. Let $A$ be a set obtained by choosing one and only one member from each orbit different from a singleton. Suppose that $X$ is finite. Then we have the equation
$$
|X| = |X^G| + \sum_{x \in A} [G : \operatorname{Stab}(x)] .
$$
Here again, each term under summation in the right-hand side is greater than $1$ and divides $|G|$.

This equation is also called the class equation or class formula for the action $\star$.

Proof: If $x \in A$, then $\operatorname{Orb}(x) \neq \{x\}$. Further,
$$
\operatorname{Orb}(x) \cap \operatorname{Orb}(y) = \varnothing
$$
for all $x,y \in A$ with $x \neq y$.

Since $X$ is the union of all orbits,
$$
X = X^G \cup \left( \bigcup_{x \in A} \operatorname{Orb}(x) \right) .
$$
Since distinct orbits are disjoint and $|\operatorname{Orb}(x)| = [G : \operatorname{Stab}(x)]$, we have
$$
|X| = |X^G| + \sum_{x \in A} [G : \operatorname{Stab}(x)] .
$$

Now we give some examples and applications to the structure theory of finite groups.

Example 1. Let $G$ be a group and let $X = G$. Define
$$
g \star x = gxg^{-1}, \qquad g,x \in G .
$$
It is easy to verify that $\star$ is an action of $G$ on $G$. This action is called the inner conjugation of $G$ on $G$.

The isotropy group of the inner conjugation of $G$ on $G$ at $x$ is
$$
\operatorname{Stab}(x)
= \{g \in G \mid gxg^{-1} = x\}
= \{g \in G \mid gx = xg\}
= C_G(x) ,
$$
where $C_G(x)$ is the centralizer of $x$ in $G$.

The isotropy group of the inner conjugation of $G$ on $G$ is
$$
\ker \rho
= \operatorname{Stab}(G,X)
= \{g \in G \mid gxg^{-1} = x,\forall x \in G\}
= \{g \in G \mid gx = xg,\forall x \in G\}
= C(G) ,
$$
where $\rho$ is the corresponding representation, and $C(G)$ is the center of $G$.

The representation $\rho$ determined by the inner conjugation of $G$ on $G$ is given by $\rho(g) = f_g$, where $f_g$ is the inner automorphism of $G$ determined by $g$. Thus $\rho(G)$ is the group $\operatorname{Inn}(G)$ of all inner automorphisms of $G$. By the fundamental theorem of homomorphisms, we have
$$
G/C(G) \cong \operatorname{Inn}(G) .
$$

The orbit through $x \in G$ of the inner conjugation of $G$ on $G$ is the set
$$
\operatorname{Orb}(x) = \{gxg^{-1} \mid g \in G\}
$$
of all conjugates of $x$ in $G$, which is also called the conjugacy class of $G$ determined by $x$.

The fixed point set of the inner conjugation of $G$ on $G$ is the set
$$
G^G = \{x \in G \mid gxg^{-1} = x,\forall g \in G\} = C(G) .
$$

Let $A$ be a set obtained by choosing one and only one member from each orbit different from a singleton. By the class formula, we have
$$
|G| = |C(G)| + \sum_{x \in A} [G : \operatorname{Stab}(x)] .
$$
This equation is also called the classical class equation of $G$.

Example 2. Let $G$ be a group and let $X$ be the set $S(G)$ of all subgroups of $G$. Define
$$
g \star H = gHg^{-1}, \qquad g \in G,\ H \le G .
$$
It is easy to verify that $\star$ is an action of $G$ on $S(G)$. This action is called the inner conjugation of $G$ on the subgroups of $G$.

The isotropy group of the inner conjugation of $G$ on the subgroups of $G$ at $H$ is
$$
\operatorname{Stab}(H)
= \{g \in G \mid gHg^{-1} = H\}
= N_G(H) ,
$$
where $N_G(H)$ is the normalizer of $H$ in $G$.

The orbit through $H \in S(G)$ of the inner conjugation of $G$ on the subgroups of $G$ is the set
$$
\operatorname{Orb}(H) = \{gHg^{-1} \mid g \in G\}
$$
of all conjugates of $H$ in $G$, which is also called the conjugacy class of $G$ determined by $H$.

The fixed point set of the inner conjugation of $G$ on the subgroups of $G$ is the set
$$
S(G)^G
= \{H \in S(G) \mid gHg^{-1} = H,\forall g \in G\}
= \{H \in S(G) \mid H \lhd G\} ,
$$
which is the set of all normal subgroups of $G$.

Let $A$ be a set obtained by choosing one and only one member from each orbit different from a singleton. By the class formula, we have
$$
|S(G)| = |S(G)^G| + \sum_{H \in A} [G : N_G(H)] .
$$

Example 3. Let $G$ be a group and let $X = G/H$. Define
$$
g \star xH = gxH, \qquad g,x \in G .
$$
It is easy to verify that $\star$ is an action of $G$ on $G/H$. This action is called the left multiplication of $G$ on $G/H$.

The isotropy group of the left multiplication of $G$ on $G/H$ at $xH$ is
$$
\operatorname{Stab}(xH)
= \{g \in G \mid gxH = xH\}
= \{g \in G \mid x^{-1}gx \in H\}
= xHx^{-1} .
$$

The isotropy group of the left multiplication of $G$ on $G/H$ is
$$
\ker \rho
= \operatorname{Stab}(G,G/H)
= \bigcap_{x \in G} \operatorname{Stab}(xH)
= \bigcap_{x \in G} xHx^{-1} ,
$$
where $\rho$ is the corresponding representation. Thus $\bigcap_{x \in G} xHx^{-1}$ is a normal subgroup of $G$. Further, it is the largest normal subgroup of $G$ contained in $H$. This subgroup is called the core of $H$ in $G$, and it is denoted by $\operatorname{Core}_G(H)$. By the fundamental theorem of homomorphisms,
$$
G/\operatorname{Core}_G(H)
$$
is isomorphic to a subgroup of $\operatorname{Sym}(X)$.

Now we apply the theory of group actions to the structure theory of finite groups.

Proposition 4. Let $G$ be a group of order $p^n$, where $n \geq 1$ and $p$ is a prime. Let $H$ be a nontrivial normal subgroup of $G$. Then
$$
H \cap C(G) \neq \{e\} .
$$
In particular,
$$
C(G) \neq \{e\} .
$$

Proof: Since $H \lhd G$, $G$ acts on $H$ through inner conjugation. The fixed point set is
$$
H^G
= \{h \in H \mid ghg^{-1} = h,\forall g \in G\}
= \{h \in H \mid gh = hg,\forall g \in G\}
= H \cap C(G) .
$$

Let $A$ be a set obtained by choosing one and only one member from each orbit different from a singleton. Consider the class formula:
$$
p^r = |H| = |H \cap C(G)| + \sum_{x \in A} [G : \operatorname{Stab}(x)] .
$$
Each term under summation in the right-hand side is greater than $1$ and divides $|G|$.

Since $H \neq \{e\}$, we have $r \geq 1$. Thus the left-hand side is divisible by $p$, and the second term on the right-hand side is also a multiple of $p$. Hence $p$ divides $|H \cap C(G)|$. Since $e \in H \cap C(G)$, it follows that $H \cap C(G)$ contains at least $p$ elements.

Taking $H = G$, we obtain $C(G) \neq \{e\}$.

Corollary. Let $G$ be a group of order $p^2$, where $p$ is a prime. Then $G$ is abelian.

Proof: Consider $C(G)$. By the above proposition, $|C(G)| = p$ or $p^2$. If $|C(G)| = p^2$, then $C(G) = G$, and so $G$ is abelian.

Now we show that $|C(G)| \neq p$. Suppose that $|C(G)| = p$. Then
$$
|G/C(G)| = p ,
$$
and so $G/C(G)$ is cyclic. Hence $G$ is abelian, and therefore $G = C(G)$. So
$$
|C(G)| = |G| = p^2 ,
$$
which is a contradiction.

Proposition 5. Let $G$ be a group of order $p^n$, where $n \geq 1$ and $p$ is a prime. Let $H$ be a subgroup of $G$ with $H \neq G$. Then
$$
H \neq N_G(H) .
$$

Proof: The proof is by induction on $n$.

If $n = 1$, then $G$ is cyclic of prime order. Hence $H \neq G = N_G(H)$.

Assume that the result is true for groups of order less than $p^n$. Let $G$ be a group of order $p^n$.

If $C(G) \nsubseteq H$, then there exists $x \in C(G)$ such that $x \notin H$. Since $xH = Hx$, we have $x \in N_G(H)$. Thus $H \neq N_G(H)$.

If $C(G) \subseteq H$, consider the group $G/C(G)$. By Proposition 4, $C(G) \neq \{e\}$, and hence
$$
|G/C(G)| < p^n .
$$
By the induction hypothesis, for the subgroup $H/C(G)$ of $G/C(G)$ with $H/C(G) \neq G/C(G)$, we have
$$
N_{G/C(G)}(H/C(G)) \neq H/C(G) .
$$
Then there exists $x \in G \setminus H$ such that
$$
\overline{xHx^{-1}} = \overline{H} .
$$
By the correspondence theorem, we have
$$
xHx^{-1} = H ,
$$
which implies that $x \in N_G(H)$. Therefore $H \neq N_G(H)$.

Corollary. Let $G$ be a group of order $p^n$, where $n \geq 1$ and $p$ is a prime. Let $H$ be a subgroup of $G$ of order $p^{n-1}$. Then $H$ is a normal subgroup of $G$.

Proof: By the above proposition,
$$
H \subset N_G(H)
$$
and
$$
H \neq N_G(H) .
$$
Since $|H| = p^{n-1}$ and $N_G(H)$ is also a subgroup of $G$ whose order is greater than $p^{n-1}$ and divides $p^n$, it follows that
$$
|N_G(H)| = p^n .
$$
Thus $N_G(H) = G$, and therefore $H \lhd G$.

Proposition 6. Let $G$ be a finite group, and let $p$ be the smallest prime dividing $|G|$. Then every subgroup of $G$ of index $p$ is normal in $G$.

Proof: Let $H$ be a subgroup of $G$ of index $p$, where $p$ is the smallest prime dividing $|G|$. Then $G/H$ contains $p$ elements. Consider the action $\star$ of $G$ on $G/H$ defined by
$$
g \star xH = gxH .
$$
The isotropy group of this action is $\operatorname{Core}_G(H)$, which is the largest normal subgroup of $G$ contained in $H$. Also,
$$
G/\operatorname{Core}_G(H)
$$
is isomorphic to a subgroup of $\operatorname{Sym}(G/H) \cong S_p$.

Since
$$
|G/\operatorname{Core}_G(H)|
= [G:H][H:\operatorname{Core}_G(H)]
= pq ,
$$
we have
$$
pq \mid p! \Rightarrow q \mid (p-1)! .
$$
Since $p$ is the smallest prime dividing $|G|$ and $q$ divides $|G|$, we must have $q = 1$. Thus
$$
H = \operatorname{Core}_G(H) \lhd G .
$$

Corollary. If $G$ has no subgroups of index $2$, then every subgroup of index $3$ must be normal in $G$.

Proof: Let $H$ be a subgroup of $G$ of index $3$. Consider $\operatorname{Core}_G(H)$. By the fundamental theorem of homomorphisms,
$$
G/\operatorname{Core}_G(H)
$$
is isomorphic to a subgroup of $\operatorname{Sym}(G/H) \cong S_3$.

If $G/\operatorname{Core}_G(H)$ is isomorphic to $S_3$, then since $S_3$ has a subgroup of index $2$, $G$ must also have a subgroup of index $2$ by the correspondence theorem, contradicting the hypothesis.

Since
$$
|G/\operatorname{Core}_G(H)| = [G:H][H:\operatorname{Core}_G(H)] \geq 3 ,
$$
it follows that
$$
|G/\operatorname{Core}_G(H)| = 3 ,
$$
which implies that
$$
H = \operatorname{Core}_G(H) \lhd G .
$$

The cover image of this article was taken in Singapore.