Anarchism rejects coercion. Pacifism rejects violence, including coercion. They go hand-in-hand. Why would you expect anarchism to be the opposite of pacifism?
Probably because its easy to conflate Anarchism, a clear ideology, with anarchy, a state of social/political/governmental organization that may coincide with Anarchist ideology, but may also more closely resemble Hobbes' state of nature: "solitary, poor, nasty, brutish, and short."