You know how when you pop bubble wrap, you squeeze it until part of the bubble fails and ruptures and all the air now can escape? It's that, but replace the bubble with your skin and the air inside the bubble with icky grossness.
Also that opening you just made? It goes two ways, and bacteria just LOVE openings in your skin, so try to resist the urge to pop.
There is a thin layer of epidermis that is rupturing essentially. Pressure builds up till it finds the path of least resistance out the top of the pore/zit.