When the wave hits the hole, a portion of it continues into the chamber and some of the waves are reflected. The waves pass through the chamber, impact the rear wall of the muffler and bounce back out of the hole. The length of the chamber is calculated such that the wave exits the cavity immediately after the next wave is reflected from outside the chamber. Ideally, the high pressure portion of the wave came from the chamber will align with the low pressure portion of the wave that was reflected from outside of the chamber wall and the two waves will cancel each other out.
The animation below shows how the resonator works in a simplified muffler.
Waves canceling inside a simplified muffler
In fact, the sound from the engine is a mixture of sounds of many different frequencies, and since many of these frequencies are dependent on the engine speed, the sound almost never reaches the correct frequency. The resonator is designed to work best in the frequency range where the engine makes the most noise; however, even if the frequency is not the frequency at which the resonator is tuned, it still produces some destructive interference.
Some cars, especially luxury cars where quiet operation is a key feature, have another component in the exhaust that looks like a muffler, but is called a resonator. This device works just like the resonator chamber in the muffler -- the dimensions are calculated so that the waves reflected by the resonator help cancel out certain frequencies of sound in the exhaust.
There are other features in this muffler that help it reduce the sound level in different ways. The main body of the muffler consists of three layers: two thin metal layers with a thick, slightly insulating layer between them. This allows the body of the muffler to absorb some pressure pulses. Moreover, the inlet and outlet tubes entering the main chamber are perforated with holes. This allows thousands of tiny pressure pulses to bounce in the main chamber, which cancels out to some extent, in addition to being absorbed by the muffler housing.