According to Wikipedia:

The purpose of the ritual was to establish the consummation of the marriage, either by actually witnessing the couple’s first sexual intercourse, or far more often symbolically, by leaving before consummation. It symbolized the community’s involvement in the marriage.

So in western Europe at least, people weren’t directly watching