They make these yellow plastic hanging bee & wasp traps that you put some juice in and the crawl in and can't get out and die. They sell them at hardware stores. They are most effective in the early spring. You try to catch the queen before she breeds. You can use them all summer though.
Or you can just swat them with a flyswatter and hope that you don't miss.