In my previous village in Oxfordshire, we were plagued by many hare coursers. These included organised criminals from London and Birmingham who beat up several concerned locals and threatened many of us with Hollywood levels of violence (to us specifically and our families and houses more generally). The police weren't very helpful.... If you wish to go hare coursing, all you need to do is (1) pick a large field with several hares and more than one way out for a vehicle, (2) turn up with a couple of 4x4s and more than one dog (so you can bet on the outcome). The combination of secrecy and simplicity makes it difficult for the antis to organise themselves effectively. Not to mention the very real threat of serious harm if you were to object.
Protesting against fox-hunting is a much more genteel occupation. It's easy to find out where the hunt is starting from and where they hope to go. Those involved are often dressed in bright colours, make loud braying noises and aren't particularly covert. Horses and horse boxes are conspicuous. The (pro) hunt followers usually include a number of muscular individuals wearing paramilitary clothing who like to work up an appetite for lunch by beating up a vegan. However, mostly, it's no worse than (and possibly a substitute for) a Saturday afternoon's rugby.
I don't have a problem with shooting or hunting if you eat what you kill (and minimise the distress you cause to the prey). Similarly, shooting to control the numbers of wild animals (eg wild deer which have no natural predators in the UK) is probably the only way to stop numbers getting further out of control. Both of these could be described as ethical activities, even if the participants enjoy the process.
However, if the way in which you hunt is modified to maximise the hunters' enjoyment at the expense of the prey (driven shoots, hare-coursing, fox-hunting), and even more if the prey are bred specifically to be slaughtered (any game birds, especially pheasant, partridge and grouse), I think there's a big ethical issue.