I'd tell Barney Fife that he has a good point. I"ll be sure not to mow my lawn when kids are near by. Then I'd mow when I want, and see what he says. If he comes back, I'll just say that I stop when the kids come by. This is the only time that I have available for mowing.
If he still complains, have him mow it as others have suggested.