A waiver was one of my original suggestions when we first started. This would follow very close to the turbine waiver. Basically you have to get an experienced FPV pilot to check you out every now and then to make sure you're current with the rules.
We have a few clubs here in the DC Metro area that do not allow FPV flying. This is for the same reasons you explain here. It's mostly due to the counties hand in their fields management. I can understand their fear since many of them have lost several fields due to development. I would like to leave it open to the field safety officer. If they feel it's something the field is equipped for then have at. There are many fields in my area that are just not equipped for FPV flying. Usually these are existing electric fields in close proximity to houses.