...Trail Guides, Backpacker Magazine, detailed Trip Reports in sensitive areas, and GPS Way Point files on the Internet :(
These things drive people to places they would not find on their own, and it puts unneeded pressure on the wilderness.
I agree with Art, not a simple cut and dry solutions.
Generally I leave them be, but in some instances I knock them down or even do more creative things with them.
Sometimes I follow faint Indian trails that have cairns on them. I don't need the cairns to prevent getting lost -- I know where I am going, but they sometimes help me walk the easiest routes. I often hike in trail-less areas, and it is not uncommon to come across many, many cairns; many which end up in places that cannot be easily navigated, or is a much more difficult route. Be skeptical of them.
An example was a trip I did last year. I wanted to hike to a summit and there were no trails on my map. I found a ridgeline that looked promising. When I got near it, there was a sign that warned, "use trail not maintained" at the mouth of a canyon. The trail went up the canyon and there were plenty of cairns. But the route looked too difficult versus climbing out of the canyon to the ridgeline. So after about a 1/2 mile of canyon and boulder work, following cairns, I climbed up to the ridge and found an easy trail with zero cairns on it and was able to get to the summit.
For the most part, I don't mind cairns marking old and rarely used trails. I object to new trails being made, and cairns creating additional foot traffic that results in a new trail.
Interestingly that Steve Roper, in "The Sierra High Route" says, "I recommend that High Route backpackers dismantle ducks wherever they are found -- unless there's a very good reason for them." The point is that the areas of the route that are cross country should remain cross country routes, and no defined trail should be constructed either by plan or heavy use. This enables each hiker to hike his or her own route and overall there is less impact. And Roper's suggestion will hopefully minimize any impact due to his publication of the book. I guess that is LNT or walking softly.
Earlier, someone mentioned Carey's Castle in the vein of should we destroy this man made rock house? The answer is no, it is 80 years old. But here is the rest of that story... little is known about Carey. He dug a mine in that area of Joshua Tree National Park and it is now part of a designated wilderness. Not easy to find. About 30 years ago I was doing a lot of hiking in the area, learning and seeing as much of this small mountain range as I could. I stumbled across Carey's Castle by accident; it was not a publicly known place at the time. Occasionally in the JTNP backcountry I would run into a ranger who pretty much had the same ideas as to what wilderness should be, as I have. I asked him about the rock house. He said it was called Carey's Castle and was only known to a few park employees. A few years later a picture of it was included in a trail guide, but no directions or even the area of its location was revealed. It became sort of a needle in the haystack search for some people... they didn't even know where to start if they wanted to find it. Then came Trail Guides and Internet postings with fairly explicit instructions on how to get there. What has happened in the 30 years since I ran across it? Many of the artifacts inside the house are gone - stolen. Many Indian artifacts, including arrowheads, that could be found near the house are gone - stolen. So when I am in this area, I either relocate the ducks or disassemble them into arrows pointing the wrong way. If you want to visit Carey's Castle, then get a friggn' topo map and figure it out. If you can't navigate there by yourself, you shouldn't be hiking there.
Last month I led a group of BPLers on a hike in Anza Borrego. A small section of this route is across a faintly marked Indian trail, hard to follow and the cairns help -- but you do not need them. I would not knock these down. The trail is almost impossible to find. Now if someone built cairns in the area at the beginning of the trail to tell people there is a trail up there, I would knock them down in a heartbeat. If you want to hike the area, get a friggn' topo map like I did. If you can't navigate without the cairns, you shouldn't be hiking there.
In other areas I am familiar with, I do not knock down new cairns the first time I see them. Someone may be relying on them to get back. However on subsequent trips I might take them down, and if I see a trail is developing along the new cairns, I will definitely take them down.
Is this right or wrong? Each person must decide for themselves.