For a number of years, I've been pointing out the unaccountable power of Google's search engine employees to marginally screw over individuals they don't like. One of the weirder examples is Google's intermittent but long-running petty campaign against the high-brow blogger Dennis Mangan.
If you go to Bing and type in "Dennis Mangan," the first his is his blog, Mangan's.
But if you go to Google and type in "Dennis Mangan," you don't get his blog on the first page of responses. Ironically, you just get other bloggers wondering why Google is messing with Dennis.
Same with my blog.
ReplyDeleteA lot of it just won't show up.
But if you go to Google and type in "Dennis Mangan," you don't get his blog on the first page of responses.
ReplyDeleteMight have something to do with the word Dennis not appearing anywhere on the front-page of his blog.
B.B.
This morning on Fox, an expert of was detailing the very subject although the major example he used involved how, for example, if it wishes (ha!) Google could control which candidate in any election, could get the first and the most positive hits on any search.
ReplyDeleteI thought of you, Steve. You've been talking about this for a long while.
B.B.: Incompetence, not malice, eh?
ReplyDeleteAnd the serial problems with Pat Buchanan are merely a Whiskey-like inability to recall how to spell his name?
Google, like the NSA is vulnerable to the people who staff it. Switzerland had to give up bank secrecy, because it was too easy for the French and German governments to bribe clerks to give them massive amounts of data on tax evaders on USB sticks; Bradley Manning famously downloaded his data onto CDs.
ReplyDeleteThe kind of person who works at a Google, NSA etc. in IT staffing positions is young, naive, idealistic, unsophisticated in how human beings behave, gifted in computer programming and operations, and believes in independence of action because that has been the key to his success. That is, self-taught, constantly upgrading skills, latching onto the new thing, and pushing boundaries.
Google famously rewards its staff for pushing boundaries. So they do. So far Google has not paid a price for pushing boundaries by their staff. BUT ..
Already many European and Asian nations are considering creating their own versions of Google and banning same, a two-fer: create their own national champions whose owners and staff will be indebted, and keep privacy/security concerns around Google, Apple, Twitter, Facebook, Dropbox, Skype, AOL, etc at bay. No more risks from government people inadvertently leaking secrets in say, China, because Google is banned. Along with Apple, and the rest. If the NSA is in bed with them (and they are), reasonable Chinese and Russian and French security services might wonder what backdoors are built into, say Itunes, Dropbox, Skype, etc.
There is not much of a barrier to entry to create national versions of same, most of these services use open source frameworks and languages: Python, Ruby, PHP, etc. and there is no shortage globally of guys who can rip/copy what say, Google built.
Steve, I've moved to Wordpress because I don't like Google deleting my blog over politics. Which happened. You really should bit the bullet, convert to Wordpress, and maybe run your own domain. I know Blogger has some nice stuff, but Wordpress is even nicer. You could probably find someone for a very reasonable price to do the conversions and move nearly everything to your own hosted site.
I think I remember Mangan writing that he voluntarily selected an option in the blogging software that would prevent his posts from showing up in public search results. Of course I can't find where he wrote this for that exact reason.
ReplyDeleteWhen Mangan's blog went down a couple years ago, you speculated it was Google. I'm not so sure...
ReplyDeleteNever mind, I didn't notice anything...
LOLOLOLLOLOL!
If you run an independent web site, there are a lot of things you can do to get knocked down or out of Google's index: overloading your pages with keywords, (over)using certain kinds of tags, buying links from other sites, etc. But when your blog is on their freakin' site, most of that isn't possible, and the rest isn't anything a popular blogger would mess with anyway.
ReplyDeleteAnd no, just having his first name missing from his home page doesn't account for it. It might have 10 years ago, but the search engines are far more sophisticated than that now. They routinely figure out connections more difficult than that. Duckduckgo.com puts him at #1, by the way.
His robots.txt file (just add that to his main URL to see it) has an extra section for "googlebot" that I don't see on other blogspot sites I've checked. It should only be blocking indexing of his /search pages, and not his home page, but it's not unheard of for googlebot to get confused about something like that. It's also possible that it was misconfigured before and has now been fixed, but hasn't taken effect yet.
But if I had to guess, based on my experience with Google's search people....there's a reason I didn't put my blog there.
A reader brought this up to Mangan shortly after his recent return to blogging. IIRC, he responded that he had deliberately disabled search indexing on his blog-- reasons unstated-- and it appears that, for the most part, Bing doesn't turn up entries from his blog or provide descriptions from them. (A Bing search for site:mangans.blogspot.com yields much of the following: "We would like to show you a description here, but the site you’re looking at won't allow us.") Note also that he has been blogging and commenting by his surname alone since he reemerged.
ReplyDeleteThat Bing turns up Mangan's blog right away and Google does not may actually be a demonstration of Google's stricter observance of the robots.txt protocol. At least in this case. Indeed, one somehow very easily imagines the search giant only too happy to misplace him.
Mangan is high-brow!? Only if you refer to his hairline.
ReplyDelete"Steve, I've moved to Wordpress because I don't like Google deleting my blog over politics. Which happened. You really should bit the bullet, convert to Wordpress"
ReplyDeleteOh, don't do that. We'll never hear from you again. Wordpress is 100x more complicated to use than Blogger. It really is horrible. And I say that as someone who made the switch from Blogger to Wordpress.
Stick with something simple to use so you can focus on the content of what you post. The only platform I can think of that's easier to use than Blogger is Tumblr, but, by design, it doesn't come with a native comment system (you can easily add Disqus to it though).
I like how his photo is the first two hits of a Google Image Search for his name. In other words, they want this horrible unperson's identity to be widely know, but they don't want anyone to actually know what he is saying for fear people agree with him.
ReplyDeleteBut if I had to guess, based on my experience with Google's search people....there's a reason I didn't put my blog there.
ReplyDeleteCould you share some of your experience with Google's search people?
Remember when Mangan voluntarily took a blogging hiatus? This was after Google made his blog disappear for a while, before making it visible again.
ReplyDeleteI wonder if Mangan purposely tried to make his blog not appear in the search engines' listings to try to be a little more anonymous--in the same vein that some bloggers only let invited bloggers see their blogs.
If it's Mangan's own choice to maintain a lower profile on the internet, perhaps you should delete this post, Steve.
ReplyDeleteThe only platform I can think of that's easier to use than Blogger is Tumblr, but, by design, it doesn't come with a native comment system (you can easily add Disqus to it though).
ReplyDeleteDisqus is horrible.
Remember when Mangan voluntarily took a blogging hiatus? This was after Google made his blog disappear for a while, before making it visible again. I wonder if Mangan purposely tried to make his blog not appear in the search engines' listings to try to be a little more anonymous--in the same vein that some bloggers only let invited bloggers see their blogs.
ReplyDeleteSomeone should ask him about that. I'd be curious to know of his experience and what he thought when it was over.
Who is Dennis Mangan?
ReplyDeleteThe new Emmanuel Goldstein?
Could you share some of your experience with Google's search people?
ReplyDeleteHa! That's funny.
Here is the part of Mangan's robots.txt file that's keeping Google from indexing it. (No, I didn't hack anything to get it; a robots.txt file is public on all sites that have one.)
User-Agent: *
Disallow: /search
Disallow: /
What that does is, for all search bots (the *), it disallows indexing of pages under /search, and then it disallows indexing of pages under / (which includes pages under /search). That's odd, so I'd question whether it's as intended, but I don't know how this stuff is automated by their software.
Beneath that is this section:
User-Agent: googlebot
Disallow: /search
Allow: /
That should allow access to all pages except those beneath /search, but only for search bots calling themselves "googlebot." They actually call themselves "Googlebot," which may not match that. That section might also need to be above the section that matches all bots. (It's been a while since I worked with this stuff.)
So I'd say Googlebot is obeying that first section, and not seeing or not matching the second one. Whether that's what he intended, I don't know. The other search engines aren't really obeying the robots.txt, which is supposed to control whether pages are indexed, not whether a description is shown.
So this doesn't look nefarious to me, unless someone at Google broke his robots.txt file.
@dave - "Wordpress is 100x more complicated to use than Blogger."
ReplyDeleteno, it's not.
@anonymous - "Disqus is horrible."
yes, it is!
Since when has preening condescension equated with "highbrow"?
ReplyDelete"If it's Mangan's own choice to maintain a lower profile on the internet..."
ReplyDeleteWhy even bother to have a blog then?
Neither google nor blogspot have given me any grief whatsoever. Yet.
ReplyDeleteI can't say the same for Komment Kontrol.
You ask, I answer:
ReplyDeletehttp://isteve.blogspot.com/robots.txt
http://mangans.blogspot.com/robots.txt
Since Komment Kontrol seems to be in a magnanimous mood... I'd like to know if Steve has delegated others to moderate for him. That would explain some of the... moodiness.
ReplyDeleteI stand corrected:
ReplyDeletehttp://dictionary.reference.com/browse/highbrow
2.
a person with intellectual or cultural pretensions; intellectual snob.
Actually Mangan checked himself out of search engines.
ReplyDeleteCheck out its robots.txt http://mangans.blogspot.com/robots.txt, The command 'User-agent: * Disallow: /' means all robots are forbidden to index any page.
Compare with yours.
For some reason, Mangan's is blocked at my workplace, but you are not, nor is anyone similar.
ReplyDeleteOther people have pointed this out above: he's disabled the Google spider from creeping his blog. Check his /robots.txt file. In other words, a false flag ;)
ReplyDeleteIt just wouldn't be iSteve without Svigor complaining about Komment Kontrol.
ReplyDeleteCould you share some of your experience with Google's search people?
ReplyDeleteSorry, the first time I read this to mean "share with Google's search people my experiences," as if that's possible, which is part of the problem. Unless you're a big player, it's impossible to get answers from them.
Here's how it works: You start a web site about Red Widgets, and over the years you build it into a nice little side business, adding content to draw visitors and monetizing it, usually with ads (like Google's AdSense). You put some effort into SEO (search engine optimization), following the latest recommendations on making your site appealing to search engines so it'll rank well and bring as much traffic as possible. Traffic -- half or more of which comes from Google searches, because it's so dominant -- builds to the point where you're making a nice chunk of cash. Life is good.
One day your traffic (and revenue) drops by 50% or more. You start checking things out, and see that your traffic from Google has almost disappeared. You do some searches on Google for "red widgets" and see that your site, which used to rank #2, is nowhere to be found. By doing extremely detailed searches like "red widgets for blue trucks in mytown usa" you finally get some of your pages to come up; so they're still indexed, but something is driving them several pages deep where no one sees them.
Now, if you're a huge site, or a big media site like the NY Times that could embarrass them, you call a contact at Google and they fix it. But since you're nobody, you go to their Webmaster Tools and try to report it and ask about it. You are ignored. So you go to their support forum, where you find a bunch of sycophants ("volunteer" tech support) who respond to complaints the way people on a Star Wars forum respond to a Star Trek troll. They say you must be doing some nefarious "black-hat" SEO, because good sites that offer good content just don't have these problems.
You get a couple of suggestions and follow them, but they don't help. You try some other forums on the topic and get more suggestions -- often contradictory -- and they don't help either. You read lots of articles by people who study Google's results and try to figure out what they're looking for, but their ideas don't help either. You finally break down and pay an SEO company thousands of dollars to audit your site, but they don't guarantee results, and they don't help either. You pour time and money into adding new features and content to your site, and gradually get half of the lost traffic back through volume -- and then one day you have another 50% drop for no discernible reason.
Throughout all this, your rankings at Yahoo, Bing, and other sites have been stable; only Google is bothered by whatever it's bothered about. But you can't support the business you've developed on a fraction of the traffic, so eventually you sell the site at a fraction of what it used to be worth, and write it off as bad luck.
That's why I use other search engines. Google -- intentionally or not -- can kill a web site dead, and that wouldn't be the case if there were better competition in the search field.
To summarize my earlier comment: Mangan's robots.txt actually looks like it's trying to allow indexing by Google and not by other search engines. The * setting for all search engines is Disallow all, while the one for "googlebot" is Allow. But it's not set up quite right, so it's having the opposite effect of what appears to be the intention.
ReplyDeleteso then, which is most gay: google, wordpress or disqus?
ReplyDeleteIt just wouldn't be iSteve without Svigor complaining about Komment Kontrol.
ReplyDeleteSo why doesn't Komment Kontrol trash the comments about Komment Kontrol? Is it just a tease?
It just wouldn't be iSteve without Svigor complaining about Komment Kontrol.
ReplyDeleteI always post under this handle.
As mentioned above, Mangan chose to opt out of Google search. He said as much in a comment thread there. Google is just respecting his wishes.
ReplyDeleteNB: You can still use the search functionality on his site, which is indexed separately from Google's search database.
I always post under this handle.
ReplyDeleteDon't take the accusation to heart. Levinson wouldn't be Jewish if he didn't try to blacken someone's name every now and then.
The same thing happens with "White History".
ReplyDeleteThe 1st result on Google is an obscure, racist, anti-white tumblr page with articles about how evil white people are. Then some articles about why we don't need White History Month.
Bing returns websites that are relevant to white history.
Oh, and compare Google's treatment of "Black History".
I don't consider Mangan's blog to be high brow. It may just be the people who comment on his blog or it may be him, but I see it as just an extension of angry people ranting hypocritically.
ReplyDelete"I can't say the same for Komment Kontrol."
ReplyDeleteHow many times I gotta tell ya it's Whim?
I like how Ron Unz built a snotty post around the fact dahlia ended one of her comments Loling. Apparently see does that on every comment. Lol Unz sucks.
ReplyDeleteBlogger recently added a feature to delist your blog from search engines. I did it with mine. Perhaps Dennis Mangan did it with his.
ReplyDelete