More change to believe in
Truthfully, I don't understand the significance of this sort of geekery, but it sure sounds like a new transparency in governance. The robots.txt file from whitehouse.gov yesterday had some 2400 lines of code that looks like this:
[More posts daily at The Detroit News.]
User-agent: *The new code today under the Obama administration
Disallow: /cgi-bin
Disallow: /search
Disallow: /query.html
Disallow: /omb/search
Disallow: /omb/query.html
Disallow: /earmarks/search
Disallow: /earmarks/query.html
Disallow: /help
User-agent: *Supporting material at the link for you geeks. Maybe I'm just overly impressed by the mysteries of coding, but it looks like a huge difference to me.
Disallow: /includes/
That's it! BTW, the robots.txt file tells search engines what to include and not include in their indexes.
[More posts daily at The Detroit News.]
Labels: President Obama
posted by Libby Spencer at 3:06 PM
7 Comments:
Dunno, looks like geek to me.
LOL. Me too.
Shorter does not mean better.
Apparently, the old file had lots of point directories that were not to be searched. And some of them, such as cgi or those multiple search and query.html entries, contains programming-related stuff, not a content, and clearly have nothing for indexing by search engines.
The new file disallows indexing for anything under /includes/ directory. Anything! As long as new guys put their stuff there, it is invisible. Who knows what percentage of the stuff will go there. Perhaps, everything.
To sum, one huge prohibition is much worse than a sum of small targeted ones.
Btw, wikipedia explains robots convention.
Anon, this is way over my pay grade but if you read the link where I got it, he posits the exact opposite view you're pushing here. And common sense would suggest that disallow nothing means everything is searchable.
In contrast, you did notice there was 2400 lines of code that disallowed searches under Bush. Your argument doesn't make sense.
Speaking from personal experience, I found trying to search anything on that site under Bush was impossible. I haven't tried it since Obama was sworn in, but I'm willing to bet I'll be able to find what I'm looking for.
I've done plenty of websites for years and this doesn't mean anything except a new web person came in and uploaded a bunch of changes to the website, which probably included a new robots.txt file as part of the upgrade.
Using this as some reason to gush over Obama's supposed transparency (when the White House press people can't even politely respond to a reporter)...well, I am continually amazed that people have flip-flopped from hating Bush with an insane passion to jumping with joy every time Obama sneezes. Some middle ground here, please!
Oh, and a PS, based on that snippet of code you posted, it looks like they were disallowing the search function. There's a perfectly reasonable and non-sinister reason for this: Search adds a lot of load to a server. In many websites this is the first thing that gets taken off (throttled) when a site gets too much traffic.
Much better to use an external Google search function...saves on bandwidth.
And if they truly had anything they wanted to hide, they wouldn't put it up on a website in the first place.
Common sense, really.
So please...enough with the conspiracies or kudos. If the government wants to hide stuff from you, they'll be more savvy than just using a robots.txt file.
Anon, I'm a complete technodope. None of this means anything to me. I went by the post because the guy seemed to know what he was talking about.
In any event, I'm hardly swooning every time Obama sneezes. I've criticized him on the few things that I thought were really bad. Otherwise, this is the hardly the biggest step towards transparency he's made so far. Let's not discount the progress because we're too stuck in criticism mode from the last 8 years to believe that change is even possible.
For the moment, it's been three weeks now and the I imagine the economy and the occupations are rather daunting problems that require a whole lot of attention, so if Obama is not perfect and OMG can't change everything in a month, I'm willing to cut him a little slack on the small stuff.
As I've been saying repeatedly, if and when the admin shows a serious pattern of obfuscation and or miscreant behavior, I'll be the on the front lines of the torch and pitchfork crowd.
Post a Comment
<< Home