hoodwink.d enhanced
RSS
2.0
XHTML
1.0

RedHanded

Using Etags With Open-uri #

by why in inspect

If you just joined us, we’re fawning over Tanaka-san’s code lately. He’s the author of an entire repository of red magic. One of his masterpieces is open-uri, a library that in now enshrined in Ruby 1.8.0.

With open-uri, you can access a file over FTP or HTTP and read from like it was any other IO handle (using read.) For HTTP resources, you can access headers through the meta accessor:

 require 'open-uri'
 open( "http://redhanded.hobix.com/index.xml" ) do |feed|
   p feed.last_modified
   p feed.meta['etag']
 end

The ETag header is great, as it offers a unique hash for the content being served. If your web server returns an ETag, you can issue a conditional GET.

Especially common in RSS feed readers. This technique was also used in the old RAA-Install to check if the RAA feed had changed since the last install. You just send the web server the old etag under the If-None-Match header. The web server will send you a 304 status unless the file is new. This prevents the whole file from being sent if it is old.

 require 'open-uri'

 # 1st request -- save the ETag
 etag = nil
 open( "http://redhanded.hobix.com/index.xml" ) do |feed|
   etag = feed.meta['etag']
 end

 # 2nd request -- only retrieve the file if it has changed
 begin
   open( "http://redhanded.hobix.com/index.xml", 
         "If-None-Match" => etag ) do |feed|
     puts "File has changed: #{ feed.read.length } bytes read" 
   end
 rescue OpenURI::HTTPError
   puts "No file available or file has not changed." 
 end

You can also use the modification time, with the help of the Time#rfc2822 method which formats dates for HTTP.

 require 'open-uri'

 # 1st request -- save the modification time
 mtime = nil
 open( "http://redhanded.hobix.com/index.xml" ) do |feed|
   mtime = feed.last_modified
 end

 # 2nd request -- only retrieve the file if it has changed
 begin
   open( "http://redhanded.hobix.com/index.xml", 
         "If-Modified-Since" => mtime.rfc2822 ) do |feed|
     puts "File has changed: #{ feed.read.length } bytes read" 
   end
 rescue OpenURI::HTTPError
   puts "No file available or file has not changed." 
 end
said on 18 Jan 2005 at 19:02

Woah, that’s groovy. I just made a mental note try out open-uri. I just love Ruby…

...still having trouble “sneaking Ruby through the system” however. As in, here at work. I’m giving a talk on “Getting Started With Ruby” this Friday. Wish me luck or, maybe, do a little dance, or, whatever.

said on 19 Jan 2005 at 01:53

...do a little dance,....

You mean the hamster dance ?

said on 19 Jan 2005 at 04:21

Shouldn’t that be duck dance ?

said on 19 Jan 2005 at 07:50

Agreed, definitly a duck dance. Come on lets all do the waddle.

said on 19 Jan 2005 at 09:59

So, “open” is a global def? Ought to be URI .open, or have some namespace attached. Tsk, tsk.

said on 19 Jan 2005 at 10:09

Good question, Daniel. For convenience, open-uri aliases and replaces Kernel::open to allow opening of files, pipes, URIs from a single method.

However, you can use OpenURI.open_uri if you know you’ll be accessing a URI .

 OpenURI.open_uri( "http://redhanded.hobix.com/index.xml" ) do |feed|
   mtime = feed.last_modified
 end

You can also access open-uri power through a URI object:

 uri = URI.parse( "http://redhanded.hobix.com/index.xml" )
 uri.open do |feed|
   p feed.last_modified
 end

Look at these options. It’s all so well thought out and neatly aligned.

said on 19 Jan 2005 at 10:12

Very ha, _why cult follower.

said on 19 Jan 2005 at 10:48

Shame on me for not looking closer. Thanks _why.

said on 26 Nov 2005 at 22:35

Why isn’t this stuff in the Ruby Docs? I have been looking forever to find this stuff. Not very fun…

said on 26 Jan 2006 at 07:22

Instead of using mtime.rfc2822 wouldn’t you want to usemtime.httpdate

Comments are closed for this entry.