Twitter is shifting into JVM

Last year the company announced that both its back-end message queue and Tweet storage had been re-written in Scala, and in the spring of 2010 the search team at Twitter started to rewrite the search engine. As part of the effort, Twitter changed the search storage from MySQL to a real-time version of Lucene. More recently the team announced that they were replacing the Ruby on Rails front-end for search with a Java server they called Blender. This change resulted in a 3x drop in search latencies.

via infoq.com

# Posted via email from opportunity__cost

Build Your Own Backup System on Linode

I have recently finished moving couple of our virtual servers instances from Slicehost to Linode, and I'm quite happy with them so far. Very flexible with their Web interface, decent support and about two times less expensive than Slicehost (which adds up pretty quickly if you run multiple instances). One more good thing is that you can actually build _your own_ backup subsystem using their API and allocated storage space, without paying anything extra (while you can still pay to use pre-built one :). E.g. here is a quick Ruby script that, run weekly, creates a copy of a disk. You need to keep enough of free space though, however, this wasn't an issue for me (I had about twice as much free for every instance).

 

require 'rubygems'

require 'linode'

 

api_key = 'SecretKeyGoesHere'

linode_id = 12345

disk_id = 67890

backup_disk_id = 0

backup_label = 'weekly'

 

l = Linode.new(:api_key =>; api_key)

 

# 1. Find the backup named 'XXX' and remove it first

result = l.linode.disk.list(:LinodeId => linode_id)

result.each do |i|

    backup_disk_id = i.diskid if i.label == backup_label

end

p "Backup disk named #{backup_label} # #{backup_disk_id}"

 

# 2. Remove old backup

remove_job = l.linode.disk.delete(:LinodeId => linode_id, :DiskId => backup_disk_id) if backup_disk_id != 0

p "Remove job status: #{remove_job}"

 

# 3. Duplicate disk

duplicate_job = l.linode.disk.duplicate(:LinodeId => linode_id, :DiskId => disk_id)

p "Duplicate job status: #{duplicate_job}"

 

# 4. Rename duplicated disk as 'XXX'

rename_job = l.linode.disk.update(:LinodeId => linode_id, :DiskId => duplicate_job.diskid, :Label => backup_label)

p "Rename job status: #{rename_job}"

 

P.S. I am so pissed off with broken markdown support at Posterous, that I am ready to give up and move my blog away from it.

# Posted via email from opportunity__cost

New routes in Rails 3

Here goes: everything you knew about working with routes in Rails 2… is history! With Rails 3, you’ve got to roll up your sleeves, unlearn what you learned, and route the new way around. And this time, it’s faster, cleaner and a lot more Ruby-like.

via engineyard.com

Great overview of the routes functionality in new Rails 3, compared to what (you might got used to) in Rails 2.

# Posted via web from opportunity__cost

Ruby Quicktips

Random Ruby and Rails tips.
This blog is dedicated to deliver short, interesting and practical tidbits of the Ruby language and Ruby on Rails framework.

via rubyquicktips.tumblr.com

Nice collection of tips and tricks on Ruby (+oR). Not that big (about 20 so far?), but nice thing to keep in bookmarks.

# Posted via web from opportunity__cost

How programmers see each other...

via rubyinside.com

How different programmers see each other. Pretty cool.

# Posted via web from opportunity__cost

Rails in a Nutshell

Rails in a Nutshell

via rails-nutshell.labs.oreilly.com

This is the draft of upcoming O'Reilly book. Feel free to read and comment. :)

# Posted via web from opportunity__cost

Some Ruby stuff

This morning I've got a questions from a friend and really couldn't give him a good answer. It's about a rather simple Ruby structure.

module Test
 def self.included(base)
  class << base
   def tester
    puts 'some message' + "\n"
   end
  end
 end
end

class Test2
 include Test

 tester
end

Speaking about the following block:

 def self.included(base)
  class << base
   def tester
    puts 'some message' + "\n"
   end
  end
 end

So what does really happen here? We're overriding the self.included which is called whenever Test module is included by other class or module (Test2) and Test2 is passed as the single argument to it.

Moving forward, whenever we're getting a callback to self.included, we're extending the class (Test2) with tester method which does something.

What I was wondering about is why tester is called there. Because if we miss including Test there we'll get NameError: undefined local variable or method `tester' for Test2:Class exception.

So basically, my real question is what make sense of using such a structure of the code? I bet the example I was given is not complete enough to understand this.

What do you think?

Issues with RoR streaming from controller

Well, I am facing some issues with "on the go" streaming data from controller using Ruby on Rails. I've tried almost every instance to query about this problem and try to find an answer. Nothing works so far.

I've posted multiple questions on the Twitter. Previously, I was getting replies very quickly. But today is not my day.

I've tried IRC. I have to acknowledge it's dead now. Very dead, even that it keeps tens of people signing on and off every single minute, as well as keeping more then 500 of signed people on.

I've tried the discussion group on Google. I didn't receive any comments back to my issue, yet. But I still hope. :)

I don't know if my blog will be any better — actually, I doubt so as it's never been a RoR lair. But I'll give it a try and post my problem here. Hopefully, I'll get it nailed some day and share the solution with you.

So, I need to stream a lot of data from the controller and there is no need for a view. Data generates on the go, and can take pretty long time to complete (say, from 1 minute to 3-4 hours). I doubt that it worths generating a string (the output can be hundreds of megabytes) and pass it to the view (ouch).

The way to use render :text => ... is not going to work for me, as it can be called only once in controller, so not a real "on the go" streaming.

Neither store to file and then streaming it with render :file works. Several simultaneous uses of the controller can generate pretty huge files for every instance, which is no-go (tens and hundreds of gigabytes of free space for temporary files?). Anyway, it's not "on the go" as well. (Well, partially. It'll be streaming once the file is created. :)

The other way is to render :text = Proc.new { |r,o| ... }, however, somehow it doesn't seem to be working for my case. First of all, it complains about some random each ... do blocks (that work just fine when the blocks are not passed to Proc.new). When I'm commenting them out, I'm getting things like that:

Wed May 13 17:34:39 -0700 2009: Read error: #
/Library/Ruby/Gems/1.8/gems/activesupport-2.3.2/lib/active_support/dependencies.rb:414:in `load_missing_constant'
/Library/Ruby/Gems/1.8/gems/activesupport-2.3.2/lib/active_support/dependencies.rb:96:in `const_missing'
/Users/white/repo/vurvimport/app/controllers/do_controller.rb:380:in `preparesql'
/Users/white/repo/vurvimport/app/controllers/do_controller.rb:630:in `rendersql'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.2/lib/action_controller/response.rb:153:in `call'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.2/lib/action_controller/response.rb:153:in `each'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.2/lib/action_controller/vendor/rack-1.0/rack/chunked.rb:37:in `each'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.2/lib/action_controller/vendor/rack-1.0/rack/handler/mongrel.rb:74:in `process'
/Library/Ruby/Gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:159:in `process_client'
/Library/Ruby/Gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:158:in `each'
/Library/Ruby/Gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:158:in `process_client'
/Library/Ruby/Gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:285:in `run'
/Library/Ruby/Gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:285:in `initialize'
/Library/Ruby/Gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:285:in `new'
/Library/Ruby/Gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:285:in `run'
/Library/Ruby/Gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:268:in `initialize'
/Library/Ruby/Gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:268:in `new'
/Library/Ruby/Gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:268:in `run'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.2/lib/action_controller/vendor/rack-1.0/rack/handler/mongrel.rb:34:in `run'
/Library/Ruby/Gems/1.8/gems/rails-2.3.2/lib/commands/server.rb:111
/Library/Ruby/Site/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
/Library/Ruby/Site/1.8/rubygems/custom_require.rb:31:in `require'
script/server:3

I think the similar situation is also mentioned in tickets. I'm not sure that this is really a bug though. I'll keep an eye on that.

Ruby trick

For those who are trapped into the same problem as I am.

obj.atr

Block below is valid for the block above:

def x(obj, atr)
    return obj.send(:atr)
end

Have fun! Please, let me know how will you use this code, if you find it useful at all. In example, I can see that it can be used for something like this:

p User.find(:first, :condition => {:userid => 1}).name

This will drop an exception, if there is no such user with id 1. However, you can call it through the wrapper:

def wrp(obj, atr)
  if obj.nil?
  then
    return "N/A"
  else
    return obj.send(:atr)
  end
end
 
...
 
p wrp(User.find(:first, :condition => {:userid => 1}), name)

This might be awkward, but works well enough.

P.S. Here is the original question posted in Twitter and replies to it. (Thanks to Tweeplies.)

Case senstive SELECTs

While working with the legacy code and migration process to REST on Ruby on Rails (from the mix of shell scripts, C and other types of binaries), I've got into trouble debugging one part of the code that was ultimately going into loop.

What it worked out to, is that we had a number of case sensitive IDs, like JONHSMITH, JohnSmith and johnsmith, reflecting the different types of account information. I'm not going to guess you why did it happen, but it always does, if there is a possibility of human interaction with the naming process ;)

What was more important is how to make the code work and to stay away of any table alterations. While there a huge amount of discussion in the Internet about that, the easiest way that I found to be is just using BINARY operator for a SELECT clause.

For example, SELECT * FROM accounts WHERE BINARY user LIKE 'JohnSmith'; gives you exactly JohnSmith's record.

You can take your time and refactor the code and table structure later, after the things are working and nobody's trying to ask you the same question for many times per day ;)