opportunity__cost

Jul 4 2011

Twitter is shifting into JVM

Last year the company announced that both its back-end message queue and Tweet storage had been re-written in Scala, and in the spring of 2010 the search team at Twitter started to rewrite the search engine. As part of the effort, Twitter changed the search storage from MySQL to a real-time version of Lucene. More recently the team announced that they were replacing the Ruby on Rails front-end for search with a Java server they called Blender. This change resulted in a 3x drop in search latencies.

via infoq.com

highscalability java jvm ruby rubyonrails twitter

0 Comments

Nov 22 2010

Build Your Own Backup System on Linode

I have recently finished moving couple of our virtual servers instances from Slicehost to Linode, and I'm quite happy with them so far. Very flexible with their Web interface, decent support and about two times less expensive than Slicehost (which adds up pretty quickly if you run multiple instances). One more good thing is that you can actually build _your own_ backup subsystem using their API and allocated storage space, without paying anything extra (while you can still pay to use pre-built one :). E.g. here is a quick Ruby script that, run weekly, creates a copy of a disk. You need to keep enough of free space though, however, this wasn't an issue for me (I had about twice as much free for every instance).

require 'rubygems'

require 'linode'

api_key = 'SecretKeyGoesHere'

linode_id = 12345

disk_id = 67890

backup_disk_id = 0

backup_label = 'weekly'

l = Linode.new(:api_key =>; api_key)

# 1. Find the backup named 'XXX' and remove it first

result = l.linode.disk.list(:LinodeId => linode_id)

result.each do |i|

backup_disk_id = i.diskid if i.label == backup_label

end

p "Backup disk named #{backup_label} # #{backup_disk_id}"

# 2. Remove old backup

remove_job = l.linode.disk.delete(:LinodeId => linode_id, :DiskId => backup_disk_id) if backup_disk_id != 0

p "Remove job status: #{remove_job}"

# 3. Duplicate disk

duplicate_job = l.linode.disk.duplicate(:LinodeId => linode_id, :DiskId => disk_id)

p "Duplicate job status: #{duplicate_job}"

# 4. Rename duplicated disk as 'XXX'

rename_job = l.linode.disk.update(:LinodeId => linode_id, :DiskId => duplicate_job.diskid, :Label => backup_label)

p "Rename job status: #{rename_job}"

P.S. I am so pissed off with broken markdown support at Posterous, that I am ready to give up and move my blog away from it.

code linode ruby script slicehost vps xen

0 Comments

Apr 3 2010

New routes in Rails 3

Here goes: everything you knew about working with routes in Rails 2… is history! With Rails 3, you’ve got to roll up your sleeves, unlearn what you learned, and route the new way around. And this time, it’s faster, cleaner and a lot more Ruby-like.

via engineyard.com

Great overview of the routes functionality in new Rails 3, compared to what (you might got used to) in Rails 2.

overview rails rails 2 rails 3 routes ruby rubyonrails

0 Comments

Feb 12 2010

Ruby Quicktips

Random Ruby and Rails tips.
This blog is dedicated to deliver short, interesting and practical tidbits of the Ruby language and Ruby on Rails framework.

via rubyquicktips.tumblr.com

Nice collection of tips and tricks on Ruby (+oR). Not that big (about 20 so far?), but nice thing to keep in bookmarks.

dev quick rails ruby tip trick

0 Comments

Dec 21 2009

How programmers see each other...

Media_httpwwwrubyinsidecomwpcontentuploads200912languagefanboysjpg_vgddniaiyndbeag

via rubyinside.com

How different programmers see each other. Pretty cool.

c fanboys fun images java php ruby

0 Comments

Oct 20 2009

Rails in a Nutshell

Rails in a Nutshell

via rails-nutshell.labs.oreilly.com

This is the draft of upcoming O'Reilly book. Feel free to read and comment. :)

dev rails ruby rubyonrails

0 Comments

Oct 4 2009

Scrubyt in the house

Okay, I figured out the problem with Scrubyt. And guess what, it was a bad gem. You know, I'm starting to hate all this stuff already. I don't want to mess with gem package versions, and it looks like I'm getting into this too often nowadays.

Whatever, lets go through the problem.

Here is my code:

1 require 'rubygems'

2 require 'scrubyt'

3

4 google_data = Scrubyt::Extractor.define do

5 fetch 'http://www.google.com/ncr'

6 fill_textfield 'q', 'ruby'

7 submit

8

9 result 'Ruby Programming Language'

10 end

11

12 google_data.to_xml.write($stdout, 1)

13 Scrubyt::ResultDumper.print_statistics(google_data)

I was running mechanize-0.9.3 (I think it was the most recent one). The problem was solved with uninstalling 0.9.3, and getting 0.8.5:

white@iwhite:~> sudo gem install mechanize -v=0.8.5

Successfully installed mechanize-0.8.5

1 gem installed

And now lets try the code:

white@iwhite:~> ruby s.rb

<root>

<result>Ruby Programming Language</result>

<result>Download Ruby</result>

<result>Ruby - The Inspirational Weight Loss Journey on the Style Network ...</result>

<result>Ruby (programming language) - Wikipedia, the free encyclopedia</result>

<result>Ruby - Wikipedia, the free encyclopedia</result>

<result>Ruby on Rails</result>

<result>Ruby&#39;s Diner - rubys.com</result>

<result>Ruby Central</result>

<result>Ruby Annotation</result>

<result>[Ruby-Doc.org: Documenting the Ruby Language]</result>

<result>Blog posts about ruby</result>

/Library/Ruby/Gems/1.8/gems/activesupport-2.3.4/lib/active_support/dependencies.rb:440:in `load_missing_constant': uninitialized constant Scrubyt::ResultDumper (NameError)

from /Library/Ruby/Gems/1.8/gems/activesupport-2.3.4/lib/active_support/dependencies.rb:80:in `const_missing'

from s.rb:13

</root>

Wow, it works.

Well, we do have an error with ResultDumper, but that's on the easy side and can be ignored. ResultDumper was indeed dropped, should be rewritten and back in the future. But it's not a big deal though.

I know also that some people were able to tweak the line 4 of the code:

4 google_data = Scrubyt::Extractor.define :agent => :firefox do

and the requests would be made through Firefox, but it didn't work for me:

white@iwhite:~> ruby s.rb

/Library/Ruby/Gems/1.8/gems/firewatir-1.6.2/lib/firewatir/firefox.rb:271:in `set_defaults': Unable to connect to machine : 127.0.0.1 on port 9997. Make sure that JSSh is properly installed and Firefox is running with '-jssh' option (Watir::Exception::UnableToStartJSShException)

from /Library/Ruby/Gems/1.8/gems/firewatir-1.6.2/lib/firewatir/firefox.rb:161:in `initialize'

from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/navigation/agents/firewatir.rb:17:in `new'

from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/navigation/agents/firewatir.rb:17:in `included'

from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/navigation/agents/firewatir.rb:16:in `module_eval'

from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/navigation/agents/firewatir.rb:16:in `included'

from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:20:in `include'

from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:20:in `define'

from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:19:in `class_eval'

from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:19:in `define'

from s.rb:4

But this is a different and I actually never tried to hunt it down. But I think it's an easy fix anyways. :)

Have fun!

gem mashups rails ror ruby screen scrapping scrubyt

0 Comments

Oct 1 2009

Problem with Scrubyt

Well, working on one of the projects, I came over the Scrubyt. It wasn't something very new to me, as we played with mashups for a while at Atomkeep and Everytalks. But I really loved the execution piece. It's simple. It's Ruby. It's Web scrapping bundle of tools.

Media_httpblogspcworldcomphoneconnectionarchivesmashupjpg_pmhykakjnacnacy

Well, only theoretically though. It never worked for me.

I've tried to run the sample code:

white@iwhite:~> cat scr.rb
require 'rubygems'
require 'scrubyt'
google_data = Scrubyt::Extractor.define do
   #Perform the action(s)
   fetch 'http://www.google.com/ncr'
   fill_textfield 'q', 'ruby'
   submit
   #Construct the wrapper
   link "Ruby Programming Language" do
   url "href", :type => :attribute
   end
   next_page "Next", :limit => 2
end
puts google_data.to_xml

And, guess what, it doesn't run.

white@iwhite:~> ruby scr.rb
/Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/navigation/agents/mechanize.rb:178:in `fill_textfield': undefined method `[]=' for nil:NilClass (NoMethodError)
   from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/navigation/navigation_actions.rb:27:in `eval'
   from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/navigation/agents/mechanize.rb:178:in `fill_textfield'
   from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/navigation/navigation_actions.rb:27:in `fill_textfield'
   from scr.rb:8
   from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:75:in `instance_eval'
   from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:75:in `initialize'
   from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:32:in `new'
   from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:32:in `define'
   from scr.rb:4

I've found several error reports similar to this one, but none was answered and even discussed. Any ideas?

P.S. Well, yes, I tried different codes, but while it eats up the fetch, it never works with submit.

gem mashup ruby scraping screen scrubyt

0 Comments

Twitter is shifting into JVM

Build Your Own Backup System on Linode

New routes in Rails 3

Ruby Quicktips

How programmers see each other...

Rails in a Nutshell

Scrubyt in the house

Problem with Scrubyt

About Olexandr Prokhorenko

Search Blog

Tags