A brief intro to Red5. Overview, creating an app, interacting with the app, installing Red5 on FC, and using SharedObjects.
These magnets are shared across all current viewers on the system. Go ahead, open two browsers! Continue reading...
<object height='400px' width='500px'>
<param>
<param>
<param>
<param>
<embed src='http://elctech.com/assets/2008/1/25/fridge.swf' height='400px' width='500px'>
</object>
Red5 is an Open Source Flash Server written in Java that supports:
Streaming Audio/Video (FLV and MP3),
Recording Client Streams (FLV only),
Shared Objects,
Live Stream Publishing,
Remoting”
Red5 is essentially an open source alternative to Adobe’s Flash Media Server. Both technologies leverage Adobe’s RTMP (Real Time Message Protocol) to stream small binary fragments in real time. The packets are multiplexed and therefore only one persistent connection is needed for each user. Objects are serialized via AMF (Action Message Format) which is what we will be using in this tutorial to create SharedObjects, which as the name suggests, are objects (in this case refrigerator magnets!) that are shared amongst all viewers of the application.
Download Red5
Once installed, navigate the Red5 root directory to get started.
Just the important stuff...
In the root Red5 directory you’ll find red5.jar which holds a pre-compiled jar holding the red5 java classes. You’ll use this later to compile your main application class.
The directory webapps is where you will be creating new Red5 applications. All new applications have a consistent structure which is laid out as a template for you in doc/templates
Creating a new Red5 application is simple. We just create a new folder in webapps/<new>.
We will create an app called elcMagnets so we create a directory for it and now have webapps/elcMagnets. Red5 expects us to also have a directory inside our new application named WEB-INF/ so we create that as well, and now have webapps/elcMagnets/WEB-INF.
There are 4 default configuration files in doc/templates/myapp/WEB-INF that we will modify and use in our app, so we can copy those over to our elcMagnets directory. (log4j.properties, red5-web.properties, red5-web.xml, web.xml)
You can read up about all the properties and handlers specified in these files here.
For now, we only want to customize these files for our current app. We need to change the webapp.contextPath specified in red5-web.properties to reflect our newly created app. Take note of the virtualHosts parameter and remember to deal with it on deployment to a remote host.
webapp.contextPath=/elcMagnets
webapp.virtualHosts=localhost, 127.0.0.1
The only other configuration file we will modify for now is red5-web.xml. For this we need to specify the handler that is invoked when users connect to the app. You can read up on handlers here.
For the purposes of this demo app we will only be creating a web.handler. We will end up writing Application.java as the web.handler inside the a package called package org.red5.server.webapp.elcMagnets so we will set our web.handler to org.red5.server.webapp.elcMagnets.Application.
<bean id=”web.handler”
class=”org.red5.server.webapp.elcMagnets.Application”
singleton=”true” />
Lastly we will comment out the sample service bean since we will not be using that in this demo.
<!-- <bean id=”myhandler.service”
class=”the.path.to.my.ServiceHandler”
singleton=”true” /> -->
To keep things organized, we will create the directories lib/, classes/, src/ inside webapps/. Next we will create Application.java inside of src/.
As you recall, Application.java will serve as our web.handler, and therefore needs to extend ApplicationAdapter. (For more info on handlers, see the ‘HOWTO-NewApplications.txt’) In short, it allows you to stack functionality on top of existing events (connect, disconnect, join, leave, start, stop) or create your own.
Our example handler will be trivial, but illustrate calls to the Red5 server.
package org.red5.server.webapp.elcMagnets;
import org.red5.server.adapter.ApplicationAdapter;
public class Application extends ApplicationAdapter {
public String high5Red5(int how_many)
{
String highFives = “”;
if(how_many==0){ return “I dont like you anyways, bro.”; }
for(int i=0;i<how_many;i++){ highFives += “*Smack*\n”; }
return highFives;
}
}
Later, we can directly call this handler in ActionScript...
nc.call("high5Red5",nr,5);
// Or more generally...
netconnection.call("function",responder,arguments.....)
What we want to end up with is a nicely packaged up elcMagnets.jar located in lib/
So from src/ we compile Application.java to the classes/ directory.
javac -classpath ../../../../red5.jar -d ../classes Application.java
Now lets create our jar file. First we need our MANIFEST.MF. If you aren’t familiar with it...
Manifest-Version: 1.0
Ant-Version: Apache Ant 1.7
Created-By: 1.5.0_06-b05 (Sun Microsystems Inc.)
Class-Path: ../../../../red5.jar
(you can try yum/port install ant, but seem to remember building it)
jar -cmf MANIFEST.MF ../lib/elcMagnets.jar ../classes/org/red5/server/webapp/elcMagnets/Application.class
You should now be able to start up your Red5 server and have it load up our new elcMagnets application.
Flash AppNext we will create an AS3 flash app to interact with our Red5 application and utilize SharedObjects. Fridge.as (Document Class)
(I'm just going to post code, comment if you need any explanation.)
// =======================================
// Cary Dunn <cdunn@elctech.com>
// ELC Technologies http://www.elctech.com
//
package
{
import flash.events.*;
import flash.display.MovieClip;
import flash.text.TextField;
import flash.errors.IOError;
import flash.system.Security;
import caurina.transitions.Tweener;
import flash.net.*;
import com.elctech.Magnet;
public class Fridge extends MovieClip
{
NetConnection.defaultObjectEncoding = ObjectEncoding.AMF0 ;
private var nc:NetConnection = null;
private var alphabet:Array = new Array;
private const LETTERS:Array = ["A","B","C","D","E","F","G","H","I","J","K","L","M","N","O","P","Q","R","S","T","U","V","W","X","Y","Z",",","!"]
public function Fridge():void
{
nc = new NetConnection();
nc.addEventListener(NetStatusEvent.NET_STATUS,netStatusHandler);
var nr:Responder = new Responder(onHighFives,null);;
nc.connect("rtmp://localhost/elcMagnets");
nc.call("high5Red5",nr,5);
}
private function onHighFives(resp:String):void
{
trace(resp);
}
private function netStatusHandler(event:NetStatusEvent):void
{
trace(event.info.code);
if(event.info.code == "NetConnection.Connect.Success")
{
createAlphabet();
}
}
private function createAlphabet():void
{
for(var i:uint = 0; i < LETTERS.length; i++)
{
var new_magnet:Magnet = new com.elctech.Magnet(LETTERS[i],nc,i);
addChild(new_magnet);
alphabet.push(new_magnet);
}
}
}
}
com.elctech.Magnet.as
// =======================================
// Cary Dunn <cdunn@elctech.com>
// ELC Technologies http://www.elctech.com
//
package com.elctech
{
import flash.display.MovieClip;
import flash.display.DisplayObject;
import flash.display.DisplayObjectContainer;
import flash.geom.Rectangle;
import flash.events.*;
import flash.display.Stage;
import flash.text.TextField;
import flash.net.*;
import flash.utils.Timer;
import caurina.transitions.Tweener;
public class Magnet extends MovieClip
{
private var _id:uint = 0;
private var _so = null;
private var _nc:NetConnection = null;
private var _l:String = null;
static internal var dragging:Boolean = false;
static const MAGNET_COLORS:Array = [0xFF3333, 0x3399FF, 0x339966, 0x993399, 0xFF9933]
static internal const ELASTICITY:Number = .5;
static internal const FRICTION:Number = .5;
private var ax:Number = 0;
private var ay:Number = 0;
private var vx:Number = 0;
private var vy:Number = 0;
public function Magnet(l:String, nc:NetConnection, i:uint):void
{
this.letter.text = l;
this.letter.textColor = MAGNET_COLORS[Math.round(Math.random()*(MAGNET_COLORS.length-1))];
this.x = Math.round(Math.random()*390)+67;
this.y = Math.round(Math.random()*260)+20;
trace("New Magnet " + l);
_l = l;
_nc = nc;
_id = i;
_so = SharedObject.getRemote("letter_"+_id, _nc.uri, true);
_so.addEventListener(SyncEvent.SYNC,syncEventHandler);
_so.connect(_nc);
this.buttonMode = true;
this.addEventListener(MouseEvent.MOUSE_DOWN, function(evt:Event):void {
dragging = true;
stage.addEventListener(MouseEvent.MOUSE_UP, stopDragging);
evt.target.parent.addEventListener(Event.ENTER_FRAME, updatePosition);
});
this.addEventListener(MouseEvent.MOUSE_UP, stopDragging);
}
private function stopDragging(evt:Event):void
{
this.removeEventListener(Event.ENTER_FRAME, updatePosition);
stage.removeEventListener(MouseEvent.MOUSE_UP, stopDragging);
dragging = false;
}
private function updatePosition(evt:Event):void
{
ax = (stage.mouseX-this.x+Math.random()*5)*ELASTICITY;
ay = (stage.mouseY-this.y)*ELASTICITY;
vx += ax;
vy += ay;
vx *= FRICTION;
vy *= FRICTION;
this.x += vx;
this.y += vy;
this.scaleX = (100-Math.abs(ax)*2)/100;
this.scaleY = (100-Math.abs(ay)*2)/100;
this.rotation = ax*3;
_so.setProperty("currentPosition", {x: evt.target.x, y: evt.target.y});
}
private function syncEventHandler(evt:SyncEvent):void
{
trace("Sync for letter "+_l);
if(dragging){ return; }
Tweener.addTween(this, {x:_so.data.currentPosition.x, y:_so.data.currentPosition.y, time:0.5, transition:"easeOutBack"});
}
}
}
Download Project Source
This section documents my trials and tribulations with installing Red5 on an Amazon EC2 instance running a standard FC4 image. This is a nasty (but working) implementation of this.
Some of the rpms are unnecessary, but some of the yum packages were failing on me. Anyways, this might help someone...
$ yum install gcc
$ rpm -ivh ftp://rpmfind.net/linux/fedora/core/updates/4/i386/libstdc++-devel-4.0.2-8.fc4.i386.rpm
$ rpm -ivh ftp://rpmfind.net/linux/fedora/core/updates/4/i386/gcc-c++-4.0.2-8.fc4.i386.rpm
$ yum install fedora-rpmdevtools
$ fedora-buildrpmtree
$ cd /etc/yum.repos.d/
$ wget http://www.jpackage.org/jpackage.repo
---->>> download jdk 5.0 update .bin from <a href="http://java.sun.com/javase/downloads/index.jsp" target="_blank">sun</a>, scp it up to server (requires accepting of TOS)
---->>> cp jdk-....bin rpmbuild/SOURCES
$ wget http://mirrors.dotsrc.org/jpackage/1.7/generic/non-free/SRPMS/java-1.5.0-sun-1.5.0.14-1jpp.nosrc.rpm
$ rpmbuild --rebuild java-1.5.0-sun-1.5.0.14-1jpp.nosrc.rpm
$ cd ~/rpmbuild/RPMS/i586/
$ (echo config gpgcheck 0; echo localinstall java-1.5.0-sun*.rpm; echo run) > yum-cmd
$ yum shell yum-cmd
$ rm yum-cmd
$ cd ~/
wget http://archive.apache.org/dist/ant/binaries/apache-ant-1.7.0-bin.tar.gz
$ cd /usr/local/
$ tar -zxf ~/apache-ant-1.7.0-bin.tar.gz
$ mv apache-ant-1.7.0 ant
$ vim /etc/profile
PATH=$PATH:$HOME/bin:/usr/local/ant/bin
export PATH
$ source /etc/profile
$ tar -zxvf red5-0.6.3.tar.gz
$ cd red5-0.6.3
$ make
$ make install
$ /usr/lib/red5/red5.sh
At this point if you can hit http://{ip here}:5080 you are golden.
Thanks to some loud complaining from the community (including myself), GitHub will soon support OpenID.
Awesome.
It looks like Nick Kallen’s wildly popular has_finder plugin will be making its way into Rails 2.x in the form of named_scope. Observe:
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | class User < ActiveRecord::Base named_scope :active, :conditions => {:active => true} named_scope :inactive, :conditions => {:active => false} named_scope :recent, lambda { { :conditions => ['created_at > ?', 1.week.ago] } } end # Standard usage User.active # same as User.find(:all, :conditions => {:active => true}) User.inactive # same as User.find(:all, :conditions => {:active => false}) User.recent # same as User.find(:all, :conditions => ['created_at > ?', 1.week.ago]) # They're nest-able too! User.active.recent # same as: # User.with_scope(:conditions => {:active => true}) do # User.find(:all, :conditions => ['created_at > ?', 1.week.ago]) # end |
All the goodness you’ve come to love in has_finder is now available as named_scope – plus you get some extra goodies too. User.all is given to you for free as an alias for User.find(:all).
AdvancedFor those with more discriminating needs, don’t forget some of these has_finder tidbits:
Passing ArgumentsPass in arguments to your named scopes to specify conditions (or other props) at run-time.
| 1 2 3 4 5 | class User < ActiveRecord::Base named_scope :registered, lambda { |time_ago| { :conditions => ['created_at > ?', time_ago] } end User.registered 7.days.ago # same as User.find(:all, :conditions => ['created_at > ?', 7.days.ago]) |
Extend named scopes (in a similar fashion to association extensions).
| 1 2 3 4 5 6 7 8 9 10 | class User < ActiveRecord::Base named_scope :inactive, :conditions => {:active => false} do def activate each { |i| i.update_attribute(:active, true) } end end end # Re-activate all inactive users User.inactive.activate |
You can also pass around scopes as first class objects using scoped (a named scoped provided to you for free) as a way to build hairy queries on the fly.
| 1 2 3 4 5 6 7 8 9 | # Store named scopes active = User.scoped(:conditions => {:active => true}) recent = User.scoped(:conditions => ['created_at > ?', 7.days.ago) # Which can be combined recent_active = recent.active # And operated upon recent_active.each { |u| ... } |
named_scope is a truly great feature. If you haven’t started using it yet, do so. You won’t know how you lived without it. Major thanks goes out to Nick.
tags: ruby, rubyonrails
Since the initial release of the teaser screencast, a lot of people have asked about Passenger’s (a.k.a. mod_rails) performance compared to Mongrel and Thin. Here’s the benchmark that you’ve all been waiting for.
In this benchmark, we compare the following software:
System specification:
Test applications:
Typo and Eldorado both use SQLite 3. Petstore uses MySQL, because for some reason Petstore performs very badly when SQLite 3 is used.
People have also recommended Lovd By Less as test application, but I couldn’t get it working (database setup failed).
The applications, when served by Passenger, are setup at the following test domains:
Both the Mongrel cluster and the Thin cluster are setup at http://mongrel_cluster.test/. (Only either Mongrel or Thin is running during the benchmark, not both at the same time.)
And of course, we always use the production environment.
Backend configuration:
First we benchmark Typo served by Mongrel clusters. Because the actual application code is loaded by Mongrel during the first request, we do not want to count the first request’s time. So we run the following command to “warm up” the Mongrels, and discard its result:
ab -n 1000 -c 100 http://mongrel_cluster.test/Next, we test 10,000 requests with 100 concurrent users:
ab -n 10000 -c 100 http://mongrel_cluster.test/Result:
Concurrency Level: 100 Time taken for tests: 32.628480 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Total transferred: 54890000 bytes HTML transferred: 51480000 bytes Requests per second: 306.48 [#/sec] (mean) Time per request: 326.285 [ms] (mean) Time per request: 3.263 [ms] (mean, across all concurrent requests) Transfer rate: 1642.83 [Kbytes/sec] receivedWe repeat these steps Petstore and Eldorado.
Petstore:
(for Petstore we benchmark the URI “/shop/viewCategory.shtml?category=DOGS” because it’s written in the INSTALL file)
Eldorado:
Concurrency Level: 100 Time taken for tests: 70.889388 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Total transferred: 41130000 bytes HTML transferred: 35530000 bytes Requests per second: 141.06 [#/sec] (mean) Time per request: 708.894 [ms] (mean) Time per request: 7.089 [ms] (mean, across all concurrent requests) Transfer rate: 566.60 [Kbytes/sec] received Benchmark: Thin clustersThe benchmarking steps for Thin are the same.
Typo:
Concurrency Level: 100 Time taken for tests: 25.795470 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Total transferred: 54910000 bytes HTML transferred: 51480000 bytes Requests per second: 387.66 [#/sec] (mean) Time per request: 257.955 [ms] (mean) Time per request: 2.580 [ms] (mean, across all concurrent requests) Transfer rate: 2078.78 [Kbytes/sec] receivedPetstore:
(for Petstore we benchmark the URI “/shop/viewCategory.shtml?category=DOGS” because it’s written in the INSTALL file)
Eldorado:
Concurrency Level: 100 Time taken for tests: 71.311031 seconds Complete requests: 10000 Failed requests: 5 (Connect: 0, Length: 5, Exceptions: 0) Write errors: 0 Non-2xx responses: 5 Total transferred: 41137070 bytes HTML transferred: 35518568 bytes Requests per second: 140.23 [#/sec] (mean) Time per request: 713.110 [ms] (mean) Time per request: 7.131 [ms] (mean, across all concurrent requests) Transfer rate: 563.34 [Kbytes/sec] received Benchmark: PassengerThe benchmarking steps for Passenger are the same.
Typo:
Concurrency Level: 100 Time taken for tests: 25.131980 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Total transferred: 55200000 bytes HTML transferred: 51260000 bytes Requests per second: 397.90 [#/sec] (mean) Time per request: 251.320 [ms] (mean) Time per request: 2.513 [ms] (mean, across all concurrent requests) Transfer rate: 2144.92 [Kbytes/sec] receivedPetstore:
(for Petstore we benchmark the URI “/shop/viewCategory.shtml?category=DOGS” because it’s written in the INSTALL file)
Eldorado:
Concurrency Level: 100 Time taken for tests: 72.587018 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Total transferred: 41660000 bytes HTML transferred: 35530000 bytes Requests per second: 137.77 [#/sec] (mean) Time per request: 725.870 [ms] (mean) Time per request: 7.259 [ms] (mean, across all concurrent requests) Transfer rate: 560.47 [Kbytes/sec] received SummaryHere’s a nice summary of the requests per second, in a table and in a graph:
| Mongrel | 306.48 | 91.09 | 141.06 |
| Thin | 387.66 | 103.23 | 140.23 |
| Passenger | 397.90 | 95.60 | 139.27 |

I was pleasently surprised by the results. During Passenger’s development, our performance baseline was Mongrel behind mod_proxy. According to earlier (simple and naive) tests, Passenger performed similar to Mongrel. I expected Thin to be a bit faster than Passenger. These new benchmarks however suggest Passenger is faster than Mongrel, and is on par with Thin.
But you know what they say: there are lies, damn lies, and statistics. Your hardware and software is different, so if you benchmark it yourself, you’re likely to get (slightly?) different results.
There is a classic tradeoff between threaded servers and event driven servers. Event driven servers tend to be much faster than threaded servers when all the requests are fairly fast. But the event model falls down if you have long requests like file uploads or reporting actions. This is because the long action blocks the event loop, effectively keeping other requests from running.
There are two new event driven ruby webservers at your disposal these days, Thin and Ebb. Both of these servers support the Rack web server interface that merb uses. Until now both of these servers were not the best choice for file uploads or long blocking actions but that’s all changing.
Both ebb and thin have added a deferred?(env) method to their rack adapter interface. Both webservers will call this method on your Rack @app object before they call your call(env) method. This allows your rack adapter to determine if the request should be run in its own thread if it’s slow.
I’ve just committed support for this deferred_actions construct in merb-core. If you want to run on thin or ebb but you have say a file upload action and some slow reporting actions you could add this to your init.rb in your app:
Merb::Config[:deferred_actions] = ["/uploads/create", "/reports/longaction"]What this means is that all of your actions will run in fast event driven mode except for requests to /uploads/create and /reports/longaction. Any request for either of these urls will be served from a newly spawned thread, all other requests will be served by the main event loop.
This allows us to have the best of both world. Combining threaded and event driven styles to use the strengths of both makes a lot of sense here.
Cheers to the authors of ebb and thin for making the same interface. All of this requires the HEAD versions of ebb or thin so if you want to play along at home you will need to build thin or ebb from source on github: ebb and thin
We’ve released the 0.9.1 version of merb to the merbivore.com gem server. This release has a lot of polish and is getting very close to being stable api wise after the big 0.5.3 → 0.9.x refactoring.
You can see the big changelog here.
I’m pretty happy with how the codebase is shaping up, this was a major refactoring of merb and we’ve come out with a very clean system. Performance is improved quite a bit from older 0.5 versions of merb.
We’ve split the code base into multiple parts, merb-core, merb-more and merb-plugins. merb-core is the heart of the system, it has the rack abstraction along with the dispatcher, router, controller and view layers. You can make very fast, small footprint services and apps with just merb-core.
merb-more has a bunch of add-ons for core. More consists of: merb-action-args merb-assets merb-builder merb-gen merb-haml merb-mailer merb-parts merb-testAnd merb-plugins consists of:
merb_activerecord merb_datamapper merb_helpers merb_param_protection merb_sequel merb_stories merb_test_unitWow that’s a lot of gems! Merb is built in a modular way that allows you to cherry pick features so you never have to load code you aren’t going to use. This helps keep the memory footprint down when building service style apps. But still allows for all the advanced features you want.
You see, Merb is built on rubygems and Merb plugins are just rubygems so plugins have the same standing as built-ins code loading wise. Merb is just begging you to peek under the hood to see how it ticks ;)
To make things easier to get started we still have a merb gem that will install all of merb-core and merb-more for you. You should uninstall all of your old merb gems before you install the new version.
$ sudo gem install merb -y --source http://merbivore.comMerb development has moved to GitHub for source control and Lighthouse for ticketing.
The best place to get merb questions answered is still #merb on irc.freenode.net. We will have a wiki and a google group up shortly.
Let us know what you think, kick the tires and all that. After it settles for a few days we will push it to rubyforge, hopefully by this weekend.
RailsConf this year has a pre-conference tutorials days which was optional and it’s starting now. However attendance seems already pretty big and there was a quite long queue to the registrations desks. I am now at Denny’s for my breakfast and will skip the first tutorial and meet with Tony to apply the finishing touch to our talk. We have a lot of code to present, I hope we can cover all the details in the 3.5 hours we have. We will publish the slides and the source code of the 10 apps we are going to present, so if you didn’t make it to RailsConf or our talk you still can get a glimpse at what we are presenting. The slides will have code extracts, but during the talk we will not use them as we will run and show the apps, so there is still good value if you attend the talk :-). See you there!
Enjoy, Daniel.
Episode 20. Info on JRuby, Deploying Rails Applications, Matz' tech talk, the new git bundle for TextMate, some tutorials, a though-provoking post by Jay Fields, and some great information on scaling. As always, a big thanks to Chu Yeow for the edge Rails updates.
Subscribe via iTunes - iTunes only link.
Download the podcast ~18 mins MP3.
Subscribe to feed via RSS by copying the link to your RSS Reader
Deploying Rails Applications book complete!
The “Deploying Rails Applications” book by Ezra Zygmuntowicz, Bruce Tate, and Clinton Begin is now complete. It’s been available as a beta book for a few months now, but this final release includes a chapter on Apache and Nginx scaling, setting up your own XEN installation, and setting up Mysql Master Slave, and Master Master replicated databases.
Google Tech Talk with Matz
Google put up a tech talk from Matz on YouTube. Matz explains the different versions of Ruby and goes in to Ruby 1.9 quite a bit. It’s about 50 minutes and he gives some really interesting info out.
Thomas Enebo's Announcement of JRuby 1.1 RC2
Charlie Nutter on the new release and what's next
Last saturday Thomas Enebo announced the release of JRuby 1.1 RC 2, which represents a focus on speed and refinement. With this release most benchmarks are pretty close or are sometimes faster then Ruby 1.9, it also reduced overall memory usage.
Rush, the Ruby Shell
Two weeks ago we talked about Heroku, the web application that allows you to code and deploy rails applications without doing any configuration. Adam Wiggen of the Heroku team just released Rush, a replacement for bash and ssh which uses Ruby syntax.
Farm work to EC2 using processor_pool and Sanatra
Last week Ari Lerner on the CitrusByte blog posted an awesome tutorial about how to offload image processing onto an EC2 cloud with an S3 storage backend using Sinatra with the processor pool gem.
Rails Messaging Tutorial
If you ever find yourself coding up a web application with a user messaging system, you should definitely check out the Rails Messaging Tutorial by NovaWave Solutions. This tutorial does a good job of getting you started on the right foot with several specifications that aren’t really intuitive from the get go. It’s also a great read if you’re new to rails and you want to read someone else’s code to improve.
Biggest Rails App
Last week we talked about a Rails website that is getting 100 million page views a month. Well, we found one that takes more, and this one isn’t using any page caching. “Friends For Sale” is a facbook application written in Rails, and is currently receiving 300 Million page views/month.
Git Bundle for Textmate
If you’re starting to use git and you already use Textmate you may want to pickup the new and improved Git bundle released last week.
Learn Git 10 Different Ways
While we’re on the topic of Git, Rob Sanheim put up a blog post titled “Learn Git 10 Different Ways” which, contrary to what you might thing, links you to 10 different resources for learning git.
Skinny Request, Fat Backend
Eric Allam, a fellow Orlando Ruby programmer wrote up a really interesting article last week entitled "Skinny Request, Fat Backend." The article explains why it’s important that as rails developers we keep our requests as skinny as possible, because of how rails locks every time a request comes in. By using cache columns, ensuring calls to webservices don’t timeout, offloading image manipulation and sending emails, and lastly he gives an example of how to isolate a controller action outside of rails by creating a small Rake app which uses thin.
El Dorado – Community Web Application
You know how PHP has all of those community systems.. with a message board, events, files, and user system? Now Ruby has something similar called “El Dorado” which is perfect if you need to throw together a website to communicate with your friends, or maybe even for your Ruby Users Group.
Gems in Rails
Last week Jay Fields wrote up an interesting article where he brings up the fact that Rails doesn’t really play well with Gems.
Mumble your Tests
If you’re developing in linux and are envious of all us Mac users who use growl for our autotest notifications, over on the Caffinated code blog this week you’ll find a script that will allow you to patch Mumbles to do the same thing.
Rails Undo Redo
There’s an easy way to implement undo and redo in your rails app thanks to Pascal on the Nano Rails blog. Pascal recently released the Rails Undo Redo plugin, which allows you to give exactly this type of functionality to your models and controllers.
Radiator: Build status on your Beta Brite
The guys over on the Something Nimble blog were doing a project where they needed to see progress of their continuous build status, like whether it succeeded or failed. They figured out how to hook up the results of the build to show on a giant LED and then released the application they use to interface with it called Radiator.
YUI 2.5.0 Released
For you javascript fans, Yahoo has released version 2.5.0 of the YUI library with lots and lots of goodness and changes.
Merb 0.9.3 is mostly (but not only) a bugfix release. We’ve not only added several features very useful for developers who want to use Merb for web services, but fixed many bugs, greatly improved spec coverage and quality, made documentation better, and did some performance-related work.
diffstat shows 95 files changed, 3359 insertions(+), 837 deletions(-)
Engine Yard is looking for one or two kick ass Erlang programmers to help work on our next generation cloud computing platform. Knowledge of ruby/xmpp/ejabberd/linux is a plus.
If you think you fit the bill then please email me directly at ezra@engineyard.com.
Erector is nearing its initial beta release.
Erector is a Builder-like view framework, inspired by Markaby but overcoming some of its flaws. In Erector all views are objects, not template files, which allows the full power of object-oriented programming (inheritance, modular decomposition, encapsulation) in views.
The latest Rails Podcast features Pivotal's Nathan Sobo talking about Treetop.
Treetop is a language for describing languages. Combining the elegance of Ruby with cutting-edge parsing expression grammars, it helps you analyze syntax with revolutionarily ease.
There are a few technical achievements in Passenger that we haven’t actively marketed. But we’d like the community to be aware of them, so we’ve written this blog post.
Not too long ago, Thin announced support for Unix domain sockets. This gave Thin an incredible speed boost. Switchpipe soon followed, with alpha support for Unix domain sockets.
It has always surprised me that the Ruby/Rails web containers didn’t support Unix domain sockets until early 2008. Unix domain sockets aren’t exactly rocket science or exotic: the X Window System has used them for client-server communication on localhost for as long as I can remember. Database systems such as MySQL prefer to use Unix domain sockets on localhost as well. It’s also well-known that Unix domain sockets are faster than TCP sockets.
There’s one down side to Unix domain sockets though: you have to remove them. Usually, server processes remove them during exit. But if the system crashes, or if those processes crash, then the socket files are never removed, and the system administrator will have to do that manually. Ouch, maintenance overhead.
Passenger uses Unix sockets extensively, and has done so since the first release. But there’s one difference compared to Thin and Switchpipe: Passenger uses Unix sockets in the abstract namespace (as opposed to on the filesystem) whenever possible. Abstract namespace Unix sockets have filenames, just like regular Unix sockets, but they do not appear as files on the filesystem. This means that after a reboot, or if Apache crashes, no stale files will be left on the filesystem. This way, the system administrator doesn’t have to manually remove stale files if something went wrong.
Passenger strives for a concept that we call “zero maintenance”. We believe that software should Just Work(tm), and system maintenance overhead should be as low as possible. The system administrator shouldn’t have to worry about stuff like stale files, if he doesn’t have to. Ideally, the system administrator should be able to forget that he ever installed Passenger at all. The usage of abstract namespace Unix sockets is one of the mechanisms that we use to achieve that goal.
PID files (or the lack thereof)Mongrel and Thin write so-called PID files to the filesystem. PID files contain the PIDs of background processes, and are used for shutting down those processes. But if the system crashes, or if those background processes crash, then the PID files will never be removed, and they become stale PID files. Early Mongrel versions refuse to start if there are stale PID files, forcing the system administrator to remove them manually. Passenger on the other hand doesn’t use PID files at all. If one shuts down Apache, then all Rails processes are shut down as well.
So what happens if Apache crashes? The Rails processes will exit as well. We use so-called “owner pipes” in Passenger, pipes that are shared between Apache and the Rails processes. Owner pipes are essentially a mechanism for reference counting processes. If Apache crashes, then the owner pipe is automatically closed. The Rails processes will detect this, and will shut down gracefully.
Today I had a few hours to spare and decided to try out Passenger. This blog is highly a high traffic website, but it has some crazy RewriteRules that I figured would test the limits of the module.
Everything appears to be working perfectly, and it took less than 5 minutes to set everything up. I’m seriously impressed at how simple this was. I’ll confess to being skeptical at first, but so far the package lives up to the promises on its website.
If you notice anything broken let me know. Now I have to figure out how to kill the rest of my spare time.
There is a great tutorial/screencast about a new merb feature that I really like a lot called merb-slices. This is now part of merb-more, contributed by a long time friend of mine, Fabien Franzen(worked on ez-where with me)
merb-slices are “Little slices of MVC cake”. These are self contained merb apps with models, controlers, views and assets that you can distribute as rubygems. You can mount a merb-slice at a specific point in your router definition and you can override any part of the slice up in your main app. So in a way these are similar to what Rails-Engines promise, except merb-slices are built into the framework and will not break when merb itself is updated.
Check out the tutorial/screencast for a peek at how merb-slices work.
I usually don’t blog about stuff that everyone else is obviously blogging about already. But I am making an exception, just this once, because I am excited about GitHub.
GitHub is a social source control platform based on Git. Git is a whole different animal than SVN. Git has no concept of source and checked out copy. You can commit locally, you can easily branch and merge, you can push some branches to some central repos, and keep other to yourself for now, its amazingly dynamic. But I’m not here to talk about Git so much.
The first part that makes GitHub cool is the social thing. It’s been really hard for me to get into social networks. I try every once and a while and find it hard to convince my peers to join me on said network. I can upload a widget, or have a friends list, but I never seem to find any functionality that keeps me using the network for any length of time. GitHub is finally a social network I can get into. You don’t have “friends”, you have contibutors, watchers and forkers. You can add people to your project as contributors, people can simply keep an eye on your project by watching it, or they can fork your code to make their own modifications in their own sandbox. It’s completely addicting to see how many “Watchers” I have today. As of this writing the Fleximage watcher page lists 9 watchers. Not bad for less than 1 week in the wild.
Now, lets say someone wants to add some useful functionality to Fleximage. They can fork my code, without my permission, and work on their useful feature. They then let me know saying “I did some cool stuff, check it out”. I can clone their working repo, test it out, and integrate the diff with my code with just a few commands. Even if someone forks my code in order to overlay a monkey watermark on every image, that’s still fine. They can keep their forked little sandbox for themselves or whoever else wants to use it and I never have to pull in their changes at all.
The second part that is awesome is what this does for open source. The pricing model really encourages open source. Open source hosting is free. Private repositories cost money. Let that sink in a minute. This means that its easier and cheaper to release your project open source than not to. Previously, to open source a project you needed server side repository hosting, need a website for your project, in order to get those free you needed the convoluted rubyforge interface or ad laden source forge. This made it more time consuming and costly than a private codebase. So the “default” mode for most apps was private, because it was easier.
What GitHub did, was to make open source easier and cheaper than closed source. So going forward my apps will be, by default, open source. Thats a big deal. Now the little nuggets that I write every now and again will be opensourced on GitHub. Some of them may turn out to be nothing. But other may get forked by someone far smarter than me, enhanced, and eventually turned into something totally cool. Such scenarios never would be possible if there wasn’t an easy and free open source hosting platform like GitHub.
GitHub, you guys rock. The worldwide open source community has been greatly enhanced by the presence of GitHub. Give it some time and I think GitHub will be the defacto open source standard.
I’m not going to lie and tell you that I’m great at task management. Frankly, I suck and I blame it on technology. If only there were a task system that could light a fire under my butt. We all know there isn’t, but the system that I’ve found to work best for me is Todoist. Todoist is really light and fast. Also, it has a simple API so it’s very extensible and already has a lot of tools surrounding it (quicksilver, launch, dashboard widget, mobile, gmail integration, etc).

The problem that I have with task management is two fold. First, I need reminders. My programming brain is sharply trained to scoot information in and out, only keeping what is immediately necessary. It’s great to have a to do list but if it doesn’t remind me, it barely works. The good news is that todoist has a premium service that is only $3/month (yes only $3) that allows for reminders in any form you like (twitter, sms, email, jabber, etc). First problem solved.
The second problem that I have is that sometimes a reminder is not enough. I personally need to see my upcoming projects, tasks and calendar items in front of me anytime I can. It hit me the other day that I look at my desktop a lot. This got me wondering if I could use the todoist API to put my upcoming, date sensitive tasks right there, on the desktop, in front of me whenever I’m on my laptop.
The SolutionI did some googleing and found Geektool, which I had installed/uninstalled before but this time seemed to have a purpose. The end result is in the screen cap below.

To do this, I created a quick and dirty ruby script to hit the todoist api and put out a simple list of all time sensitive tasks. 1) I put the following script at ~/bin/todoist.rb.
require 'yaml' require 'rubygems' require 'rio' require 'json' require 'active_support' def todoist_query(*args) token = args.pop queries = %Q{["#{args.flatten.join('","')}"]} url = "http://todoist.com/API/query?queries=#{queries}&token=#{token}" JSON.parse(rio(URI.encode(url)).read) end lines, now = [], Time.now config = YAML::load(rio(File.join(ENV['HOME'], '.todoist')).read) format = '%Y-%m-%dT%H:%M' days = (now..now.advance(:days => config['days'] || 3)).step(1.day) todoist_query("overdue", days.map { |d| d.strftime(format) }, config['token']).each do |type| next if type['data'].size == 0 lines << (type['type'] == 'date' ? Time.parse(type['query']).strftime('%b %d').gsub(/\s0/, ' ').upcase : type['type']) type['data'].each { |i| lines << " - #{i['content']}" } lines << "\n" end puts linesLike I said, it’s quick and dirty. 2) Now setup geektool to run the script at a given interval (I chose 2 minutes).

3) I created a file in my home directory to store my todoist token and the number of days I want to show. The main reason I separate the token from the ruby script is in case I feel like using it somewhere else on my system with another script. If my token ever changes, I can update it in one place and all the scripts will still use correct one. Anyway…I put the file at ~/.todoist and it looked something like this:
token: foobarbaztoken days: 4Now every two minutes the script runs and updates based on how I am updating my tasks at todoist.com. My tasks are continually in front of me, which helps keep them in my mind so that I actually get stuff done.
Other UsesSome other uses I could see valuable for this would be calendaring. Google Calendar has an API so you could hit that and show your upcoming events right by your tasks. Maybe there is some site that posts a lot of updates and it’s hard to keep up with by feeds? Just use feedtools and a tinye ruby script to show the latest posts for your perusal when you get a few seconds. Just some thoughts.
Anyone else do something along these lines? Have some code to share or thoughts on how to do it better? Let me know.
Here are my slides from my keynote at Mountain West Rubyconf this morning:
http://www.slideshare.net/ezmobius/merb-core/
GitHub has officially launched! No more invites required to sign up.
In preparation for their launch we set them up on a shiny new Engine Yard cluster. With all the adoption of git/github as well as Rails itself about to switch to github we wanted to make sure everyones favorite git repo hosting can scale as far as folks want to take it.
So go Sign Up already and fork your favorite projects to your hearts desire!
The ability to track whether or not your active record objects have been modified or not becomes a lot easier with the new dirty object functionality of edge rails. It’s dead simple, and pretty slick, to use:
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 | article = Article.find(:first) article.changed? #=> false # Track changes to individual attributes with # attr_name_changed? accessor article.title #=> "Title" article.title = "New Title" article.title_changed? #=> true # Access previous value with attr_name_was accessor article.title_was #=> "Title" # See both previous and current value with attr_name_change accessor article.title_change #=> ["Title", "New Title"] |
Beyond the clever attribute based accessor methods, you can also query to object directly for its list of all changed attributes. (Continuing from the previous example):
| 1 2 3 4 5 | # Get a list of changed attributes article.changed #=> ['title'] # Get the hash of changed attributes and their previous and current values article.changes #=> { 'title' => ["Title", "New Title"] } |
Note: Once you save a dirty object it clears out its changed state tracking and is once again considered unchanged.
| 1 2 3 | article.changed? #=> true article.save #=> true article.changed? #=> false |
If you’re going to be modifying an attribute outside of the attr= writer, you can use attr_name_will_change! to tell the object to be aware of the change:
| 1 2 3 4 | article = Article.find(:first) article.title_will_change! article.title.upcase! article.title_change #=> ['Title', 'TITLE'] |
And coming down the pipe is a feature that will make the most of this functionality – partial SQL updates that will only update attributes that have changed…
tags: ruby, rubyonrails
So after being in the works for almost 2 years now and after going through many rewrites as Rails deployment tech has changed, my book is finally done! I just have to thank all of my coauthors who put in tons of work to get this thing as polished as it is now. Especially Bruce Tate and Clinton Begin, thanks guys!
New in this release is the chapter on Apache and Nginx scaling as well as a performance chapter. ALso included is setting up your own Xen installation and setting up Mysql Master → Slave and Master → Master replicated databases.
If you were holding off on getting this book until the advanced content was in there then now is the time to snap it up. This is the final beta release and the book will be going to print fairly soon. I really like the way the book has come together and I think it is a wealth of knowledge about deploying Rails applications.
\m/ Yay! Writing a book was a lot more work then I thought it would be and I could not have done it without my wonderful co-authors and all of the great feedback from the early beta readers.
Thanks to everyone who provided feedback. You can grab your copy of the book here:
Deploying Rails Applications
Rack is an awesome webserver abstraction that distills the idea of a ruby web app down to an object or Proc that has a call method. The call method takes the rack environment, which is all of the cgi style headers for the current request, and returns an array of [status, headers, body]. The status is a number like 200 or 404, the headers is a hash of header key value pairs for the response and the body is the output from your application, The body must respond to each and yield lines or chunks of strings to be flushed down the socket to the client.
Here is the most basic example of a rack app using the rackup builder DSL for mounting and running rack apps:
rack_app = Proc.new do |env| # env here has all the headers you would expect [200, {"Content-Type"=>"text/html"}, "hello world!"] end run rack_appThis app will return “hello world!” to the client. Pretty simple eh?
Now that merb-core is all based on rack, there are some very interesting things you can do with this knowledge. In the config/ directory of a freshly generated merb-core app you will see a rack.rb file. By default the file just contains this:
run Merb::Rack::Application.newThis will run the main merb rack app class that handles the standard dispatching of requests through the whole merb framework. Now merb-core is small and fast, but what if you have some certain requests that don’t really need the router or controllers/views of a full merb stack.
Say we need to handle a ton of concurrent file uploads, and we don’t want to invoke the full merb stack just for these uploads. The solution is to use the config/rack.rb file in your merb app along with the Rack::Cascade middleware to mount a simple Proc object to handle the uploads instead of merb proper. Here is what a rack.rb file for this would look like:
uploader = Proc.new do |env| request = Merb::Request.new(env) if request.path =~ /\/images\/upload/ #file uploads can get the params from request.params and do whatever # you want with it, this allows for multiple concurrent uploads # with very minimal overhead, doesn't go through the merb # framework at all params = request.params FileUtils.mv params[:file][:tempfile].path, Merb.root / 'public' / 'uploads' / params[:file][:filename] headers = {"Content-Type"=>"text/html", "Location" => "/images"} [302, headers, "You are being redirected"] else [404, {}, "No Upload here, move along"] end end merb = Merb::Rack::Application.new run Rack::Cascade.new([uploader, merb])Rack::Cascade works by trying to call each app you specified in order and actually use the results from the first one that does not return a 404 error. So what our new uploader Proc/app does is creates a Merb::Request object and checks if the request.path matches /images/upload. If it does match then you can use code in here to handle the file upload and place it wherever you want. Once you’ve done whatever you need to the file upload you can redirect by setting the Location header like in the above example, or you can render out a page or return JSON or really anything you like.
Having the power of merb-core and also the power of raw rack access conveniently in the same app is very powerful. I think this is one very compelling thing merb-core has going for it now that we are all rack based.
Hopefully you enjoyed this installment of what is new and cool in merb. Hope to write some more articles shortly about special features in merb-core that go undernoticed.
So we are just getting started on mod_rubinius here at EY. We’ve hired Eero Saynatkari ( rue in the #rubinius irc channel) full time to work on the project.
The architecture for mod_rubinius is still up in the air at this point. We do know that it will be rack based so the interface from mod_rubinis into ruby apps will be via rack. Other then that we don’t yet know what the best way to architect the platform will be. It could be an embeded rubinius VM inside the apache processes. Or it could be a process manager that manages separate rubinius VM’s, or a combination of both of these approaches.
We would like to hear from you folks.. What would you like to see in a mod_rubinius? How is deployment of ruby apps painful for you and how would you like it to work in a perfect world?
We’d appreciate some feedback so please leave feature requests and ideas in the comments.
Want to work for the best and with the best? Are you interested in merb, rails, rubinius, and other ruby projects? Are you passionate about learning what you don’t know?
This position requires direct interaction with customers through the phone, ticketing system, irc, and email. You will be supporting some of the largest rails deployments on the internet. Can you handle it?
Desired: Experience with memcached, merb, monit, postgres, swiftiply.
Required: Solid ability to speak and write English. Good Attitude. Great work ethic. Experience with capistrano, mongrel, mysql, nginx, rails, subversion. Basic networking knowledge.
This position is right someone who isn’t afraid of hard work, loves facing new challenges each day, and wants to contribute to Engine Yard’s projects.
We offer comprehensive dental and medical insurance as well as the ability to work from home (or anywhere). Stock options are around the corner too.
Drop us a line at info@engineyard.com and tell us why you would be a great addition to our team. If you are the right person—we will send you an offer immediately. It’s a bonus if you are on the West Coast, maintain open source projects, or have a great community presence.