As a trivial example of the design scenario I was working with, consider the case of an object-oriented vegetable garden. Vegetables come in all shapes, sizes, and colors, but we might want to say that all vegetables should be green unless we’ve said otherwise. We might start modeling our vegetable garden with a Vegetable class, and we could set a color attribute on it with a default value of "green". Lettuce, which happens to be green, could inherit that attribute from Vegetable. Eggplant, however, should redefine color to be "purple".

While certainly a contrived and flawed example, it demonstrates the behavior I was looking for. Let’s see how Ruby’s various flavors of class attributes can help us solve this design problem — or not.

First up, class variables:

class Vegetable
  @@color = 'green'
  def color
    @@color
  end
end

class Eggplant < Vegetable
  @@color = 'purple'
end

Vegetable.new.color  # => "purple"
Eggplant.new.color   # => "purple"

I wasn’t expecting that! Apparently class variables are shared among subclasses, so you can’t redefine their value in subclasses without changing the value in the base class.

Next up, class instance variables:

class Vegetable
  @color = 'green'
  class << self
    attr_reader :color
  end
  def color
    self.class.color
  end
end

class Lettuce < Vegetable
  # no need to set @color here, since lettuce is green ... right?
end

Vegetable.new.color  # => "green"
Lettuce.new.color    # => nil

No love here, either: class instance variables are not accessible from subclasses at all. Probably for the better, since the code needed to access class instance variables from instances is even uglier than that needed to access class variables from instances.

Class constants are right out:

class Vegetable
  Color = 'green'
  def color
    Color
  end
end

class Eggplant < Vegetable
  Color = 'purple'
end

Vegetable.new.color  # => "green"
Eggplant.new.color   # => "green"

Class constants are statically bound, so the polymorphic call to Vegetable#color from an Eggplant instance references the Color constant defined in Vegetable, not the one defined in Eggplant.

Giving up on the class attributes approach, I resorted to defining the attributes at the instance level. I considered explicitly setting a @color instance variable in the class initialize method, but then the attribute wouldn’t be constant. Instead, the simplest implementation that does what I want seems to be to use methods that return constant values:

class Vegetable
  def color
    'green'
  end
end

class Lettuce < Vegetable
end

class Eggplant < Vegetable
  def color
    'purple'
  end
end

Vegetable.new.color  # => "green"
Lettuce.new.color    # => "green"
Eggplant.new.color   # => "purple"

So as it turns out, each of Ruby’s class attribute mechanisms behaves differently in subclasses. I’m sure class variables, class instance variables, and class constants have their utility, but they aren’t useful for defining constant attributes shared by all instances of a class, but which can be redefined in subclasses.

SOAP support, which ought to be fully baked in Ruby by now, is still somewhat painful to work with. In Perl, SOAP just works. When I wrote our release orchestration tool a year ago, it took way longer than it should have to get Ruby talking to the SOAP iControl interface on our BigIP load balancers. By contrast, it took all of five minutes to get the Perl sample working — and that includes time spent installing the SOAP::Lite CPAN module.

Using Rails for the first time in a recent project, I was immediately struck by how little work is required to get a web app off the ground. I almost felt guilty for writing so little code. But a lot of the clever Rails magic that’s supposed to make life easier, didn’t. While error messages like, “Expected foo.rb to define Foo” seem pretty straight-forward, they are maddening when foo.rb does indeed define Foo. For their next trick, the Rails developers ought to use their meta-programming fu to produce intelligible error messages!

We recently ported a Rails app to JRuby, and straight away we ran into bugs. JRuby couldn’t call Java correctly, and it had a file descriptor leak in Net::SSH that caused the site crawler component of our application to go belly-up after a few hours. And we should have known better than to try talking to Oracle from JRuby on Rails. The activerecord-jdbc-adapter component had myriad issues — goofy things like "uninitialized constant ActiveRecord::VERSION", improper column name quoting, and incorrect integer datatype coercions. Finally we gave up and ported the database to MySQL.

I understand that Ruby and its libraries are open-source efforts written mostly by unpaid enthusiasts, so I try not to get too upset when things don’t work correctly. I wish I had the time to jump in and submit patches to fix issues when I run into them.

Tracing the application’s system calls with truss revealed that the process was getting EPERM errors while trying to read the CSS files, which live on NFS. One of our more clever engineers decided to start up the application manually, not via the code deployment tool, and found that the CSS loaded just fine when the Java process was invoked directly from the shell. He compared user and group ids, as reported by ps, of JVMs started by our tool and those started manually and found no differences. Hmm.

When looking at the processes’ /proc/<pid>/cred files, however, some differences were apparent. The cred file contains binary data and is best viewed with od:


$ od -X /proc/$$/cred
0000000 00002716 00002716 00002716 0000000a
0000020 0000000a 0000000a 00000002 0000000a
0000040 0000000e
0000044


The file consists of a sequence of 32-bit id values in the following order:

* uid
* euid
* suid
* gid
* egid
* sgid
* supplemental group ids …

You can see how that maps to decimal ids by comparing with id output:


$ id -a
uid=10006(clay) gid=10(staff) groups=10(staff),14(sysadmin)


[Solaris geek aside: remember when you wanted to be a member of the sysadmin group so you could run the handy-dandy admintool?]

So what we noticed was that while the manually started JVM and the JVM launched via our code deployment tool had identical uid/euid/sgid and gid/egid/sgid values, they had different supplemental group id lists. Notably, the JVM running under the code deployment tool still had a gid of 0 in its supplemental group list. Letting our Java application servers traipse around the filesystem with elevated privileges is perhaps not the best “feature” we’ve ever implemented.

Trust but verify might be a good foreign policy, but our NFS server wasn’t having any of it. It thoroughly distrusted the Java app servers claiming to have elevated privileges, and rewarded them with EPERMs for their trouble. Root squash is, after all, a pretty common NFS security measure.

As it turns out, I had implemented a new feature in the code deployment agent to make it switch user id on startup. Previously we handled the user switch by launching the tool under su, but that approach prevented the tool from writing its pid file to the root-owned /var/run directory. The solution, I thought, was just to call setgid() followed by setuid(). We tested that code by verifying the user and group ids with ps, and it seemed to work just great.

Quick: what’s wrong with this?

    def HostUtils.switch_user user
      pwent = Etc::getpwnam(user)
      Process::GID::change_privilege(pwent.gid)
      Process::UID::change_privilege(pwent.uid)
    end

Maybe several things, but certainly one thing is that I’ve completely neglected supplemental group ids. I should have written:

    def HostUtils.switch_user user
      pwent = Etc::getpwnam(user)
      Process::initgroups(user, pwent.gid)
      Process::GID::change_privilege(pwent.gid)
      Process::UID::change_privilege(pwent.uid)
    end

That call to Process::initgroups makes all the difference. After making the change, the apps could access NFS and our test site looked all pretty again. Good thing we caught it when we did!

Turns out this is a fairly common problem, and I feel especially dumb for overlooking something so obvious. Live and learn.