Monthly Archives: April 2013

Private Accessors in Ruby

This is a post about writing private method accessors in Ruby, but it takes me a while to get around to saying that because there’s a ton of backstory, which is of course my favorite part. If you want to skip ahead to the code, I have provided a means of conveyance compatible with your hypermedia information consumption device. (Be warned: you’ll miss the part about the monkey.)

The Principle of Least Access

So there’s this great idea called the Principle of Least Access. It is found in computer science, information security, and other fields. It’s more frequently called the Principle of Least Privilege, but the names are interchangeable and when I learned it POLA was an acronym you could pronounce.

Yeah, that’s exactly the kind of motivation that I would choose to make a design decision over. There’s a reason POODR is my favorite book of all time, and “pronounceable acronym” is reason number two. You can guess what reason number one is. (Hint: It’s also number two).

Anyway, POLA can be summed up nicely as

A thing must be able to access only the information and resources that are necessary for its legitimate purpose.

In programming, this rule is often misquoted as simply “make everything private until something else needs it”. This isn’t a bad rule of thumb, but it’s not a great one, and at first blush it really puts private methods at odds with testing code.

But Testing Private Methods is Haaaaaard!

My relationship with POLA has been rocky. I used to think it was scripture. Then I thought it was idiotic. Then I came to ruby where everything is either public or can be accessed anyway. We even had a splendid bout of drama in the ruby community over whether or not you should test private methods. I fought bravely on the side of “yes you should”, even going so far as to write some code with a humorous name to make it easier to test those private methods.

The two tenets of my argument were that

  1. You should test things that can break. If you’re going to put really complicated logic in private methods in your code, then they can break and you should test them, and
  2. Testing private methods through the public interface adds impedance to the testing process, and results in low-value tests that are very hard to construct.

But you know what’s really funny? When I really got into it with a bunch of people, even the people furiously certain that their stance on testing private methods was correct, I found out something crying out loud funny:

Ruby programmers don’t write private methods.

Test ‘Em? Let’s Not Even Write ‘Em

For the most part, we just… don’t. We know that a determined user can get at our guts, so we make no pretense of security or safety in marking something protected or private. If we’re having a good design day, we extract the private functionality to a class. If we’re having a bad design day, we shrug and say “Eh” and just leave everything public. The important thing is, when we came in from C++ or Java, we had to check at the door this notion that our encapsulation would protect us, and for most of us that was the whole point of private methods to begin with.

I watched Sandi Metz talk about creating high-value tests a while ago, and she talks about private methods–and about testing them. Her take was that private methods are a good place to put code that is uncertain or subject to change on a whim; that way no external code would be affected. Since the code is so volatile, any tests that touch it will break often, so if the purpose of a test is to provide lasting reassurance, it doesn’t make sense to test them.

One of the reason I love Sandi so much is that she gets the concept that everything is a tradeoff. When she was on Ruby Rogues I asked her about the case where I sometimes will use tests to figure out complicated code. I’ll inch forward bit by bit, and lock each bit down with a short-term test. She considered this for a long moment, and then announced her new rule for testing private methods:

“Don’t test private methods unless you really need to. In which case, test them as much as you want.” –Sandi Metz

And THAT, dear readers, is how I found myself poring through POODR and pondering the fact that keeping everything public is a very, very bad idea.

Making Everything Public Is Killing Us

Every public method, variable and accessor on a class is a message that can be fired at that object from anywhere in the system. I know this will seem strange to some rubyists, but we really have to get away from this notion of making everything public–or, at least, making things public that other objects have no business knowing about. Let’s quote Sandi again:

Each [object in a hard-to-maintain system] exposes too much of itself and knows too much about its neighbors. This excess knowledge results in objects that are finely, explicitly, and disastrously tuned to do only the things that they do right now…

The roots of this new problem lie not in what each class does but in what it reveals. –Sandi Metz (emphasis hers)

The more I read the more I realize that each object has a sort of “surface area”, which is its public interface, where it can be interacted with. I’m used to referring to applications with too many dependencies as “monolithic”, literally meaning “one piece of stone”. This word makes me think of how all the dependencies in the system are interlocked and hard (as stone) to break apart. But Sandi has an even better metaphor: She draws this little picture of all these objects, and then she connects them with a kitten’s ball of yarn. Instead of talking about interlocking dependencies, she starts talking about the messages going back and forth, shooting all around the system. Viewing the system as an uncontrolled net of messages, Sandi pulls out an even better word: a woven mat.

So… okay. I’m convinced. It’s time to reduce the surface area of my classes. It’s time to start writing private and protected methods. More importantly, it’s time to start writing private accessors.

Okay, But There’s a Secret Reason I’ve Hated POLA All These Years

There’s a cost to the Principle of Least Access, however. It is simply this: if you start out making everything private, and then find out you need to make something public, there’s a refactoring cost. It’s not big, but it’s there, and it’s one of the kinds of hassle that I am very finely tuned to sniff out. I have literally spent 5 years learning how to design applications to not need private methods rather than writing them.

It’s not that you can’t do it. Here’s what it looks like in ruby:

class Monkey
  attr_accessor :poo_count

  def need_to_reload?
    poo_count < 1

  def throw! victim
    self.poo_count -= 1
    onlookers.each {|onlooker| onlooker.amuse! }
    reload! if need_to_reload?

Look at those first three lines inside Monkey. YUCK! Either we need to change ruby itself, or Sandi’s book needs to stop being so awesome.

(Admit it, I had you at “change ruby itself”, didn’t I.)

Adding Private Accessors to Ruby

As I write this blog post, I am dumbfounded that nobody else has done this already. What I’m about to show you is so obvious that I am convinced that it is proof that we as rubyists just don’t care much for private methods and accessors. Because this turns out to not be so hard at all:

module PrivateAttrAccessor
  def private_attr_accessor(*names)
    attr_accessor *names

class Monkey
  extend PrivateAttrAccessor
  private_attr_accessor :poo_count

  # ... rest of code unchanged

Also, if we’re willing to commit to using private and protected accessors, then we can move that call to extend PrivateAttrAccessor up into the Object class and now Monkey just looks like this:

class Monkey
  private_attr_accessor :poo_count

  # ... rest of code still hilarious

BAM. There you go. No, wait… HERE you go. That’s the full set of scoped accessors (protected_attr_reader, private_attr_writer, etc) ready to go. The file is 20 lines of documentation, 38 lines of code, and about 290 lines of test code. This feels like about the right ratio of code to test for code that is designed to be stabbed right into the very heart of ruby’s object system. If you are running ruby 2 (and if not, what is wrong with you?) you can just run the file and MiniTest will autorun and test the module. If you are running an out-of-date Ruby, such as 1.9.3, you can still run the file, just make sure you gem install minitest first.

I only had one real concern as I wrote that module: Inside the class, I am declaring everything in public scope, but inside the private accessor method, I tell ruby to use private scope. Would ruby stay stuck in private scope when it returned, or would the scope reset back to public?

Answer: It resets!

Sorry, sorry, that was pretty anticlimactic. I should have said something super profound about ruby’s access scoping and how it interacts with ruby’s lexical scope handling, but the short answer is I have no clue and the even better answer is I don’t need to have one. Rather than pick the ruby source code apart, I just wrote some unit tests specifically to ensure that the scope remains unchanged, and it does.

Give those scoped accessors a whirl and let me know what you think. If you just type ruby scoped_attr_accessor.rb minitest will execute its test suite. Or you can require the file in your project and it will quietly patch Object and you’re all set.

I haven’t touched the deep-downs of ruby with a monkeypatch in literally hours, so I’m unsure if this idea is awesome or just terrible. What do you think? Should I publish this as a gem or should I delete it, burn my laptop, and exile myself to Tibet? Update: I chose the non-Tibetan-exile option. Type gem install scoped_attr_accessor or get the source code here.

Start Small. Start Growable.

Languages Should Be Growable

One of the fun things in Computer Science is finding new and mind-blowing stuff that turns out to be 15 years or more old.

I just watched Guy Steele’s 1998 OOPSLA talk, Growing a Language. If you haven’t watched it, go watch the first ten minutes and you’ll be hooked for the rest of the talk. I don’t want to give anything away but there’s a huge bomb that he drops in the first ten minutes that not only kept me riveted for the rest of the talk, but then made me rewind it and watch it again. (Remember how you watched Sixth Sense, and the bomb gets dropped at the end and you had to watch it again? And then the director explained his use of the color red in the movie, and you had to go watch it a THIRD time to see that the bomb was constructed right there in front of your face the whole time? Yeah, Guy’s talk is like that.)

In his talk, Guy discusses whether a language should be large or small; a large language lets you say many things easily but requires that you and your listener both learn many words before you can say anything. A small language lets you both start talking immediately, but requires that you spend a lot of time creating new words before you can say anything interesting.

I won’t tell you whether Guy thinks you should create a large language or a small one–in fact, Guy won’t tell you either. But he does make it clear that a good small language must be growable, and a good large language must be both well-groomed and still growable. He even goes so far as to say that most good large languages are the ones that started small and grew over time, with good cultivation.

Bless his misguided heart, he then says that Java is a good language. I have to point out that this establishes his crazy-person street cred right there, but in a way, he’s right: Java DID start small enough to be easily understood. It grew slowly, and with careful curation. This was in 1998, and Guy then goes on to point out that Java is fundamentally broken and ungrowable unless they add some growth mechanisms to the language, such as generics and operator overloading. Most Java programmers today think of these as “having always been there” in the language, and they’re probably part of the reason Java is not only still around, but a dominant language in the industry today.

Applications Should Start Small… and be Growable

So I’m working on an new app right now, and I want to do some good OO design up front to ensure that the app looks and works well. But I’m stuck trying to figure out where to start. Funnily enough, I opened Sandi Metz’ book, POODR (Practical Object-Oriented Design In Ruby) for some guidance, and I found this astonishing guidance right there at the top of chapter two:

“What are your classes? How many should you have? Every decision seems both permanent and fraught with peril. Fear not. At this stage your first obligation is to take a deep breath and insist that it be simple. Your goal is to model your application, using classes, such that it does what it is supposed to do right now and is also easy to change later.”

Sounds familiar, doesn’t it?

Libraries Should Be Growable… or Well-Grown

I’m a fan of RSpec. If you’ll permit me stretching Guy’s language metaphor, it’s a big language for testing, with many words. I can test very complicated ideas without extending the language, and someone who has learned RSpec’s language can read my complicated ideas without learning new words.

MiniTest is a very tiny testing language. It has only a few words. As a programmer not used to constructing new words in my testing language, I initially found MiniTest to be insufferably repetitious. Each test used ten words or so, and nine of them were identical in all of my tests. When presented with this frustration, Ryan Davis shrugged with annoyance and snapped “so write an abstraction method!” It wasn’t until I watched Ryan write tests at MountainWest RubyConf this year that I realized that he does this all the time. This means that a) he was not kidding b) he was not being dismissive and c) that adding words to minitest’s language is in fact exactly how MiniTest expects to be used.

Interestingly, while I think RSpec’s large language is elegant and well-curated, many programmers feel that RSpec has grown in the wrong direction, or has at least become buried by overgrowth. Ryan felt that even Test::Unit had too much cruft in it, let alone RSpec, so rather than prune the language back, he started fresh, started small, and most importantly, started growable.

When Ryan spoke at MWRC, he created a new testing word that I felt did not make much sense. Even watching him define it I thought “Okay, I understand the abstraction, but that word is horrible. It doesn’t communicate what the word does at all.” That’s the drawback to small languages: naming things is hard, and small languages require you to name things from the start. Had I been pairing with him we’d have had a splendid argument about the name of the abstraction method he wrote. But that sort of fits into Guy’s logic as well: growth should be carefully curated. As you grow, you’ll create new words, and those words should be easy to learn and understand or you can’t communicate well.

Growth Should Be Curated

I’m gritting my teeth as I type this, but I have to own up to it: The growth of Java has been well curated. C# has also been well-groomed, even if I think the language designers have carefully and consistently solved all the wrong problems with the language.

I think PHP is probably the poster child for bad growth*. That’s in addition to its internal syntax inconsistencies; I’m just talking about the language’s internal methods. For a quick example, see how many different clumps of consistency you can find just in the string functions. For a longer example, read @eevee‘s rant, PHP: A Fractal of Bad Design. (TL;DR? Fine, just click on the rant but don’t read it–just scroll down to see HOW LONG it is.) PHP’s problem is twofold: they added things inconsistently, and they were unwilling to prune things back out of the core once they had been added. PHP has grown much faster than Java and C#, because the maintainers were willing to make mistakes rather than deliberate for years in committee, but like Java and C#, PHP hasn’t gone back and fixed mistakes once made.

In my opinion, Ruby sort of gets a B+ on growth curation. A lot of words are unnecessary synonyms (count and size and length, for example) while other words are occasionally synonyms that suddenly change meaning when you’re not expecting them to (Array#count and ActiveRecord::Base#count, for example). Some things in the language are pretty bizarre edge cases (The Kernel#test method, for example) and in the new Ruby 2 release one major feature (Refinements) was brought in under strident protest. But by and large the methods across the entire language are consistent with each other, and when they vary it is usually to be consistent with some external protocol that they are modeling. Ruby 2 was willing to break backward compatibility in order to fix mistakes and grow sufficiently. I also cut Ruby some slack because it’s growing extremely fast in comparison to other languages, and having the breaking changes in Ruby 1.9 for 3 years before committing to Ruby 2 let the community keep pace.

So Grow, But Grow Carefully

And that’s sort of my whole point here: Size and growth are key tenets at every level of abstraction: for a single application, a broadly applicable test suite, or an entire language, two rules I’m drilling into my head right now are

  1. Start Small. I can’t say “Always Start Small” because sometimes the problem you need to solve is big. But it is fair to say “Always Start As Small As Possible”.

  2. Start Growable. This one IS gospel for me. However big I choose to start, I think growability is essential for success.

  3. (Bonus Rule) Curate your growth, but don’t be so afraid of growing wrong that you become afraid to grow. Be willing to grow and then weed.

And if all else fails, start over with a new small thing and start growing again.

* Note: Every time I kick PHP’s tires I get hate mail from offended PHP programmers who assume I’ve never used the language. The fact that PHP programmers are the only group worse than rubyists when it comes to language fanboyism is a topic for another day, but for now, let me just say $$dispatcher->$$method. If you’ve never written your own framework in PHP, that little snippet of code means I am better at PHP than you. I have pushed out, much like an agonizing stool the day after an all-night taco binge, a little over a million lines of PHP code. It is the foul tongue of Mordor, but I have earned the right to dislike it purely on its lack of merit. If you like it, that’s fine. It’s your choice. I’m not saying PHP doesn’t have its good points or that I can’t write clean code in it. I’m just saying it’s not worth it to me.