Hacker News new | ask | show | jobs
Learn how to test Rails using Test::Unit instead of Rspec (whatdoitest.com)
45 points by genericsteele 4843 days ago
14 comments

I hate to be that guy, but the Rspec tests should be written more like the following:

    describe 'Article' do
      let(:article) { Article.new }
  
      describe '#slug' do
        before(:each) do
          article.title = "Testing with Test::Unit"
        end
  
        it 'does not contain non-alphabetical characters' do
          article.slug.should_not match(/[^\w-]/)
        end
    
        it 'does not contain capital letters' do
          article.slug.should_not match(/[A-Z]/)
        end
    
        it 'replaces spaces with hyphens'
      end
    end
Note: I almost certainly misinterpreted the first Regex. 'now you have two problems,' etc. etc.
one step further:

    describe Article do
      describe '#slug' do
        before do
          subject.title = "Testing with Test::Unit"
        end

        its(:slug) { should_not match(/[^\w-]/) }
        its(:slug) { should_not match(/[A-Z]/) }
        it 'replaces spaces with hyphens'
      end
    end
I like and use RSpec, but I think these improvements are proving the guy's point that its DSL is forbidding for beginners. I write RSpec more like the original post because it's less magic.
Not really. At least RSpec's creator doesn't think so (http://blog.davidchelimsky.net/2012/05/13/spec-smell-explici...).
neat, I didn't know you could do that. thanks for sharing!
Not at all. Feel free to be _that_ guy. First thing I noticed about the example was that it was badly written RSpec - which undermines an article advising on what you should/shouldn't do for testing.
Even better with the new syntax:

    expect(article.slug).to match(/[^\w-]/)
I didn't understand why the author used the unnecessary regex assertions, which are less clear and don't add any additional checks beyond:

  article.slug.should == "testing-with-test-unit"
I agree with you, I can't stand when I see @article and stuff strewn through out the tests. It's just asking for test pollution.
Some jarring issues with testing in the Rails community, and these aren't things I recognized until about 3 years into it all (in other words, recently) and finally had the wherewithal and exposure to other tools/paradigms where I could finally criticize my tools.

* Testing is an advanced topic, yet it seems to be misleadingly sold in Rails tutorial literature as something that everyone else effortlessly does with the same ease of "gem install hairball".[3]

* Good luck finding resources on Test::Unit/Minitest.[1]

* Where the documentation?

* Rails invents its own definitions for "unit tests", "functional tests", and "integration tests". Even though they already exist in testing vernacular. [2] Unfortunately confusing when Rails is your first experience with testing and you think "unit test" means "model test" and "integration test" means fumbling fruitlessly with your jasmine/capybara frankensuite because you're lost like I was and tried to stitch together a few Railscasts because you just don't know any better.

* Instead of embracing tests that don't depend on the Rails stack, the Rails community embraces tools like Spork that keep a Rails process alive. Because Rails takes too long to boot to have a test-driven cycle with tests that depend on Rails. The concept of service objects and wrapping Rails/3rd party libs is brand new to me and I only encountered it after I was good enough to contribute to large Rails apps and read their source code.

[1]: At least Ruby 1.9 changed the name (in essence) to "minitest" which we can finally google. "test unit"? Not so easy to google.

Also, here's the best resource for Minitest I can find: http://www.mattsears.com/articles/2011/12/10/minitest-quick-.... It's a cheatsheet on someone's blog.

Frankly, there are much better arguments for sticking with Rspec. For one, it's the testing library that the community uses and you need community support. The popular gems you'll be using probably have more wiki info on Rspec testing too (like Devise). Rspec is documented and even illustrates best practices. http://betterspecs.org/.

That Minitest is built in to Ruby was never a convincing reason to use it. I already have development dependencies. Trust me that testing isn't going to be the thing that introduces the first 3rd party dependency into my app. Fucking "awesome_print" is already in my Gemfile, so it's not a big deal to add a gem that makes testing better for me.

And for all the hate that Rspec gets for its DSL, Test::Unit/Minitest's DSL (assertion statements) is just as bad and Minitest::Spec introduces an Rspec DSL anyways.

The only way I want to write tests is by writing my assertions in pure Ruby without having to lug around a bunch of different assertion variants just so the testing library knows what kind of error output to show me.

The ONLY solution I've found is a library called Wrong (https://github.com/sconover/wrong) and it works with both Minitest and Rspec.

It lets you write assertions like this:

    include Wrong
    assert { "abc" == "abd" } 
    assert { cookie_jar.empty? }
    assert { dog.respond_to?(:walk) }
    assert { boy.is_a? Human }
Like, actual Ruby. And it will figure out on its own how to display diffs in error output.

Meanwhile, here's Minitest:

    assert_equal "abc", "abd"
    assert_empty cookie_jar
    assert_responds_to dog, :walk
    assert_instance_of Human, boy
    # I know these because I'm looking at a cheatsheet
Here's Rspec:

    "abc".should == "abd"
    cookie_jar.should be_empty
    it ->(dog){responds_2}.(:walk).(&:does?)
So all this talk about avoiding DSLs yet you don't. And it wasn't until blowmage (of minitest) told me about Wrong in IRC that I found what I wanted -- the only real way to avoid DSLs and verbosity in Ruby testing that I've come across.

[2]: Xavier Shay brings this up in his talk http://www.confreaks.com/videos/815-larubyconf2012-rails-sus...

[3]: Oh yeah, and after three years of Ruby and Rails, it wasn't until I was exposed to functional programming (Martin Odersky, Rich Hickey), OO design (Sandi Metz), and testing screencasts (Gary Bernhardt) that I finally understood how to even fundamentally write code that could be tested. In other words, it took high level a-ha moments for me to be able to write tests that didn't just cripple my workflow and waste my time and slow down my learning process. That's where I found out that testing is an advanced topic, not a bullet point you can throw into a newbie tutorial on how to generate a Rails scaffold.

God yes.

Test::Unit is simple, comes baked in, and because it's not some faux-English DSL your brain isn't constantly fighting with itself making stupid and incorrect assumptions about syntax because 'hey, it looks like English'.

The worst testing-related decision I ever made was using Michael Hardt's Rails Tutorial (which has RSpec baked in) to learn Rails in the first place. Great tutorial otherwise, but it set my testing back immensely - it pretty much meant I did no testing at all. When I finally bit the bullet and decided to learn how to use Test::Unit, I was astounded at how easy it actually was.

Most Rails resources online assume familiarity with so many not-out-of-the-box tools it's a bloody miracle anyone can learn Rails at all. It's a real problem.

> it's not some faux-English DSL your brain isn't constantly fighting with itself making stupid and incorrect assumptions about syntax because 'hey, it looks like English'.

I'm doing the same Rails tutorial right now and I constantly feel like this when it comes to writing the RSpec stuff. It just doesn't feel intuitive and logical, in spite of (or maybe because of) its efforts of being just that. The tutorial doesn't really dive into the specifics of it, it's just "Monkey write, Monkey do" without being able to grasp the concepts of it. This completely leaves me in the dark when it comes to writing my own tests.

Yes! Michael Hardt's tutorial is the culprit for me as well.
I write tests primarily with Rspec for personal projects and Cucumber at work. I haven't worked with unit test at all. I like both Rspec and Cucumber. I use Rspec for personal projects because they are more backend Rails side heavy web app. And my company uses Cucumber because we found writing tests based on user behavior to make more sense.

My problem with Rspec is in its error message, whether it is showing why the test failed or that the test code syntax is wrong. I noticed that Rspec is more prone to showing cryptic errors than Cucumber. I've seen some mind boggling error messages that doesn't really help user debug the code. For devs beginning TDD, I can understand why working with Rspec could be annoying. Even now, Rspec doesn't feel second nature to me.

> Just from skimming over the tests, it’s obvious that Rspec is more readable.

And that's where I'd disagree and my tastes differ, I never quite got the Ruby fetishism for almost-sort-kinda English DSLs. It's still baffling to non-developers and I'd much rather read a more regular programming language syntax, never mind that abusing the syntax can lead to some weird error messages. (Not saying that RSpec/Cucumber fall into that trap, as I never had that much experience with them. Like I said, kinda turn-off for me)

A bit like Lisp macro abuse or too much parsing in Tcl.

Computers don't understand english very well. It's easier for us to learn to talk to them, than to teach them how to understand us.

DSLs are helpful for some, but sometimes they get in my way. A good example is formtastic (http://github.com/justinfrench/formtastic) in ActiveAdmin. I already understood forms when I had to implement this, so it just dragged me down.

If using a DSL makes your life easier, by all means use it.

I think the bigger issue here is that new developers of Rails don't understand the best practices for testing. I switched from PHP to Ruby, then learned rails over the past year and it had a significant learning curve.

First I tried testing models and controllers, skipping views. Now I use integration tests with Capybara because it effectively tests the whole stack without writing redundant test code.

What's not clear is the pros and cons of each approach. Over time you end up using your own approach that works best for you. It's one thing to know that you need to "test your code", but with all the various testing libraries created, it can quickly become overwhelming.

In an ideal world, I'd like to learn more about the various approaches. Test::Unit vs Rspec, vs all the additional testing add-ons and how everything works together. When you end up spending just as much time writing tests as you do writing code, you quickly realize how critical getting the right approach to testing really is.

> First I tried testing models and controllers, skipping views. Now I use integration tests with Capybara because it effectively tests the whole stack without writing redundant test code.

Does this mean you primarily rely on integration tests at the exclusion of model unit tests?

If that's the case, I'm of the opinion you would be well served by revisiting your unit tests. Controller tests are kinda taken care of by integration tests, but you won't get solid coverage of your models with integration tests alone.

At a minimum, model unit + integration (skipping controllers), IMHO. If for no other reason than running an integration test suite can be time consuming, and you don't always need your feedback loop to be that long.

I do integration tests + model tests
My theory is that new Rails developers spend more time learning how to test than what to test. This article isn't necessarily about which framework is better, but more about which is better as a testing learning tool.

It's been pretty cool working on the testing book because I'm approaching it from a different angle. I'm not testing something, I'm showing someone how to test. I've got to dig into Test::Unit and MiniTest more than I ever have, and it's been great.

There isn't a wrong way to test, so long as tests are being written.

There's most definitely inefficient ways of testing. You can often cut the time of your testing if done right.
Fundamentally disagree with much of this article. Not just what _that_ guy said: https://news.ycombinator.com/item?id=5418460

When I learned Rails a few years ago, I was learning programming from scratch apart from some very early fumblings in BASIC and FORTRAN 20 years ago now. I was completely unfamiliar with the idea of testing. I had to learn it from the ground up. RSpec wasn't just more understandable it was, and remains, better documented. It's not just the official documentation. It's that there are tons of examples of how to test different use cases, what one should test, best practices.

Learning to test isn't just learning literally what a few methods do. It's understanding how to use those in myriad mix and match situations in order to provide good test coverage.

RSpec is easier to read before the test, it produces beautiful output that is readable and expressive, when you write good tests and test the right things.

After trying to struggle with Test::Unit and writing uncovered code for long stretches because I couldn't understand what to do, and then discovering RSpec, reading great blog posts, write-ups and documentation on it and finding comfort in RED-GREEN-REFACTOR I would only recommend RSpec for someone starting from scratch, as I did.

Oh... and I cannot recommend this resource highly enough.

As RSpec documentation goes this set of slides open in a window will help you tremendously:

http://kerryb.github.com/iprug-rspec-presentation/#1

It uses simple example and hits all the major points well.

You do realize that if you're using Ruby 1.9+ you're actually using Minitest, right? It's just a backwards compatible API. Don't confuse people with all the talk about Test::Unit.

Minitest::Spec is just an RSpec-like DSL built on top of Minitest, which is the successor to Test::Unit.

Thank you for the question "What do I Test"

As a developer, that is my number one question. Not cucumber/rspec/testunit, but really what do I test so the tests can be fast, I can test my core functionality and I don't overlap rails.

Pretty sure `define` is not a keyword in RSpec. The author means `describe`.
Fixed. Thanks.
Minitest is horribly coded. Wouldn't go near it with a nineteen foot pole.
I was a little disappointed in this article. What I've always wanted to know is: Why do Rails developers use Rspec when they can use MiniTest::Spec? I was hoping the OP would address that, since MiniTest is the default Ruby testing suite he refers to. Do Rails devs continue to use Rspec because it's what they're used to? Or is Rspec a dependency for other testing tools and libraries?

I don't agree at all with this premise by the OP:

> Just from skimming over the tests, it’s obvious that Rspec is more readable. The Rspec DSL was designed for us humans to be able to understand what is happening at a glance. This is where Rspec gets my vote. Unfortunately, we aren’t talking about reading or sharing tests. We are talking about writing tests, and this is where Rspec has a heafty learning curve.

I think the MiniTest::Spec API is both easy to read and write...(I'm assuming the OP is criticizing both Rspec and MiniTest::Spec here). I'm just saying this from personal experience, as I learned both the Spec and standard testing syntax at the same time.

I've been using Minitest, Mocha, and RR lately after using Rspec for many years, and find myself really missing Rspec.

At a high level, I would say that Rspec is just much more thought out. The other libraries all seem to be written by someone who looked at Rspec and though "Ugh, this is too much. I just need X."

But I find, if I'm doing serious TDD, I'm gonna need pretty all of Rspec. And what happens is that the other libs just don't integrate that well.

For example, you may be using one test runner (minitest) with a mock from some other library (mocha) and assertions from a third (test::unit). The question is: are those things all meant to work well together? Are your mocks scoped correctly within your test runner? Is your database cleaner working properly with your before and after hooks?

The answer is sometimes "no". But with Rspec, everything works well together. It just fits. Generally I'm a big fan of focused libraries, loosely coupled. But the state of things with testing is that I don't think we've yet found that good loose coupling.

The second issue is that Rspec just seems to have much richer assertions. I can do things like:

    expect {
      delete_zombie_users
    }.to change(User, :count).by(-1)
Or:

    thingy.length.should < 4
And mocking is also underdeveloped in other libs. Mocha, for example, can't stub constants. In Rspec I can do:

    User.stub(:all => [fred, betty, sue])
Or

    stub_const("File::MAX_SIZE", 10**10)
And it works great. I find I can easily stub out parts of ActiveRecord and write good targetted unit tests. With Mocha I find myself using any_instance all the time because I can't target things on the structure I really want to target them to.

Some of these are available in other expectation libraries, but I've found Rspec to have a really nice, comprehensive set of them.

All in all, I just feel like David Chelimsky and the Rspec folks have really thought about testing in a way the other libs' authors haven't. I keep learning things about TDD and finding out that Rspec has already considered them. With other libs the opposite often seems true.

OP here. MiniTest::Spec is great, and knocks out two of my complaints in the article: * Expectations are readable and writable * The docs are very easy to digest: http://bfts.rubyforge.org/minitest/MiniTest/Expectations.htm...

I don't recommend MiniTest::Spec for the same reason I didn't recommend Rspec. It requires some extra configuration out of the box. The actual process of switching from Rspec to Test::Unit went Rspec -> MiniTest -> Test::Unit. There's just more stuff that's neccessary to start writing tests.

It's the same reason that new rails devs shouldn't swap out ERB with HAML before they build their first application. Get to know the defaults, then decide.

I use TDD most of the time when I'm developing, when I'm writing tests I'm really writing specs. I'm almost thinking out loud, what is this thing I'm about to create actually supposed to do? When writing like this the rspec syntax is far more natural, it matches the internal dialogue that's going on in my head. And the syntax is designed to encourage exactly this, thinking about the code you're about to write as though you were describing it to someone else.
I started testing with Test::Unit+Shoulda, and then moved to RSpec when I found that it was basically the same thing with less hackery. This was being Minitest existed.

Since then, I've stuck with RSpec because it's got a ton of community support, and consistently lets me write concise tests. I never feel like I have to unduly repeat myself with RSpec; I constantly felt like that when I was writing testunit tests.

Seconded; I love the community around RSpec, and it works perfectly well for me.
My biggest problem was the matchers and mock support in MiniTest::Spec. I think RSpec is further ahead here with better support for both. Avdi Grimm agrees: https://twitter.com/avdi/status/180394364357443584
The website rejected my email address as being invalid (j@ckjennin.gs)…
Sorry about that. It lazily checked for two characters before the @ instead of one. It should work now: http://whatdoitest.com/
I <3 Test::Unit
My experience:

I just started the process of converting our few hundred Test::Unit tests to Rspec simply because Test Unit documentation is terrible. Our tests need a lot of work and I'm not that experienced at writing test scripts. I have to learn this as I do it, and I struggled to find resources for learning Test::Unit.

Every blog post, tutorial and guide I can find was written for Rspec.

As much as I'd love to continue using Test::Unit and save myself the hassle of converting from one to the other, I found learning Test::Unit to be one giant brick wall after another.

No one is discussing it, no one is blogging about it, no one is writing stack overflow questions about the issues. (I say, in a discussion about a blog post about it... but this is the exception not the rule, sadly)

For me it was a simple decision: If I want community support, I need to be using the tool the community is actually supporting.

(Not that Rspec documentation is that much better-- I've now resorted to just reading public github repos and looking for spec/ directories to scrutinize and learn from. If anyone has links to any good repos that use rspec tests, I'd love it if you could send them my way! I need more good repos to learn from!)

"because Test Unit documentation is terrible"

Are you confusing Test Unit documentation with 'How/What to test' documentation? I can't think of any documentation required if what you use is a group of asserts.

Test::Unit in a nutshell: `test "should do something"; ...; end` is what gets run in each of the files. `setup` is called before each test. `teardown` is called after each test. use `assert_*` at your discretion.

You know, recently I had reason to look into the teardown method (DatabaseCleaner, dontchaknow), and I never did find anything conclusive that it was something that test-unit used. Sure, it was mentioned in many many DatabaseCleaner questions/answers/threads, but nothing I could really rely on since invariably these were also using Rspec. That is, my unanswered question was "Is teardown solely an Rspec thing?"
Setup and teardown are in the Rails testing guide.

http://guides.rubyonrails.org/testing.html#setup-and-teardow...

I suppose so, but the readability of the guides are an (perhaps-idiosyncratic) issue for me.
"For me it was a simple decision: If I want community support, I need to be using the tool the community is actually supporting."

Truth. And the community is really split on this one. The side that makes the decisions about what goes into rails prefer unit tests, while a large chunk of the users prefer spec-based.

This was my experience too. I tried to use Test::Unit because I was learning from scratch, Rails 2.something, and Test::Unit was "The Rails Way". But I was fumbling in the dark. The community around RSpec picked me up and carried me to the testing promised land.
This. That was totally my experience too, after freaking out learning rails the friendly rspec syntax and natural readability felt like I'd reached a pub while lost in the desert.
Hear hear. I am dreading the technical debt I have in test-unit, and meanwhile the (Rails) world is moving on to Minitest and the world appears to be settling on Rspec syntax regardless. Nowhere to be found is even a conversion HOWTO, so test-unit starts feeling even more like a ghetto, and increases my conversion anxiety, which is all too bad, since I like what I consider to be the simplicity of test-unit.
>I am dreading the technical debt I have in test-unit, and meanwhile the (Rails) world is moving on to Minitest

Minitest and Test::Unit are part of the same package since 1.9, with Test::Unit implemented as a simple compatibility layer on top of Minitest:

http://www.rubyinside.com/a-minitestspec-tutorial-elegant-sp...

The Minitest assertions are quite similar to the old Test::Unit assertions, a large number are identical:

http://rubydoc.info/stdlib/minitest/1.9.3/MiniTest/Assertion...

Right, and I've seen many pages like the rubyinside one leaving me still hungry with questions like, "So, no more test_helper.rb or what?" Couldn't find that in either of those links.

This is really more of a general documentation quibble on my part though, since I'm the dope who can't read through my questions and find the answers in the source (or wherever). I've been planning on making a blog post about 1:1 mapping of concepts between Test::Unit and Minitest learned from my own transformation, but as I've mentioned, I haven't been able to undertake that yet.

The docs really are lacking in this case; I expected the Test::Unit docs would document what's being stubbed and not in MiniTest but there's nothing there on that as near as I can tell.

As a rule, I miss the thoroughness of CPAN/Perl docs. Some ruby packages have amazing docs but often I feel at sea, particularly with regard to high level overviews and examples.