Be awesome. Every day.

Testing Faraday clients for APIs in your Rails app

I am working on two applications that extensively communicate with each other using a JSON API. One application is for inventory management where products and stock quantity gets management. The other application is a web store frontend. Both are Ruby on Rails applications.

The inventory management application exposes the JSON API so I can do realtime data synchronization in the web store frontend. For this, I am using Faraday as the http client.

Here's a sample of a call:

class LineItem < ActiveRecord::Base

  def create_in_vacustock
    connection = vacustock_connection

    response = connection.post "/api/v1/delivery_assignments/#{order.external_id}/assignment_items.json", { assignment_item: { ean8: product.external_id, amount: quantity, external_line_item_id: id, custom_code: customer_article_code } }
    assignment_item = response.body['assignment_item']

    update_attribute(:external_id, assignment_item['id'])
  end

  private

  def vacustock_connection
    return Faraday.new(SystemSettings.vacustock_endpoint_url, :params => { :user_credentials => SystemSettings.vacustock_api_key} ) do |faraday|
      faraday.request :url_encoded

      faraday.response :json
      faraday.adapter Faraday.default_adapter
    end
  end

end

The create_in_vacustock method is used to create an AssignmentItem in the inventory application using a POST request with some JSON as parameters.

You can see that I created a private method vacustock_connection that actually initializes the Faraday client. This is important for two reasons. First, I can re-use that method and initialize connections for the update_in_vacustock and delete_from_vacustock methods as well. These methods are not shown in the sample. Second, I will use that method for stubbing out the Faraday client in my test that I will show you now.

Faraday is awesome for it has built-in stubbing for requests. To use these stubs in our test we could use the test adapter like this and mock it out using [Mocha]:

test "create_in_vacustock creates an assignment item" do
  line_item = LineItem.create(amount: 20)

  vacustock_connection = Faraday.new do |builder|
    builder.response :json

    builder.adapter :test do |stubs|
      stubs.post("/api/v1/delivery_assignments/1/assignment_items.json", {line_item: { amount: 20 } } ) { [200, {}, nil] }
    end 
  end

  line_item.stubs(:vacustock_connection).returns(vacustock_connection)

  line_item.create_in_vacustock
end

In this test case, Faraday will check if the stubbed URL is requested by the method create_in_vacustock.

If we wanted to test another case with a different URL endpoint we could refactor initialization of the vacustock_connection Faraday client into a private method on our test class and add another stubbed URL to the list of stubs like so:

test "create_in_vacustock creates an assignment item" do
  line_item = LineItem.create(amount: 20)

  line_item.stubs(:vacustock_connection).returns(vacustock_connection)

  line_item.create_in_vacustock
end

test "update_in_vacustock updates an assignment item" do
  line_item = LineItem.create(amount: 30)

  line_item.stubs(:vacustock_connection).returns(vacustock_connection)

  line_item.update_in_vacustock
end

private

def vacustock_connection
  Faraday.new do |builder|
    builder.response :json

    builder.adapter :test do |stubs|
      stubs.post("/api/v1/delivery_assignments/1/assignment_items.json", {line_item: { amount: 20 } } ) { [200, {}, nil] }
      stubs.update("/api/v1/delivery_assignments/1/assignment_items/1.json", {line_item: { amount: 30 } } ) { [200, {}, nil] }
    end
  end
end

There are three problems with this second approach.

  1. It's not very nice. We are adding stubs for different test cases into this single method which means the test cases won't be fully isolated from each other.
  2. In the case of the test case for the update of the line item we will need to hardcode the endpoint address with it's id for the line item which will not be possible if you're using fixtures or factories to set up the test data.
  3. Using this method it is impossible to stub for varying return results. For instance, if you want to test for a different status code or a different JSON response.

I found a pretty clean solution that I am now using throughout the tests to set up my API connection to VacuStock where stubbed requests can be isolated between test cases and where I can properly set endpoint URLs from the generated fixtures I use.

Here's how it looks in a test case:

test "update_in_vacustock should update an assignment item in vacustock" do
  line_item = LineItem.create(amount: 120)

  vacustock_connection do |stubs|
    stubs.put("/api/v1/delivery_assignments/1/assignment_items/#{line_item.id}.json", { line_item: { amount: 120 } }) { [200, {}, nil]
  end

  line_item.stubs(:vacustock_connection).returns(vacustock_connection)

  line_item.update_in_vacustock

end

You'll see that vacustock_connection has become a method that takes a block where I can define my stubs for this specific test case. I moved this method into my test_helper.rb so this method is accessible for all models and controllers where I do API things. Here it is:

class ActiveSupport::TestCase

  # other stuff…

  def vacustock_connection
    @connection ||= Faraday.new do |builder|
      builder.response :json

      builder.adapter :test do |stubs|
        yield(stubs)
      end
    end
  end

end

When the method is called for the first time, it initializes a new Faraday client with the test adapter using the stubs you defined in your test case. It will set itself to the @connection instance variable.

After the first time it is called, the @connection instance variable will be present and it will just return it.

If you have any questions or feedback about this code, please let me know!

Read this article →

Quick tip: Use OS X Keychain together with GitHub for Mac for https URLs

Just a quick tip for all the users of the offical GitHub for Mac client that are getting tired of typing in their credentials each time you use the git client in your Terminal manually.

Run this command to have your OS X Keychain remember your password so you don't have to type it in another time:

git config --global credential.helper osxkeychain

From: Is there a way to skip password typing when using https:// github on Stackoverflow.

Read this article →

Production Code: Ensuring I clean up in a Worker class that runs a shell command

I am working on an application that lets students verify certain assignments they worked. The validation is done in the background by a Ruby script. In one particular example, I want to know if someone created a public repository on GitHub. If so, the method should return true. If not, it should return false.

The following code creates a placeholder directory where I can clone repositories into. Then, it tries to clone the repository based on the given repository URL and returns the status of the shell command to see if it worked.

require 'grit'

class GithubRepositoryCreated

  def unlocked?(player)
    return false if player.git_repository_url.blank?
    tmp_repositories_url = "/tmp/repositories"
    checkout_path = File.join(tmp_repositories_url, player.id.to_s)

    FileUtils.mkdir_p(checkout_path)

    output = `git clone #{player.git_repository_url} #{checkout_path} 2>&1`

    return $?.success?
  end

end

There are two problems with this piece of code.

The first one is the biggest security flaw in mankind, since anyone could execute code on my server by putting something dangerous in player.git_repository_url. Let's solve this by calling the String#shellescape method on this string.

Second, there is no way of cleaning up the directory in the unlocked? method. I can't put any code after the return statement since that won't be executed if the return is called. I could put the cleanup code before the return statement but than I would still have a problem if some exception was raised that would not reach the cleanup code.

This is where ensure comes in handy:

require 'grit'

class GithubRepositoryCreated

  def unlocked?(player)
    return false if player.git_repository_url.blank?
    tmp_repositories_url = "/tmp/repositories"
    checkout_path = File.join(tmp_repositories_url, player.id.to_s)

    FileUtils.mkdir_p(checkout_path)

    output = `git clone #{player.git_repository_url.shellescape} #{checkout_path} 2>&1`

    return $?.success?
  ensure
    FileUtils.rm_r(checkout_path)
  end

end

No matter what happens, ensure is always called for the method. So, in case of a normal return or in exceptional cases, the checkout_path will always be removed after this job is done.

Here are two tests I wrote for this class:

require 'test_helper'

class GithubRepositoryCreatedTest < ActiveSupport::TestCase

  test "should return false on non existing repository" do
    player = players(:michiel)
    player.git_repository_url = '"; touch /tmp/hello; '
    validator = GithubRepositoryCreated.new

    assert !File.exists?('/tmp/hello')
    assert !validator.unlocked?(player)
  end

  test "should clone a repository" do
    player = players(:michiel)
    validator = GithubRepositoryCreated.new

    assert validator.unlocked?(player)
  end

end
Read this article →

Edition Press Preview: Sharing and linking individual magazine pages

Two weeks ago, we showed our first preview of Edition Press. There is another thing that is pretty damn cool when you create an iPad magazine using Edition Press. This thing comes natural to our daily browsing of web pages; and in an iPad magazine it adds something very convenient. It is the ability to link to individual pages.

When you build an iPad magazine with Edition Press; each page in your magazine gets its own web address. This makes it easy to link to, bookmark, and share an individual article. When someone opens such a link, the magazine opens at the linked article, and you can still swipe to the next and previous page. Just like opening a a book at a certain page.

You can see in the next screenshot that the browser address bar and the title of the tab is updated after swiping to this particular page.

Screenshot of link to individual page

Bookmarking pages on the iPad will also work seamless. This is because the reader's address bar is updated when he swipes to another article. So, when they choose to bookmark or share the page on Twitter/Facebook using iOS, the correct address will be used.

Here is a screenshot that shows that a bookmark is added for the page you have swiped to, and are currently reading.

Screenshot of iOS bookmark individual page

Linking to individual pages is great for bookmarking, sharing and saving a page to external services like Pocket.

We are inviting a small bunch of people to be our early users. Please let us know if you would like to get a chance.

Read this article →

Production code: I am concerned

So I tried to give ActiveSupport::Concern in an app that has been living since Rails 2.x. We only just upgraded it to Rails 3.0, then 3.1 and now we are rolling on 3.2.

I've always liked showing what I really do instead of showing examples and now I would like to follow Alexander Zaytsev example of showing production code.

While David Heinemeir Hanson talks about making your fat models slim with Concerns, I am going to use it to DRY up a piece of controller code I have laying around.

The app is an e-commerce webshop app that has a products listings (or catalog) view laying around in a few different places. There is some logic in some controller actions that sorts the products based on a few given things in the params hash.

Here is the legacy bulky controller index action of our ProductsController before the refactoring:

There is some stuff in there that I won't explain right now. If you want to know, please post a comment.

The bit I want to refactor is line 22-41 since I am also using the exact same duplicate code in an OrganizationsController that shows an account page with the same list, only with featured products in it.

Before refactoring I ended up with for the OrganizationsController:

As you can see, there is A LOT of bulky code in this controller as well that is being moved from the legacy shit it is to good code.

For this blog post, we are looking at line 39-58 that we are going to put in a concern.

Following new Rails 4 conventions I will create an app/controllers/concerns/product_sortable_listing.rb with the ProductSortableListing module:

I created a private method sort_products_from_params that takes the AREL relation and returns it, with potential ordering and sorting added based on what's in the params.

Now, here's the refactored after for the ProductsController and the OrganizationsController, but concerned.

I am still thinking about how to let someone know where a method lives. I solved that with a comment above the line now, but that feels as if I am telling something that should be implicit anyways. Do you have any good suggestions?

Read this article →

New PressPass 0.1.2 released: Fix for NoMethodError on Ruby 1.8.7

Yesterday, I published PressPass 0.1.2 on RubyGems. This minor release contains a bug fix to make PressPass work on Ruby 1.8.7, which is the default if you are using Mac OS X.

If you are seeing an error like this:

MacBook-Air-van-Michiel:~ michiel$ presspass _0.1.1_ new my_blog
/Users/michiel/.rvm/gems/ruby-1.8.7-p371/gems/presspass-0.1.1/lib/presspass/cli/new_project_generator.rb:58:in `create_project_directory': undefined method `exists?' for Dir:Class (NoMethodError)
    from /Users/michiel/.rvm/gems/ruby-1.8.7-p371/gems/presspass-0.1.1/lib/presspass/cli/new_project_generator.rb:25:in `run'
    from /Users/michiel/.rvm/gems/ruby-1.8.7-p371/gems/presspass-0.1.1/lib/presspass/cli.rb:17
    from /Users/michiel/.rvm/rubies/ruby-1.8.7-p371/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
    from /Users/michiel/.rvm/rubies/ruby-1.8.7-p371/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
    from /Users/michiel/.rvm/gems/ruby-1.8.7-p371/gems/presspass-0.1.1/bin/presspass:3
    from /Users/michiel/.rvm/gems/ruby-1.8.7-p371/bin/presspass:19:in `load'
    from /Users/michiel/.rvm/gems/ruby-1.8.7-p371/bin/presspass:19
    from /Users/michiel/.rvm/gems/ruby-1.8.7-p371/bin/ruby_noexec_wrapper:14

Then this update fixes it. Please install the new version using:

gem install presspass

And use this command to generate a new installation.

presspass new my_blog

What is PressPass?

PressPass is a command line tool for generating WordPress installations, theme and plugins. It also allows developing WordPress sites alongside your Rails projects if you are using Pow by 37signals

Of course, suggestion, feedback and help requests welcome!

Read this article →

Edition Press Preview: Swiping pages and navigation

Edition Press is a WordPress framework theme that lets you create amazing iPad magazine themes. Allow users to swipe between articles, open videos from YouTube or Vimeo and more. All of this from within WordPress. We will be officially making it all available in January 2013.

Adding pages to the magazine

In our first release, you create a page in your magazine by creating a regular WordPress page. For example, you can have a cover page, a table of contents page, a few article pages and maybe an advertisement somewhere in between.

The next screenshot shows how you can write a page using the editor, and how it shows up on the iPad.

Preview image showing translations of pages into articles on the iPad

So, how do you manage the pages in the magazine? You might want to change the order of the articles while writing them or you might want to add a new page in a later moment.

Reordering pages, articles and advertisements

You can add, remove and reorder the pages in your magazine using the WordPress menus functionality. Just add a page to the menu using the content widgets on the left, save the menu and the page will show up the next time someone opens the magazine.

When you want to change the order of the pages in the magazine, just drag the page to the correct position within the menu, save it, and you're done!

Preview image showing how the main menu looks like for a sample magazine

Read this article →

WordPress 3.5 on Pow with the new PressPass release

Hi guys,

I just published a 0.1.1 version of the Presspass gem. Presspass is a little command line tool that lets you easily download and install WordPress installations. This new version supports WordPress 3.5 by default.

An extra feature is that you can use it alongside Pow (http://pow.cx) so you can run Rails apps and WordPress sties alongside each other on .dev domains without having to install a webserver like Apache or nginx on your development machine.

Installation

Install the gem:

gem install presspass

Generate a clean WordPress installation:

cd Code;
presspass new wordpress_site

Then link to Pow

ln -s ~/Code/wordpress_site ~/.pow

Open up http://wordpress_site.dev/ in your browser and you'll get the standard WordPress installation wizard page.

Try it out! If you need help, you can reply in the comments or post an issue on GitHub.

Thanks!

Read this article →

Preview: Showing people how we work

I always enjoy screencasts, behind the scenes blog posts and samples in guides and books. One reason is because they tell me how to fix certain specific problems. The more important reason is: I love to see how other people do their work.

But, when you don't have the tech, the CPU, the bandwidth or you are working in little pieces and bits on a project, screen casting is not an option.

Because of this, I created a little web app that lets you show others how you work in a bit more indirect way. I wanted to build this web app for myself since a while now.

I am calling it my devlog and you can see a simple preview I've put up on http://devlog.apphakker.nl/.

Devlog Preview for Apphakker

It looks like a simple blog. I can post short notes on my thinking direction and what kind of problem I am solving right now.

It also pulls in GitHub commits in realtime, so you can see when I push some commits and what kind of commit messages I am writing. Since apphakker is a public product, you can also view the code changes on GitHub.

A few things I am going to add over the next couple of days is realtime updates. This way, you don't have to refresh for new updates. I'll also add in deploys that I do with capistrano.

More stuff you'd like to see from me while I am developing?

Read this article →

Most startups are expensive hobbies

Yesterday I read an article about Kraft splitsing into two companies. The article was written by a Dutch source of interesting 'entrepreneurial' news.

Kraft is a big corporation with popular snack food brands. Splitting the company logically results in two companies, of which one is new.

The Dutch new sources dubbed the new company 'the biggest startup in Europe'. A startup with over 300 office locations and all kinds of products available in supermarkets.

Are you kidding me? This is not a startup.

In the past couple of weeks I met all kind different people that are working on a project of their own. Most of the times they already have some users. But when asked about their next steps most of them talk about the next product release. They might even be looking for investment so they can spend more time working on their project.

That's not a startup, but an expensive hobby.

So we start calling nice product ideas a startup. Right next to that an existing corporation with 300 offices and existing brands is a startup as well. I really think something is off and we're completely losing what the term startup is about.

Don't get me wrong, I really love most app ideas I hear. We work on side projects as well. It's within the nature of an entrepreneur to come up with stuff and build something nice. But not every project should be called a new startup.

The best definition of what a startup is comes from Steve Blank: 'a startup is an organization formed to search for a repeatable and scalable business model.'

From now on when people ask what we do I completely remove the term 'startup' from my story. We are building new businesses. Building a business, to me, implies creating value for customers and making money to keep the lights on.

Read this article →