5 Jan 2012

Hosting your own gem server, simply

So why would you want to host your own gem server?
Well recently, I had to share some in-house code between different proprietary rails apps. We needed this code and all it’s dependencies to be easily installed on different development machines and we wanted to support multiples versions of the code in the Rails apps.

Does this sound like a job for rubygems and bundler? Yes? Well that’s what I thought too, except for fact that for bundler to find a gem, it needs to be available in one of the sources in your Gemfile.

Like I mentioned, this was all private code, so I couldn’t just push it to rubygems.org or rubyforge (also, it would be of no use to anyone outside our organisation).

But luckily, a bundler source can be any valid gem repository, even one you host yourself.
To host your own Rubygem server you don’t need to install anything, all you need to do is type:

gem server

This will host all the gems you have installed. This is all nice and easy, but it can get a cumbersome to maintain in a real development environment. It doesn’t let you ‘push’ gems to the server and it provides no information about the gems it hosts.

So basically I wanted all the convenience of rubygems.org, but without releasing my code to the world. After some searching, it seemed like I had one of two options:

  • hosting the opensource rubygems.org server myself. As you can see from their development setup guidelines, this is quite an undertaking. It’s quite a complex app with lots of dependencies and it’s built with a lot of assumptions to how it’s used on the public site. This might be viable for you in a big organisation if you needed some of the nicer features of the web ui, but seemed like total overkill for my situation.
  • Using geminabox. It’s a Sinatra app that allows you to host your ruby gems on your own server. It also lets you push gems, host multiple versions and even has a nifty little web ui. Perfect.

Their github page has all the info to get you up and running, but if you don’t feel like clicking the link, I’ll run you through what I did.

First up, I setup a dedicated server vm and installed Ruby 1.9.3, RVM and bundler. After creating a directory, gemset and .rvmrc for my gem server, I installed geminabox:

gem install geminabox

This will install sinatra too. I then created a directory for my gem data:

mkdir ./geminabox-data

All that was needed now was a rackup file:

 require "rubygems"   require "geminabox" Geminabox.data = "./geminabox-data" run Geminabox 

And that’s it! I could now easily start my gem server with

rackup config.ru

This starts up your gem server that should look something like this:

If you need robustness, you’ll probably want to host this using something like passenger and nginx, but I’ll leave that as an exercise to the reader.

To start pushing gems to your server, you also need to install geminabox on your client development machines:

gem install geminabox 
gem inabox pkg/my-awesome-gem-1.0.gem

Once your gems are hosted you can consume them using bundler by just adding a source to your Gemfile:

source "http://mygemserver:9292"

Or add it to your gem sources:

gem sources --add http://mygemserver:9292

If you have never written a gem before, checkout the excellent railscast on bundle gem and  this post by Yehuda Katz with more detail on .gemspecs.

As always, questions and comments are welcomed. Let me know what you think!

(Edit) Using git

After some insightful comments, (see below), I have realised that that I have  failed the mention one of the easier options available to ‘host’ your own gem, just by using git. Bundler understands git, so in your Gemfile, you can just add a reference to your git repo. You can also point to a branch , ref and tag.

You can get more detail on git in Gemfiles here.

9 Responses to “Hosting your own gem server, simply”

  1. What are the advantages of this vs installing from your private source control repo, something like the following in your Gemfile…

    gem ‘smallworld’, :git => ‘git@git.assembla.com:yadayada.2.git’, :tag => “0.9.1”

    It’s a cool exercise either way, nice job and thanks for sharing!

    • michael says:

      I did explore that option, but had a few issues with it:

      You still can’t just install a gem by adding it to your gem sources and doing a ‘gem install’

      I like using gems’ built in versioning without having to rely on git tags and such.

      We have a weird setup with our source control, we have one big repo with multiple gems inside, bundler doesn’t really like that. This was basically a showstopper for me, although most people won’t have this issue.

      Another nice thing about geminabox is that I can easily checkout my gems using web ui, without having to dig into the source repo.

      You are right however, using git in bundler is definitively the simplest way, and I should have mentioned it.

      Thanks for the comment!

  2. Josh Cheek says:

    I think the git solution is better. If you host your own gem server with private gems, and there is a name conflict with a public gem (ie someone releases a public ruby gem with the same name as yours) then Bundler can’t tell the difference, you can’t tell it which one to install. It thinks they are both the same and instead uses versions to decide which gem to install, making the dependencies volatile. (e.g. say you put gem ‘mygem’, ‘~> ‘1.2.3’ and then someone else releases a public mygem version 1.2.6, bundler will install theirs not yours)

    Also, when developing, we make tons of changes to our gems. Having to rely on version numbers is a PITA. Instead, we just point it at the local path and leave it alone (for CI, anyway. For development, we point it at local paths so that we can edit all the way through without having to talk to git or to the gem server) It’s annoying, too, in that you have to change your Gemfile whenever you develop, but have to make sure not to commit it, but it’s still way better than having to jump through hoops.

    • michael says:

      Thanks Josh, I actually didn’t know that bundler would get confused with name clashes. Luckily I named my gems to be pretty unique to our team, but now I at least know to avoid it in future.

      Although I agree on your thoughts about making a lot of changes to your gems while developing, using only git lets the gem developer commit ‘unstable’ code that might break consuming code in unexpected ways.
      I prefer being a bit more explicit and using a ‘push-pull’ model, meaning gem developers can commit code to git without worrying about breaking anything and only push a gem at a stable point.
      You can also only use a new version of a gem when you want it. If you don’t want to explicitly use version numbers in your Gemfile, it just uses the latest stable gem.

      Sure, this can be done with git tags, but gems already have good versioning.

      All in all, I do actually agree with you. git is by far the simplest way of ‘hosting’ gems. It is the first thing I tried, however it just didn’t work well for my particular situation. It’s actually good to know that there are so many options available to you.

      Thanks for the comment!

  3. [...] Hosting Your Own Local RubyGems Server Want to have your own in-house RubyGems server? It's easy and Michael Erasmus shows you how in this post. [...]

  4. I am also using geminabox and like it very much.

    One extra step I’ve taken is to monkey patch bundler so it’s “rake release” task so that publishes to my private gemserver instead of rubygems.org. In case the code below is not readable in this comment you can see what I mean at http://www.alexrothenberg.com/2011/09/16/running-a-private-gemserver-inside-the-firewall.html

    module Bundler
    class GemHelper
    def rubygem_push(path)
    gem_server_url = ‘http://gems.intranet.mycompany.com’
    sh(“gem inabox ‘#{path}’ –host #{gem_server_url}”)
    Bundler.ui.confirm “Pushed #{name} #{version} to #{gem_server_url}”
    end
    end
    end

  5. michael says:

    Ah, brilliant idea Alex! I was actually quite worried about running ‘rake release’ by accident, so I’ll definitely give this a try.

Leave a Reply