tag:blogger.com,1999:blog-89179311155542259732024-03-18T19:44:19.908-07:00Sleepless CodingA simple blog about the coding I do when I can't sleep.scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.comBlogger33125tag:blogger.com,1999:blog-8917931115554225973.post-69045968316732040962012-01-15T18:22:00.000-08:002012-01-15T18:22:45.080-08:00My Blog Has MovedI will no longer be posting at this blog. The posts and comments will remain intact, but all new posts will happen at by new site: <a href="http://scottwb.com/">http://scottwb.com</a>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com1tag:blogger.com,1999:blog-8917931115554225973.post-71665326942157693692011-09-04T11:33:00.000-07:002011-09-04T11:33:28.017-07:00Recreate a single version of a CarrierWave::Uploader fileCarrierWave supports a simple way to recreate all the versions of an uploaded file. This comes in handy when you add a new version to your uploader, or when you change the parameters of one of them. For example, assume you have a <span style="font-family: "Courier New",Courier,monospace;">Post</span> class, with a CarrierWave mounted uploader on the <span style="font-family: "Courier New",Courier,monospace;">photo</span> column, and you have version called <span style="font-family: "Courier New",Courier,monospace;">thumb</span>. If you decide to change the resolution of the thumb version, and need to go back and regenerate all existing thumbs, you can do something like this:<br />
<br />
<pre class="ruby" name="code">Post.all.each do |post|
post.photo.recreate_versions!
end
</pre>
<br />
What if you have 5 different versions on this Uploader, and you only changed one of them? The above method will download all the versions of every <span style="font-family: "Courier New",Courier,monospace;">Post#photo</span> from storage (think S3 or Cloud Files), regenerate all of them - <i>even the ones that haven't changed</i> - and re-upload all of them to storage.<br />
<br />
That's a lot of wasted time, CPU, and bandwidth.<br />
<br />
To address this, I've created a simple monkey patch for CarrierWave to provide a new method called <span style="font-family: "Courier New",Courier,monospace;">recreate_version!</span>. This method works just like <span style="font-family: "Courier New",Courier,monospace;">recreate_versions!</span>, except that it takes a parameter specifying a single version to recreate, rather than recreating all of them.<br />
<br />
To use this, first drop this monkey-patch in somewhere that it will get loaded. Above your Uploader class will suffice.<br />
<br />
<br />
<script src="https://gist.github.com/1193236.js?file=carrierwave_recreate_version.rb">
</script><br />
<br />
Now your script to recompute all the thumbs can look like this:<br />
<br />
<pre class="ruby" name="code">Post.all.each do |post|
post.photo.recreate_version!(:thumb)
end
</pre>
<br />
This will recreate and re-upload only the <span style="font-family: "Courier New",Courier,monospace;">thumb</span> versions of your <span style="font-family: "Courier New",Courier,monospace;">Post#photo</span> uploads.<br />
<br />
As an added side benefit, the <span style="font-family: "Courier New",Courier,monospace;">recreate_version!</span> method cleans up after itself. The <span style="font-family: "Courier New",Courier,monospace;">recreate_versions!</span> method downloads into a temporary cache dir, but leaves it up to the caller to clean that up. This new method assumes that if it had to download and cache the uploaded file and its versions, then it should clean them up immediately.<br />
<br />
This method has turned out to save me a lot of time in regenerating large numbers of uploaded file versions. If there's any interest in this, I might work up a pull request with test cases and try to get it into the mainline CarrierWave gem release.<br />
<br />scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com7Kirkland, WA47.6814875 -122.208735347.638726500000004 -122.2876993 47.7242485 -122.1297713tag:blogger.com,1999:blog-8917931115554225973.post-45106406794603188662011-08-23T22:04:00.000-07:002012-02-17T00:21:42.045-08:00Adding filters to stock Devise controllersHave some before_filters or after_filters you want to add to some or all of the Devise controller actions without having to subclass all the Devise controllers?
<br />
<br />Here's a little trick I use: directly call the <span style="font-family:courier new;">before_filter</span> or <span style="font-family:courier new;">after_filter</span> class methods in the <span style="font-family:courier new;">devise.rb</span> initializer. For example, I have a <span style="font-family:courier new;">prepare_for_mobile</span> method that I want to apply as a before_filter to all of the Devise controller actions. This is what I have at the bottom of <span style="font-family:courier new;">config/initializers/devise.rb</span>:
<br />
<br /><script src="https://gist.github.com/1167350.js?file=devise.rb"></script>
<br />
<br />Note that this is exactly the same as if you had added the before_filter call directly inside these controllers. That means if you wanted to be more selective, you could just as easily do something like:
<br />
<br /><script src="https://gist.github.com/1167352.js?file=devise.rb"></script>
<br />
<br />
<br /><span style="font-style: italic;">IMPORTANT NOTE:</span> This tactic only works reliably in the production environment. That is because in development mode, the Devise controller classes get reloaded on each request, but this config file that we've edited does not.
<br />
<br />
<i><b>UPDATED 2/16/2012:</b> See my <a href="http://scottwb.com/blog/2012/02/16/revisited-adding-filters-to-stock-devise-controllers/">follow-up post</a> where I revisit this issue with a solution that works in both production and development environments.</i>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com1tag:blogger.com,1999:blog-8917931115554225973.post-7015330551775783412011-08-21T14:21:00.000-07:002011-08-21T14:44:59.226-07:00Rackspace Cloud Files SSL Failures Over ServiceNetI recently switched from using the public interface on my Rackspace Cloud Server to using the private interface (a.k.a. ServiceNet) for uploading files from my instance to Cloud Files. When I made this switch, I started noticing a number of intermittent "Connection reset by peer" errors.
<br />
<br />I have reproduced this problem with both the `Fog` and `ruby-cloudfiles` gems.
<br />
<br />To upload to Cloud Files, I'm using CarrierWave, which uses Fog, which uses Excon. With those, the error looks something like this:
<br /><pre name="code" class="ruby">Excon::Errors::SocketError: Connection reset by peer - SSL_connect
<br />
<br />[GEM_ROOT]/gems/excon-0.6.4/lib/excon/connection.rb:264:in `connect'
<br />[GEM_ROOT]/gems/excon-0.6.4/lib/excon/connection.rb:264:in `open_ssl_socket'
<br />[GEM_ROOT]/gems/excon-0.6.4/lib/excon/connection.rb:243:in `connect'
<br />[GEM_ROOT]/gems/excon-0.6.4/lib/excon/connection.rb:282:in `socket'
<br />[GEM_ROOT]/gems/excon-0.6.4/lib/excon/connection.rb:156:in `request'
<br />[GEM_ROOT]/gems/fog-0.9.0/lib/fog/core/connection.rb:20:in `request'
<br />[GEM_ROOT]/gems/fog-0.9.0/lib/fog/storage/rackspace.rb:102:in `request'
<br />[GEM_ROOT]/gems/fog-0.9.0/lib/fog/storage/requests/rackspace/put_object.rb:19:in `put_object'
<br />[GEM_ROOT]/gems/fog-0.9.0/lib/fog/storage/models/rackspace/file.rb:59:in `save'
<br />[GEM_ROOT]/gems/fog-0.9.0/lib/fog/core/collection.rb:50:in `create'
<br />[GEM_ROOT]/gems/carrierwave-0.5.6/lib/carrierwave/storage/fog.rb:229:in `store'
<br /></pre>
<br />I never really narrowed down why this is happening, but it seems to happen only after a period of inactivity. I.e.: it fails once, then works fine as long as I keep making requests, but after 10+ minutes, the next request fails again.
<br />
<br />Since I am only storing files to Rackspace Cloud Files via CarrierWave, the most expedient thing for me to do was to wrap the storage call in CarrierWave with some exception handling to retry these failures. It's a hack, but it works like a charm and saved me at crunch time:
<br />
<br /><script src="https://gist.github.com/1161210.js?file=carrierwave_fog_hack.rb"></script>
<br />
<br />Even if you're not using CarrierWave or Fog, this concept could work for you. Of course, you may use more operations that just storing files, and you'll probably have to wrap those too.
<br />
<br />scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com1tag:blogger.com,1999:blog-8917931115554225973.post-47971904053545761632011-08-18T14:33:00.000-07:002011-08-18T15:06:57.623-07:00Performance of Rackspace Cloud Files Upload Over Private Network<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGsTS-78pOdfkXn3YtMUpUhKsTiSRYYi9owTp73PqtiFWcgc3q3va1PfL2xNBhE7KjNmURxay7sR7trJR8keexV2UBq4B-I0OJr5A5HDL2307VHpGHBbVZD9eRhRplfrjr42-QotHq-MlL/s1600/rackspace.png"><img style="float:left; margin:0 10px 10px 0;cursor:pointer; cursor:hand;width: 199px; height: 200px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGsTS-78pOdfkXn3YtMUpUhKsTiSRYYi9owTp73PqtiFWcgc3q3va1PfL2xNBhE7KjNmURxay7sR7trJR8keexV2UBq4B-I0OJr5A5HDL2307VHpGHBbVZD9eRhRplfrjr42-QotHq-MlL/s200/rackspace.png" alt="" id="BLOGGER_PHOTO_ID_5642320490402839842" border="0" /></a>
<br />
<br />
<br />Does using the private network interface on a Rackspace Cloud Server instance to transfer data into Rackspace Cloud files save you anything?
<br />
<br />I am certain it saves you money. You pay for outgoing bandwidth on your Cloud Server instance's public IP address, but not on the private one. But, I've also read that it is much faster to use the private IP, or "ServiceNet" as they call it. "Lightning fast" even. So, I tried make a quick-and-dirty benchmark to see for myself.
<br />
<br />In short, I don't think it makes much difference, considering the margin of error. Here are the results of a typical test run that shows the timings of uploads of a 1MB file alternating over the public and private interfaces, with a cumulative time printed out at the bottom:
<br />
<br /><a href="https://gist.github.com/1155335">https://gist.github.com/1155335</a>
<br />
<br />That example is the most sane run of many test runs I did with this script. What is more disconcerting is that both public and private have wild outliers. Sometimes a transfer that normally takes a tenth of a second can take 30 seconds or even two minutes!
<br />
<br />Check out this test run: <a href="https://gist.github.com/1155349">https://gist.github.com/1155349</a>
<br />
<br />Whoa! The 31st upload took 132 seconds! In my experience over multiple runs, the private snet almost always comes out way ahead. It seems like it is just as likely to have 10-second outliers, but less likely to have 30+ second outliers.
<br />
<br />If you are interested in reproducing my tests, you can get the code at: <a href="https://github.com/scottwb/cloudfiles-snet-bench">https://github.com/scottwb/cloudfiles-snet-bench</a>
<br />
<br />On caveat to this test, other than generally not being very scientific, is that I am not using large files (which is the use case I am actually interested in). It is possible that large transfers may get big speed gains over the private servicenet.scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com1tag:blogger.com,1999:blog-8917931115554225973.post-87975606602815312512011-08-16T23:35:00.000-07:002011-08-16T23:48:24.411-07:00Rails incorrectly parses HTTP Accept header for single media with rangeTry hitting your Rails app with an HTTP header like this:
<br />
<br /><pre name="code" class="ruby">Accept: text/html;q=0.7
<br /></pre>
<br />
<br />You will most likely get an exception that looks something like this:
<br />
<br /><pre name="code" class="ruby">Missing template home/index with {:locale=>[:en, :en], :formats=>["text/html;q=0.7"], :handlers=>[:rhtml, :erb, :haml, :rxml, :builder, :rjs]} in view paths "/path/to/my_app/app/views"
<br /></pre>
<br />
<br />This is because the ActionPack <span style="font-family:courier new;">Mime::Type</span> class is buggy in the way it parses this header when there is only one media type specified, and that media type has a range specified (the ";q=0.7"). This is perfectly valid HTML, and ActionPack actually handles the ranges correctly when there are more than one type specified.
<br />
<br />An issue has been open for a while on this: <a href="https://github.com/rails/rails/issues/736">https://github.com/rails/rails/issues/736</a> -- working patches have been proposed but not applied, yet the issue got closed for some reason.
<br />
<br />I prefer not to patch my own version of Rails, so instead I am using a small monkey patch that works around this problem. I've created a file named <span style="font-family:courier new;">config/initializers/rails_issue_736.rb</span> that fixes the <span style="font-family:courier new;">Mime::Type.lookup</span> method to only use the part that comes before the semi-colon:
<br />
<br /><script src="https://gist.github.com/1150932.js?file=rails_issue_736.rb"></script>
<br />scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com3tag:blogger.com,1999:blog-8917931115554225973.post-91700155506071329182011-08-15T14:42:00.001-07:002011-08-15T15:03:43.905-07:00MailMatic: HTML emails using Haml and SassDeveloping static HTML emails is a pain in the butt. For best results, you need to inline all of your styles at HTML attributes instead of using real CSS (externally or inline). You also typically find yourself being very repetitive with your markup if you have a bunch of static emails that share common elements.
<br />
<br /><a href="https://github.com/scottwb/mailmatic">MailMatic</a> to the rescue.
<br />
<br />MailMatic is a Ruby gem, that is really just a mashup of tools, that aims to ease the burden of creating and maintaining static HTML emails. It uses <a href="https://github.com/staticmatic/staticmatic">StaticMatic</a>, <a href="http://haml-lang.com/">Haml</a>, <a href="http://sass-lang.com/">Sass</a>, and <a href="http://compass-style.org/">Compass</a> to let you author your HTML emails with all of the facilities you enjoy while creating your dynamic site: templates, partials, helper functions, inline Ruby code, reuse of markup and styles, etc. It uses <a href="https://github.com/alexdunae/premailer">Premailer</a> to convert the HTML/CSS generated from Haml/Sass into self-contained HTML files that inline all of their styles. Premailer even gives you warnings about styles that may not work nicely in certain email readers.
<br />
<br />Armed with this, you can accelerate your ability to crank out static HTML emails, and lessen the burden of maintaining them.
<br />
<br />To get started with MailMatic, check out its installation and usage instructions on github at: <a href="https://github.com/scottwb/mailmatic">https://github.com/scottwb/mailmatic</a>.
<br />scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com3tag:blogger.com,1999:blog-8917931115554225973.post-10813640610123831042011-07-19T21:18:00.000-07:002011-07-20T09:15:32.111-07:00The ask_geo gem: find the time zone for a given latitude and longitude<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://upload.wikimedia.org/wikipedia/commons/thumb/9/9f/Tz_map_world2009r_efeledotnet.png/800px-Tz_map_world2009r_efeledotnet.png"><img style="float:left; margin:0 10px 10px 0;cursor:pointer; cursor:hand;width: 800px; height: 408px;" src="http://upload.wikimedia.org/wikipedia/commons/thumb/9/9f/Tz_map_world2009r_efeledotnet.png/800px-Tz_map_world2009r_efeledotnet.png" alt="" border="0" /></a><br /><br /><br />The <a href="http://code.google.com/apis/maps/index.html">Google Maps API</a> has all kinds of great services like geocoding and reverse geocoding, but it lacks an API to determine a timezone from a latitude and longitude. This kind of service would come in handy, for example, when you are picking a point on Earth and scheduling an event there in the point's local time. It could also be useful for automatically determining a website visitor's current local time based on the geolocation obtained from their browser or IP address.<br /><br />There is a service called <a href="http://www.geonames.org/export/web-services.html">GeoNames</a>, which comes close to serving this need. However, its look-ups are based on "closest point of interest", which means that you can get errors when you are near borders, and some locations may not return any results at all.<br /><br />There is another service called <a href="http://www.earthtools.org/webservices.htm">EarthTools</a> which doesn't have this problem, but it only returns the current offset from UTC at that point. That is great if you only care about the current local time, but leaves you stranded if you need to account for Daylight Savings Time on some future or past date.<br /><br />After a bit of searching, I came across a great free service called <a href="http://www.askgeo.com/">AskGeo</a>. AskGeo provides a very simple JSON-over-HTTP API (or XML if you prefer to party like it's 1999). AskGeo uses a real time zone map, so every point on earth returns a result, and returns real <a href="http://en.wikipedia.org/wiki/Tz_database">Olson IDs</a> for time zones, so you can look up DST transitions, leap seconds, etc.<br /><br />I gave AskGeo a try and it works great...and it's fast! Since there was no ready-made Ruby gem that provided a client API, I decided to create one.<br /><br />Introducing the ask_geo gem: <a href="https://github.com/scottwb/ask_geo">https://github.com/scottwb/ask_geo</a><br /><br />It can be installed as a <a href="https://rubygems.org/gems/ask_geo">ruby gem</a>. See <a href="https://github.com/scottwb/ask_geo">the github page</a> for installation and usage instructions.scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com3tag:blogger.com,1999:blog-8917931115554225973.post-85541002860041891462011-07-10T18:10:00.000-07:002011-07-10T18:26:58.534-07:00Custom RSpec be_square matcher for CarrierWaveIt seems to me that when using CarrierWave for image uploads, particularly user avatars, it would be a pretty common thing to generate at least one square-cropped version of the image. CarrierWave ships with some nice custom matchers to test exact dimensions and such, but I found I wanted a way to just test that an image was square.<div><br /></div><div>I did this using by building a custom matcher on top of the ones CarrierWave provides. If you have the conventional <span class="Apple-style-span" >spec_helper.rb</span> that requires <span class="Apple-style-span" >support/**/*.rb</span>, then you can just drop this somewhere like <span class="Apple-style-span" >support/matchers/carrierwave_matchers.rb</span>:</div><div><br /></div><br /><script src="https://gist.github.com/1075177.js?file=carrierwave_matchers.rb"></script><br /><div><br /></div><div>Then, you can use this like:</div><div><br /></div><br /><script src="https://gist.github.com/1075180.js?file=avatar_uploader_spec.rb"></script><br /><div><br /></div>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com1tag:blogger.com,1999:blog-8917931115554225973.post-52920509063100866972011-05-31T11:48:00.000-07:002011-05-31T12:16:23.841-07:00Encrypted Cookie Store that works with Rails 3.0If you like the benefits using Rails's CookieStore, but want to store possibly sensitive data in the cookie, then encrypted_cookie_store is for you. It works like CookieStore, but encrypts the cookie payload so that it cannot be read by the client.<div><br /></div><div>If you are using Ruby on Rails 3.0.0 or higher (but not yet Rails 3.1), then you'll need my fork of encrypted_cookie_store that fixes it for Rails 3.</div><div><br /></div><div>To install it:</div><div><br /></div><pre class="ruby" name="code">gem install scottwb-encrypted_cookie_store</pre><div><br /></div><div>Then add to your bundler Gemfile:</div><div><br /></div><pre class="ruby" name="code">gem 'scottwb-encrypted_cookie_store', :require => 'encrypted_cookie_store'</pre><div><br /></div><div>Then read the rest of the <a href="https://github.com/scottwb/encrypted_cookie_store/blob/master/README.md">installation/configuration instructions</a>.<br /><div><br /></div><div><span class="Apple-style-span" style="font-size:large;">Background</span></div><div><br /></div><div>For Rails 2.3, the folks at Phusion created <a href="https://github.com/FooBarWidget/encrypted_cookie_store">encrypted_cookie_store</a>, which you'd install as a plugin, and it worked great. I used it for a long time.</div><div><br /></div><div>However, they never updated it for Rails 3. That's where <a href="https://github.com/twoism-dev">Ben Sales</a> came in. He <a href="https://github.com/twoism-dev/encrypted_cookie_store">forked this project</a> and made it work for Rails 3...in the pre-release days, that is. Ben did all the work to get it packaged up as a gem and updated it to work with Rails 3 railties and initializers.</div><div><br /></div><div>Unfortunately, <a href="https://github.com/rails/rails/commit/25f7c030e4ea440ea6c2a84c92118299753392d9#actionpack/lib/action_dispatch/middleware/session/cookie_store.rb">sometime between Rails 3.0.0.beta3 and 3.0.0.beta4</a>, the layout of AbstractStore and CookieStore changed quite a bit, pushing a lot of the functionality out to Rack, and breaking the encrypted_cookie_store gem.</div></div><div><br /></div><div>That's where I come in. I basically did the minimal amount of work required to get it to work with Rails 3.0 (tested on 3.0.0, 3.0.7, and 3.0.8.rc4), got all the specs working again, and created a new gem called 'scottwb-encrypted_session_cookie'.</div><div><br /></div><div>It doesn't work in Rails 3.1, but I'll probably remedy that once Rails 3.1 officially releases. I'm also happy to accept patches if anyone else onces to tackle that.</div><div><br /></div><div>This is a nice gem. Maybe some day I'll make a push to clean it up and lobby to have it as one of the packaged options that ships with Rails...</div><div><br /></div>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com3tag:blogger.com,1999:blog-8917931115554225973.post-6143638540935814352011-03-30T07:57:00.000-07:002011-03-30T08:03:42.536-07:00How to delete a remote git tagI suppose this is probably taboo. You're not supposed to mess with tags once you've pushed them upstream. Still, there are times when it is necessary, and unfortunately `git push --tags` does not push the local removal of tags upstream. It always takes me forever to figure this out, so I figured I'd write it down. Maybe someone else will find it useful.<br /><br /><pre name="code" class="ruby"><br /> git tag -d tag-to-delete<br /> git push origin :refs/tags/tag-to-delete<br /></pre><br /><br />This will remove that tag from the remote repository. You can verify this by pulling from the remote again and seeing that it does not get pulled back down.scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com1tag:blogger.com,1999:blog-8917931115554225973.post-1676406143340164062011-02-10T20:09:00.000-08:002011-02-10T20:19:25.455-08:00Rails-like number_with_delimiter in javascriptThe number_with_delimiter view helper is Rails is nice. I wanted to do the same thing in Javascript. I went to just grab it from Rails and port it to js. But first I googled and found someone had already done exactly that. I took his code and made it extend Number.protoype so you can do something like:<br /><br /><pre name="code" class="javascript"><br />var num = 12345678;<br />var str = num.number_with_delimiter();<br />alert(str);<br /></pre><br /><br />This will show that str is "12,345,678".<br /><br />Her's my version of the code:<br /><br /><script src="https://gist.github.com/821904.js?file=number_with_delimiter.js"></script><br /><br />Based on code from: <a href="http://kevinvaldek.com/number-with-delimiter-in-javascript">http://kevinvaldek.com/number-with-delimiter-in-javascript</a>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com0tag:blogger.com,1999:blog-8917931115554225973.post-31295921496718842042011-01-27T21:48:00.000-08:002011-01-27T22:18:07.106-08:00YUI Compressor breaks CSS3 Media Query SyntaxIf you use the CSS3 Media Query Syntax for something like this:<br /><br /><pre name="code" class="css"><br />@media screen and (max-device-width: 480px) {<br /> body {font-size: 18px;}<br />}<br /></pre><br /><br />You will find that when you compress this with <a href="http://developer.yahoo.com/yui/compressor/">YUI Compressor</a> (as is done when packaging your assets with <a href="http://documentcloud.github.com/jammit/">Jammit</a>), that the output breaks this syntax. YUI Compressor removes the whitespace between the "and" and the opening parenthesis, which does not conform to <a href="http://www.w3.org/TR/css3-mediaqueries/">the CSS3 standard</a>, and renders those styles unusable in most browsers (tested in Chrome and Safari, read about others hitting this in Firefox).<br /><br />There's a YUI Compressor bug filed here that covers this: <a href="http://yuilibrary.com/projects/yuicompressor/ticket/2528053">http://yuilibrary.com/projects/yuicompressor/ticket/2528053</a><br /><br /><h3>My Workaround</h3><br /><br />I am invoking YUI Compressor via an asset-packager called Jammit. In a <a href="http://sleeplesscoding.blogspot.com/2011/01/jammit-compass-and-safari-cant-we-all.html">blog post yesterday</a>, I noted some problems I encountered with Jammit not handling duplicate @charset directives generated by Sass/Compass, and outlined a quick hack workaround I used to deal with it.<br /><br />Today, encountering this @media-related bug leads to an additional workaround I had to add to that. So, I figured I'd share a bit of my resulting rake task that combines all of these.<br /><br /><ul><br /> <li>Use compass to compile my Sass to CSS <span style="font-style:italic;">without compressing it (this is important)</span></li><br /> <li>Use sed to remove all @charset directives (dangerous, but I know I don't need them in this case). Take care to make sure the differences in how the -i parameter to sed works between OS X and Linux.</li><br /> <li>Use jammit to package up everything.</li><br /> <li>For each permutation (e.g.: normal, datauri) CSS package jammit generated, use sed to add the missing space back in to offending @media directives.</li><br /> <li>For each of those CSS files we sed-edited, re-gzip them. (Jammit made gzipped versions originally, but we have to regenerate them since we changed the contents.)</li><br /></ul><br /><br />Here's the final rake task:<br /><br /><script src="https://gist.github.com/799901.js?file=assets.rake"></script>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com0tag:blogger.com,1999:blog-8917931115554225973.post-21795816647901433722011-01-26T15:17:00.000-08:002011-01-26T15:52:28.246-08:00Jammit, Compass, and Safari: Can't we all just get along?When <a href="http://documentcloud.github.com/jammit/">Jammit</a> is used with CSS files generated by <a href="http://compass-style.org/">Compass</a>, the resulting CSS file may not work with Safari. This is because every <a href="http://sass-lang.com/">Sass</a> file that includes Compass will add the following line to the top of the generated CSS file:<div><br /></div><div> <pre class="code"> @charset "UTF-8";</pre></div><div><br /></div><div>That happens because something in Compass requires UTF-8, and Sass will generate that line if anything it is processing requires UTF-8. This is all fine...until you try to combine more than one of these standalone-css files with an asset packager such as Jammit.</div><div><br /></div><div>Once Jammit combines all these CSS files, you end up with one CSS file that has multiple @charset directives in it, which according to the <a href="http://www.w3.org/International/questions/qa-css-charset">W3</a>, is a no-no. Generating a CSS file with more than one @charset directive (and anywhere other than the very first line) is clearly against the W3's spec...but the only browser it seems to break is Safari.</div><div><br /></div><div>But here's the thing: why do my CSS files need to have the @charset added by Sass when they don't actually get output with any non-ASCII characters?</div><div><br /></div><div>In the end, my quick stop-gap solution was to remove them myself. I used to let Jammit do all the driving. During deployment it would force the regeneration of the CSS files and package them. Now I have to break out the Compass step and add my own sed command in order to remove all the @charset directives:</div><div><br /></div><div><br /><script src="https://gist.github.com/797776.js?file=jammit.sh"></script><br /></div><div>I think each one of these components has something to improve:</div><div><br /></div><div><b>Safari:</b></div><div>If you don't support multiple @charset directives, I support you on that front. The spec says you're right. However...can't you just raise an error that says "ERROR: multiple @charset directives" or something? That would be much better than just randomly disabling half the CSS styles.</div><div><br /></div><div><b>Sass:</b></div><div>Sass shouldn't add a @charset line if the generated CSS does not need it. If a mixin that uses UTF-8 is included but not used, and is not represented in the generated CSS, there is no need to add a @charset directive on its behalf.</div><div><br /></div><div><b>Compass:</b></div><div>Compass is probably doing nothing wrong here. Though I do question why it needs to use UTF-8 characters. Maybe that's for a feature I haven't used yet.</div><div><br /></div><div><b>Jammit:</b></div><div>In the end, I think Jammit is the real culprit here. It takes in N perfectly-valid CSS files and outputs one invalid CSS file. Jammit should remove duplicate @charset directives, and if they are conflicting, it should convert the charset of the conflicting stylesheet and into the charset being used by the output file.</div><div><br /></div><div><br /></div>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com4tag:blogger.com,1999:blog-8917931115554225973.post-27870720504036448792011-01-25T08:38:00.000-08:002011-01-25T09:25:41.374-08:00Workaround an IE bug: Redirecting subdomains to www while preserving virtual hosts (Nginx, Rails)I ran into an interesting redirect configuration problem that was exacerbated by Internet Explorer's handing of cookies, which led me to move all subdomain redirection logic out of my Rails app and into Nginx. In the end, this feels like a good thing anyway. If you just want the conclusion, skip to the solution section.<br /><br /><h3>Background</h3><br /><br />My original setup was to have nginx have virtual hosts for www.example.com, test.example.com, and staging.example.com, where the www.example.com was the default one that would catch anything else such as foo.example.com, or just example.com. Then the Rails app at www.example.com would check the URI in a before_filter and redirect to www.example.com, using a solution similar to the one described here: <a href="http://stackoverflow.com/questions/327122/redirect-myapp-com-to-www-myapp-com-in-rails-without-using-htaccess">http://stackoverflow.com/questions/327122/redirect-myapp-com-to-www-myapp-com-in-rails-without-using-htaccess</a>.<div><br /></div><div><h3>Problem</h3></div><div><br /></div><div>The problem arose when an Internet Explorer user hit the site via http://example.com (i.e.: without the "www." prefix). Here is what happens when someone hits example.com in this setup:</div><div><ol><li>The default virtual host for www.example.com picks up the request and hands it to the Rails app.</li><li>The Rails app's before_filter sees that the request.host does not start with 'www', and does a redirect.</li><li>Somewhere the ActionController internals set a session cookie on the domain "example.com" -- even though this before_filter redirects BEFORE any code tries to touch the session. (Perhaps this is an artifact of using CookieStore.)</li><li>The browser stores the session cookie for example.com, the follows the redirect to www.example.com, sending that cookie (which is basically an empty session at this point).</li><li>The user logs in. The Rails app sets stuff in the session and returns a response that sets the new session cookie with the new session, for www.example.com.</li><li>The browser now has two session cookies: One not-logged-in one for example.com, and one logged-in one for www.example.com.</li><li>The next page load is where the problems start...</li></ol></div><div>On Safari, Chrome, and Firefox, the next page load, on www.example.com, only sends the session cookie that is for www.example.com...and all is well. However, on Internet Explorer (tested on IE8 and IE9 beta), the browser sends BOTH session cookies, which both have the same name. In my observation, Rails always choose the first one, which is always the LEAST specific one - the one for example.com. Now, even though the user just logged in, he is not logged in.</div><div><br /></div><div>I don't know what the HTTP RFC for user agents says about this, but I am just gonna go ahead and call this in IE bug. According to an <a href="http://blogs.msdn.com/b/ieinternals/archive/2009/08/20/wininet-ie-cookie-internals-faq.aspx">MSDN FAQ about IE's cookie-handling</a>, IE is different than other browsers, and they're just fine with that (see Q3 on that page). Also see Q1 where they say they don't even try to support the RFC for cookies.</div><div><br /></div><div>Similar problems have been documented on the web. I read <a href="http://arstechnica.com/civis/viewtopic.php?f=20&t=1126381">somewhere</a> that if your Rails app redirects before touching the session, that you won't have this problem. That's not the case for me, but I suspect that may have to do with using CookieStore. I can imagine that CookieStore touches the session by reading it when it comes time to generate the cookie, and that this gets triggered even after a redirect.</div><div><br /></div><div><h3>Redirect with Nginx</h3></div><div><br /></div><div>Since I could not get Rails to NOT set a cookie for example.com, I decided I would do this redirect in nginx. There are plenty of <a href="http://www.debrice.com/tip/view/13/">examples</a> on the web about redirecting example.com to www.example.com, or vice-versa. However, most of them don't work for my needs for one particular reason: virtual hosts.</div><div><br /></div><div>In my scenario, I can't redirect ALL requests to non-www subdomains to the www-subdomain, because I have other virtual hosts on specific sub-domains that should not be re-routed. However, I can't simply enumerate the ones that I want redirected, since that is meant to be a catch-all. Rather than try to maintain a regular expression that matches all subdomains except the ones I setup virtual hosts on, I decided to just test the hostname inside the default virtual host, as shown in the conclusion section below. This way, specific subdomains get routed to the right virtual host, anything else gets routed to the www.example.com virtual host, and within that one, anything that isn't www.example.com gets redirected to www.example.com.</div><div><br /></div><div><h3>Solution</h3></div><div><br /></div><div>The final solution: this nginx config mockup shows how I can have specific subdomains handled by their virtual hosts, and everything else redirected to the www subdomain.</div><div><br /></div><div><br /><script src="https://gist.github.com/795176.js?file=nginx_subdomain.conf"></script><br /></div>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com2tag:blogger.com,1999:blog-8917931115554225973.post-91706564876721340112011-01-25T08:10:00.000-08:002011-01-25T08:23:08.336-08:00Using tcpdump to sniff HTTP traffic from a specific hostThis is mostly just a reminder to myself about my preferred parameters to <span class="Apple-style-span">tcpdump</span> on linux, so that I don't have to keep reading the man page.<div><br /></div><div><pre name="code">tcpdump -c 20 -s 0 -i eth1 -A host 192.168.1.1 and tcp port http</pre></div><div><br /></div><div>The parameter breakdown:</div><div><ul><li><span class="Apple-style-span">-c 20</span>: Exit after capturing 20 packets.</li><li><span class="Apple-style-span">-s 0</span>: Don't limit the amount of payload data that is printed out. Print it all.</li><li><span class="Apple-style-span">-i eth1</span>: Capture packets on interface eth1</li><li><span class="Apple-style-span">-A</span>: Print packets in ASCII.</li><li><span class="Apple-style-span">host 192.168.1.1</span>: Only capture packets coming to or from 192.168.1.1.</li><li><span class="Apple-style-span">and tcp port http</span>: Only capture HTTP packets.</li></ul></div>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com17tag:blogger.com,1999:blog-8917931115554225973.post-68527321233800611702010-12-10T23:10:00.001-08:002010-12-10T23:19:43.376-08:00Hate the emacs tilde backup files?I do...but for decades I have been afraid to turn them off in case I ever needed the backup file, even though I have never used them once. Finally I snapped and tried to figure out how to turn them off. While looking this up, I came across a better option: store them all in a single hidden directory, instead of littering the whole filesystem with them. There is a nice package called "backup-dir" that even provides the ability to keep multiple versions of your backup files, while keeping them all out of sight and out of mind, and automatically cleaning up old versions. Learn more at:<div><br /></div><div><a href="http://www.emacswiki.org/emacs/BackupDirectory">http://www.emacswiki.org/emacs/BackupDirectory</a></div><div><br /></div><div>To see how I pulled in the <span class="Apple-style-span" >backup-dir</span> package and set it up in my init.el file, see <a href="https://github.com/scottwb/dotfiles/commit/54e6a05d061e05aae3b4a320fb23290857656f49">the change to my dotfiles project</a>.</div><div><br /></div><div><br /><br /></div>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com1tag:blogger.com,1999:blog-8917931115554225973.post-40187486157965903362010-12-08T18:21:00.000-08:002010-12-08T18:34:06.565-08:00Creating/Deleting local and remote branches with gitIn the course of my git-based workflow, more often than not, I use branches like this:<br /><br /><div><ul><li>Create a new local branch starting from the HEAD of my current working branch.</li><li>Immediately push it upstream to the remote origin so that it exists in both places.</li><li>Set the local branch to track the remote branch.</li><li>Commit/push/repeat</li><li>Merge the branch back into master (or wherever it started from)</li><li>Delete the local branch.</li><li>Push to delete the remote branch.</li></ul>Since this seems pretty common, and some of the commands to do this are a bit difficult to remember sometimes, I created two simple scripts called <a href="https://github.com/scottwb/dotfiles/blob/master/bin/git-branch-create">git-branch-create</a> and <a href="https://github.com/scottwb/dotfiles/blob/master/bin/git-branch-delete">git-branch-delete</a> to help with this. Now my workflow looks like:</div><div><br /></div><div><ul><li><i>git branch-create feature_branch</i></li><li>Commit/push/repeat.</li><li>Merge branch back into parent branch.</li><li><i>git branch-delete feature_branch</i></li></ul></div><div><br /></div><div>To use these scripts via the above commands, download the scripts above and place them somewhere in your path. I even created short aliases for them like so:</div><div><br /></div><script src="https://gist.github.com/734254.js?file=gistfile1.sh"></script><div><br /></div>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com0tag:blogger.com,1999:blog-8917931115554225973.post-53680373984832771912010-12-04T11:35:00.000-08:002010-12-04T12:06:24.380-08:00Installing curb gem on Mac OS X 10.6 (Snow Leopard)If you have previously installed curl with something like:<div><br /></div><pre name="code"> sudo port install curl +ssl</pre><div><br /></div><div>and then try to install the curb gem like:</div><div><br /></div><pre name="code"> sudo gem install curb</pre><div><br /></div><div>and you find yourself getting an error that looks something like this:</div><br /><div><pre name="code"><br /> In file included from /opt/local/include/curl/curl.h:44,<br /> from curb.h:12,<br /> from curb.c:8:<br /> /opt/local/include/curl/curlrules.h:144: error: size of array ‘__curl_rule_01__’ is negative<br /> /opt/local/include/curl/curlrules.h:154: error: size of array ‘__curl_rule_02__’ is negative<br /> lipo: can't open input file: /var/folders/wX/wX64Cb+PGjG-EXuklO+I+k+++TI/-Tmp-//ccKIrqTY.out (No such file or directory)<br /> make: *** [curb.o] Error 1<br /></pre></div><div><br /></div><div>Then, I think I have a solution for you. <span style="font-weight:bold;"><span style="font-style:italic;">Change your curb install command to this instead:</span></span></div><div><br /></div><pre name="code"> sudo env ARCHFLAGS="-Os -arch x86_64 -fno-common" gem install curb</pre><div><br /></div><div>It seems a number of people on the web suggest re-installing curl as a universal binary using something like:</div><div><br /></div><div><pre name="code"> sudo port install zlib +universal<br /> sudo port upgrade --enforce-variants openssl +universal<br /> sudo port install curl +universal</pre></div><div><br /></div><div><i><b>I do not like this approach.</b></i> The original problem is that you installed an x86_64 version of curl (which is the default, <i>as it should be</i>) but you are trying to install a universal version of curb (which is <i>unfortunately</i> the default). Instead of re-installing curl to be universal, which is wasteful and takes forever, install curb using the x86_64 flags to build it. This is what you want, it works, and it only takes a few seconds.</div><div><br /></div>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com5tag:blogger.com,1999:blog-8917931115554225973.post-39165915693271817482010-09-21T12:08:00.000-07:002010-09-21T12:29:58.975-07:00Huge speed gain migrating large tables in RailsHave you ever needed to make multiple column adds/removes/changes to a large table in Rails in a single migration? If so, you've probably noticed that each of these individual changes issues a single ALTER TABLE command to the database, which can take a long time on a large table. At least with MySQL, combining these all into a single ALTER TABLE dramatically reduces the amount of time it takes for your migration to run.<div><br /></div><div>Rather than write custom SQL for this, the folks at <a href="http://devblog.xing.com/">XING</a> have created a very cool Rails plugin called <a href="http://github.com/xing/alter_table">alter_table</a>. Their blog post, <a href="http://devblog.xing.com/ruby/alter-table-rails-plugin/">Alter Table Rails Plugin</a>, gives more background on this. Check it out, it's a good, quick read.</div><div><br /></div><div>One thing they didn't mention in their post was what kind of performance gains they observed. I was curious, so I did a quick-and-dirty experiment.</div><div><br /></div><div>Consider an existing table named "songs" that has 330K records in it. First, I tried just adding a column named "foo" using the normal migration style we're all accustomed to:</div><div><br /></div><script src="http://gist.github.com/590351.js?file=gistfile1.rb"></script><div><br /></div><div>On my MacBook Pro, this is what the timings for this looked like:</div><div><br /></div><script src="http://gist.github.com/590352.js?file=gistfile1.txt"></script><div><br /></div><div>Now, after rolling that back, installing the alter_table plugin (as per their instructions), I rewrote this migration to use the new alter_table method:</div><div><br /></div><script src="http://gist.github.com/590353.js?file=gistfile1.rb"></script><div><br /></div><div>Now, on the same machine, this migration takes roughly half the time because it does all the alterations in a single pass. Here are the new timings:</div><div><br /></div><script src="http://gist.github.com/590354.js?file=gistfile1.txt"></script><div><br /></div><div>In real life, I had a table with almost 10 million rows that I needed to do 10 alterations to. Using this plugin, I cut my migration time down by almost 10x! If you find yourself doing more than one alteration on the same table in a single migration, I highly recommend checking out alter_table. You can find their source code at:</div><div><br /></div><div><blockquote><a href="http://github.com/xing/alter_table">http://github.com/xing/alter_table</a></blockquote></div><div><br /></div>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com0tag:blogger.com,1999:blog-8917931115554225973.post-77393535351391438672010-08-23T13:39:00.001-07:002010-08-23T13:48:32.974-07:00Enumerate all Rails ActiveRecord::Base model classesI couldn't quickly find an easy way to enumerate all ActiveRecord::Base-derived model classes in my Rails project (for some simple analysis scripts), so I whipped up this quick and dirty solution.<div><br /></div><div><br /><script src="http://gist.github.com/546281.js?file=enum_models.rb"></script><br /></div><div><br /></div><div>I extended ActiveRecord::Base just out of convenience, so I can call <span class="Apple-style-span" style="font-family:'courier new';">ActiveRecord::Base.each_model_class{|klass| ...}</span> easily. You could apply the same technique in another class if you don't like extending Rails core classes.</div><div><br /></div><div>The "rescue nil" is just a quick hack to catch the cases where you have tables that don't map to classes directly.</div><div><br /></div><div>NOTE: This does NOT enumerate STI-based model classes that don't have their own tables.</div>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com2tag:blogger.com,1999:blog-8917931115554225973.post-65292043140090744242010-08-07T11:44:00.000-07:002010-08-07T14:04:23.858-07:00Rails 2.3 ActiveSupport::Cache::MemoryStore Freezes Objects<div style="text-align: left;">Using <span class="Apple-style-span" style="font-family:'courier new';">MemoryStore</span> for caching in Rails 2.3.X has what I would consider a show-stopper bug. When you write an object to the cache using <span class="Apple-style-span" style="font-family:'courier new';">Rails.cache.write</span>, the original object gets frozen. When you read that object from the cache, it is also frozen. So, if you do something like read/write-through a cache for all your ActiveRecord models, you'll never be able to modify any instances of your models in your code.</div><br />This same behavior is not present in other cache storage classes such as <span class="Apple-style-span" style="font-family:'courier new';">ActiveSupport::Cache::MemCacheStore</span>. I don't think I'd ever use <span class="Apple-style-span" style="font-family:'courier new';">MemoryStore</span> in production...but it is great for testing. When running all of my rspecs and cucumber features, I have those environments set to use <span class="Apple-style-span" style="font-family:'courier new';">MemoryStore</span>. The problem is, many of my tests break when they try to modify objects that were frozen by the cache.<br /><br />Here's a simple example demonstrating this. Look at the WTFs on lines 10 and 14 of the output below:<br /><script src="http://gist.github.com/513051.js?file=gistfile1.txt"></script><br /><br /><b><span class="Apple-style-span" style="font-size:large;">The Attempted Fix</span></b><br />It turns out this bug was filed (and partially fixed) a long time ago. Rails <a href="https://rails.lighthouseapp.com/projects/8994/tickets/2655-railscache-freezes-all-objects-passed-to-it">ticket #2655</a> was filed back in May 2009 about this exact issue. Yehuda Katz and Carl Lerche from EngineYard <a href="http://github.com/rails/rails/commit/1026d7706ffb467eac3cee8142d964bc2d30baa8">committed a fix</a> for this (to duplicate the object into the cache instead of freezing it) on July 1, 2009.<br /><br />This is great. It fixes line 10 in the example output above. However, it is not quite enough if you are putting <span class="Apple-style-span" style="font-family:'courier new';">ActiveRecord::Base</span> objects into the cache because the object's <span class="Apple-style-span" style="font-family:'courier new';">@attributes</span> hash needs to be dup'd too. It turns out, the "carlhuda" pair was well aware of this and <a href="http://github.com/rails/rails/commit/16dc139caa9286638785469304a69ab77e4fe9b5">committed a fix</a> for that too, just moments before the other one.<br /><br />Another problem still remains though. Their fix still involves freezing the duplicate object in the cache and directly returning that frozen object. We need one more change to the <span class="Apple-style-span" style="font-family:'courier new';">read</span> method to duplicate the object on the way out too. I made <a href="http://github.com/scottwb/rails/commit/46f73da7815b9ee9c84d871ef50a356d849161a3">such a fix</a> and updated the <span class="Apple-style-span" style="font-family:'courier new';">test_store_objects_should_be_immutable</span> test for <span class="Apple-style-span" style="font-family:'courier new';">MemoryStore</span> to be consistent with that of <span class="Apple-style-span" style="font-family:'courier new';">MemCacheStore</span><br /><br />The final problem is that carlhuda's commits never got merged into the Rails 2.3 branch. A quick look at the <a href="http://github.com/rails/rails/network">github network graph for the rails project</a> on July 1, 2009 tells the story.<div><br /></div><div><span class="Apple-style-span" style="color: rgb(0, 0, 238); -webkit-text-decorations-in-effect: underline; "><a href="http://img.skitch.com/20100807-c5m47abjaffmbe32u6up74dt3s.png"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhemT96nvG25aHtuM3Z-5CtO2uWnjSVcvEYCGgHh4vj39Dt0MFQBEDOVqtzVXlP2uz1ZhJTf0LluZNpIMXGF1zxLelO_ncQ9HamTuRkbG4xoJUxmTdZ3eZZeHyZhu3Yk3-qdv_WhfKrSqmQ/s400/The+rails+Network+-+GitHub.png" border="0" alt="" id="BLOGGER_PHOTO_ID_5502768140640845202" style="display: block; margin-top: 0px; margin-right: auto; margin-bottom: 10px; margin-left: auto; text-align: center; cursor: pointer; width: 400px; height: 102px; " /></a></span></div><div><div style="text-align: center;"><span class="Apple-style-span" style="font-family:arial;"><span class="Apple-style-span" style="font-size:x-small;">Click Image To Enlarge</span></span></div></div><div><br /></div><div>That yellow branch looks like wycats and carllerche were pairing on a bunch of bug fixes around this time. Notably the two short red arrows I drew here show that they cherry-picked two of those and merged them into the <span class="Apple-style-span" style="font-family:'courier new';">2-3-stable</span> branch. Then, shortly after that, they make a few more fixes that never got merged into the <span class="Apple-style-span" style="font-family:'courier new';">2-3-stable</span> branch. The two I circled in red are the two fixes I mentioned above.<br /><br /><b><span class="Apple-style-span" style="font-size:large;">Completing The Fix</span></b><br />I don't know if the Rails team is planning any more updates to the 2.3 line, but if they do release a 2.3.9, I vote for including these two commits, plus the one I applied. So, I forked the repo and created a <a href="http://github.com/scottwb/rails/tree/2-3-cache-fixes">topic branch</a> off of <span class="Apple-style-span" style="font-family:'courier new';">2-3-stable</span> that cherry-picks the two carlhuda commits and includes my additional commit.<br /><br /><b><span class="Apple-style-span" style="font-size:large;">A Workaround</span></b><br />Since I'd rather not maintain my own patched version of Rails, for now I am just monkey-patching <span class="Apple-style-span" style="font-family:'courier new';">MemoryStore</span> and <span class="Apple-style-span" style="font-family:'courier new';">ActiveRecord::Base</span> like so:<br /><script src="http://gist.github.com/513093.js?file=gistfile1.rb"></script></div>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com3tag:blogger.com,1999:blog-8917931115554225973.post-42104055973937548782010-07-05T14:02:00.000-07:002010-07-05T15:03:08.887-07:00Install a Rails App's dependencies where 'rake gems:install' fails.<span class="Apple-style-span" style="font-family:'lucida grande';">Using </span><span class="Apple-style-span" style="font-family:'courier new';">rake gems:install</span><span class="Apple-style-span" style="font-family:'lucida grande';"> to install the gem dependencies of your Rails 2 app just doesn't seem to work. There are a number of problems you may encounter:</span><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><span class="Apple-style-span" style="font-size:large;"><b>Problem 1: It depends on Rails.</b></span></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';">You have to manually install Rails before you can use </span><span class="Apple-style-span" style="font-family:'courier new';">rake gems:install</span><span class="Apple-style-span" style="font-family:'lucida grande';">. Ok, so this might not be that big of a deal, but consider this scenario:</span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><ul><li><span class="Apple-style-span" style=" ;font-family:'lucida grande';">Your app is running in production on rails 2.3.4.</span></li><li><span class="Apple-style-span" style=" ;font-family:'lucida grande';">You upgrade your app to rails 2.3.5, including the </span><span class="Apple-style-span" style="font-family:'courier new';">RAILS_GEM_VERSION</span><span class="Apple-style-span" style=" ;font-family:'lucida grande';">.</span></li><li><span class="Apple-style-span" style=" ;font-family:'lucida grande';">You deploy your app.</span></li><li><span class="Apple-style-span" style="font-family:'lucida grande';">Your deployment script calls </span><span class="Apple-style-span" style="font-family:'courier new';">rake gems:install</span><span class="Apple-style-span" style="font-family:'lucida grande';"> to keep the dependencies up to date with each new release.</span></li><li><span class="Apple-style-span" style="font-family:'lucida grande';">That fails with something that looks like:</span></li></ul></div><div><div><script src="http://gist.github.com/464719.js?file=gistfile1.txt"></script></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><span class="Apple-style-span" style="font-size:large;"><b>Problem 2: Sometimes, for some reason, it just doesn't work.</b></span></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';">Sometimes (most times) you add a new </span><span class="Apple-style-span" style="font-family:'courier new';">config.gem</span><span class="Apple-style-span" style="font-family:'lucida grande';"> command to </span><span class="Apple-style-span" style="font-family:'courier new';">environment.rb</span><span class="Apple-style-span" style="font-family:'lucida grande';">. The next deployment runs </span><span class="Apple-style-span" style="font-family:'courier new';">rake gems:install</span><span class="Apple-style-span" style="font-family:'lucida grande';"> and it fails with the complaint that you are missing the very gem you are hoping it will install for you. For example, in a working Rails 2.3.5 project, add this line to environment.rb:</span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><blockquote>config.gem 'sanitize', :version => '1.2.1'</blockquote></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';">Then, run </span><span class="Apple-style-span" style="font-family:'courier new';">rake gems:install</span><span class="Apple-style-span" style="font-family:'lucida grande';">, and it fails with a "Missing these required gems:" message, e.g.:</span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><script src="http://gist.github.com/464711.js?file=gistfile1.txt"></script></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';">WTF? It's telling me to run the command I just ran, to install the missing gem that's preventing that command from running.</span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><span class="Apple-style-span" style="font-size: large;"><b>Problem 3: It depends on your Rails environment.</b></span></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';">This can create a circular dependency where some file (particularly vendored plugins/gems) can require a gem to be loaded before the task to install it runs. For example, consider that some file in your </span><span class="Apple-style-span" style="font-family:'courier new';">vendor</span><span class="Apple-style-span" style="font-family:'lucida grande';"> directory does this:</span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><blockquote>require 'gdata'</blockquote></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';">Then naturally, you add this to environment.rb:</span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><blockquote>config.gem 'gdata'</blockquote></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';">The next time you run </span><span class="Apple-style-span" style="font-family:'courier new';">rake gems:install</span><span class="Apple-style-span" style="font-family:'lucida grande';">, it will fail with a "no such file to load" error, e.g.:</span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><script src="http://gist.github.com/464717.js?file=gistfile1.txt"></script></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><span class="Apple-style-span" style="font-size: large;"><b>The Solution: rails_gem_install</b></span></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';">I have created a new tool called </span><span class="Apple-style-span" style="font-family:'courier new';">rails_gem_install</span><span class="Apple-style-span" style="font-family:'lucida grande';"> to help alleviate this problem. Use this instead of </span><span class="Apple-style-span" style="font-family:'courier new';">rake gems:install</span><span class="Apple-style-span" style="font-family:'lucida grande';">, and you should be much more successful at getting all your Rails app's dependencies installed without manual intervention.</span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';">To install:</span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><blockquote>gem install rails_gem_install</blockquote></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';">To use:</span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"></span></div><blockquote><div><span class="Apple-style-span" style="font-family:'courier new';">cd my_rails_app</span></div><div><span class="Apple-style-span" style="font-family:'courier new';">RAILS_ENV=production rails_gem_install</span></div></blockquote><div><span class="Apple-style-span" style="font-family:'courier new';"></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';">This will install all the gems required to run your app in the production environment, <i>including Rails</i>. Replace </span><span class="Apple-style-span" style="font-family:'courier new';">rake gems:install</span><span class="Apple-style-span" style="font-family:'lucida grande';"> in your deployment scripts with </span><span class="Apple-style-span" style="font-family:'courier new';">rails_gem_install</span><span class="Apple-style-span" style="font-family:'lucida grande';">, and as you change </span><span class="Apple-style-span" style="font-family:'courier new';">config.gem</span><span class="Apple-style-span" style="font-family:'lucida grande';"> requirements, those gems will be properly installed.</span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';">See the <a href="http://github.com/scottwb/rails_gem_install">github project page</a> for more details.</span></div></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><span class="Apple-style-span" style="font-size: large;"><b>How It Works</b></span></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';">The main principle behind this tool is that it does not depend on Rails. It creates its own module named Rails that provides some of the functionality from the real Rails that it needs, up until the point that Rails is installed. Then, it only loads some very specific parts of Rails that implement the gem dependency and installation mechanisms, without actually running the app's </span><span class="Apple-style-span" style="font-family:'courier new';">Rails::Initializer</span><span class="Apple-style-span" style="font-family:'lucida grande';"> and loading all of its environment, plugins, etc.</span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';">First, it runs a simple </span><span class="Apple-style-span" style="font-family:'courier new';">rake -T</span><span class="Apple-style-span" style="font-family:'lucida grande';"> to see if it complains about Rails missing. It parses the output of this, and if necessary, installs the indicated version of Rails.</span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';">Next, it parses out the </span><span class="Apple-style-span" style="font-family:'courier new';">config.gem</span><span class="Apple-style-span" style="font-family:'lucida grande';"> statements and uses those to ensure all the listed gems requirements are met, and installs those that aren't. It does this without actually loading Rails or the app environment. This carries us most of the way.</span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';"><br /></span></div><div><span class="Apple-style-span" style="font-family:'lucida grande';">Finally, it runs the <span class="Apple-style-span" style="font-family:'courier new';">rake gems</span> command and parses its output to detect all the kinds of errors described above and install the corresponding gems. This step is repeated until </span><span class="Apple-style-span" style="font-family:'courier new';">rake gems</span><span class="Apple-style-span" style="font-family:'lucida grande';"> does not complain anymore.</span></div>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com6tag:blogger.com,1999:blog-8917931115554225973.post-81931183980888954602010-07-01T20:37:00.000-07:002010-07-01T21:24:56.681-07:00Hudson CI behind an Nginx Reverse Proxy with SSL<div>Here is a quick example nginx configuration to reverse proxy on an HTTPS virtual host to a Hudson CI server running on localhost. When I first tried to do this, the management page displays an error about the configuration being wrong. There are instructions for <a href="http://wiki.hudson-ci.org/display/HUDSON/Running+Hudson+behind+Apache">Running Hudson behind Apache</a> that were helpful, and this <a href="http://hudson.361315.n4.nabble.com/Hudson-behind-an-Apache-Reverse-Proxy-w-SSL-td370997.html">email thread</a> that seems to suggest terminating SSL at Hudson, not at the reverse proxy. Well, after a bit of tinkering, I worked out this configuration for nginx that worked out great:</div><div><br /></div><div><br /><script src="http://gist.github.com/460906.js?file=hudson_nginx.conf"></script><br /></div>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com1tag:blogger.com,1999:blog-8917931115554225973.post-25883764542855706542010-04-22T01:37:00.000-07:002010-04-22T01:44:17.814-07:00Find the SHA1 of the latest commit in a git branchHere's a way to get the SHA1 of the last commit in a given branch in a way that can be used in a script easily. For humans <span class="Apple-style-span" style="font-family:'courier new';">`git log branchname`</span> is probably fine - just read the first line. For a shell script that wants to get that value into a variable, here's a quick way to do that:<div><br /></div><div><br /><script src="http://gist.github.com/374977.js?file=get_git_branch_sha.sh"></script><br /></div><div><br /></div><div>Seems like there ought to be an easier way...but not that I could find in less time than it took to write this command. </div>scottwbhttp://www.blogger.com/profile/07534230274246537772noreply@blogger.com3