Sunday, September 4, 2011

Recreate a single version of a CarrierWave::Uploader file

CarrierWave supports a simple way to recreate all the versions of an uploaded file. This comes in handy when you add a new version to your uploader, or when you change the parameters of one of them. For example, assume you have a Post class, with a CarrierWave mounted uploader on the photo column, and you have version called thumb. If you decide to change the resolution of the thumb version, and need to go back and regenerate all existing thumbs, you can do something like this:

Post.all.each do |post|
  post.photo.recreate_versions!
end

What if you have 5 different versions on this Uploader, and you only changed one of them? The above method will download all the versions of every Post#photo from storage (think S3 or Cloud Files), regenerate all of them - even the ones that haven't changed - and re-upload all of them to storage.

That's a lot of wasted time, CPU, and bandwidth.

To address this, I've created a simple monkey patch for CarrierWave to provide a new method called recreate_version!. This method works just like recreate_versions!, except that it takes a parameter specifying a single version to recreate, rather than recreating all of them.

To use this, first drop this monkey-patch in somewhere that it will get loaded. Above your Uploader class will suffice.




Now your script to recompute all the thumbs can look like this:

Post.all.each do |post|
  post.photo.recreate_version!(:thumb)
end

This will recreate and re-upload only the thumb versions of your Post#photo uploads.

As an added side benefit, the recreate_version! method cleans up after itself. The recreate_versions! method downloads into a temporary cache dir, but leaves it up to the caller to clean that up. This new method assumes that if it had to download and cache the uploaded file and its versions, then it should clean them up immediately.

This method has turned out to save me a lot of time in regenerating large numbers of uploaded file versions. If there's any interest in this, I might work up a pull request with test cases and try to get it into the mainline CarrierWave gem release.

5 comments:

Jon Horton said...

Thanks for the script! This is exactly what I'm looking to do on a current project.

Will this also handle passing multiple versions like:
post.photo.recreate_version!(:thumb1, :thumb2) ?

scottwb said...

John - no this monkey-patch doesn't allow multiple versions in a single call. It would be pretty easy to extend to do that though. Just make it take an array of variable number of params and loop over those values for calling the `send(version).store!` method.

Short of that, if you want to recreate *all* versions, CarrierWave already has a method for that. Instead of calling my recreate_version!, call the built-in recreate_versions! method (note the plural).

The maintainers actually asked me to roll this all in to the existing recreate_versions! method, but I never got around to it. Maybe I should get on that...

Garrett said...

I was testing this out and with storage :file. It deletes all other files except the version being recreated (bad). Once I commented out the line that did that. It still processed all versions?? (bad)

I got around needing it using conditional versions and custom methods.

I suppose it wouldn't be linked from carrierwave site unless it worked. Maybe only as intended with :fog?

scottwb said...

Garrett - Indeed, I have only tried this with :fog (using Rackspace CloudFiles), so I cannot really speak to how it works with :file or EC2 for that matter. This is why I never made an effort to submit a pull request to get this into mainstream CarrierWave. I did talk it over with the maintainers a bit and they suggested they liked the idea.

As I recall, the way this works was by using the internal cache, which I can easily imagine being circumvented when using storage :file. I still plan to someday investigate making a real patch and will be sure to test it with all storage providers.

Until then, sorry I don't have any better answers for you.

Anonymous said...

Hi,
I've tried this monkeypatch and ran into a curious problem: *all* versions are reprocessed when calling recreate_version! with a single version. Are you aware of this issue, or do you know an easy work-around?

Thanks