Gzip: prep for s3 deploy

#1

Hi! I’m planning on deploying a middleman site to Amazon S3 (via s3cmd), and have not found a good way to generate gzip’d text files that work well with a standard s3 configuration.

The core of the issue is that the middleman gzip extension emits files with a .gz suffix, but when using s3 without an additional server in between, there’s no opportunity to detect an accepts gzip header, and serve the .gz file instead of the normal uncompressed version.

What I’d like is to configure the gzip extension to emit gzip’d files without the .gz extension, and simply avoid hosting the uncompressed files on s3 altogether. I’m not interested in accommodating clients that don’t understand gzip.

Would an option like this be considered as a feature for the gzip extension, or should I look into other ways to implement this?

Thanks!

#2

I am not sure, if it helps you. I dealt a bit with gzip file handling to meet the requirements of an apache server when it comes to content negotiation. Maybe s3 can do something like this as well.

# this part ensures, that gz-files are delivered by the Apache server using content negotiation.
# without renaming the i.e. "index.html" to "index.html.txt" it will receive priority and "index.html.gz" will not be used in any case
after_build do |builder|
  Dir.chdir(::Middleman::Application.server.inst.build_dir) do
    files = Dir.glob('**/*.gz')
    files.map { |file| file[0..-4] }.each do |f| # find corresponding non-gz file by removing ".gz" at the end
      unless File.directory?(f)
        output_filename = f + '.txt'
        File.rename(f, output_filename)
        builder.say_status :mv, output_filename
      end
    end
  end
end
#3

Cool - thanks! I’ve done something similar to this in a separate deploy script: https://gist.github.com/liamstask/9097615

replacegzip() essentially strips the .gz extension so s3 can serve the file directly.

this works ok for now, but I’m seeing s3cmd transmit the files more often than it should, so it would be nice if middleman could support this directly to avoid re-writing the .gz files after every build if they haven’t changed.