tompaton.com/blog

Posts tagged with aws

Blog architecture revisited

2017-06-16 posts aws

I'm bailing on the original blog architecture for now.

Using AWS Lambda is just more complicated than it should be and I'd rather just get this thing working.

I reverted a bunch of the complicated changes I had made so far in my static generation script and that was before I'd even figured out how to handle the mako templates or bundle the whole thing for deployment or hook up the API gateway...

Using Zappa seemed like a possible solution, but even that seems like overkill.

So the new process is:

  1. content in a github repo, initially directly in a single jsonfeed file
  2. posting to my bot would have a checkout of that repo and update the file, push it to github
  3. generate.py script pulls the latest version and processes the feed file into static html
  4. nginx serves static html
  5. profit
Nginx proxy to AWS S3 with url rewrites

2017-06-07 posts nginx aws

There seems to be plenty of information on how to use Nginx to proxy content stored in an AWS S3 bucket , but it took me a long time to figure out how to also get url rewriting to work in conjunction with this.

I wanted to rewrite the $uri to allow links to omit index.html from directories and also to omit .html extensions, and came up with the following (this goes in the server block, before the location block below): 

rewrite ^/somepath/(.+)\.(html|json|rss|css)$ /somepath/$1.$2 last;
rewrite ^/somepath(|/.+)/$ /somepath$1/index.html last;
rewrite ^/somepath/(.+)$ /somepath/$1.html last;

This works as follows:

  1. If $uri has a known extension, rewrite as itself and break out of the rewrite block.
  2. Add index.html to directories and break out of the rewrite block.
  3. If we've made it this far, it doesn't have an extension and isn't a directory, so assume a .html extension.

Note that the list of extensions in line 1 needs to include everything you serve from S3, otherwise a .html extension will be added and it will 404.

Once that's done, you can then pass any requests to /somepath/ through to S3 (I have the bucket permissions set to public-read, you may need to use a different proxy_pass url): 

location /somepath/ {
  proxy_intercept_errors on;
  proxy_hide_header x-amz-id-2;
  proxy_hide_header x-amz-request-id;
  proxy_pass https://s3.<region>.amazonaws.com/<bucket>/;
}

When debugging, it may help to enable logging of rewites: 

rewrite_log on;
error_log /var/log/nginx/<somepath>.error.log notice;