tompaton.com/blog

Posts tagged with posts

Org-mode formatting improvements

2018-03-13 posts code emacs

org-mode is so much better with some rich formatting. It really makes it appealing as a text format with embedded and actionable code — whether that is as a source file for literate-programming or just some notes with code snippets.

I've added the following to my .emacs, based on Org as a Word Processor: 


;; org-mode formatting improvements

(require 'org-bullets)
(add-hook 'org-mode-hook (lambda () (org-bullets-mode 1)))
(add-hook 'org-mode-hook (lambda () (whitespace-mode -1)))

(setq org-hide-emphasis-markers t)

(let* ((variable-tuple (cond ((x-list-fonts "Source Sans Pro") '(:font "Source Sans Pro"))
                             ((x-list-fonts "Lucida Grande")   '(:font "Lucida Grande"))
                             ((x-list-fonts "Verdana")         '(:font "Verdana"))
                             ((x-family-fonts "Sans Serif")    '(:family "Sans Serif"))
                             (nil (warn "Cannot find a Sans Serif Font.  Install Source Sans Pro."))))
       (base-font-color     (face-foreground 'default nil 'default))
       (headline           `(:inherit default :weight bold :foreground ,base-font-color)))

  (custom-theme-set-faces 'user
                          `(org-level-8 ((t (,@headline ,@variable-tuple))))
                          `(org-level-7 ((t (,@headline ,@variable-tuple))))
                          `(org-level-6 ((t (,@headline ,@variable-tuple))))
                          `(org-level-5 ((t (,@headline ,@variable-tuple))))
                          `(org-level-4 ((t (,@headline ,@variable-tuple :height 1.1))))
                          `(org-level-3 ((t (,@headline ,@variable-tuple :height 1.25))))
                          `(org-level-2 ((t (,@headline ,@variable-tuple :height 1.5))))
                          `(org-level-1 ((t (,@headline ,@variable-tuple :height 1.75))))
                          `(org-document-title ((t (,@headline ,@variable-tuple :height 1.5 :underline nil))))))

And then I start the .org files with the following parameters: 


#+STARTUP: indent
#+STARTUP: odd
#+STARTUP: hidestars
#+STARTUP: showall
#+TITLE: document title
Open tabs

2017-11-17 posts bookmarks

It seems it's taking rather too long for me to read these pages, so I'm going to post the links here and close the tabs.

These are outliers (the 99th percentile is 4 months), but that leaves 50 odd pages that were open for a couple of years and the worst for nearly 3.5 years!

Getting new posts from friends on Facebook with Selenium

2017-08-16 posts python code

I made a little script that uses Selenium to create a single page containing all the new posts from my Facebook friends.

This lets me see everything that is posted without having to seek it out and wade through the ads and other things Facebook is trying to recommend to me.

The output is currently very rough, but it'll do for now - in particular, it just screenshots the post, rather than trying to extract the content and re-render it.

Staging of blog updates

2017-06-17 posts

  1. Symlink generated output to next_release/
  2. Run ./release.sh to move next_release/ to released/ and released/ to prev_release/
  3. Run ./release.sh --rollback to revert to prev_release/

    4. Time-delay safety feature — ./release.sh will no-op until next_release/ is an hour old. Can then run this from a cron job and there's a window to catch issues before they are publicly visible. This will also effectively batch RSS/JSON feed updates to be kind to readers.

Blog architecture revisited

2017-06-16 posts aws

I'm bailing on the original blog architecture for now.

Using AWS Lambda is just more complicated than it should be and I'd rather just get this thing working.

I reverted a bunch of the complicated changes I had made so far in my static generation script and that was before I'd even figured out how to handle the mako templates or bundle the whole thing for deployment or hook up the API gateway...

Using Zappa seemed like a possible solution, but even that seems like overkill.

So the new process is:

  1. content in a github repo, initially directly in a single jsonfeed file
  2. posting to my bot would have a checkout of that repo and update the file, push it to github
  3. generate.py script pulls the latest version and processes the feed file into static html
  4. nginx serves static html
  5. profit
Nginx proxy to AWS S3 with url rewrites

2017-06-07 posts nginx aws

There seems to be plenty of information on how to use Nginx to proxy content stored in an AWS S3 bucket , but it took me a long time to figure out how to also get url rewriting to work in conjunction with this.

I wanted to rewrite the $uri to allow links to omit index.html from directories and also to omit .html extensions, and came up with the following (this goes in the server block, before the location block below): 

rewrite ^/somepath/(.+)\.(html|json|rss|css)$ /somepath/$1.$2 last;
rewrite ^/somepath(|/.+)/$ /somepath$1/index.html last;
rewrite ^/somepath/(.+)$ /somepath/$1.html last;

This works as follows:

  1. If $uri has a known extension, rewrite as itself and break out of the rewrite block.
  2. Add index.html to directories and break out of the rewrite block.
  3. If we've made it this far, it doesn't have an extension and isn't a directory, so assume a .html extension.

Note that the list of extensions in line 1 needs to include everything you serve from S3, otherwise a .html extension will be added and it will 404.

Once that's done, you can then pass any requests to /somepath/ through to S3 (I have the bucket permissions set to public-read, you may need to use a different proxy_pass url): 

location /somepath/ {
  proxy_intercept_errors on;
  proxy_hide_header x-amz-id-2;
  proxy_hide_header x-amz-request-id;
  proxy_pass https://s3.<region>.amazonaws.com/<bucket>/;
}

When debugging, it may help to enable logging of rewites: 

rewrite_log on;
error_log /var/log/nginx/<somepath>.error.log notice;
Python, Go and Rust bot comparison

2017-06-01 posts code

After implementing my bot in Python, I thought I'd try porting it to other languages as an opportunity to learn a bit of Go and Rust.

Both Go and Rust are statically typed and compile to a single self-contained binary, which is appealing as there is no need to maintain a virtualenv and install dependencies on the server.

The parsing code for "rain" messages is a good starting point to compare the bots. Each is using a "parser combinator" library to parse the messages.

Quick thoughts: Python & Go are both pretty sane. There's almost certainly a better way in Rust, but I found it considerably harder.

The tests are stored in different places in each version so the total line count is not comparable, and the Rust version includes "ignore case" code that is built into Python's parser combinators, and that I added into a fork of the Go parser library.

Blog architecture outline

2017-06-01 posts

  1. content in a github repo, initially directly in a single jsonfeed file
  2. posting to my bot would have a checkout of that repo and update the file, push it to github
  3. github webhook would call amazon lambda
  4. lambda would process the feed file into static html
  5. push html to s3
  6. set up nginx to proxy the blog/ path to s3
  7. profit