github.com/aw
Fork me on GitHub

in-progress

PicoLisp Libraries

aw/picolisp - A collection of libraries for PicoLisp

I accidentally discovered PicoLisp while searching for examples on Rosetta Code. As it turns out, this language is my dream language. It was my first introduction to LISP programming, but I seem to get the hang of it rather quickly. (Un)fortunately PicoLisp was missing some important libraries, so I fixed that by creating a few and then open sourcing them:

unit: Unit Testing framework
bcrypt: bcrypt password hashing
https: HTTP(S) client
json: JSON encoder/decoder
nanomsg: Official Nanomsg ffi bindings

source: https://aw.github.io/picolisp

libraries unit-testing bcrypt https json nanomsg picolisp

in-progress

RFC Compliant HTTP caching library

aw/CacheRules - HTTP caching library aimed at being RFC 7234 compliant

I spent hours searching for an RFC-compliant HTTP caching library with a liberal open source license, and which _only_ handled caching (i.e: no proxying). Despite my attempts at using/improving other people's half-baked libraries, I decided to do it myself and created something that actually works.

paradigm: functional
coverage: 100%
dependencies: 0
compliance: full
license: mpl-2.0

I can't stress the importance of unit/integration tests, and aiming for 100% code coverage from the start. It's much easier when your code is purely functional, because you can safely assume a function will always do what it should based on the given inputs. source: https://github.com/aw/CacheRules

http caching ruby

merged

Allow Hubot to send colours over XMPP

markstory/hubot-xmpp - XMPP adapter for Hubot

All I wanted was for Hubot to send coloured text into a Jabber/XMPP chat room. The xmpp adapter wasn't fully standards compliant (XEP-0071), so I tried to move it in that direction. It took me an hour to figure out how ltx (the xml parser) works, ~10 minutes to implement this change, and a few minutes to write the tests.

parsedMsg = try new ltx.parse(msg)

This line is subtle but quite powerful. It actually does 2 things: 1. tries to parse the msg and assign it to a variable 2. leaves the variable undefined if it can't (eg: invalid XHTML) Nice. source: https://github.com/markstory/hubot-xmpp/pull/76/files#diff-8d2355e589161f82f78443e903896e26R352

hubot xmpp coffeescript

in-progress

Decision table parser

aw/ruby-decision-table - Parse a Decision Table in Ruby

In early 2015 I discovered a thing called "decision tables" which seems to be a concept from the 70s used to define business rules and simplify decision-making. I strongly believe this concept is still valid today, and wanted to write a Ruby implementation to parse said tables for another project I'm working on.

result = answers.each_index.map(&columns).reduce(:&).compact

Map/Reduce in all its beauty. This line maps a set of values to a multi-dimensional array, reduces it by combining into one array, and compacts by removing nil values. I think they call this "functional programming". source: https://github.com/aw/ruby-decision-table/blob/master/parse.rb#L35

functional map/reduce ruby

merged

Documentation update and fix

github/github-services - Official GitHub Services Integration

This was the 2nd pull request to fix a breaking change introduced by GitHub. The initial one was a bit of a mess and they eventually released a helper to make it easier. This PR updates the commando.io service to use that helper, and updates the documentation.

github api ruby

merged

Add commando.io service

github/github-services - Official GitHub Services Integration

Justin asked me to create the official commando.io GitHub services integration. A few days were spent writing and testing before making the pull request, but it took nearly 2 weeks to be accepted and merged.

config_value('server').empty? ^ config_value('groups').empty?

One of two values must be set, but not both.. I guess most people would use a fancy conditional to check for this. I opted for Ruby's XOR operator. source: https://github.com/github/github-services/pull/945/files#diff-58f4ed4d6f61e7096d616732fcc6dadfR75

github api ruby

merged

Add special keys (Packerisms)

jedi4ever/veewee - Easing the building of vagrant boxes

Packer is a tool built to be identical (almost) to Veewee. It had certain useful keys which Veewee didn't, so I added them. The project maintainer wanted to make it even more flexible by allowing a broad scope regex. I never got around to finding a good implementation, so we left it as is.

unless keycode=="wait"
  ...
else
  ...

This is a classic example of what NOT to do in Ruby. "unless / else" is difficult to grok. I replaced that code with a simple switch/case statement. source: https://github.com/jedi4ever/veewee/pull/989/files#diff-9297be6091052d6f7c1fce8574f554e5L32

veewee packer ruby

merged

Make special scancodes keys case-insensitive

jedi4ever/veewee - Easing the building of vagrant boxes

I was constantly being forced to use case-sensitive keys in my Veewee config files. It was very error-prone, so I made a slight change to solve that problem. I've never had a pull request merged so quickly.

if thestring.match(/^#{key}/i)

Ruby has a "start_with?()" String method, but who really knows how it works? A simple regex does the job with minimal effort, and is much more flexible. source: https://github.com/jedi4ever/veewee/pull/988/files#diff-ecc1975498c6479fad63df6b9977a3b9R73

veewee regex ruby

merged

Add Jidoteki.com to developer category

jdavis/twofactorauth - List of sites with two factor auth support

I implemented two-factor logins in Jidoteki using a software token and wanted to see it listed on twofactorauth.org.

2fa jidoteki yaml

merged

Fix xmpp-muc service

github/github-services - Official GitHub Services Integration

I run my own Jabber server and wanted to receive notifications in a chat room. The GitHub/XMPP-MUC integration was a great addition. It was finally "almost" there but still had a few tiny bugs, which I was happy to fix.

rescue StandardError => e

Many Ruby developers use begin/rescue statements but often forget to catch the most basic errors: StandardError. This line prevents many headaches and broken code. source: https://github.com/github/github-services/pull/926/files#diff-94b02515e6f15c17ddbde2d7d50c0cbaR94

github xmpp ruby

merged

Replace the equal signs in base32 string

guyht/notp - Node One Time Password library

Google Authenticator isn't happy with base32-encoded strings containing equal signs (padding?). I made sure the code's documentation explains clearly how to generate the proper two-factor secret for that.

2fa javascript

closed

Better random string generation

markbao/speakeasy - Easy two-factor authentication for node.js.

I needed an easy way to perform two-factor authentication in Node.JS.This library was using its own wacky random string generation algorithm. That's usually a bad sign. I attempted to improve the code, but my changes were just as ugly, so I decided to use a different library instead.

return crypto.randomBytes(length/2);

I suggested this line because in Node.js, and in any language actually, it's best not roll your own cryptography. source: https://github.com/markbao/speakeasy/pull/28/files#diff-25f2b5307c2beaac2abf001ea7ed5e62R230

cryptography 2fa javascript

merged

Enable AMQP over SSL, topic exchanges, and graphite format

mrtazz/statsd-amqp-backend - StatsD backend to emit stats to AMQP

This backend implementation was quite limited and only working over HTTP. I re-wrote most of it and introduced some new features, including HTTPS support, writing to AMQP topic exchanges instead of queues, and pushing data in Graphite format. It was a lot more work than I initially planned, but totally worth it. I still use this in production to push tens of thousands of metrics per day. I'm now also the official maintainer of this library (for obvious reasons).

conn.publish(amqpQueue, JSON.stringify(data));

Sometimes it's not about the code you write, but the code you remove. In this case, it was important to avoid writing directly to a queue. AMQP exchanges allow for a sort of "broadcast" message to be sent to multiple subscribers. source: https://github.com/mrtazz/statsd-amqp-backend/pull/1/files#diff-7fd25e9b16d2235a9d2c8fcc0f89fd11L56

amqp statsd javascript

closed

Fix apache and support Debian 7.x

obfuscurity/synthesize - Easy installer for Graphite and StatsD

I wanted to support Debian 7.x as well as the default Ubuntu 13. Jason didn't want these changes, so after a bit of back-and-forth and constructive discussion, I decided to close the pull request.

<IfModule mod_authz_core.c>
  Require all granted
</IfModule>
<IfModule !mod_authz_core.c>
  Order allow,deny
  Allow from all
</IfModule>

Apache's conditionals allow support for backwards-compatible configurations. This means you can have a v2.2 config in a v2.4 config file and it'll work on both versions. Cool. source: https://github.com/obfuscurity/synthesize/pull/1/files#diff-57adb371bf27eeda311c061acb8aaf59R14

apache bash yaml

in-progress

Ansiblize Synthesize

obfuscurity/synthesize - Easy installer for Graphite and StatsD

Jason Dixon, a fellow ex-GitHubber wrote a long bash script for setting up a Graphite server. It was epic, but I didn't want to manage a bash script. I ported the entire script over to Ansible as a pet project, and it proved to be quite popular. I still run a private Graphite server based on this Ansible setup.

`ansible-playbook install.yml -i graphite.inventory`

From the documentation, this shows how simple it is to setup a Graphite server using these Ansible scripts. It was just as easy as the bash script, but also much friendlier to people wanting to tweak their setup. Afterall, it's just YAML.. source: https://github.com/aw/synthesize/compare/obfuscurity:master...ansible#diff-5a5d069dcad99a9f8109ac77d1b9860cR8

ansible bash graphite yaml

merged

Websockets support via socket.io

shutterstock/rickshaw - JavaScript toolkit for creating interactive real-time graphs

I wanted to use Rickshaw to generate real-time graphs using JSON data received over web sockets. A working example didn't exist, so I created one from scratch. It took a few hours of testing because I was completely new to socket.io and Rickshaw. I never ended-up using this library, but it was a fun experiment.

"license": "MIT"

Most of my code is MIT licensed. In the past, I wouldn't put much thought into choosing an appropriate license, but the more you release and use open source, the more you need to care about these types of things. source: https://github.com/shutterstock/rickshaw/pull/312/files#diff-a463e2dee6255401fb545017d2e11816R15

rickshaw websockets javascript

closed

Enable custom prefix helper

akoenig/express-lingua - An i18n middleware for the Express.js framework

Yet another unmaintained Node.JS project. This library forces you to use its own prefix all over your i18n code. I wrote a small patch to allow for a custom prefix, which I set to 't' (t for translate). I don't use this library anymore.

i18n unmaintained javascript

merged

Official Ubuntu 12.04 cloud image contains VBox Additions

garethr/vagrantboxes-heroku - Repository for http://www.vagrantbox.es

I was a bit annoyed I had to make this pull request. The official Ubuntu cloud image was not supposed to contain VirtualBox additions, but in fact it did and it was not very easy to find out. This confirms my original opinion about trusting public vagrant boxes.

vagrant documentation html

merged

Add IPv6 support

reinh/statsd - A Ruby Statsd client

I remember this implementation like it was yesterday. Living in Tokyo, I had a chance to discuss this problem with a core Ruby developer. He gave me some good ideas and I was able to implement this change in just a few lines. The code in its current state is not thread-safe, but I use it exclusively with IPv6 addresses (not hostnames), and only one instance at a time, so not an issue for me. There seems to be a pending major revision which adds thread-safety.

def addr_family
  Addrinfo.udp(@host, @port).ipv6? ? Socket::AF_INET6 : Socket::AF_INET
end

It blows my mind how simple this was: detect if the host is ipv6 or not, then return the correct socket. No other changes necessary. source: https://github.com/reinh/statsd/pull/46/files#diff-179756856cd1fe6003145224a9f4f979R397

statsd ipv6 ruby

merged

Implement to_ary to avoid calls to method_missing

intridea/grape - An opinionated micro-framework for creating REST-like APIs in Ruby

I always do my local development in DEBUG mode. I need to see all warnings and errors generated by my code and the libraries I use. In this case, I kept seeing repeat "method missing" warnings, and a quick search landed me on someone else's solution. I copied the fix and take no credit for it haha.

def to_ary
  nil
end

This is such a hack only Ruby would allow this. It's basically a hack on top of a monkey-patch. Once again, I take no credit for it. source: https://github.com/intridea/grape/pull/466/files#diff-64d352c6d46153e1e4ff964b0ff4ecdeR23

hack monkeypatch ruby

merged

Add repository field to play nice with others

shutterstock/armrest - A high-level HTTP / REST client for Node

I use the armrest library for making simple HTTP requests. It works, but using it as a module in a module was impossible due to the missing "repository" field.

"repository": {
  "type": "git",
  "url": "https://github.com/shutterstock/armrest.git"
}

NPM and its package.json doesn't require all fields to be entered, but some things are necessary even if they aren't mandatory. This was one. source: https://github.com/shutterstock/armrest/pull/3/files#diff-b9cfc7f2cdf78a7f4b91a753d10865a2R21

http documentation json

closed

Add optional IPv6 support

reinh/statsd - A Ruby Statsd client

This was an ugly patch and I'm happy it wasn't merged. I attempted to add IPv6 support by forcing the addition of a 3rd config argument, but it was just bad. I took a step back and re-attacked the problem at a later date.

- def initialize(host = '127.0.0.1', port = 8125)
+ def initialize(host = '127.0.0.1', port = 8125, ipv6 = false)

Here's a classic example of how NOT to make pull requests on open-source code. It adds an extra useless flag for something which could obviously be guessed. Don't do this. source: https://github.com/reinh/statsd/pull/38/files#diff-179756856cd1fe6003145224a9f4f979L116

statsd ipv6 ruby

merged

Add configuration option to listen on an IPv6 address

etsy/statsd - Simple daemon for easy stats aggregation

I'm a huge fan of IPv6, and have been using it in production for a few years. StatsD is really awesome but only handled IPv4. Adding IPv6 support only took a few minutes with the help of a config flag and a short ternary operation.

/* By default, if the `address_ipv6` configuration option is not set (or set to `false`), the socket will use IPv4. */

Good documentation is invaluable, and so are backwards-compatible changes. This line in the README makes it clear that a simple config flag can enable IPv6, but the setting is entirely optional and an old config file will work regardless. source: https://github.com/etsy/statsd/pull/266/files#diff-04c6e90faac2675aa89e2176d2eec7d8R428

statsd ipv6 javascript

in-progress

Replace IPv4 with IPv6 in Redis 2.6

antirez/redis - Redis is an in-memory database that persists on disk.

I needed to use Redis on an internal IPv6-only network, but in 2013 that was not possible. Rather than wait for official support, I wrote the patch myself. I followed an online guide on porting applications to IPv6, but I couldn't be bothered to make it support IPv4 as well, that's why I never created a pull-request.

strcpy(ipbuf,inet_ntop(AF_INET6, &sa.sin6_addr, host, sizeof(sa)));

I initially wrote this patch on FreeBSD and couldn't compile on Linux. To be honest I had no idea what I was doing; I was just happy it worked. source: https://github.com/aw/redis/compare/antirez:2.6...2.6-ipv6#diff-aa7b6bb078b34b6800290132958ea20eR181

redis ipv6 C

merged

Make the 'stdout/stderr/exit code' available during exceptions

jedi4ever/veewee - Easing the building of vagrant boxes

When you execute a shell command from with your code, it's always important to capture the STDOUT, STDERR and EXIT CODE. I made this small change to ensure we can get this information on SSH commands.

class SshResult < RuntimeError

Ruby allows you to extend the RuntimeError class in order to create your own Error class. This is a "standard" way of extending the language without monkey-patching or re-inventing the wheel. source: https://github.com/jedi4ever/veewee/pull/539/files#diff-30574f6a59a4d793d846820c991a3092R6

veewee ssh ruby

published

How to get a job at GitHub

free eBook - A free eBook for people wanting to work at GitHub

I woke up one day with an email from my friend Thierry. He sent me a link to a job ad for a Sysadmin at GitHub. It was perfect for me, but I knew competition would be tough (everyone wanted to work at GH). To differentiate myself, I wrote and published this free eBook after spending the day reading everything about their infrastructure and trying to learn all I could about the famous company. My plan worked, I was soon interviewed by TPW, and flown to SF for an in-person interview with the team. I joined the Ops team in August 2011. https://github.com/blog/913-alex-williams-is-a-githubber

github ebook job pdf

published

Web Scaling vol. 1 - Small Architectures

free eBook - A free eBook on how to scale a web site

I had a dream of writing my own eBooks, publishing them and getting paid handsomely for doing nothing at all. As it turns out, it's not that easy. I wrote the eBook to provide advice to developers and small business owners on how to scale their 1-server setup to something a bit more tolerant. I had plans to write volume 2 and 3 for larger setups, but never got around to it. The initial eBook was sold for $30 USD and it sold quite well for the first two years, but in 2014 the numbers started to drop near 1 sale per month. That's when I decided to offer the eBook completely FREE for anyone to download. I wrote this eBook using Confluence and wish I had used a more "open" system, since now it's almost impossible for me to make revisions.

scaling ebook pdf