An Experiment - Write Code Every Day
If you missed John Resig’s recent post about writing code everyday I highly recommend it.
He is a busy guy with a full time job (at Khan Academy), a few open source side projects (the author of jQuery), a wife, and a few hobbies. How to sustainably get stuff done on his open source side projects without his wife leaving him? He decided to start writing (non work) code for 30 minutes every day. This by itself is not a revolutionary idea. What blew my mind out of my nose and on to the table are the benefits he encountered:
- Minimum viable code – No time for more than that.
- Small but continuous progress – No anxiety about not getting stuff done.
- Free time on the weekends – Instead of working all weekend to catch up from doing nothing during the week.
- Lowered cost of context switching – Compared to resuming work on a side project just on the weekends.
- Brain solves side project issues in the background
Wow, I need to do this too. So this is another experiment and here are the rules.
- I will write code for a minimum of 30 minutes each day.
- I must push working code every day.
- I will write for a minimum of 10 minutes each day.
- I must publish a blog post at least once a week.
Codecube.io Now Supports Perl
Codecube.io is a jsfiddle type service which runs Perl code (and other languages) and shows the results in your browser.
The website is written in Go and runs your code inside a Docker container. It originally had support for C, Go, Python, and Ruby. I was looking for an excuse to play with Docker and Go so I submitted a pull request which added support for Perl.
See also:
- The original Codecube blog post which is a good introduction to Docker.
- The Codecube github repo.
- My previous post on Go.
- My previous post on Docker.
Reading Code - Plack::Builder
- Part 1 – An Overview
- Part 2 – plackup Architecture
- Part 3 – PSGI Application Architecture
- Part 4 – Plack::Builder
Plack::Builder provides a domain specific language (DSL) for middleware developers. It looks like this:
use Plack::Builder;
my $app1 = sub { ... };
my $app2 = sub { ... };
builder {
enable "Deflater";
enable "Session", store => "File";
enable "Debug", panels => [ qw(DBITrace Memory Timer) ];
mount "/narwhale" => $app1;
mount "/unicorn" => $app2;
};
How does it work? With three artful tricks.
Artful trick #1
The first artful trick is the builder block.
sub builder(&) {
my $block = shift;
...
}
The & is a function prototype. Perl offers some limited compile time checking
for parameters passed to subs. Here is what perldoc perlsub says about &:
An “&” requires an anonymous subroutine, which, if passed as the first argument, does not require the “sub” keyword or a subsequent comma.
So if I try to pass builder() a scalar or an array or anything thats not an
anonymous subroutine, I will get a compile time error. But if I pass it an
anonymous subroutine, the compiler will allow things to continue.
Artful trick #2
The next artful trick is that Plack::Builder implements the DSL keywords as subs and then exports those subs.
package Plack::Builder;
use strict;
use parent qw( Exporter );
our @EXPORT = qw( builder enable enable_if mount );
...
sub enable {...}
sub enable_if {...}
sub mount {...}
# etc
Actually thats 90% of the whole thing isn’t it? Now its starting to look obvious. But lets continue.
Artful trick #3
There is one more interesting idea here. Notice that if I use enable,
enable_if, or mount outside of a builder block I will get an
error. This works because the DSL keywords are subs which run code references.
By default those code references refer to code which croaks an error. But when
builder runs, those references are temporarily replaced with real working
code.
Here’s some simplified code to illustrate how it works.
our $_enable = sub { Carp::croak(...) }; # << default code reference
sub enable { $_enable->(@_) }
sub builder(&) {
my $block = shift;
...
local $_enable = sub {...}; # << temporarily assign real working code
...
my $app = $block->();
...
}
Reading Code - PSGI Application Architecture
- Part 1 – An Overview
- Part 2 – plackup Architecture
- Part 3 – PSGI Application Architecture
- Part 4 – Plack::Builder
PSGI Applications
The PSGI spec defines a PSGI application.
A PSGI application is a Perl code reference. It takes exactly one argument, the environment, and returns an array reference containing exactly three values.
The three values are a status, headers, and a body. Here is an example:
my $app = sub {
my $env = shift;
return [
'200',
[ 'Content-Type' => 'text/plain' ],
[ "Hello World" ], # or IO::Handle-like object
];
};
The PSGI environment hash
The PSGI environment hash is a hashref with many keys. But mostly it is the data (headers, body, etc) from an HTTP::Request which has been parsed and put into a hash for convenient access.
Middleware
A middleware component takes a PSGI application and runs it, passing in the PSGI environment hash. But before it runs the app, it gets to modify the environment if it wants to. And after running the app, it can modify the response if it wants to.
Plack::Middleware
Middleware is a wrapper around a PSGI app. More than one middleware can be wrapped around an app, creating a series of layers like an onion. What makes the middleware onion a somewhat unusual construct is the event driven / callback nature of it. Lets look at how its implemented.
All middleware inherits from Plack::Middleware which is an itsy bitsy (teeny
weeny) module. The middleware onion is created with just 2 short subroutines
(notice the call() and prepare_app() subs are written by middleware authors):
sub wrap {
my($self, $app, @args) = @_;
if (ref $self) {
$self->{app} = $app;
} else {
$self = $self->new({ app => $app, @args });
}
return $self->to_app;
}
sub to_app {
my $self = shift;
$self->prepare_app;
return sub { $self->call(@_) };
}
How do these subs work together? The middleware onion is sometimes constructed as follows:
my $app = MyWebApp->new->to_app;
$app = Plack::Middleware::A->wrap($app);
$app = Plack::Middleware::B->wrap($app);
$app = Plack::Middleware::C->wrap($app);
But it might be more clear to write it this way
my $app0 = MyWebApp->new->to_app; # $app0->($env) runs the web app
$app1 = Plack::Middleware::A->wrap($app0); # $app1->($env) calls P::M::A->call() which calls $app0->($env)
$app2 = Plack::Middleware::B->wrap($app1); # $app2->($env) calls P::M::B->call() which calls $app1->($env)
$app3 = Plack::Middleware::C->wrap($app2); # $app3->($env) calls P::M::C->call() which calls $app2->($env)
# When the server receives a request it calls $app3->($env)
So when an event occurs — for example the PSGI server sees a new request — it passes the event to the app. The app is a chain of callbacks which run each other. This is clearly an example of event driven programming.
Plack::Component and Plack::App
Plack::Middleware inherits from Plack::Component. So the most common use of Plack::Component is in middleware.
Plack::Component can also be used as a tool for creating PSGI applications. It has a light dusting of code, but mostly its an interface which is implemented by modules in the Plack::App namespace. For example Plack::App::File is a web app which serves static files from a root directory, and Plack::App::URLMap is a web app which maps multiple web apps to multiple urls.
But notice that I am not required to use Plack::Component to create a PSGI application. A PSGI application is just a code reference. The PSGI spec does not say that a PSGI application is a reference to code that inherits from Plack::Component.
The nice thing about using Plack::Component to build my app is that it
provides a common interface for all PSGI apps. Whenever I see $app, I
can rely on that behavior. This is clearly important for middleware. And it
feels good from a design point of view.
But its not required and it adds some complexity.
Reading Code - Plackup Architecture
- Part 1 – An Overview
- Part 2 – plackup Architecture
- Part 3 – PSGI Application Architecture
- Part 4 – Plack::Builder
Plack::Runner and plackup
plackup starts a PSGI server which executes a PSGI application. However
the script itself is just a very small wrapper around Plack::Runner which does
all the heavy lifting. Plack::Runner
- parses the command line options.
- instantiates the chosen loader class (which is in the Plack::Loader namespace).
- instatiates the chosen server library (which is in the Plack::Handler namespace).
- starts the PSGI server and passes it a PSGI application
Plack::Loader
Loaders are responsible for instantiating and running the PSGI server. Here are
the more interesting capabilities a $loader object has:
$loader->guess()guesses which server library should be loaded by looking at command line opts, $ENV, and %INC.$loader->load()instantiates the server library and returns the object.$loader->run()starts the server.
The Plack::Loader namespace contains 3 kinds of loaders:
- Plack::Loader::Delayed – delays compilation of the web app until the first request occurs
- Plack::Loader::Restarter – reloads the server if any files are changed
- Plack::Loader::Shotgun – foreach request, forks a child which compiles the web app and runs it
I can choose which loader I want using plack --loader
Plack::Handler
The PSGI spec tells me that PSGI defines the interface between an application and a server. Because the PSGI spec is (intentionally) very minimal, there is a good deal of wiggle room to interpret how an application and a server might want to play together.
A library in the Plack::Handler namespace is the place where the application meets the server. This layer contains all the wiggling.
Lets say I wrote a new server called AngryBrontosaurus and I want to be able to
use it with plackup --server AngryBrontosaurus. I could implement a small
class like this:
package Plack::Handler::AngryBrontosaurus
use strict;
use AngryBrontosaurus;
sub new {
my $class = shift;
bless { @_ }, $class;
}
sub run {
my ($self, $app) = @_;
AngryBrontosaurus->new->run($app, $self);
}
Then, to make sure AngryBrontosaurus and Plack::Handler::AngryBrontosaurus correctly implement the PSGI spec, I should also test my code with Plack::Test::Suite.
use Test::More;
use Plack::Test::Suite;
Plack::Test::Suite->run_server_tests('AngryBrontosaurus');
done_testing;
Notice that while the Plack::Handler namespace contains classes for several PSGI servers like Plack::Handler::Starman or Plack::Handler::Twiggy, it also includes some classes like Plack::Handler::Apache2 and Plack::Handler::FCGI. Clearly Apache2 was not written with PSGI compliance in mind, but there is glue in the Plack::Handler::Apache2 layer to enable it to speak with PSGI compliant applications.
Sequence diagram
This diagram describes how Plack::Runner, Plack::Handler, and Plack::Loader interact.
Reading Code - Plack
I write lots of code. But I want to be better and faster at reading code. When I had the privilege of working with @ranguard I discovered he is a code reading cheetah and I always envied that. So I’ve decided to practice by reading the source code of various CPAN modules. I’m starting with Plack.
- Part 1 – An Overview
- Part 2 – plackup Architecture
- Part 3 – PSGI Application Architecture
- Part 4 – Plack::Builder
Plack
Plack describes itself as a set of tools for using PSGI (the Perl Server Gateway Interface).
The earliest release of Plack on CPAN is version 0.9000 from 2009-10-13. There were releases several times a week for the first two years. Impressively, in 2012 releases were still happening roughly once a week. In 2013 things seem to have cooled and now releases happen about once a month.
The code itself is written tersely and with attention to detail. There are almost no comments. In fact 43 out of 71 files have fewer than 3 comments. Of course the code is very well written which makes comments less necessary and nowadays there is quite a bit of excellent POD as well.
The code also makes heavy use of callbacks (code references). That is to say
its heavily event driven. Which makes sense given the event driven nature of web
servers. For me this gave the code a JavaScript flavor. Take for example
Plack::Util::foreach which works just like
jQuery.each() by iterating over an array
calling a code reference on each item.
Plack::Util::foreach([1,2,3], sub { print shift }); # prints "123"
Background reading
The most important thing to read is the PSGI spec. This is the problem Plack was built to solve. Its clear and well written but perhaps also a little boring and lacking in context. Still I found it very helpful to refer back to while reading the code.
Getting started
~/code ⚡ git clone git@github.com:plack/Plack.git
~/code ⚡ cd Plack
The first thing I noticed is a cpanfile containing a list of the project
dependencies. Because understanding and running the tests is often useful
when reading new code I installed the dependencies using
Carton and ran the tests.
~/code/Plack ⚡ carton
~/code/Plack ⚡ prove -rl t
Who works on Plack?
Lets get a feel for who is involved in the project.
~/code/Plack ⚡ git shortlog --summary --numbered | head
1567 Tatsuhiko Miyagawa
70 Kazuho Oku
68 Tokuhiro Matsuno
20 Daisuke Murase
20 Jesse Luehrs
19 yappo
16 Karen Etheridge
16 Mark Stosberg
12 hiratara
11 Stevan Little
How big is it?
~/code/Plack ⚡ tree lib | tail -1
17 directories, 70 files
~/code/Plack ⚡ cloc . 2>/dev/null | tail -13
-------------------------------------------------------------------------------
Language files blank comment code
-------------------------------------------------------------------------------
Perl 82 2889 3341 5309
Bourne Shell 8 55 138 251
YAML 1 0 0 19
HTML 3 2 0 14
Python 1 2 1 6
Javascript 1 0 0 1
CSS 1 0 0 1
-------------------------------------------------------------------------------
SUM: 97 2948 3480 5601
-------------------------------------------------------------------------------
What is the lib directory structure like?
Unfortunately with 71 files in the lib directory its hard to grok whats happening in a single glance so maybe thats not a great question. Instead I found it helpful to group the code by functionality. I came up with 3 major categories which look like this:
- Category 1 – Modules for loading and running PSGI servers
- plackup
- Plack::Handler
- Plack::Handler::*
- Plack::Loader
- Plack::Loader::*
- Plack::Runner
- Category 2 – Modules for building PSGI apps
- Plack::App::*
- Plack::Builder
- Plack::Component
- Plack::Middleware
- Plack::Middleware::*
- Category 3 – Modules for testing
- Plack::Test
- Plack::Test::*
- Plack::HTTP::Message::PSGI
- Plack::LWPish
However I also got crazy and went ahead and listed everything in the lib directory along with a brief description. I guess it might be handy for reference purposes. The numbers (1), (2), (3) below correspond to the three categories I listed above.
~/code/Plack ⚡ tree lib
lib
├── HTTP
│ ├── Message
│ │ └── PSGI.pm # (3) Converts an HTTP::Request to a PSGI env hash
│ └── Server
│ └── PSGI.pm # (1) Reference PSGI web server; no deps; not usually for prod
├── Plack
│ ├── App # (2) These libs inherit from Plack::Component; they are PSGI web apps
│ │ ├── Cascade.pm # Foreach request, tries a number of PSGI apps until one is successful
│ │ ├── CGIBin.pm # Creates many PSGI apps for a directory with many CGI scripts (uses WrapCGI.pm)
│ │ ├── Directory.pm # Serves a directory of files
│ │ ├── File.pm # Serves a file
│ │ ├── PSGIBin.pm # Create PSGI apps from a directory of .psgi files
│ │ ├── URLMap.pm # Maps a url to a PSGI app
│ │ └── WrapCGI.pm # Creates a single PSGI app from a single CGI script
│ ├── Builder.pm # (2) A DSL for building Plack Middleware
│ ├── Component.pm # (2) A (optional) tool for building PSGI web apps
│ ├── Handler
│ │ ├── Apache1.pm
│ │ ├── Apache2
│ │ │ └── Registry.pm
│ │ ├── Apache2.pm
│ │ ├── CGI.pm
│ │ ├── FCGI.pm
│ │ ├── HTTP
│ │ │ └── Server
│ │ │ └── PSGI.pm # A Plack::Handler for HTTP::Server::PSGI
│ │ └── Standalone.pm # Alias for Plack::Handler::HTTP::Server::PSGI
│ ├── Handler.pm # (1) Instantiate and run PSGI compatible servers
│ ├── HTTPParser
│ │ └── PP.pm # Parse HTTP headers with XS
│ ├── HTTPParser.pm # (1) Parse HTTP headers; used by HTTP::Server::PSGI
│ ├── Loader
│ │ ├── Delayed.pm # Delay compilation of the PSGI app until the first request
│ │ ├── Restarter.pm # Restart the server when a watched file changes
│ │ └── Shotgun.pm # Recompile the PSGI app for every request
│ ├── Loader.pm # (1) Load PSGI compatible web servers
│ ├── LWPish.pm # (3) Light version of LWP for testing
│ ├── Middleware
│ │ ├── AccessLog
│ │ │ └── Timed.pm # Write access logs but can handle a fake File::IO body
│ │ ├── AccessLog.pm # Write access logs
│ │ ├── Auth
│ │ │ └── Basic.pm # Basic authentication
│ │ ├── BufferedStreaming.pm # Enable streaming for servers that don't
│ │ ├── Chunked.pm # Implements part of HTTP/1.1 - chunked HTTP transfer encoding
│ │ ├── ConditionalGET.pm # Implements part of HTTP/1.1 - Conditional GET
│ │ ├── Conditional.pm # Runs the specified middleware if a specified condition is met
│ │ ├── ContentLength.pm # Adds a Content-Length header if possible
│ │ ├── ContentMD5.pm # Sets the Content-MD5 header when the body is an arrayref
│ │ ├── ErrorDocument.pm # Show different error documents for different HTTP errors
│ │ ├── Head.pm # Delete any response body from HEAD requests
│ │ ├── HTTPExceptions.pm # Redirect to an error page when HTTP::Exceptions are caught
│ │ ├── IIS6ScriptNameFix.pm # Fix for IIS
│ │ ├── IIS7KeepAliveFix.pm # Fix for IIS
│ │ ├── JSONP.pm # Change JSON responses to JSONP if a callback param is specified
│ │ ├── LighttpdScriptNameFix.pm # Fix for Lighttpd
│ │ ├── Lint.pm # Checks input/output for compliance w/PSGI spec
│ │ ├── Log4perl.pm # Log with Log::Log4Perl
│ │ ├── LogDispatch.pm # Log with Log::Dispatch
│ │ ├── NullLogger.pm # Don't log anything
│ │ ├── RearrangeHeaders.pm # Fix for very old MSIE and broken HTTP proxy servers
│ │ ├── Recursive.pm # Allows the app to forward the request to a different (url) path
│ │ ├── Refresh.pm # Similar to Plack::Loader::Restarter but less effective
│ │ ├── Runtime.pm # Sets the 'X-Runtime' HTTP response header = app's response time
│ │ ├── SimpleContentFilter.pm # Filter response content
│ │ ├── SimpleLogger.pm # Logs messages
│ │ ├── StackTrace.pm # Displays a stacktrace when a PSGI app dies
│ │ ├── Static.pm # Serve static files
│ │ ├── XFramework.pm # Adds an X-Framework HTTP response header
│ │ └── XSendfile.pm # Adds an X-Sendfile HTTP response header
│ ├── Middleware.pm # (2) Wraps PSGI apps; can modify incoming requests / outgoing responses
│ ├── MIME.pm # A list of MIME types (mostly)
│ ├── Request
│ │ └── Upload.pm # A subclass of Plack::Request for file uploads
│ ├── Request.pm # (2) Low level request obj for middleware and web apps
│ ├── Response.pm # (2) Low level response obj for middleware and web apps
│ ├── Runner.pm # (1) The guts of plackup -- uses Plack::Loader and Plack::Handler
│ ├── TempBuffer.pm # For backward compat. Saves data in memory or to a file if its big;
│ ├── Test
│ │ ├── MockHTTP.pm # Test a PSGI app without using a server (faster)
│ │ ├── Server.pm # Test a PSGI app using a very small server (fast)
│ │ └── Suite.pm # Ensure the web server complies with the PSGI spec
│ ├── Test.pm # (3) A factory for generating test objects
│ ├── Util
│ │ └── Accessor.pm # Light version of Class::Accessor for backward compat
│ └── Util.pm # Misc but important utilities used throughout the code base
└── Plack.pm # No code here -- just pod and a version number
Look Around Assertions in Perl Regular Expressions
When Perl’s regex engine evaluates a string, it moves from left to right, one letter at a time checking the match at each position. That position is called the current match position.
Look around assertions allow you to match a specific pattern before or after the current match position without moving the match position.
Look ahead assertions
Look ahead assertions match the text after the current match position
(without moving the match position). They look like (?=pattern).
my $job = "space cowboy";
$job =~ /space (?=cow)/; # matches
$job =~ /space (?=cow)cow/; # also matches
Look behind assertions
Look behind assertions match the text before the current match position
(without moving the match position). They look like (?<=pattern).
my $job = "space cowboy";
$job =~ /(?<=space) cowboy/; # matches
$job =~ /space(?<=space) cowboy/; # also matches
Positive and negative look ahead assertions
Positive look ahead assertions are look ahead assertions which match when their
subpattern matches. They look like (?=pattern).
my $job = "space cowboy";
$job =~ /space (?=cowboy)/; # matches
Negative look ahead assertions are look ahead assertions which match when their
subpattern fails. They look like (?!pattern).
my $job = "space cowboy";
$job =~ /space (?!mooseboy)/; # matches
Positive and negative look behind assertions
Positive look behind assertions are look behind assertions which match when their
subpattern matches. They look like (?<=pattern).
my $job = "space cowboy";
$job =~ /(?<=space) cowboy/; # matches
Negative look behind assertions are look behind assertions which match when
their subpattern fails. They look like (?<!pattern).
my $job = "space cowboy";
$job =~ /(?<!earth) cowboy/; # matches
For more details see perldoc perlre. I also recommend the DuckDuckGo
regex cheat sheet.
Vim Cheat Sheet
I created a Vim cheat sheet instant answer for DuckDuckDuckGo. If you search for vim cheat sheet or vim help you will get a result like this:
The data is from rtorruellas’s excellent vim cheat sheet (also available on github) which has a responsive mobile friendly layout and is also available in Japanese.
An Experiment - Living With Less Than 200 Things
- Stuff is expensive to buy. Especially when I end up only using things once or twice like books, dvds, dress shoes, and ties.
- Stuff is expensive to store. One time I had to rent a storage locker because I couldn’t take my things when I moved to China. Some people get big houses with walk in closets and a three car garage.
- Stuff is expensive to maintain. Broken stuff needs to be fixed. Owning a house means I need to fix the roof. Owning a car means I need to change the tires.
- Stuff is time consuming to maintain. I have to dust my stuff.
A bigger house means more sweeping.
A bigger yard means more mowing. Owning a car means cleaning it and
changing the oil and scraping ice off the windows.
I need to go to the store to find replacement parts, batteries, light bulbs, cleaning supplies, and duct tape. - Stuff is stressful to own. I might lose it or dent it or scratch it or drop it or step on it. Or it might catch on fire. Or someone might steal it. Sometimes I get so stressed I buy insurance for my stuff.
- Stuff reduces my freedom. Its hard to move to new place because I have to carry all my stuff with me.
Therefore I have decided to live with less stuff. So I hereby declare war on my stuff. And also on my things. Here is my 4 point plan to free myself from the tedious tyranny of too many things:
- Point 1: Keep a comprehensive inventory of all my stuff.
- Point 2: Ruthlessly reduce the number of things in my inventory to less than 200.
- Point 3: No new stuff unless I throw out some old stuff first.
- Point 4: Have a beer.
If I were hard core I would throw out everything I didn’t use in the last 6 months. Maybe I’ll be hard core at some point in the future.