Hi, I‘m Martin. You should follow me: @martinklepsch

June 2015 Managing Local and Project-wide Development Parameters in Leiningen

Little tip. Long headline.

In any project there are often settings that are specific to the context the project is run in (think of an environment parameter) and then there are parameters that are specifc to the developer/workstation they’re run on. This is a guide to separate these two things nicely in Leiningen-based Clojure projects.

So you have a project setup that uses environ to determine the context the project is run in (development vs. production).

; in project.clj:
(defproject your-app "0.1.0-SNAPSHOT"
  ; ...
  :profiles {:dev {:env {:environment "development"}}})

Now you also want to use environment variables (or anything else thats supported by environ) to store AWS credentials to access Amazon S3. You don’t want to commit these credentials into version control, therefore you can’t add them to project.clj. The way to go is to create a file profiles.clj in your project to store workstation specific information. Naively this could look something like this

{:dev {:env {:aws-access-key "abc"
             :aws-secret-key "xyz"
             :s3-bucket "mybucket"}}}

If you run your project with this profiles.clj you will be able to access your AWS credentials. You might also notice that (environ/env :environment) is nil. That wasn’t intended.

The problem here is that Leiningen will override keys in profiles defined in project.clj if the same profile has also been defined in profiles.clj. To recursively merge Leiningen profiles combine them like so:

; in project.clj:
(defproject your-app "0.1.0-SNAPSHOT"
  ; ...
  :profiles {:dev [:project/dev :local/dev]
             :project/dev {:env {:environment "development"}}})

; in profiles.clj
{:local/dev {:env {:secret-key "xyz"}}}

Now both, :envrionment and :secret-key should be defined when you retrieve them using environ.

This is largely inspired by James Reeves’ duct Leiningen template.

April 2015 Formal Methods at Amazon

I saw this paper being mentioned again and again in my Twitter feed. Basically not even knowing what “formal methods” really means I was intrigued by the claim that it’s easy to read. And it has been.

The paper describes how Amazon used a specification language to describe designs of complex concurrent fault tolerant systems finding bugs and verifying changes in the process.

The specification language (TLA+) is not focus of the paper, rather the authors concentrate on describing benefits, problems and the path of adopting formal specification of system designs in an engineering organization.

TLA+, stands for Temporal Logic of Actions and “is especially well suited for writing high-level specifications of concurrent and distributed systems.”

Reading how they use it at Amazon I’m under the impression that it works very similar to generative testing dumping a ton of basically random (according to some rules) data into a system and checking if the desired properties are maintained. Often the term “model checker” is used.

Download the original paper or a copy of it with some passages highlighted that I found particulary interesting.

February 2015 (lisp keymap)

A while back I wanted to setup hotkeys for the various apps I use. Mostly because I was annoyed by the cognitive effort you need to make to figure out how often you need to press Alt + Tab exactly to get to the app you want.

It seemed like a good idea to use Capslock as a modifier key. This way I could be sure I wouldn’t override any other keybindings. Figuring out how to do this I stumpled upon an excellent post by Steve Losh “A Modern Space Cadet”. It’s described in detail how to set Capslock to Hyper - a fifth modifier key. I then created bindings like Hyper + S which will focus Safari etc. Exactly what I was looking for.

Then I found something in his post I wasn’t looking for: instructions to map my shift keys to parentheses. It sounded crazy at first but doing mostly LISP-y stuff these days I tried it anyways.

Now I wouldn’t want to live without it anymore. It’s just so much easier than Shift + {9,0}. Also the Shift keys still work as they do usually when pressed in combination with other keys.

A few days ago I was typing some stuff at a collegues computer and it immediately felt cumbersome when having to type a parenthesis.

January 2015 CLJSJS - Use Javascript Libraries in Clojurescript With Ease

In Clojure, Java interoperability or “interop” is a core feature. In Clojurescript, interop with Javascript libraries does not work out-of-the-box across optimization modes. Extern files or “externs” required for advanced optimizations are often hard to find.

To fix this a few newly found friends and I created CLJSJS. CLJSJS is an effort to package Javascript libraries with their respective extern files and provide tools to integrate them into your project.

My personal hope is that this will make it easier for newcomers to get started with Clojurescript.

Also existing solutions like deps.clj (more here) only address the problem of Javascript dependencies partially. Maybe CLJSJS can serve as a vehicle to find some “pseudo-standard” for this kind of stuff.

Thanks to Juho Teperi, Micha Niskin & Alan Dipert for their contributions and ideas so far. Now go and check out the project homepage or jump straight into the packages repo and learn how you can contribute.

Announcement post and discussion on the Clojurescript mailinglist

November 2014 Why Boot is Relevant For The Clojure Ecosystem

Boot is a build system for Clojure projects. It roughly competes in the same area as Leiningen but Boot’s new version brings some interesting features to the table that make it an alternative worth assessing.

Compose Build Steps

If you’ve used Leiningen for more than packaging jars and uberjars you likely came across plugins like lein-cljsbuild or lein-garden, both compile your stuff into a target format (i.e. JS, CSS). Now if you want to run both of these tasks at the same time — which you probably want during development — you have two options: either you open two terminals and start them separately or you fall back to something like below that you run in a dev profile (this is how it’s done in Chestnut):

(defn start-garden []
    (print "Starting Garden.\n")
    (lein/-main ["garden" "auto"])))

Now there are issues with both of these options in my opinion. Opening two terminals to initiate your development environment is just not very user friendly and putting code related to building the project into your codebase is boilerplate that unnecessarily can cause trouble by getting outdated.

What Boot allows developers to do is to write small composable tasks. These work somewhat similar to stateful transducers and ring middleware in that you can just combine them with regular function composition.

A Quick Example

Playing around with Boot, I tried to write a task. To test this task in an actual project I needed to install it into my local repository (in Leiningen: lein install). Knowing that I’d need to reinstall the task constantly as I change it I was looking for something like Leiningen’s Checkouts so I don’t have to re-install after every change.

Turns out Boot can solve this problem in a very different way that illustrates the composing mechanism nicely. Boot defines a bunch of built-in tasks that help with packaging and installing a jar: pom, add-src, jar & install.

We could call all of these these on the command line as follows:

boot pom add-src jar install

Because we’re lazy we’ll define it as a task in our project’s build.boot file. (Command-line task and their arguments are symmetric to their Clojure counterparts.)

(require '[boot.core          :refer [deftask]]
         '[boot.task.built-in :refer [pom add-src jar install]])

(deftask build-jar
  "Build jar and install to local repo."
  (comp (pom) (add-src) (jar) (install)))

Now boot build-jar is roughly equivalent to lein install. To have any changes directly reflected on our classpath we can just compose our newly written build-jar task with another task from the repertoire of built-in tasks: watch. The watch-task observes the file system for changes and initiates a new build cycle when they occur:

boot watch build-jar

With that command we just composed our already composed task with another task. Look at that cohesion!

There Are Side-Effects Everwhere!

Is one concern that has been raised about Boot. Leiningen is beautifully declarative. It’s one immutable map that describes your whole project. Boot on the other hand looks a bit different. A usual boot file might contain a bunch of side-effectful functions and in general it’s much more a program than it is data.

I understand that this might seem like a step back at first sight, in fact I looked at it with confusion as well. There are some problems with Leiningen though that are probably hard to work out in Leiningen’s declarative manner (think back to running multiple lein X auto commands.

Looking at Boot’s code it becomes apparent that the authors spent a great deal of time on isolating the side effects that might occur in various build steps. I recommend reading the comments on this Hacker News thread for more information on that.

When To Use Boot, When To Use Leiningen

Boot is a build tool. That said it’s task composition features only get to shine when multiple build steps are involved. If you’re developing a library I’m really not going to try to convince you to switch to Boot. Leiningen works great for that and is, I’d assume, more stable than Boot.

If you however develop an application that requires various build steps (like Clojurescript, Garden, live reloading, browser-repl) you should totally check out Boot. There are tasks for all of the above mentioned: Clojurescript, Clojurescript REPL, Garden, live reloading. I wrote the Garden task and writing tasks is not hard once you have a basic understanding of Boot.

If you need help or have questions join the #hoplon channel on freenode IRC. I’ll try to help and if I can’t Alan or Micha, the authors of Boot, probably can.

« 1 2 3 4 5 6 7 »