Following the instructions in the project’s README.md file, I cloned the project

1

git clone https://github.com/zsh-users/zsh-syntax-highlighting.git

then I added the line

1

source /path/to/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh

to my .zshrc file.

Unfortunately, when I tried re-instantiating my terminal, I got the following error message

1

/Users/olleicua/dev/zsh/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh:76: parse error near `done'

I spent a good forty-five minutes googling variations on “zsh-syntax-highlighting parse error near `done’” with and without my zsh version and OS. Finally it occurred to me to try the search without zsh-syntax-highlighting since it my be an issue with my zsh setup specifically and I found this mailing list archive. The mailing list suggested that one of the built-in zsh keywords that was being used in a control structure had been overwritten by an alias, in this case fi. I reviewed the section of the plugin that was bugging out and noticed that it was using the keyword do in its control structures and recalled that I had aliased do to cd into my downloads directory. I changed the alias to dow and now the plugin works fine.

The moral of this story is to always use which before naming an alias or binary in your path. which looks for the specified name in it’s list of built-in commands, in your defined aliases and in your path. It then tells you if it found it and if so where. In this case

1
2

$ which do
do: shell reserved word

It doesn’t complain when you overwrite things with aliases because you want to be able to do things like

1

alias rm="rm -rf"

if you’re feeling foolhardy.

Coloring zsh

Once I had gotten this to work, I was so happy with myself that I decided to make my zsh more colorful in general. I had to do a bit of trial and error because I’m using both a prompt and a right-prompt. Before this change my shell looked like this

and was generated by the following in my .zshrc file

1
2
3
4

PROMPT='
$ '
 
RPROMPT='| %~ | %D{%k:%M} | %h | Jack |'

I was able to add colors by changing this to

1
2
3
4

PROMPT='%F{green}
$ %f'
 
RPROMPT='%F{green}|%f %F{cyan}%~%f %F{green}|%f %F{cyan}%D{%k:%M}%f %F{green}|%f %F{cyan}%h%f %F{green}|%f %F{cyan}Jack%f %F{green}|%f'

In this case %F{COLOR}TEXT%f is the syntax for coloring text in zsh prompts. The result was

This semester, in an attempt further my education in the art of making great software, I began looking at how software is made in the open source world. I chose to look at this field for several reasons. I’ve been quite interested in open source software as long as I’ve known about it. I appreciate the social/economic benefits of software that is made by individuals who want it to be of quality as opposed to corporations that simply want to make money (and are therefore frequently motivated to compromise quality for business reasons e.g. planned obsolescence etc.). My experience with academic vs. professional programming also leads me to believe that people are far more likely to cut corners when they’re getting paid. Another reason for looking into open source was that large-scale software is often made in teams. I had had very little experience working on software with a group and, being at tiny isolated liberal arts school with a similarly tiny computer science department, I realized that this was a key missing component of my education. This would also give me an opportunity to compare my coding to others’ so as to get a sense of how I need to improve.

I began the semester thinking I would be contributing to Mozilla Firefox and I spent the first few weeks looking into this. In the process I was exposed the XUL and was able to build a very simple Firefox extension as well and getting a glimpse at the guts of Firefox. I quickly realized that my lack of familiarity with XUL, C++, and Firefox development in general was going to make contributing difficult and that I would likely spend more time exploring these technologies than learning about open source. For this reason, I switched my focus to WordPress, a project I was already much more familiar with. I had previously dabbled in creating plug-ins for WordPress so contributing to the core in PHP (a language I am quite familiar with) seemed far less daunting. Although I would still love to someday learn C++ and XUL and explore the Mozilla community, I’m glad I made this decision because it allowed me to actually engage with a community the maintains a popular CMS used by millions of bloggers. I made several small contributions to the swiftly approaching release of WordPress 3.3 (as of the time of this writing there is a single ticket impeding the release). My contributions included three major patches and a bug report. I got the opportunity to interact with several of the movers and shakers of the community and the experience was, on a whole, intimidating yet gratifying. I intend to continue contributing to WordPress.

I also spent a some time exploring some of the tools of open source software such as version control systems. Although WordPress uses Subversion, the software world at large seems to be slowly moving towards distributed version control systems like Git and Mercurial. I’ve become somewhat familiar with Mercurial and I hope to find more excuses to work with it in the future. I also looked a bit at software licenses. In particular, I looked at the differences between restrictively open “free” licenses like GPL and LGPL and much simpler ones like the MIT license. I didn’t spend much time on this subject as it doesn’t directly relate to programming and would probably be more interesting to a law student but it was good to get a sense of what exactly is meant by terms like open and free when they are applied to software.

I also spent some time this semester looking at subjects tangential to open source but directly related to becoming a well rounded programmer. For example, I began formally acquainting myself with Perl. Perl seems to be quite powerful though intimidatingly open-ended. I’ve also begun learning Ruby, which I feel will soon replace Python as my go-to scripting language. Finally I dove into Emacs which is increasingly my default text-editor. I’ve spent the semester continuing to improve my general coding skills and using open source as focal point. I set out to get exposure to a larger world of software technology specifically in the open source community and I feel that I have done this. This is of course not to say that I’m done; the software world is enormous and there will always be more to learn.

Contributing

I did make some initial attempts to contribute to Mozilla Firefox. I found the process inviting, with many helpful tutorials on how to get started. The main barrier to entry for me was that most of the tickets marked as “Good first bugs” in Bugzilla were like Greek to me. They often referred to aspects of C++ or the Mozilla core that I wasn’t familiar with and assumed that I was. There were a couple of bugs that I was able to understand well enough to reproduce. The first was one involving Firefox’s handling of <wbr /> tags, which was inconsistent with other browsers. There was some disagreement as to whether the behavior of Firefox was in fact incorrect. Given the clear lack of agreement this seemed like a bad place to start. The other bug involved a button in the bookmark interface that needed to be disabled in a certain circumstance. This seemed like something that I might be able to fix but before I got the chance to a patch was submitted by someone else. This was about the time I decided to switch to looking at WordPress.

WordPress was slightly less initially inviting. Unlike with Firefox, there was no “so you want to contribute to WordPress” page with helpful step by step instructions. There was a general description of how to get involved, which I followed. The main difference was my familiarity with the underlying technologies. About half of the tickets in trac were clear enough that I could begin looking into them without much trouble at all and the other half would become more straight forward if I looked at them more. My first contribution was to ticket #18314 which was about cleaning up duplicate entries in the admin style-sheet. I took a look a the offending CSS file and quickly found a section where a large block of identical rules was being applied separately to three different selectors. I merged the identical rules into one block, made a patch using svn diff and submitted it to trac. My patch was soon committed and so I decided to do a once over of the CSSticket #18866. It referred to a bug in the error page displayed if the wp-config.php file is missing. Another bug having to do with certain situations where the URL of the appropriate style-sheet for the error page might not be accessible had necessitated putting the CSS rules for the page in-line in the HTML head. A couple of rules had been forgotten so I tracked down the file that was generating the page and added the necessary rules. I found it necessary to include a reference to a background gradient image, causing the same problem as including a style-sheet would have, except that the difference between the page with and without this gradient was barely noticeable so this seemed like the solution most likely to work most of the time. Another contributor suggested using CSS gradients but, as they are not supported by Microsoft Internet Explorer prior to version 10, my patch was used.

I also fixed a bug in the “Press This” interface, ticket #18989. Press This is a bookmarklet that opens a pop-up window allowing bloggers to easily post about content they find while browsing other sites. In addition to automatically pasting a link to the site into the post the bookmarklet grabs any selected text as well as providing an interface to include in-line images from the page. A change from another ticket had caused a bug in the image including functionality. Everything seemed to work fine but the image was not pasted into the post. Another contributor had pointed to the offending bit of code. It seems that an HTML input tag with the same id had been added to the interface and the Jquery selector that got the image URL to insert it into the post was now no longer specific enough. I added a more specific selector but this didn’t fix the problem. By adding console.log() statements, I was able to follow a relatively complex chain of seemingly unnecessary functions and discover a place where the HTML to be inserted into the post was being passed as the only argument to function that expected it as the second argument. Not wanting to break another aspect of the application, I passed the parameter as both the first and second arguments and the bug was fixed. If I had been more comfortable with general workings of the editing interface I’d have looked further to see why this was so excessively complex but this felt like a much larger project so I just made it work in a way that was unlikely to cause conflicts. It strikes me that this sort of apathy is probably what allows this code to remain as complex as it is. Perhaps I will look into cleaning it up for the next release.

I also found a bug and created ticket #19119. A major feature had been added creating fly-out menus for the main admin navigation in ticket #18382. I noticed relatively early that the changes being made had caused the collapsed menu fly-outs to be off by one pixel. Since there were a lot of developers working on this at the time I assumed that someone was on it and was worried that too many cooks might spoil the sauce. Apparently I was wrong because ticket #18382 was closed with the notice “If you encounter any bugs, please open a new ticket.” The bug was still there so I opened a new ticket. I tried submitting a patch to it but a better one was used by someone with a better understanding of the CSS involved. I also made several smaller contributions in the form of testing patches and commenting on discussions.

The Open Source Community

My initial reaction to the communities I looked at was that they were very inviting. This was unsurprising as it seems like a necessity to an organization fueled by volunteers. It was interested to see how decision were made within these communities. The Mozilla approach is a complex hierarchical structure that rewards longstanding contributors with the ability to make more serious decisions. WordPress being smaller, has a much simpler approach. There is a core WordPress team that has commit access to the Subversion repository. Anyone else can check out the repository and submit changes to Trac in the form of patches (anyone can create an account in Trac). Because WordPress is relatively small, it’s rather trivial for the core team to keep track of tickets with patches in them and commit things that need to be committed.

It was interesting to see the varying quality of documentation in WordPress code. Any function intended to be used by plug-in/theme developers is described in detail in the WordPress Codex in terms that can be understood by someone who has very little programming experience. Anything internal is lucky to have more than basic in-line comments next to it in the code. This also varies widely from file to file. Most of the files have a comment block at the top explaining the file’s purpose but this is often all you get and you have to hope the code is well written (which it only sometimes is). It’s not clear to me why there is such a discrepancy between end user documentation and developer documentation. It seems reasonable that user documentation would be prioritized as it is part of making the product usable but it seems like it would be just as important to provide support to developers so to help developers improve the core.

One thing that surprised me that seemed to be a major feature of the community was a general sense of inertia. A debate came up in the wp-hackers email list about a month ago about the differences between relative and absolute URLs. Whenever an internal image or link is added to a post or page on a WordPress site, the editor stores an absolute URL in the database. The strongest argument for this seems to be that absolute URLs will fully specify the correct location in any context and since the pages therefore always want to be served that way, they shouldn’t need to be processed for every page load. The strongest argument against this seemed to be the issue of portability. If development is at dev.example.com and a production is at example.com we shouldn’t need to fix the database any time we push the database to production. Various other arguments were made and things got a bit out of hand at times but the end result was that since the core team was mostly in the former camp nothing was going to change in the WordPress core but it was fine to implement relative URLs in a plug-in (this was not particularly satisfactory for the proponents of relative URLs since the plug-in solution turned out the be somewhat clunky). “Make it a plug-in” seems to be an unfortunately common answer to feature requests that don’t seem necessary to the use case that the core team has in mind. For example re-arranging the order of your pages is an annoyingly complex task of meticulously changing a number in a field on each page. There is a plug-in called my page order that provides a nice drag-and-drop interface for this but no one has considered incorporating it into the core. It strikes me as unfortunate that this sort of inertia exists and I hope that it isn’t as prevalent in the open source community at large as it is in WordPress as it seems like a key barrier to competition with commercial software.

Gaining Programming Skills

One of my main goals for the semester was to gain skills in group development. To this end, I spent a good deal of time acquainting myself with the relevant tools. One of the most important tools in group development is version control. The three version control systems I looked at were Subversion, Mercurial, and Git. Subversion was clearly a vast improvement on its predecessor CVS, but it seems to be somewhat outdated at this point because the advantages of distributed version control are significant. Unfortunately it is still quite prevalent. Mercurial seems to be a really nice solution as it is fast, powerful, distributed, and relatively easy to learn. Git seems to be the most powerful of the three but is harder to learn. My current preference is for Mercurial which I have started using for some of my own projects.

The experience of working with other people’s code was helpful in general. My experience with ticket #18989 in particular drove home the importance of keeping code simple and easy to follow. It also re-affirmed my appreciation for the value of debug statements.

I also spent some time looking at Software licenses. Particularly I looked at the differences between GPL, LGPL, and more permissive licenses like the MIT license. GPL is restrictively free. This sounds like a contradiction but the idea is basically that if a piece of software is licensed under GPL then any software built on top of it must also be GPL. This ensures that the use of this code will never be subject to proprietary endeavors. LGPL is less restrictive. It allows proprietary software to be built on top of it so long as the licensed software is unmodified. LGPL is common for programming languages and other situations where GPL would severely limit the products usability

I gained a lot of general knowledge/skills this semester that will help me in my goal of becoming a great software developer. In addition to gaining familiarity to the WordPress core, I got exposure to XUL, Version Control, software licenses, Perl, and Ruby. Although my semester was not as focused on contributing to open source as I originally intended it to be, I don’t see this as having been a problem. The nature of software is that making it well is often as much about knowing how to use the tools as it is about understanding the concepts. I feel like this semester gave me the opportunity to become acquainted with a number of powerful tools of the trade. Even though it didn’t necessarily coalesce into a clearly structured curriculum, it gave me a chance to learn what I need to learn.

Convolutions

I will herein describe convolution functions, if you are already familiar with the topic, feel free to skip ahead to the next heading.

The basic idea of a convolution is to combine to mathematical functions in a particular way. The concept (like most things in mathematics) is described quite well on its Wikipedia page, so I won’t go into enormous detail here. As it applies to image processing, convolution is a process of applying a grid of numerical values called a kernel to an image. A simple example of a kernel is:

The process is quite simple: look at each pixel in the input image and set the corresponding output pixel to the weighted sum/average of the neighboring pixels where the weights are determined by the values in the kernel. So for example, with the above kernel, each pixel diagonally adjacent to a given pixel will be given a weight of 1, each pixel orthogonally adjacent to the pixel will be given a weight of 2, and the pixel itself will be given a weight of 3. This particular kernel has the effect of blurring the image.

All sorts of things can be done with different kernels. One interesting use is finding edges which can be done with a kernel like:

AJAX without AJAX

The thing that excited me most in building this little program was the realization that although it uses JQuery to communicate with the server without reloading the page without using AJAX, at least not in the traditional sense. It is technically still asynchronous, and it doesn’t use XML (but modern AJAX rarely does).

When you click the process image button, a couple of things happen. First the image you selected is asynchronously uploaded using a JQuery plugin called JQuery.upload. As it turns out the only way to asynchronously upload a file is using an iframe, which is exactly what this plugin does. So not really AJAX as we typically think of it. The second thing that happens is the src attribute of the modified image on the page is set to a php script that processes the newly uploaded image and spits out an image, definitely not AJAX as we normally think of it.

That’s basically it, besides boring implementation details (personally I find implementation details fascinating but this is a blog post not documentation).

Press This and Images

WordPress has a feature called “Press This”, which allows bloggers to easily share content that they might find while browsing the web. Using the “Press This” bookmarklet pops up a new window which allows the user to post new content including anything highlighted on the page you were on. One nice feature is that when you click the add images button from this new post interface, it loads all of the images from the page you were on and gives you the option of inserting them into your post.

There was a bug created however by some unrelated change leading up to the 3.3 release. You see when you select an image from the page to insert into your post, you are prompted to add a description and then click the “insert photo” button to insert it. This button uses the JQuery to determine the url and description of the image to insert with something like:

1
2

url = JQuery('#this_photo').val();
desc = JQuery('#this_photo_description').val();

The problem was that another change had created two unrelated DOM objects with the ids this_photo and this_photo_description. Consequently the JQuery, assuming that ids are unique (as they should be) grabbed the wrong DOM objects, found no values and then promptly failed. To fix this I found the id of the parent div to the two DOM objects I wanted and replaced the code with something like this:

1
2

url = JQuery('#parent_div #this_photo').val();
desc = JQuery('#parent_div #this_photo_description').val();

This is probably not ideal but since I wasn’t sure where else these objects might be referenced by JavaScript and I didn’t want to create more bugs that might not be caught this close to a release I didn’t want to mess with changing the ids themselves.

Dynamic Functions

The API system I had set up involved using functions to represent moves and both of these games have more possible moves than it would be sensible to define individually. Fortunately Lisp is great at generating code dynamically. It took a while to find the right syntax for this and what the method I eventually used was the back-tick quoting mechanism that Jim Mahoney pointed me to. The following code generates the possible moves for my Tic-Tac-Toe program:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

(setf all-moves
      '((top-left . 0)
        (top-center . 1)
        (top-right . 2)
        (middle-left . 3)
        (center . 4)
        (middle-right . 5)
        (bottom-left . 6)
        (bottom-center . 7)
        (bottom-right . 8)))
 
(dolist (move all-moves)
  (let ((name (car move))
        (num (cdr move)))
    (eval `(defun ,name (state) (mark-square ,num state)))))

The backtick ` notation here lets me quote the list and still add in evaluated pieces later with , so that I can construct the defun statement as I want it before evaluating the whole thing with eval.

Side effects and Pointers

Another issue I found was that while the adversarial search worked fine for my stone game. I had made a fundamental assumption. A given game state for the stone game is represented by and integer and as such making a function that returns a modified copy of the game state without affecting the original (something I do a lot) is simple and requires no thought. Unfortunately this is not the case with more complicated data structures like lists.

For both tic-tac-toe and othello, my games states were represented by lists and as such I needed to use copy-list. My tic-tac-toe games states are encoded as a list of length 10 with the first element being the current player (X or O) followed by the contents of each square in left to right top to bottom order. So for example the start state is:

1

(X NIL NIL NIL NIL NIL NIL NIL NIL NIL)

and if X goes in the center the state would become

1

(X NIL NIL NIL NIL X NIL NIL NIL NIL)

A naive version of my mark-square function referenced in the above code might look like this:

1
2
3
4
5
6
7

(defun mark-square (square state)
  "Mark the designated square with the symbol of the current player's turn and toggle the turn"
  (let ((player (first state))
        (board (rest state)))
    (setf (nth square board) player)
    (cons (if (eq 'x player) 'y 'x)
          board)))

The problem with this is that (rest state) doesn’t return a copy of the state list starting with the second element, but rather it returns a pointer to the second cons cell in the state list. This means that we are dealing with same data in memory that was passed in and when we call setf, we modify the original state. More concretely if we try:

1
2
3
4
5
6

(setf state '(x nil nil nil
                nil nil nil
                nil nil nil))
(setf new-state (mark-square 4 state))
(format t "old state: ~a~%" state)
(format t "new state: ~a~%" new-state)

we will get:

1
2

old state: (X NIL NIL NIL NIL X NIL NIL NIL NIL)
new state: (O NIL NIL NIL NIL X NIL NIL NIL NIL)

This is what is known as a side-effect, a function that does something other than just return a value. It is a result of our using Common Lisp which is not a purely functional language. In a purely functional language this would not be possible. The fixed version of my mark-square function is:

1
2
3
4
5
6
7

(defun mark-square (square state)
  "Mark the designated square with the symbol of the current player's turn and toggle the turn"
  (let ((player (first (copy-list state)))
        (board (rest (copy-list state))))
    (setf (nth square board) player)
    (cons (if (eq 'x player) 'y 'x)
          board)))

The copy-list function creates new cons cells so the changes affect a new copy of the list.

My Code

I’m really happy with my code right now for this. You can download a copy of it by clicking here.

You can run it at the command line with:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132

$ cd /path/to/downloaded/directory
 
$ clisp
[]> (load "tic-tac-toe.lisp")
T
[]> (play-game tic-tac-toe 10 'computer 'sam) ; the 10 here is maximum depth for adversarial search
It is X's turn
 -----
 
 -----
computer: BOTTOM-RIGHT
 
SAM's turn
It is Y's turn
 -----
 
     X
 -----
 
choose a move from
(TOP-RIGHT BOTTOM-LEFT TOP-RIGHT TOP-LEFT CENTER MIDDLE-LEFT BOTTOM-CENTER
 MIDDLE-RIGHT TOP-CENTER): top-left
It is X's turn
 -----
 Y    
 
     X
 -----
computer: TOP-RIGHT
 
SAM's turn
It is Y's turn
 -----
 Y   X
 
     X
 -----
 
choose a move from (BOTTOM-LEFT CENTER MIDDLE-LEFT BOTTOM-CENTER MIDDLE-RIGHT TOP-CENTER): middle-right
It is X's turn
 -----
 Y   X
     Y
     X
 -----
computer: BOTTOM-LEFT
 
SAM's turn
It is Y's turn
 -----
 Y   X
     Y
 X   X
 -----
 
choose a move from (CENTER MIDDLE-LEFT BOTTOM-CENTER TOP-CENTER): center
It is X's turn
 -----
 Y   X
   Y Y
 X   X
 -----
computer: BOTTOM-CENTER
It is Y's turn
 -----
 Y   X
   Y Y
 X X X
 -----
COMPUTER wins
NIL
[]> ^D
 
$ clisp
[]> (load "othello")
T
[]> (play-game othello 5 'computer 'sam) ; the 5 here is maximum depth for adversarial search
 
SAM's turn
It is BLACK's turn
 ---------------
 
       W B
       B W      
 
 ---------------
 
choose a move from (3-4 4-3 5-6 6-5): 3-4
It is WHITE's turn
 ---------------
 
       B
       B B
       B W      
 
 ---------------
computer: 3-3
 
SAM's turn
It is BLACK's turn
 ---------------
 
     W B
       W B
       B W      
 
 ---------------
 
choose a move from (3-2 4-3 5-6 6-5): 5-6
It is WHITE's turn
 ---------------
 
     W B
       W B
       B B B    
 
 ---------------
computer: 3-5
 
SAM's turn
It is BLACK's turn
 ---------------
 
     W W W
       W B
       B B B    
 
 ---------------
 
choose a move from (2-2 2-3 2-4 2-5 4-3):
 
etc..

There is of course nothing terribly exciting or new about the sleep command but the ability to quickly solve a practical problem with simple tools is always gratifying. My egg timer is the following simple text file with executable bit set:

1
2
3

sleep 600;
osascript -e 'beep 3'
osascript -e 'tell app "System Events" to display dialog "Eggs Done!"'

Yay eggs!

1

setopt autocd

And suddenly instead of:

1

$ cd some-directory/some-subdirectory

you can just use:

1

$ some-directory/some-subdirectory

and zsh will notice that it’s a directory and do the only sensible thing: go to it.

I also have a sizable number of time saving aliases:

1
2
3

alias l="ls -lahG"
alias mkd="mkdir -pv"
alias dir="open ./"

ls -lahG is nice because it gives me so much information. The -l flag formats the output into a vertical list that tells me more about each file. The -a includes hidden files and directories that start with . like .zshrc. The -h flag displays file sizes in a more readable format so if a file is 2262261 bytes long it will say 2.2M instead of 2262261. The -G flag colors my files using a custom colorings scheme that I define in my .zshenv file.

mkdir -pv is pretty straight forward. The -p flag allows me to create nested directories with a single command so that if ~/foo doesn’t exist then mkd ~/foo/bar will create both the foo and bar directories with one inside the other. The -v just generates verbose output.

open ./ is a nifty Mac OS only command. On a mac the open command will do the same thing as double clicking the file in the Finder. It’s a handy link between my terminal and the rest of my mac. In this case I can use it to quickly switch to the mac GUI interface for this directory if I need to for some reason.

I also have a number of alias that are simply shorter names for commands that I use all the time:

1
2
3
4
5
6
7

alias m="less"
alias e="edit"
alias t="touch"
alias sq="sqlite3"
alias o="open"
alias ql="qlmanage -p"
alias c="echo -n \"\" >"

edit is a python script I made that determines my default text editor from a configuration file and then opens the file in that editor. qlmanage is a mac specific program that does the same thing as pressing space-bar while highlighting a file in the Finder.

Backspace vs. Delete

Recently I had a problem with using zsh on certain ssh servers where my delete key (the mac equivalent of windows’ backspace) was being mapped to the windows del. That is to say it was deleting the next character instead of the previous. I finally found this wiki page that explained the problem and how to fix it. All I had to do was add the following to my .zshrc file:

1
2

bindkey "^?" backward-delete-char
bindkey "^[[3~" delete-char

1
2
3
4
5
6
7
8
9
10
11
12

;; Game
(defstruct game
  "An object representing the rules of a game"
  (name "") ; The name of the game
  (initial-state nil) ; The state the game begins in
  (state-type 'list) ; A symbol representing the type of the state object
  (winning-score 1000) ; The score associated with a win
  (score? nil) ; A function that takes a state and returns a score
  (terminal? nil) ; A function that takes a state and returns a score iff the state is
                  ; terminal, otherwise nil
  (moves? nil) ; a function that takes a state and returns a list of legal moves
  (print-state? nil)) ; a function that prints out human readable version of the state

Additionally I created a function to generate a modified version of a game after a move has been made:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

(defun game-progress (game move)
  "Return a new game that is equivalent to the game but starts after move have been
 applied to its initial state"
  (make-game
   :name (game-name game)
   :initial-state (funcall move (case (game-state-type game)
                                  (list (copy-list (game-initial-state game)))
                                  (otherwise (game-initial-state game))))
   :state-type (game-state-type game)
   :winning-score (game-winning-score game)
   :score? (game-score? game)
   :terminal? (game-terminal? game)
   :moves? (game-moves? game)
   :print-state? (game-print-state? game)))

Note the use of case on line 6 to make sure that if the data-type of state is a list we are making a copy of the list, not a the original.

The other important structure I used was called node and I used it to keep track of positions in the search tree.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

;; Node
(defstruct node
  "A position in a search tree"
  (state nil)
  (moves nil)
  (score 0))
 
(defun new-node (game &key state moves)
  "Build a new node with a score generated by the game definition"
  (let ((new-state (if state state (game-initial-state game))))
    (make-node
     :state (case (game-state-type game)
              (list (copy-list new-state))
              (otherwise new-state))
     :moves (if moves moves nil)
     :score (funcall (game-score? game) new-state))))
 
(defun -- (node)
  "Return a copy of the node with the opposite score"
  (make-node
   :state (node-state node)
   :moves (node-moves node)
   :score (- (node-score node))))
 
(defun do-move (move node game)
  "Do the move to the node"
  (let ((new-state (funcall move (node-state node))))
    (make-node
     :state (case (game-state-type game)
              (list (copy-list new-state))
              (otherwise new-state))
     :moves (cons move (node-moves node))
     :score (funcall (game-score? game) new-state))))

Note the similar case statements. The real meat of the program is the alpha-beta search itself:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

;; DFS
(defun run-depth-first-search (game max-depth)
  "A wrapper function for the main DFS recursive function that should return the
 optimal move for the game"
  (let ((search-result
         (depth-first-search max-depth game
                             (make-node
                              :state (game-initial-state game)
                              :score (* 2 (- (game-winning-score game))))
                             (make-node
                              :state (game-initial-state game)
                              :score (* 2 (game-winning-score game)))
                             (new-node game))))
    (first (last (node-moves (first search-result))))))
 
; basic algorithm based on Jim Mahoney's:
; http://cs.marlboro.edu/courses/spring2010/programming/notes/mar_2.attachments/tictactoe.py_html
; which in turn credits http://en.wikipedia.org/wiki/Alpha-beta_pruning
(defun depth-first-search (max-depth game alpha beta current &optional (depth 0))
  "Recursively search the game tree down to the max-depth"
  (block loop
    (when (> depth max-depth)
      (return-from loop (list current nil)))
    (when (funcall (game-terminal? game) (node-state current))
      (return-from loop (list current t)))
    (do* ((possible-moves (funcall (game-moves? game) (node-state current)))
          (move (pop possible-moves) (pop possible-moves))
          (result nil))
        ((or (null move) (< (node-score beta) (node-score alpha)))
         result)
      (let* ((search-result (depth-first-search max-depth game (-- beta) (-- alpha)
                                                (do-move move current game) (1+ depth)))
             (effective-node (-- (pop search-result)))
             (terminal (pop search-result)))
        (when (> (node-score effective-node) (node-score alpha))
          (setf alpha effective-node))
        (setf result (list alpha terminal beta))))))

The idea behind the depth-first-search function here is that is uses alpha and beta to keep track of nodes representing the best move it’s found for each player so far. Each time through the loop checks to see if the current node is terminal or the maximum depth has been exceeded and returns in either case. The return value (lines 23 and 25) is a list with the first value being the best node below this point and the second being whether or not the node is terminal (This should probably be refactored to put this terminal flag in the node structure).

If we don’t immediately return we instead loop through the possible moves and for each one recursively call depth-first-search. However in the call (line 31) we swap (and –) the alpha and beta nodes because we want to consider the best move for our opponent.

The actual alpha-beta pruning happens on line 29 where we set the condition for do* to end the loop through current possible moves. The first condition is obvious, if we’ve checked all possible moves from this point then leave the loop. The other, (< (node-score beta) (node-score alpha)) is the alpha-beta part. It ends the loop if a move has been found that proves this move to be worse than a previous best move.

I also made a simple play-game function that runs the game:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

; Play
(defun play-game (game max-depth first-player second-player)
  "Play a game"
  (let ((terminal? (funcall (game-terminal? game) (game-initial-state game))))
    (if terminal? (progn
                    (funcall (game-print-state? game) (game-initial-state game))
                    (format t "~a wins~%"
                          (if (= terminal? (game-winning-score game))
                                  first-player second-player)))
      (if (eq 'computer first-player)
          (let ((move (run-depth-first-search game max-depth)))
            (funcall (game-print-state? game) (game-initial-state game))
            (format t "computer: ~a~%" move)
            (play-game (game-progress game move) max-depth second-player first-player))
        (progn
          (format t "~a's turn~%" first-player)
          (funcall (game-print-state? game) (game-initial-state game))
          (let* ((possible-moves
                  (funcall (game-moves? game) (game-initial-state game)))
                 (move (do ((move nil (progn
                                        (format t "choose a move from ~a: "
                                                possible-moves)
                                        (read))))
                           ((member move possible-moves) move))))
            (play-game (game-progress game move)
                       max-depth second-player first-player)))))))

One nice thing about this function is that if neither first-player second-player is set to 'computer, then two human players can play.

I made a simple game to test it out. I call the game the stone game and it goes like this. There are n stones in a pile. Two players take turns removing 1, 2, or 3 stones. The last player to remove a stone loses. I defined my game in Lisp like so:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

; moves
(defun take-1 (state)
  (1- state))
(defun take-2 (state)
  (- state 2))
(defun take-3 (state)
  (- state 3))
 
; game
(setf starting-stones 17)
 
(setf stone-game
      (make-game
       :name "stone game"
       :initial-state starting-stones
       :state-type 'int
       :winning-score 1
       :score? (lambda (state)
                 (cond
                  ((= 0 state) 1)
                  ((= 1 state) -1)
                  (t 0)))
       :terminal? (lambda (state)
                    (cond
                     ((= 0 state) 1)
                     ((= 1 state) -1)
                     (t nil)))
       :moves? (lambda (state)
                 (cond
                  ((> 2 state) nil)
                  ((= 2 state) '(take-1 take-2))
                  (t '(take-1 take-2 take-3)))) 
       :print-state? (lambda (state)
                       (format t "There are ~a stone(s) left~%" state))))

Here’s an example of the game in action:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

[]> (play-game stone-game 20 'sam 'computer)
SAM's turn
There are 17 stone(s) left
choose a move from (TAKE-1 TAKE-2 TAKE-3): take-2
There are 15 stone(s) left
computer: TAKE-2
SAM's turn
There are 13 stone(s) left
choose a move from (TAKE-1 TAKE-2 TAKE-3): take-1
There are 12 stone(s) left
computer: TAKE-3
SAM's turn
There are 9 stone(s) left
choose a move from (TAKE-1 TAKE-2 TAKE-3): take-3
There are 6 stone(s) left
computer: TAKE-1
SAM's turn
There are 5 stone(s) left
choose a move from (TAKE-1 TAKE-2 TAKE-3): take-2
There are 3 stone(s) left
computer: TAKE-2
There are 1 stone(s) left
COMPUTER wins

Note that the 20 on line 1 sets the maximum search depth for the ai not the number of stones.

My code could use some refactoring and I hope to revisit it at some point. I’m also working on making it solve tic-tac-toe.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

class DbRecord :
	table = ""
	fields = []
	def _fieldsForQuery(self) :
		out = []
		for field in self.fields :
			out.append("%s.%s" % self.table field)
		return ", ".join(out)
	def load(self, args) :
		# load a record from the db
	def save(self) :
		# save a record to the db
 
class Place(DbRecord) :
	table = "place"
	fields = ["id", "name", "description"]
 
class Person(DbRecord) :
	table = "person"
	fields = ["id", "name", "email", "place_id"]
	def getAddress(self) :
		sql = "SELECT %s FROM %s, %s WHERE %s.id = %s" %
			(Place.fieldsForQuery(),
			self.table,
			Place.table,
			Place.table,
			self.place_id)
		self.place = Place.load(sql)

For starters, the PHP equivalent of this looks quite different. Notably, there are no class variables in PHP. For example:

1
2
3

class Foo {
	$var = 1 ;
}

results in errors, while

1
2
3
4
5
6

class Foo {
	$var ;
	function __construct() {
		$var = 1 ;
	}
}

is fine.

But since I want to access these variables without instantiating the class (see for example line 25 in the above python code) this will not do. There are class constants in php for example:

1
2
3

class Foo {
	const var = 1 ;
}

but, among other issues, they cannot be used to store arrays.

This leaves us with few options. The one I chose was to make functions with the sole purpose of returning a fixed value. It ended up looking something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

abstract class DbRecord {
	function _fieldsForQuery() {
		$out = array() ;
		foreach ($this->fields() as $field) {
			$out[] = $this->table() . '.' $field ;
		}
		return implode(', ', $out) ;
	}
	function load(args) {
		# load a record from the db
	}
	function save(self) {
		# save a record to the db
	}
}
 
class Place extends DbRecord {
	function table() {return 'place' ;}
	function fields() {return array('id', 'name', 'description') ;}
}
 
class Person(DbRecord) {
	function table() {return 'person' ;}
	function fields() {return array('id', 'name', 'email', 'place_id') ;}
	function getAddress(self) {
		$sql = '
			SELECT ' . Place::fieldsForQuery() . '
			FROM ' . $this->table() . ', ' . Place::table() . '
			WHERE ' . Place::table() . '.id = ' . $this->place_id . '
		' ;
		$this->place = new Place() ;
		$this->place->load(sql) ;
	}
}

Of course this doesn’t even work. As it turns out the fieldsForQuery method called on line 27 here returns the formatted fields from Person, not from Place. The moral of the story here is probably that PHP wasn’t designed to support non-trivial complex inheritance.

It would have been simple of course if I hadn’t just upgraded my site to WordPress. If my site were a simpler PHP based site I could have done something like this:

1
2
3
4
5
6
7
8
9
10
11

< ?php
	function money($hours) {
		$hourlyRate = 35 ;
		echo number_format($hourlyRate * $hours, 2) ;
		// the 2 here specifies how many decimal places to display
	}
?>
 
...
 
Some job that takes two and a half hours costs $< ?php money(2.5) ?>

The problem with this approach is that WordPress stores the content of its pages in a database. It does however provide ways to modify the content dynamically before it’s printed. If you’re doing something small that doesn’t warrant writing a whole plugin, you can do this by editing the functions.php file in your WordPress theme.

WordPress provides a number of hooks, or events that you can attach your own php functions to to change how things work. For this project the relevant hook is called 'the_content' and can be invoked using

1

add_filter( 'the_content', 'my_function_name' ) ;

where my_function_name is the name of a function that takes a string as an argument and returns a modified version of the string.

So the idea here is that when I edit my rates page in WordPress, I should be able to add tags like

1

Some job that takes two and a half hours costs ${MONEY 2.5}

and my hooked function can use regular expressions to replace the tag with the appropriate amount. There’s a snag though; regular expressions can easily find the string using /{MONEY ([\d\.]+)}/ but performing floating point multiplication is outside the scope of a simple preg_replace call.

The solution is an excellent little function called preg_replace_callback. preg_replace_callback works like preg_replace except that it replaces the matched strings using function. A simple solution to my problem that works as of PHP 5.3 would have been to add the following to my functions.php file

1
2
3
4
5
6
7
8
9
10
11
12
13
14

function calculate_money($text) {
	$hourlyRate = 35 ;
	$text = preg_replace(
		'/{MONEY ([\d\.]+)}/', // a regex expression to find the tags
		function ($matches) {
			$hours = (float)$matches[1] ; // retrieve the parenthetical pattern
			return number_format($hourlyRate * $hours, 2) ;
		},
		$text
	) ;
	return $text ;
}
 
add_filter( 'the_content', 'calculate_money' ) ;

That nameless function on line 5 is called an anonymous function. Anonymous functions were introduced in PHP 5.3. Before PHP 5.3 you would need to use:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

function calculate_money($text) {
	$hourlyRate = 35 ;
	$text = preg_replace(
		'/{MONEY ([\d\.]+)}/', // a regex expression to find the tags
		create_function(
			'$matches',
			'$hours = (float)$matches[1] ; // retrieve the parenthetical pattern
			return number_format(' . $hourlyRate . ' * $hours, 2) ;'
		),
		$text
	) ;
	return $text ;
}
 
add_filter( 'the_content', 'calculate_money' ) ;

There are a couple of important things to note here. The first is that I’m using single quoted strings in my create_function. If I use double quotes, I need to escape all of my dollar signs with "\$" to prevent PHP from attempting to interpolate variables.

The second important thing here is that the $hourlyRate variable that I defined outside of the function must be concatenated (or interpolated) into the string effectively hard-coding it into the anonymous function we are creating. The reason this is necessary here and not in the PHP 5.3 version is that the anonymous function defined in the previous example forms what is called a closure. This means that it can see the local variables from the scope in which it was defined.

At any rate neither of these was what I wanted. You see, I don’t just have a basic rate, I also have a rate for more complex work. And sure I could have just defined two variables and had two kinds of tag but this is WordPress; I should be able to manage my site using the WordPress interface not by editing hard-coded values in functions.php.

My solution was to set up two kinds of tags, variable tags and money tags. When I edit my rates page in WordPress I could make my page look something like the following:

1
2
3
4
5
6

{VAR basicRate 35}
{VAR complexRate 40}
 
...
 
Some basic job that takes two and a half hours costs ${MONEY basicRate 2.5}

The code that I add to functions.php might look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

function calculate_money($text) {
	$vars = array() ;
	// find the variable tags and store them in $matches
	preg_match_all('/{VAR ([a-zA-Z]+) ([\d\.]+)}/', $text, $matches) ;
	// store the variables in the $vars array
	foreach ($matches[1] as $key => $var) $vars[$var] = (int)$matches[2][$key] ;
	// remove the tags so they aren't displayed on the page
	$text = preg_replace('/{VAR[^}]*}/', '', $text) ;
	$text = preg_replace_callback(
		'/{MONEY (a-zA-Z]+) ([\d\.]+)}/',
		function ($matches) {
			// retrieve the first parenthetical pattern
			$rate = $vars[(float)$matches[1]] ;
			// retrieve the second parenthetical pattern
			$hours = (float)$matches[2] ;
			return number_format($rate * $hours, 2) ;
		},
		$text
	) ;
	return $text ;
}
 
add_filter( 'the_content', 'calculate_money' ) ;

Of course prior to PHP 5.3 this would not be so simple because the anonymous function can’t see the $vars array without a closure. One solution might look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

function calculate_money($text) {
	$vars = array() ;
	// find the variable tags and store them in $matches
	preg_match_all('/{VAR ([a-zA-Z]+) ([\d\.]+)}/', $text, $matches) ;
	// store the variables in the $vars array
	foreach ($matches[1] as $key => $var) $vars[$var] = (int)$matches[2][$key] ;
	// remove the tags so they aren't displayed on the page
	$text = preg_replace('/{VAR[^}]*}/', '', $text) ;
	foreach ($vars as $key => $value) {
		$text = preg_replace_callback(
			'/{MONEY ' . $key . ' ([\d\.]+)}/',
			create_function(
				'$matches',
				'$hours = (float)$matches[1] ;
				return number_format(' . $value . ' * $hours, 2) ;'
			),
			$text
		) ;
	}
	return $text ;
}
 
add_filter( 'the_content', 'calculate_money' ) ;

Another possible pre PHP 5.3 solution is:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

function calculate_money($text) {
	$vars = array() ;
	// find the variable tags and store them in $matches
	preg_match_all('/{VAR ([a-zA-Z]+) ([\d\.]+)}/', $text, $matches) ;
	// store the variables in the $vars array
	foreach ($matches[1] as $key => $var) $vars[$var] = (int)$matches[2][$key] ;
	// remove the tags so they aren't displayed on the page
	$text = preg_replace('/{VAR[^}]*}/', '', $text) ;
	$text = preg_replace_callback(
		'/{MONEY (a-zA-Z]+) ([\d\.]+)}/',
		create_function(
			'$matches',
			'$hours = (float)$matches[1] ;
			$vars = ' . var_export($vars, true) . ' ;
			// the second argument above causes it to return instead of echoing
			$rate = $vars[$matches[2]] ;
			return number_format($rate * $hours, 2) ;'
		),
		$text
	) ;
	return $text ;
}
 
add_filter( 'the_content', 'calculate_money' ) ;