rectpara.el

Let me tell you about some code that you're not going to care about: an emacs lisp package called rectpara.el that I wrote to handle a special purpose of my own. Once upon a time, circa 1980, I was enamored of the idea that the personal computer revolution was going to allow everyone to write their own code (that's what the phrase "computer literacy" meant to me). It strikes me that rectpara.el might make a good example of this sort of thing: it does something to serve my own purposes using homebrew code that no one else was going to write. We can go over this to try to evaluate whether the whole idea of individualized, custom software makes any sense at all... but first, let's talk about exactly what it does (and the dull stuff like where do you get it and how to install it and so on, that can go down in an appendix).

the task

The core of this project is rectpara.el, which is an GNU emacs extension to work on blocks of text layed-out as chains of rectangular paragraphs, which I call "rectparas".

   For some time
   I've been inclined
   to use emacs
   picture-mode         Using small,
   to write things      floating        Which I
   like this...         rectangular     tend to call
                        paragraphs.     rectparas.
     I've always
     wanted better
     tools to do
     this... it was
     an obvious idea       It only took me
     to write a            about 10 years
     mode for it.          to get to it.

                           (And another 10
                           to publish it.)

rectpara.el functionality

GNU Emacs has always (at least as far back as version 17 or 18 or so when I came into the story) has had some features that make handling text like this easier (or at least doable): there's a "picture-mode" originally intended for doing ascii-art, and it includes some good features to cut and paste emacs rectangles.

The earliest versions of "rectpara.el" were just keystroke macros that used those picture-mode features: the macros grabbed the rectangle indicated by mark and point, moved it to another buffer for editing in text-mode, and then pasted it back where it came from. That remains the central idea behind rectpara.el: the main text is in picture-mode (or now, a variant of it called rectpara-mode), and you edit text in another buffer which is in text-mode (or now, a variant of it called rectpara-edit-mode).

The rectpara.el code has two big advantages over that old keystroke-macro approach:

1. You don't need to manually place mark and point at the beginning and end of the rectangle, instead the rectpara at point is automatically selected.
2. When the edited text is pasted back into the original buffer, it will never over write any existing text, instead it detects collisions and moves things out of the way.

Neither of those features was conceptually all that difficult, though implementing them to my satisfaction took some time, and was a gradual process. I did the auto-selection feature first, and that was so long ago I can't remember whether it was hard to get it working. The first step was just formalizing what I mean as a "rectpara": they're bounded by blank lines above and below, and at least two columns of spaces to the left and right. I can see from the code that there were quite a number of corner cases to sort out (like a rectpara is still a rectpara if it's at or near the left edge of the buffer, even though it doesn't have the two columns of spaces in that case); and since I'm a "type-two-spaces-after-a-period" kind-of-guy, I had to make sure the code wouldn't get confused by those double-spaces. I haven't touched this "select" code much in recent years, and I'm pretty sure I could do better now and make it more robust... but it works well enough that I don't care that much.

The second feature, avoiding collision and overwrite after an edit, wasn't really perfected (*knock* *knock*) until recently, but I had a crude version of it working at least ten years ago: it could only deal with vertical expansion (which is most common for me) and the way it dealt with it was an odd hack which I'll describe in a minute for you connoisseur's of odd hacks. This release of rectpara.el (the 1.0 version) now uses a more sensible recursive algorithm (and it handles horizontal as well as vertical expansion): it finds if the new rectpara is encroaching on another, and then the code checks that rectpara in turn to look for other rectparas that it will encroach on after it's moved, and those are also checked, and so on, until we've identified an entire tree that needs to be pushed over or down.

The odd hack I used to use was something like this: when the last line of a newly edited rectpara differed from where it used to be, the code would conceptually draw a line across the buffer at the end of the original rectpara, and it would grab everything that intersected the line, moving it out of the way temporarily, then insert a bunch of blank lines, moving everything below further down, then it would restore the stuff that was stashed away. This ensured that you'd never overwrite anything below what you were editing, but it also tended to shift things down that didn't need to be. I ended up doing a lot of manual fix-up, shuffling things back to where they were.

motivation

Why did I care about this enough to write this code? For a long time, I'd been writing stuff using this sort of format, largely as part of a project I've been calling the doomfiles. This project dates back to before the web existed: I was interested in the idea of hypertext, and I was impressed with Ted Nelson's point that you could implement hypertexts in many different forms, using different software. For me, doing chains of text in a 2D graphical layout was the simplest thing I could do with the software I had at hand.

About the history of the doomfiles

HISTORY

Some old material about hypertext and Ted Nelson:

HYPER

NELSON

self-assessment

Okay, what I really wanted to talk about here though is "did this make any sense?".

Remember, I've always liked the idea of a world where everyone can forge their own tools, a world where we can all go in different directions, experimenting with different ways of doing things. There are some real practical problems with this vision of things, though: writing software is hard. A life lived according to your "own personal code" is a life of struggling with your own personal bugs, bugs no one else cares about or will help you with, because they don't have much interest in your own idiosyncratic goals and desires... and they've all got problems of their own, especially if they too are writing their own code.

And there's a potential gotcha I'm well aware of from some familiarity with Ted Nelson's life: if you can at all avoid it, you really don't want to hold up what you're doing while you're trying to get some code working. Even code that seems pretty simple has a way of turning into a nightmare project-- so, under no circumstances should you say to yourself something like "there's this writing project I'd like to work on, but it'll be much eaiser if I had that code working first, so I'll do the code first, and then later...".

To pat myself on the back here, the one thing I can really congratulate myself on is the very, very gradual way I developed all of this code. The simple keystroke macros at the outset, the early emacs lisp versions with partial implementations of the most valuable features, holding off on perfecting everything until time and energy was available for it... at no point did I ever stop writing any text just because of a bug or a missing feature in my code.

(And you can say similar things about the perl tools I use on the doomfiles project. Some time I mean to give a talk about them: they're an excellent example of how bad a software design can be and still be useful.)

There's other issues I might raise about this: going in an idiosyncratic direction like this has probably cut me off from whatever like-minded community might exist out there in the world. When "blogging" software became a big deal, I wasn't particularly interested. I already knew how to write html, I put out web pages at will (often using the kind of tools I'm talking about here), why would I need someone else's "blog" software? I understood that RSS made "blogs" a little different (and I've been meaning to add RSS features to the doomfiles for ages), but I felt like I could live without that stuff. It's not like any of this is critically important, it's largely things I'm writing for my own benefit anyway, right?

And so I remain off in an a corner, having painted myself in with many hours of determined software development...

appendix

The latest code is (probably) up on github: https://github.com/doomvox/rectpara/

There's some installation instructions inside the *.el file, albiet old-fashioned ones at the moment: I'm still not up on the new(ish) emacs package management features.