My current job has me doing a lot of node.js. The documentation story in most of the community is very much “read the source”.

This is a little bit of a contrast to the Rails community, where, even in Rails 1.2.3 (when I entered the scene) I rarely had to dig through source code to find an answer to a question.

Anyway, I find myself reading a lot of node package code. Viewing the code directly on Github is OK for a simple package, but I usually want to be in my editor.

I also want these packages I download to go away, eventually. I could use Hazel, or I could shove things in /tmp/ and let the system clean it up eventually.

So I wrote a script:

This script is great coupled with LaunchBar’s Unix Executable support. I can select this script from my Launchbar, SPACE, paste a git clone path, option-return and half a second later I have the project in my editor (BBEdit).

If the package means my needs I may add it to my node project. Once I do that viewing the code is just a node edit away, but when trying to choose between 2 different-but-providing-similar-functionality packages, making a temporary clone is great.

I often find myself doing open source work. Mostly this is small improvements to already existing projects, but sometimes there are projects I’ll open source from scratch.

However, most of my work is using an open source project, and find a way to improve it. This improvement could be a bug fix, or better error messages, or improvements I’ve needed to get my job done. Sometimes clients even pay me to implement some features in an open source projects.

For the last 4 years or so most of my contributions have been via Github.

Github works by people “forking” (copying) an open source repo to their own user account, making changes, then submitting those changes back to the main project repository. This works great, except sometimes people fork projects to their user account simply to have a copy of the code.

So, you could have an account with a ton of forks - as a person browsing the account you don’t know if they’ve made contributions to the project or if they just want a copy of the code for their own.

Projects I’ve contributed to

I don’t know how to solve this problem, but I do know that I can list projects that I’ve made non-trivial contributions to, and where my code was accepted:

• Bundler, a Ruby gem
• libgit2, a C library implementation of Git
• objective-git, a Cocoa framework over libgit2
• Paver, a Python package
• twitter-bootstrap-markup-rails, a Ruby gem

My Github page shows other contributions to the open source community in large: examples, tools, things for text editors, and some original libraries of code. However, I really do enjoy the times when I can contribute a small change to an open source project that makes everyone’s life a little better.

My own projects that I’ve worked on

In addition to improving other people’s projects, Github also lets you publish your own projects. Most of mine are small projects, but I have published a few larger projects:

• the school_days gem: because some days you need to know if Date.today.school_night?
• the delegate_presenter gem: a gem based on Avdi Grimm’s thoughts on presenters. This was before the release of Objects on Rails.
• RunningBalance: a checkbook example app with API and mobile browsing considerations
• ChatChat, a Meteor play project: woh, I played with Meteor once

Goal: Creeping towards the paperless lifestyle

I’m trying to get into the paperless lifestyle this year. I’m making small changes to facilitate this.

I keep a paper notebook for scratch writing while I work. Mostly the notebook contains things I need to remember for the next 5 minutes, but can forget afterwards. Sometimes it contains more permanent notes: TODO times, thoughts, things to research

I want to be able to make these thoughts paperless. Yesterday I stumbled on a workflow that works for me.

My Workflow

Use MyScript Memo - this will do handwriting recognition and give you the text to be copied.

Send it over to the Mac via PasteBot, or save it via a Dropbox editor

But it can not do handwriting recognition on pictures

So when get idea need to save grab my iPad and stylus and enter it!!

Questions and Answers

Q: Why not Evernote?

A: I want to get the translated text, so I can make corrections to it. Evernote translates the text but I can’t go back and edit the guessed at text. I can’t even export the text: exporting the image gives me an XML document with the image data and each translated word in it’s own XML element with coordinates. I can see why this (for highlighting the text on the image), but seriously I don’t care.

Between my bad handwriting and all the technical terms I use, I really need to be able to go back and correct my text.

Also, with this workflow, the only thing that matters is the text: I don’t care about the .jpg of a picture of my writing. Evernote sees the picture as the thing that has text metadata - I want the text as the thing, with the image being metadata.

Q: Why MyScript Memo?

A: I want to be able to handwrite my notes THEN trigger recognition. I don’t want to fight with computers when I’m taking notes in a meeting.

I also want a full page to write on - not just some small 3”x3” space at the bottom of the screen. I want a notebook page that works like a notebook page in the real world.

Q: What sucks about MyScript Memo?

A: I want a bigger digital canvas. Infinite Sketchpad is awesome at this, but there’s no recognition. But Myspace Memo I can get about half a page of physical notepaper (I rewrote some notes I had laying around) so I guess that’s OK.

I believe some of these concerns are addressed in another MyScript product: MyScript notes mobile. While I get a new iPad I’ll try it out (I have an iPad 1).

I’ll probably still use Infinite Sketchpad when I’m mapping out processes or code or something.

MyScript Memo seems a little slow on my iPad 1, but is quite snappy on my iPhone 4S. I’m looking forward to when I upgrade my work iPad.

Q: But why do this in the first place? Why write something?

A: when I’m in a formal and scheduled meeting I will often take notes on my computer. I have a special document for these notes so this has become second habit to me. However there are two places where I want to use handwriting:

1. Random thoughts I have while working on something. I normally put these ideas in my scratch notebook - a 8.5x11 school notebook - but now I can put them in my iPad without swithving modes from “writing with pens on paper” to “pull up my notes file and start typing.” I’m aiming for low friction data entry.
2. I can write faster than I can type on the iOS keyboard

I’ve recently purchased a Bluetooth keyboard to try and fix issue 2, but that’s useful when I’m writing an important email on my phone or writing text longer than a tweet.

It’s still early in the process - it’ll be another week before I know if I can stick with this workflow. I use my iPad to read technical books/PDFs, sometimes as a fourth screen, and as a tool when I’m brainstorming at my whiteboard. I look forward to seeing how handwritting turned into searchable text changes how I work

A recent tweet by a SF tech recruiter put down those with 20 years experience as “[it]… only means you’re forty”

Instead of decrying 20 years experience (“Go home, old man!”) we should celebrate it.

Because 20 years experience as a computer programmer means:

• 20 years of too much stress (At least once a year or so I’m on projects that require me to work at 150% capacity for 3-5 months).
• 20 years of late projects being “solved” by adding more people, or (better yet) adding more features.
• 20 years of weekly or biweekly drama, either in your development community of choice, or internal company drama.
• 20 years of explaining to your new manager (who used to be a secretary, long haul truck driver, or sales person) how software development works and how to manage a dev team.
• 20 years of trying to follow “just make it work” orders from a manager, when the technical debt of the codebase fights you every step of the way (while said manager breaths down your neck for status updates).
• 20 years of blame.
• 20 years of fighting bugs that happened when you upgrade from one version of something to the next.
• 20 years of dealing with ever changing (potentially contradictory!) demands.
• 20 years of software cycles (server -> desktop -> server ->…)
• 20 years of refining requirements into something that can be implemented.
• 20 years of too many deadlines.
• 20 years of job requirements that boil down to “want 25 year old willing to work for peanuts and move to San Francisco”.
• 20 years of explaining things and getting rebuffed because the manager “knows better”/thinks you are just being cranky/is “willing to take that chance”/tuned out after you said the word “problem”.
• 20 years of explaining that lines of code is not a good productivity measure (nor is individual completed agile story points per iteration, or pretty much anything else).
• 20 years of dealing with cranky nerds (who are sometimes cranky at you)
• 20 years of learning to dealing with, and questioning, recruiting pitches.

If you’ve been programming for 20 years you’ve seen all of these things and probably much more. Even if you’ve been working away in some state’s IT department, migrating some legacy system from Cobol to Java for the last 6 years your experience is still valuable. Because you’ve seen it all before, and picked up programming languages like normal people pick up how to use applications.

Managers and recruiters: if you can keep your ego in check a little bit, these people can be awesome to have on your team. Why? Because:

• They know what’s going to happen after that “crunch” weekend of people pulling 20 hour days for 3 days straight
• They know that your application internally is a buggy version of half of a 50 year old programming language
• They can sometimes take a super hard problem, solve it and prove the solution by relational calculus, or graph theory, or other really hard math.
• They are probably more willing to speak up than the young guns on your team.
• They’ve seen many things before: “Oh, reactor pattern framework? Oh, been there done that, 15 some years ago!”
• They can sometimes manage working on projects with 3 near-simultaneous deadlines.
• They probably have fulfilled more roles than just developer: product development person, client relations, recruiting, evaluating new hires, coach, manager, QA tester, conference organizer, salesman.

While years of experience isn’t a great measure of how good a programmer a person is, you know the person has been around the block and (probably) experienced the majority of things on this list — they certainly deserve better than, “get off my lawn, old man!” that the recruiting/startup community seems to exude.

My new IPad stand.

Yes those are eraser covers. Works pretty well. My iPad case has good landscape support but not portrait. Yet I was reading a portrait document.

Why Divvy

Sometimes I want a set of windows regularly across the screen, and sometimes I want the ability to resize my windows in arbitrary (but common) sizes.

With Divvy if I really want to create a 1 row size window in the middle of the screen, for Bog knows what reason I can. I have yet to see this ability in other “tiled window managers”.

With Divvy I can reposition and resize a window with a simple drag. I like it.

So, what do you want to do now?

Title my windows so there is no space left on the screen. This is partially so I can see everything at one time and be efficent. It’s also partially a work around around OS X’s fucked up desktop picture “management”, but whatever. (Seriously, try setting a desktop picture across spaces and with a 2+ monitor setup. It’s impossible.)

My Divvy Setup

Pictures of my Divvy setup are here.

I have a Microsoft 4000 with a full size keypad so I’ve set up these Divvy shortcuts to be activated via the numpad.

I also have enough oddball shortcuts to remember on my system, so these are not global shortcuts. I first bring up Divvy (Control-Shift-Space for me) then hit 9 on my numpad. Namespaces are good - let’s do more of those!

What this results in

I can quickly take two windows and tile them side by side down the full length of my screen.

I can quickly take a window and resize it to take up the top or bottom, left or right, side of my screen. I can go from “two full length side by side” windows to “four quadrant windows” in 3 seconds.

I can also mix it up: I often have 2 half length windows open on the left and one, full length, window open on the right.

Why Not?

Slate? When someone writes a 100 page PragProg book on it I’ll look into it. Trying to understand Slate I’m reminded of the last time I tried to look at X11 configure files.

Size-Up? I like to have the option to create arbitrary sizes. While my standard sizes are (now) 80% of my window management needs sometimes I want something custom.

Moom? Same thing: no arbitrary sizes.

Yes, because there’s always some new thing that’s super hard. No matter how much we advance the craft there’s always a new challenge in front of us, no matter if we conquered the previous challenges or not.

Take a flip answer to that question from various eras of computing:

• 1970s: “Yes, because structured programming”
• 1980s: “Yes, because event loop and/or graphical interfaces”
• 1990s: “Yes, because networking”
• 2000s: “Yes, because pointers”
• 2010s: “Yes, because concurrency and or asynchronicity

I can imagine the following answers for the next few decades:

• 2020s: “Yes, because gestures/motions”
• 2030s: “Yes, because genetic programming”
• 2035: “Yes, because maintenance programming”
• 2040s: “Yes because damn stupid AIs”

So yes, I see the learning curve of programming continuing to be a hocky-stick like curve. Yes, it’s relatively easy to teach yourself some HTML and Javascript and build something, then things get harder and harder as you learn more and more. Even if your able to do more and more with a lower and lower skill level there will always be hard problems to solve that need an experienced programmer, and that means well paid.

And this just rounds up the technical reasons. We’ll always have people who want software built but don’t know what they want; processes that - once you dig into them - become more and more complex; and companies that want to “push the edge of what tech can do”. These people problems - that have been the same since the early days of computing - won’t be going away.

I hear this argument a lot:

Oh, you’re unemployeed or can’t find a job? Just learn computer programming - I hear Microsoft is gagging for people.

As a professional computer programmer, let me weigh in:

I spent 2 years during high school learning to program after school. I started consulting part time during my freshman year of college for a local programming consulting shop. The first year or two at that job I knew very little (looking back on things now), struggling to learn the trade. I’m glad my employers were OK with that.

Fast forward 4 years, and I started my own consultancy business out of college. Doing computer programming consultancy is hard: in addition to having technical chops you also have to do project management, manage cash flow, deal with clients who fight you about money, know basic accounting, you name it. In my own practice there were a lot of years that were “famine” years, and only a few that were “feast” years.

So maybe you think that you can get a job as a programmer with a startup, because startups are what everyone thinks about now. In my experience you need two things going for you to get a job at a startup: 1) living in SF or NYC and 2) having 2-3 years of professional experience in their language of choice. If you don’t have both of those things the pickings are pretty slim.

The thing about startups is because the team size is so small they can’t (or don’t think they can) deal with bringing on a newbie programmer and the slower output they will have.

Please don’t do DevBootcamp or CodeAcademy and think that you can put our your consultancy shingle, working for clients directly. If you really want to go that route find a more experienced freelancer to subcontract with for a year or two, and learn from them.

In one way, the world of web development is easier than the programming world I grew up in (writing C and C++ programs for desktop computers), and perhaps with laser focus on web development you could be a good developer in 6 months. HungryAcademy thinks it can train people to do that, and I have seen amazingly talented people go far with very little (comparatively) training. Then again, there’s at least 4 languages to any moderately complex website (CSS, HTML, JS, and a server-side language), so there’s a lot to cover.

But certainly it’s not a matter of just picking up “Learn Web Development in 24 hours” and a day later being able to charge clients $100/hour for websites. Your looking at half a year at least, if not 2 years at least before companies take you seriously. And probably 3 or 4 years before you really should be taking on clients yourself.

And this is assuming you have the personality and passion for programming. Having tried to train a number of people to be programmers…. it takes a certain type of person to be really good. I’m sure this is as true of other fields as it is of say the Marines.

So sure — certainly if your unemployeed and looking to get an edge up and make a long-term investment in yourself (or just learn something fun!) maybe spending 10-30 hours a week learning programming (between job searching) is a good idea. Maybe in 6-12 months you can make something interesting that people like or think shows enough potential that they’re willing to take a chance on you. But it’s not a panacea.

Especially not when applying to Microsoft, where they want the best of the best.

Github Launch is pretty cool - a command line interface for Github.

There are two things bad about it:

1. It’s dog slow on Firefox
2. It’s entirely Javascript based - I can’t build a tool that sends a command to Github launch outside the browser.

I worked around issue one by typing my command in another window, pasting the result into Github Launch, then hitting return. This kind of sucked, but whatever.

Today I realized that mostly the Github launch commands were mostly just a clean interface around the RESTful URL structure of Github.

So I wrote a search Template for LaunchBar.

The search template looks like:

http://www.github.com/*

It does require a slightly different syntax than Github launch, but it’s also super easy.

Here’s how I use it:

1. Type: “Github” in Launchbar
2. Press spacebar to trigger the text input field
3. type rwilcox/github_my_activites
4. Press return, be taken to the appropriate repository.

Now, this doesn’t provide tab completion like the Github Launch bar, but tab completion didn’t work for me anyway (dog slow on Firefox).

It’s hard and annoying to embed images into Github Markup renders.

The great thing is that since everything in Github is markdown, this trick will work everywhere.

The trick is to use an external host to host your images, then link to them by putting a ! in front of a normal Markdown link statement.

But, ugh, external host, right?

NO

Here’s how to do it step by step:

1. Put the image for Github inclusion in your Dropbox
2. In the Finder (not PathFinder) right click the file and choose Dropbox -> Share File
3. Close the share web dialog that your browser opens and copy the URL
4. Add ?dl=1 to the resulting link

I like to keep my things out of the public folder, so I just use this basic sharing feature. However, I can’t script away the annoying part as apparently part of the link is link specific, not user specific. Source: Scroll down to the ‘for our advanced users’ section of this Dropbox tech note

Alternative Way

If you have your own web space there’s the awesome Automated Transmit Uploader on Github. Although part of me would rather use Dropbox for this… MAN this workflow is really easy.

BBEdit has this useful feature: it will strip whitespace added to end of lines OR it will remove all the whitespace from lines that are *just* whitespace.

This setting is at BBEdit -> Preferences -> Text Files -> Strip trailing whitespace.

Most of the time I have this turned on because most of the time I’m writing code in BBEdit.

Except, there are some languages (programming languages and markup languages like Markdown where whitespace is significant.

We can customize BBEdit’s behavior per language via the Languages preferences item. Select your language from the Custom Language preferences, as shown in the image.

You can set language level defaults for pretty much everything text related. What we’re interested in is the Files tab and the Strip trailing whitespace checkbox.

Now, as long as your language in the BBEdit document is set to Markdown it will no longer strip whitespace

I switched to Mutt about a year ago. Mutt is pretty good, but there are a lot of convenience that you give up.

One of these conveniences that you give up with out-of-the-box Mutt is really good, inter-mailbox searching. Sure, you can use a quick search feature to find messages in the current mailbox, but sometimes I’m not sure which mailbox I put my mail.

(You see, I organize my mail in an old-school one-folder-per-topic method)

Prerequisite: Use Maildir, not mbox

For any of the techniques I talk about to work you’ll need to be using Maildir, and not .mbox files. Here’s how you do that:

Use a tool like mb2md to do this.

You’ll also want to strip the leading dot from the generated file names (well, at least I did). See a bash script that does this OR make the change to mb2md directly.

The leading dot is not just annoying (“Where did all my mail go!!!” inducing) but Spotlight won’t index files that begin with a dot.

Wait, Spotlight?!! Yes

Search the OS X way: Spotlight

Now that you’ve used maildir to categorize these messages you can search for it via Spotlight.

The bad thing is that Mutt saves its mail messages with not-human-readable names, so when you do find some messages you’re looking for you don’t really know which exact message is the one you’re looking for.

Enter QLStephen: a QuickLook plugin that allows you to look at plain text documents.

I want to find only email messages, not my search term in other documents!!!

HoudahSpot lets you save a search template. Thus you can create one for email. Here’s mine:

Text Content contains (whatever)
Where: ~/Mail

HoudahSpot is now what I use for searching mail.

Search the Unix Way: notmuch

If you don’t like Spotlight, or run your mail on Linux, great. There’s a tool called notmuch that is a mail indexer which may work for you.

There’s an awesome blog post: Mutt mail indexing on steroids! that I was working through before I found that Spotlight works.

Regardless, notmuch doesn’t support .mbox files, something I ran into early in my testing.

I do all my Ruby on Rails development with Vagrant, keeping separate virtual machines for each project I may be working on.

I use SSH magic to forward the proper ports up to my server. I’ve been using this setup for 18 months now, typing in localhost:3001 to visit my Rails server on my development machine.

Enter pow: pow’s goal is to make it really easy to visit a project server on your local machine: type myproject.dev into the browser and Pow finds myproject, boots the Rails stack and serves you a page.

The URL looks awesome and can give everyone a unified URL for development that looks great.

So, today I set up my machine to have Pow-like abilities: forwarding myproject.dev to my development server.

Step 1: Edit /etc/hosts

Edit /etc/hosts and add this line:

127.0.0.1   myproject.dev

Note that there’s a tab between 127.0.0.1 and myproject.dev

Step 2: Turn off any web server you’re running on your machine

We want to use Port 80, so nothing else can.

Step 3: Forward port 80 to port 3000 on remote machine

The /etc/hosts entry lets us go to myproject.dev and be taken to our local machine. We want to go to port 80 on our local machine and magically be taken to port 3000 on the remote machine. This is easy with some SSH magic!

Here is the SSH command to make that magic happen:

sudo ssh -F ~/.ssh/config -l vagrant my_remote_host.com  -L 80:my_remote_host.com:3000

The sudo is here because port 80 is a privileged port and we need root access to forward it.

Everything all together

Here’s how everything works:

When you visit myproject.dev in the browser it will make a request for port 80 on your local machine. SSH will see that connection and forward it to port 3000 the remote machine

Conclusion

There are some massive annoyances using this approach, which could be solved by making the remote development machine responsible for translating port 80 into port 3000. I might look into this: having two SSH connections open (one opened for my normal work and one opened via `sudo ssh) is annoying. I’ll post an update here when I find a better solution to this problem.

Even if you want to swap out Step 3 with some other step, you still need to do steps 1 and 2.

Today I decided to tag my blog posts, so I could find them later.

Unfortunately that had an side affect: all the posts look like they were posted today.

But they weren’t.

Posts from today onward will be tagged so I can find them, so this shouldn’t happen again.

A vision of what San Francisco looks like in 2075, when Big Startup leaves SV like Big Auto (effectively) left the Rust Belt

Today I discovered that my grandfather kept a diary. Not a very big one, and not very often. Files encrypted with consumer grade encryption of the day, but the house computer cracked it easily enough.

Interesting stuff. The entries in the middle were the most interesting - he had some thoughts on Big Startup.

Big Startup. There’s a boat that sailed about 30 years ago. Taking with it half the city’s tax base, and what population that could leave. Mom and Dad felt tied to their mortgage. I talked to them about it a few years ago, and I guess when I was born people actually cared about their mortgages. Their house was only underwater by 25% when I was born, they thought they could sell and get out.

Totally different time.

I read some of Grandad’s journal entries while waiting for the bus, at the ruins of Civic Center. One of the panoramas of inside of the ruined Center has been my u-desk background picture for about a year now.

Wait, I see the bus.

Ok, back now. The driverbot didn’t like the crumbled $50 I handed him for my fare, but it’s green and registers as money all right.

Last $50 I had. I’m going to have to bum rides off Liz, or hope to not get caught anywhere after dark.

Man.

Ahh, ok, home now. Home in that technically it’s for sale for $15,000, but nobody is stupid enough to buy it in this neighborhood of foreclosures and abandons, so might as well be mine. Been here maybe a year. Fridge, plumbing, computer and net still worked, so that’s an upgrade from the old place.

I added some warning detection too. Don’t really want someone to squat my squat. Nobody expects a 30 year old Rumba to be outfitted with a webcam, a stun gun, and link to the house computer. That’s what I say anyway.

Today I did some repair work down in the Mission. One in ten items that I repaired I could keep, as payment. My backpack’s half full of antiques. Mostly crap.

Gods, when was my last actual paying job? Last year when I worked the hotdog cart at the Wharf? Or was that the few months I spent ferrying tourists from SFO, keeping the saps out of gang territory?

What I hated about that job was that everyone wanted to see the Golden Gate. Stupid idiots. For one, that’s right in the middle of gang territory, for second, all the locals know the real reason why they still call it the Golden Gate - everyone pees on it.

But I guess corporate drones have to go into Facebook HQ sometimes too.

Between coding moving mostly to Mexico and South America, and the new gold rush in biotech, I’m surprised anyone comes here.

Oh, and if I see another ad for Biogenering Bootcamp I’m going to go crazy. Teach me To Bio my ass. A decade or so ago was the big u-cyber gold rush in Japan. Twenty years before that it was robotics, thirty years before that it was house computers. Before that… well, whatever.

The robotics rush - the last gasps of Big Startup.

Dad saw most of that.

Up through the house computer boom and including some of the robotics rush, you had people for the most part working for money, almost all the time. Dad pulled down a salary most of his working life.

Now a days people are lucky to get a job compensated with some stock options. Paid with actual money? I laughed the first time Dad told me that.

“Yeah, sure Dad — today half the people in this city are working 60 hour a week jobs in exchange for cafeteria meals and some blankets, and you want me to believe that you got paid money?”

Dad got sad after that. I was almost sorry I brought it up.

Gotta go. Ping from Liz - we’re doing some gold mining tonight. Hoping to get some good drops that we can sell to some engineer’s kid in Mexico or something.

I did the math today.

If I entertain 12 pre-funded startups a year, and give each 30 hours free of work, and do this every year for 10 years… I’ll have worked on 120 projects and given away in the neighborhood of $300,000 to $500,000 worth of work.

That suddenly sounds like a *lot* of risk.

It’s also about $30,000 to $50,000 worth of income per year that I’m not making. Which is a sizable chunk.

“I think you’re struggling with this project”

I’ve been in this case before. Recently, even.

Maybe I can’t put as much time into the project as was expected, or I’m struggling (or appear to be struggling) with Getting Things Done.

This last time is hard, because struggling and appearing to struggle are the same thing to external eyes. Maybe you know you’re not struggling, just surrounded by difficult people, but it’s hard to prove that…

I’ve written about this before in There’s no way to be blameless.

Today I have more concrete advice about handling situations where you’re struggling / are perceived to be struggling.

Start a developer chat

Start a developer only chat. room Use Skype’s group chat feature, AOL Instant Messager’s chat, MSN, I don’t care - just use whatever. Just developers. Call it “The Nerd Chat” or something so it’s clear that those not invited would be super bored in the chat room.

The protip here: don’t ask management about this, just do it.

On one of my projects I bugged management for two or three weeks about getting a Yammer or Campfire account. No action. I finally did it myself (started a Skype room) and Management didn’t find out about it until a few weeks later when someone let it slip. By that time the usefulness had been proved.

In this chat you can ask things like, “Hey, has anyone done anything in class Foo lately?” or, “Does someone know how X works?” and get an answer from other team members. Instantly, without asking everyone individually.

Break bugs out into behaviors before you start work

I’m a huge fan of behavior driven development. Here’s how to do it in a non-behavior driven development shop:

1. Get the ticket from the ticket tracker
2. Make a list of the behaviors you think it should have, based on ticket information, possibly user expected conventions, and what you know about the business domain of what you’re building.
3. Take this list to the appropriate person. (Manager? Product Owner? Business Analyst? Whatever’s appropriate for your project). Discuss the behaviors you derived - are you on target? Off target because they don’t want X? Maybe you suggested something off target but that spurned a refinement they did want, or launched them into a domain conversation that was important.
4. From that list of behaviors you now have a todo list for that ticket. If these make the ticket Big, break off some tasks into new tickets.

Example: let’s say you have a ticket, “Create header for the site, with this mockup”.

You get the ticket and think about the behaviors based on what’s in the ticket and what you know. Let’s say it breaks down to:

1. Should have “home”, “about”, and “blog” links
2. links should change color and background when user hovers over them
3. has our logo in upper left hand corner, 24 px away from top and left side
4. should clicking the logo take you back to the home page????

The fourth one wasn’t explicitly listed in the ticket, but it’s a behavior most web pages have.

You then start a conversation with the ticket’s owner. You learn that point 4 is right, and that a recent internal conversation has determined that the background color should be green.

Now you have five behaviors - todo items for the ticket. Implement these: these are your agreed upon deliverables.

See also: Writing Behaviors

Protip here: If you’ve done this once or twice and are still getting “you’re overcomplicating this” feedback, ok, great. Maybe you are. With the user expected conventions, well there are always bug reports from the field. It feels bad, but hey - the smug satisfaction about being right about something after all is so sweet.

Use the bug tracker

While you’re working through the header bug you find an issue: you haven’t been given art assets! You have no logo for the upper left.

File a bug, and somehow mark your “implement header” task as dependent on this new bug. Now the product owner has visibility as to why your header task isn’t getting done.

“Build logo” might not be part of your job. Or maybe it is, but someone forgot about it. I’ve had this happen, especially early in a project. In order to build this bit of functionality the client assumes the universe around it has been constructed… except they haven’t actually scheduled that bit for construction yet.

“Todo items should send an email when they are due” seems simple enough, until you realize that Todo items don’t actually have a due date yet. Maybe “todo items should have a due date” is scheduled for a month from now and nobody realized you need it now, or maybe that detail was forgotten.

Maybe you’re working through your header bug and find out that you don’t have an automated way to generate CSS from the SASS you are using. This is a nice to have, and certainly a pain to do it by hand. It’s a new idea in the system. File a bug. Hopefully you’ll get to it some day, you won’t get distracted by shaving this yack while working on this ticket.

You may need to consider phrasing the need in a business friendly way. This really depends on the team make up, but I’ve often seen situations where the product owner (a non-programmer) is making the schedule - of course they will deprioritize development tasks.

“We need an automatic system to regenerate our assets so dev and QA know they are looking at the latest work”. That’s (more) likely to get businesses attention.

Start with small tickets you know you can nail

You may end up getting yelled at for this, but it’s worked for me in the past.

Start your day off with some small tickets you know you can get done, even if chaos breaks out in the office. Nail those: get some momentum and confidence on your side.

I’ve had people yell at me before for this: “Tackle the big things first, so you know you’ll leave yourself enough time to get the big thing done”. Sure, but sometimes (especially if there’s a lot of drama in the air) I feel better getting one or two items done (and thus proving that I did something) vs getting half way through a large ticket and making no visible progress that day (aka: I didn’t change a ticket’s state to “Ready for QA”).

Bug people

Has QA been sitting on your completed tickets for three weeks before doing anything? Make sure people know this loudly, clearly, and as high as your voice can reach.

Do you need to know what’s scheduled for “beta release”, especially as the published deadline for “beta release” inches closer? Bug people!

Make sure bug not just your manager (who could be intentionally or un-intentionally) just dumping your requests into the trash, but the project owner. Make sure the technical lead of the project knows that you’re concerned about the success of the project (especially when you don’t know what you have to build!)

It’s possible people are just cranky

You did read that No Way to be Blameless entry, right? Sometimes people will just be cranky. Too many late nights (for anyone), too much stress, or they just need to rebuild some of their trust in you. Sometimes patience is important. Sometimes you have to take a breath and file the conversation away for later.

Walt Disney is credited with saying:

“I don’t make pictures just to make money. I make money to make more pictures.”

I just figured why I make money - or rather, why I need to charge for my programming service:

“I don’t do programming for people just to make money. I make money so I don’t have to take a real job” - Ryan Wilcox

Where “A real job” means worrying about being in my seat by 8:00 AM, taking lunch precisely from 12:30 to 1:00, having quarterly evaluations, worrying about being a Software Engineer I vs a Software Engineer II, if today is a tie day, or forgetting to wear dress shoes and worrying about getting written up.

Today I listened to Back To Work, Episode 71: Sorry To Lose You As A Son, about passive aggressive behavior and I remembered why I don’t want to go back to an office job.

A few years ago, when I was working for a client in DC, we had to go into the office, with typical office politics.

For example, there was a water cooler at the end of the hall. One day we discovered it and helped ourselves to some water. Turns out that water was refilled by private office workers pooling together to buy the water - it was a private water cooler. We didn’t know this, but were informed of this the next day by our manager (after, I’m guessing, she got a nasty-gram).

This seemed idiotic to me: if it takes 15 minutes to run down to the corner store to buy a bottle of water, times several thousand people, times what each person’s time costs, it would be 100 times cheaper for the organization to provide water coolers to their employees. But, typical corporate-think, nope.

So, I either charge people for my services, or I go find a corporate job. Seems like a no-brainer.

I moved to the Mutt email client in mid 2011. I wrote up some tips and tricks for using Mutt and OS X.

The interesting and somewhat annoying thing about Mutt is that sometimes features are so well hidden you only hear whispers and rumors about them.

This is one of those times.

Goal: Attach files to an email message without using Mutt’s file browser

Mutt’s has a built-in file browser which it uses to select mailboxes, save attachments, and attach files to the current outgoing message.

The reality of email in 2012 is that we’re attaching files to messages a lot. Ok, maybe “several times a week”, but hey, we’re on OS X - I don’t want to select files using some console interface.

Mutt has a solution: the Attach pseudo header

One of the “whispers and rumors” features of Mutt is the Attach pseudo header.

When it calls your editor of choice Mutt lets you edit the email headers of that message. This feature is the basis of the Attach pseudo header.

First, to get this to work, you must have edit_headers set to yes.

With edit_headers set, when you open your editor you’ll see From, To, Subject etc.

Deep in the Mutt documentation (manual section 2 subsection 2.4, second sub-sub section it talks about the Attach: header.

So, here’s how you use this:

1. Create a new message or reply to an existing one in Mutt. This will trigger your editor
2. If you want to attach a message, directly underneath the last header data you see, add a new new line that starts with Attach:. Insert the path to the file to attach to the message.

Example of Attach in use

From: Ryan Wilcox <ryan@example.com>
To: bob@example.com
Cc: 
Bcc: 
Subject: Hello world
Reply-To: 
Attach: ~/Documents/the.doc

When you exit your editor the.doc will be attached to your message: you should see it in the attachment list in the Mutt outgoing message view.

About whitespace

Like pretty much every Unix utility, Mutt’s Attach feature doesn’t handle whitespace in filenames. You must escape this whitespace using the \ character.

Introduction

This is another “things you should do when starting a Django project” blog post.

This type of blog post has been done several times:

• Andrew McCloud’s Django 1.4 Project Template
• Steve Losh’s Django Advice
• Andy McKay’s First things to do on a Django Project

I hope to build on the advice given from these articles. If you’re a Django newbie maybe it’s best to start there.

This article also assumes you’re doing the basic Django best practices like:

• All your templates should in someapp/templates/someapp/
• Make a tests folder in your app
• Set up your TEMPLATE_DIRS folder to be os.path.join(SITE_ROOT, 'templates'), but store your templates in each individual app (thanks django.template.loaders.app_directories.Loader!)
• Use a Virtual Env

If this was a college class it would be a 200 level class: it assumes you have the very basics, and need to see the next level of best practices above that.

Step 0: Have a good Vagrant configuration / Puppet Manifest

All my projects run in their own virtual machine, thanks to Vagrant. Vagrant, Puppet and Django play well together.

Here are some lines from a recently constructed Puppet manifest for a Django project:

include mini_postgres
# postgres support provided by <https://github.com/rwilcox/puppet_mini_postgres>

include python::venv
# Python support provided by https://github.com/uggedal/puppet-module-python

include nodejs
# NodeJS support provided by https://github.com/puppetlabs/puppetlabs-nodejs
# (see the modulefile for this project for prereqs)

package {"yui-compressor":
    ensure => present
}

package {"coffee-script":
    ensure => present,
    provider => 'npm',
}

package {"less":
    ensure => present,
    provider => 'npm',
}

python::venv::isolate { "/home/vagrant/.venvs/django_project":
    requirements => "/home/vagrant/Development/django_project/requirements_dev.txt"
}

package { 'foreman':
    ensure   => present,
    provider => 'gem',
}

class { "python::dev": version => "2.6" }

This will automatically install node.js and various infrastructure packages (Coffeescript, LESS) that we’ll want to use.

I consider any investment into Puppet manifests a investment into deployment target independence. If you start your Django app out on Heroku that’s great, but you may outgrow Heroku (and probably at a very inopportune time) - the Puppet manifests you create for your dev environment can be used to help boot up your production environments.

Step 1: Separate developer settings from production settings

Create a settings_local file

I have a settings_user.py file I create for every project. I add this file to the .gitignore file and add a settings_user_sample.py file to the repository. The pattern is that everyone copies settings_user_sample.py, renames it and configures everything appropriate to their machine.

settings_user.py includes things like database settings and developers specific apps.

Here’s how you include this into the main settings.py file:

USER_INSTALLED_APPS = ()

try:
    if DEBUG:
        from DjangoProject.settings_user import *
        print "using user settings"
    else:
        use_local_settings = False
except ImportError, e:
    use_local_settings = False

Wait, developer specific apps??

Yes. Apps like the Werkzeug Debugger and Django Extensions are amazing developer tools… but I don’t want them anywhere near my production site. Thus they are installed in the settings_user file like so:

USER_INSTALLED_APPS = (

    'django_extensions',

)

Notice how we have defined USER_INSTALLED_APPS in our settings.py file before we tried to import settings_user.py? That’s so we could do this, in settings.py

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',

....

    # Uncomment the next line to enable admin documentation:
    # 'django.contrib.admindocs',
) + USER_INSTALLED_APPS

Wait, don’t developer specific apps mean different required packages?

Absolutely. Thus I have the following requirements.txt infrastructure:

DjangoProject/
    requirements_common.txt
    requirements_dev.txt
    requirements.txt

requirements_common.txt is where common packages are declared: modules that we want both in production and on developer machines. Most of the Python packages in the project live in this file.

requirements_dev.txt is where developer specific packages go. This file looks (something) like:

-r requirements_common.txt

Fabric==1.4.3
django-extensions==0.9

requirements.txt are where production only packages go. Really this file should just look like:

-r requirements_common.txt

Step 2: The Asset Pipeline / Twitter Bootstrap Era

I do most of my work in Rails where we have an Asset Pipeline - a place to organize CSS and Javascript assets and deal with the precompilers for both of those languages.

Asset collection in Django

Django-Pipeline was the closest thing I could find to the Rails asset pipeline. It will handle compressing, concatenating, Javascript and CSS transpilers.

I broke the various django-pipeline settings out into a settings_assests.py file which I imported from settings.py.

Vendoring CSS/Javascript assets in Django

In Rails we put assets from external sources in the vendor/assets folder. The best way to do this with django-pipeline seems to be to structure your static folder like so:

main_app/
    models.py
    static/
        lib/
            js/
                lib/
                    jquery-someplugin.js
                main_app.js
            less/
                main_app.less
                lib/
                    bootstrap.less
                    close.less
                    ...
    views.py
    templates/

Then in your PIPELINE_CSS setting render those files with:

    'source_filenames': (
        'less/lib/bootstrap.less',

Start with a good base.html

Even after you’ve pulled the latest .less files from the Twitter Bootstrap project and manually put them in the lib folder, you’ll still need the Bootstrap boilerplate for your base.html file.

Thankfully the bootstrap-pipelined project template takes care of that. Here’s base.html. There’s also 500.html files and 404.html files.

Deploying assets on S3

Django-storages works great with django-pipeline, once you create a S3PipelineStorage class that inherits from PipelineMixin, CachedFilesMixin, and S3Storage (like the good documentation tells you to. In my experience this needs to be in a separate module.

With this properly set up, python manage.py collectfiles will upload files to S3.

Step 3: Testing

Read my article on testing your Django URLs and thank me later.

Step 4: Think about deployment

Create a fabfile (even if you’re just deploying to Heroku)

Heroku is great because you don’t have to think about normal operation tasks - no provisioning servers, setting up Apache, etc etc. Just git push and you’re done.

Except not really. When deploying Rails apps to Heroku I always forget some little thing: migrating the database, cycling the web server, you name it.

To avoid forgetting anything I made a fabfile.py:

from fabric.api import run, local

def deploy():

    # get the latest code from the server and push it
    # if we have a merge conflict then this will error and we need to resolve it
    local("git pull github master")
    local("git push github master")

    local("heroku maintenance:on")

    local("git push heroku master")
    local("heroku run python manage.py syncdb")
    local("heroku run python manage.py migrate")

    local("PROD_OVERRIDE=1 python manage.py collectstatic")
    local("heroku maintenance:off")

The PROD_OVERRIDE forces Debug to be False so that we properly upload the static assets to S3.

Error monitoring

Sentry is an error capturing and reporting tool from the guys behind Disqus. Sentry comes in a hosted and bring-your-own-server varieties.

I like Sentry because I can use it on Django projects, Rails apps, node.js apps, etc.

Heroku has an addon for Sentry that make error reporting for your Heroku app easy.

A sample logging configuration

This logging configuration will log errors to both Sentry and the console, except when running in Debug = True situations (where it will just log errors to the console)

LOGGING = {
    'version': 1,
    'disable_existing_loggers': True,
    'filters': {
        'require_debug_false': {
            '()': 'django.utils.log.RequireDebugFalse',
        }
     },
    'root': {
        'level': 'WARNING',
        'handlers': ['sentry'],
    },
    'formatters': {
        'verbose': {
            'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
        },
    },
    'handlers': {
        'sentry': {
            'level': 'ERROR',
            'class': 'raven.contrib.django.handlers.SentryHandler',
            'filters': ['require_debug_false']
        },
        'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
            'formatter': 'verbose',
        }
    },
    'loggers': {
        'RoomHunt': {
            'level': 'DEBUG',
            'handlers': ['console', 'sentry'],
            'propagate': False,
        },
        'sentry.errors': {
            'level': 'DEBUG',
            'handlers': ['sentry'],
            'propagate': False,
        },
    }
}

On Heroku

Pull in dj-database-url - it takes care of your database configuration.

Conclusion

Django is nice in that it doesn’t assume anything. Rails is “convention over configuration”, while Django takes an almost opposite approach. In practice this just means that the Django community must come up with its own best practices instead of making the Django BDFL ruler of decrees.

However, this does mean that it takes a while to get a new Django project set up. Hopefully this article will cut down your set up time for new projects.

And A plug

Have a Django project and need some help? I have some availability - feel free to contact me on my business website and maybe I can help you out!