Django 1.5 has experimental support for Python 3. This is great news, and means that we should all be upgrading our Django apps to be able to support both Python 2 and 3. Django has some documentation on how to go about this process. I recently went through this for one of my apps and, while the changes to the app itself were easy enough, I had to figure some things out myself for my testing setup.

I like to use Travis for continuous integration.

• It's easy to set up (I usually just copy a .travis.yaml file from another project and tweak a couple settings)
• It integrates really well with GitHub, even automatically running tests against people's pull requests

The .travis.yml for my projects usually looks something like this:

language: python
python:
  - "2.6"
  - "2.7"

env:
  - DJANGO="Django==1.3"
  - DJANGO="Django==1.4"
  - DJANGO="git+git://github.com/django/django.git@master#egg=django"

# command to install dependencies
install: pip install -r requirements.txt

# command to run tests
script: python setup.py test

Then I'll list my requirements in requirements.txt, and put a test command in setup.py. Travis, when given this configuration, will run tests in six different environments:

• Python 2.6 with Django 1.3
• Python 2.6 with Django 1.4
• Python 2.6 with Django's trunk
• Python 2.7 with Django 1.3
• Python 2.7 with Django 1.4
• Python 2.7 with Django's trunk

Up until now, this has served me well.

With the introduction of Python 3 support, I introduce an additional Python environment for my tests:

python:
  - "2.6"
  - "2.7"
  - "3.2"

Now Travis will run tests in 9 environments. However, in some of those environments, the tests will fail, even after I've done the work to make my app Python 3 compatible. So what happened?

The environments that Travis sets up are:

• Python 2.6 with Django 1.3
• Python 2.6 with Django 1.4
• Python 2.6 with Django's trunk
• Python 2.7 with Django 1.3
• Python 2.7 with Django 1.4
• Python 2.7 with Django's trunk
• Python 3.2 with Django 1.3
• Python 3.2 with Django 1.4
• Python 3.2 with Django's trunk

But notice that, even though my app may be compatible with Python 3, Django 1.3 and 1.4 are not. So, these environments will produce errors when even trying to install Django.

My solution to this is to introduce a couple of checks in my install and test running scripts. First, I have to make my Travis configuration a little smarter. I use a separate install script that checks whether the Django version is compatible with the Python version, and immediately exits if not:

.travis.yml:

language: python
python:
  - "2.6"
  - "2.7"
  - "3.2"

env:
  - DJANGO="Django==1.3"
  - DJANGO="Django==1.4"
  - DJANGO="Django==1.5"
  - DJANGO="git+git://github.com/django/django.git@master#egg=django"

# command to install dependencies
install: ci/install.sh

# command to run tests
script: python setup.py test

ci/install.sh:

#!/bin/bash

PYTHON_VERSION=$(python --version 2>&1)

if [[ "$PYTHON_VERSION" > "Python 3" ]]
then
  if [[ "$DJANGO" < "Django==1.5" ]]
  then
    echo "Cannot install $DJANGO on $PYTHON_VERSION"
    exit
  fi
fi

pip install six==1.2.0 mock==0.7.2 $DJANGO --use-mirrors

That line if [[ "$DJANGO" < "Django==1.5" ]] in the ci/install.sh script does a lexicographic comparison against the $DJANGO environment variable, so even when I'm using the Django trunk from git, it evaluates correctly as being greater than "Django==1.5" (upper-case "D" is less than lower-case "g"). Now, Travis will immediately exit (with an implicit exit code of 0, indicating everything is alright) from installing my dependencies before installing six and Django, which is important for the next step.

Next, in the script that runs my tests, I want to check whether Django has been installed:

runtests.py:

try:
    import six
    from django.conf import settings
except ImportError:
    print("Django has not been installed.")
    sys.exit(0)

Here, it tries to import those things that did not finish installing above. If it cannot import them, then it will immediately exit. The exit code 0 tells Travis that all the tests for this environment have passed, which, since there are no tests for this [invalid] environment, they have.

Now my project is ready for testing with Travis under Python 2 and 3!

Update (11 March 2013):

I just learned about the matrix section of a Travis config file! See here for an example.

Lately I've been creating a fair number of APIs that store some JSON in a field in a database. Often this is a good reason to use a document database like Mongo or Couch, but sometimes a relational DB will do you better (e.g., if it's just a small amount of data for each row that is free-form, but the relational aspects are many and important). If you are using a relational DB with a JSON field, you will at some point have to deserialize that JSON for use as data.

I'm concerned with the speed of my APIs, so I began wondering what the best way to do this was. When returning a single record, there aren't many choices. I load the record, pull out the JSON field value as a string, deserialize it, and put it back on as a dictionary. However, when returning a potentially large set of records, my first inclination was to process each record individually, and compose them all into a list. Something like the following:

    def get_record(...):
        record = execute_query(...)
        data = process_record(record)
        return data

    def get_many_records(...):
        records = execute_query(...)
        data = [process_record(record) for record in records]
        return data

    def process_record(record):
        json_str = record['blob']
        record['blob'] = json.loads(json_str)
        return record

    def get_many_records(...):
        records = execute_query(...)
        json_str = '[%s]' % ', '.join(record['blob'] for record in records)
        blobs = json.loads(json_str)
        for record, blob in zip(records, blobs):  # izip for Python 2.x
            record['blob'] = blob
        return records

    from timeit import Timer

    list_str_code = """
        import json
        list_str = '[' + ', '.join(['{"a": 1}']*1000) + ']'
        data = json.loads(list_str)
    """

    str_list_code = """
        import json
        str_list = ['{"a": 1}']*1000
        data = [json.loads(dict_str) for dict_str in str_list]
    """

    list_str_timer = Timer(list_str_code)
    str_list_timer = Timer(str_list_code)

    print list_str_timer.repeat(number=1000)
    print str_list_timer.repeat(number=1000)

Tomorrow, November 2nd, there is going to be a general strike in Oakland. Though I support #occupy, I'm not going to be on strike, for a few reasons:

1. I also support the organization I'm working for, think they're doing good work, and feel that it is not incongruous with #occupy (though it's certainly not the same)
2. I don't actually work in Oakland

• Help out #occupy tech support.  I've come across requests from several #occupy communities for additional IT hands (I don't know whether there is a central place where these requests are posted). If you feel so inclined, volunteer a little time to maintaining an #occupy web site, or posting minutes, etc. (if anyone knows more about what tech needs #occupy communities have, please speak up).
• Help create logistical tools. The #occupy communities that I've seen are little micro-societies and, like any society, have a number of logistical considerations and concrete needs. Are there tools that could help? For instance, I liked the idea of needsoftheoccupiers which they described as "a wedding registry meets Freecycle for the Occupy movement".
• Do something creative/educational/inspirational. #Occupy has people paying [at least superficial] attention to issues of economic [in]equality and social participation.  That doesn't happen too often, so take advantage of it by creating a visualization or an info-app that awakens peoples imagination and helps them see things that they wouldn't normally.  Things like BankMigration map and Slavery Footprint come to mind.

Update 23 Jun 2011: I have renamed the django-model-filters package django-model-blocks. It now lets you easily override the template used for a given model. Check out the changes on Github or PyPI.

Tonight I'm writing my first Django custom filter. The problem I'm trying to solve is that I want generic templates. For a given model I want to be able to set up browseable index and detail pages with minimal effort. As it stands now, say I have the following model:

    ...
    class PepulatorModel (models.Model):
        serial_number = IntegerField()
        height = IntegerField()
        width = IntegerField()
        manufacture_date = DateTimeField()
        color = CharField(max_length=32)

        def __unicode__(self):
            return u'Pepulator #%s' % self.serial_number
    ...

Now say I want my users to be able to browse my pepulators in a simple way, with the following caveats:

• They cannot edit pepulators, only view (rules out admin app)
  
• I want to define the URL structure (rules out databrowse) to be something like:
  

  
      http://www.mysite.com/pepulators/
      http://www.mysite.com/pepulators/?color=red
      http://www.mysite.com/pepulators/012345/
  

• I want to specify the base template so that it integrates well with the rest of my project (also rules out databrowse)
  

Currently, I can use the generic views ListView and DetailView, but I still have to write templates that go something like this:

    {% extends base.html %}

    {% block content %}
        <header>
            <h1>Pepulator #{{ pepulator.serial_number }}</h1>
        </header>

        <div>
            <span>Serial Number</span>
            <p>{{ pepulator.serial_number }}</p>
        </div>
        <div>
            <span>Height</span>
            <p>{{ pepulator.height }}</p>
        </div>
        <div>
            <span>Width</span>
            <p>{{ pepulator.width }}</p>
        </div>
        <div>
            <span>Manufacturer</span>
            <p>{{ pepulator.manufacturer }}</p>
        </div>
        <div>
            <span>Color</span>
            <p>{{ pepulator.color }}</p>
        </div>
    {% endblock %}

Okay, a bit verbose, but it's not going to kill me. However, now say I want to change some of the fields on my model. Well, then I have to remember to change the fields in my template as well (error-prone — this is why you don't violate DRY without good reason).

All I wanted was a simple view of my model!

So, I considered making an app that was leaner than databrowser and just provided generic templates to go with generic views. I found myself having to extend the generic views anyway, though, because there's no way to access a model instance's fields and field names without explicitly feeding them to the template's context. Then, I gleaned some inspiration from uni_forms: I'll make filters!

Now my plan is to be able to say, using the example of the Pepulator detail view above:

    {% extends base.html %}

    {% block content %}
        {{ pepulator|as_detail_html }}
    {% endblock %}

Sublime. (This must exist somewhere; but for now, I can't find it.)

So, I start off by creating my app

    $ python manage.py startapp generic_templates

Now, from the documentation on creating custom tags and filters, I see I should create a templatetags direcotry in my app. In here I'll put an __init__.py file and a module called generic_filters. This way, when I'm done, to use the filters, I'll put near the top of my template file:

    {% load generic_filters %}

I decided to start with the detail filter (as_detail_html), and to write a test first. I know generally what I want this to do, so I write the following test:

    """
    Test the generic filters
    """

    import datetime

    from django.test import TestCase
    from mock import Mock

    from django.db.models import Model, IntegerField, DateTimeField, CharField
    from django.template import Context, Template
    from generic_templates.templatetags import generic_filters as gf

    class DetailHtmlFilterTest (TestCase):

        def setUp(self):
            # Create a sample model
            class PepulatorModel (Model):
                serial_number = IntegerField(primary_key=True)
                height = IntegerField()
                width = IntegerField()
                manufacture_date = DateTimeField()
                color = CharField(max_length=32)

                def __unicode__(self):
                    return u'Pepulator #%s' % self.serial_number

            # Create a model instance
            now = datetime.datetime.now()
            self.m = PepulatorModel(
                serial_number = 123456,
                height = 25,
                width = 16,
                manufacture_date = now,
                color = 'chartreuse',
            )

            # Mock Django's get_template so that it doesn't load a real file;
            # instead just return a template that allows us to verify the context
            gf.get_template = Mock(
                return_value=Template('{{ instance|safe }}:{{ fields|safe }}'))


        def test_model_format(self):
            """Tests that a given model is formatted as expected."""

            expected_detail = (u"Pepulator #123456:[('serial number', 123456),"
              " ('height', 25), ('width', 16), ('manufacture date', %r),"
              " ('color', 'chartreuse')]") % self.m.manufacture_date
            detail = gf.as_detail_html(self.m)

            gf.get_template.assert_called_with('object_detail.html')
            self.assertEqual(detail, expected_detail)

In short, set up a model and an easy template, and check that the template is filled in correctly. Of course, since I haven't yet written my filter, this fails.

This (as_detail_html) was a straightforward method to write, but I did get tripped up because of the poor documentation available on Models' Meta classes. Here's the first go at the filter:

    from django.template import Context, Template
    from django.template.loader import get_template

    def as_detail_html(instance):
        """
        Template filter that returns the given instance as a template-formatted
        block.  Inserts two objects into the context:
          ``instance`` - The model instance
          ``fields`` - A list of (name, value)-pairs representing the instance's
                       fields
        """
        template = get_template('object_detail.html')
        fields = [(field.verbose_name, getattr(instance, field.name)) 
                  for field in instance._meta.fields]
        context = Context({'instance':instance, 'fields':fields})
        return template.render(context)

One other thing: I actually want to be able to use the filter in my templates, not call it directly in my code. I'm new here, so I write another test to make sure I understand what's going on:

        def test_filter_is_registered(self):
            """Test that the filter can be used from within a template"""

            template = Template(('{% load generic_filters %}'
                                 '{{ pepulator|as_detail_html }}'))
            context = Context({'pepulator':self.m})

            expected_detail = (u"Pepulator #123456:[('serial number', 123456),"
                " ('height', 25), ('width', 16), ('manufacture date', %r),"
                " ('color', 'chartreuse')]") % self.m.manufacture_date
            detail = template.render(context)

            gf.get_template.assert_called_with('object_detail.html')
            self.assertEqual(detail, expected_detail)

And it turns out all I have to do to satisfy it is change my module in the following way:

    from django.template import Context, Template, Library
    from django.template.loader import get_template

register = Library()

@register.filter
    def as_detail_html(instance):
        ...

Now I have a working object detail template. Yay! I figure I'll do the list the same way.

More on Github: https://github.com/mjumbewu/django-model-filters

A system is a big black box
Of which we can't unlock the locks,
And all we can find out about
Is what goes in and what comes out.
Perceiving input-output pairs,
Related by parameters,
Permits us, sometimes, to relate
An input, output and a state.
If this relation's good and stable
Then to predict we may be able,
But if this fails us—heaven forbid!
We'll be compelled to force the lid!

Even though it appears that the world has not yet heard of it (at the time I am writing this, all Google can tell me is that there is a 15 second say-nothing promotional video, and that I and one other person have tweeted it), I'm looking forward to checking this out:

Seventeen African nations gained their independence in 1960, but the dreams of the independence era were short-lived. Africa ... states of independence tells the story of some of those countries - stories of mass exploitation, of the ecstasy of independence and of how - with liberation - a new, covert scramble for resources was born.

Africa ... states of independence - Programmes - Al Jazeera English

I was looking through my Google docs for something I had noted a while ago (btw: it is helpful to have all your notes in one place—I need to do that more often), and I came across this list I made back in December of things that I consider important. I'm not sure if this is a complete list, but it is certainly still true.

It's good that I keep these things somewhere that they can be found. Now I need to work on finding and reminding myself of them on a regular basis.

• Be African
• Wear my Africanness proudly
• Study/explore my Africanness avidly
• Be a Craftsman
• Care about my craft(s)
• Strive to be the best that I can be
• Never cease to learn about what I do
• Be Ma'at-ian citizen of the Earth
• Apply my knowledge and talents to the betterment of life
• Lead a balanced life
• Not bite more than I can chew
• Be honest with myself and others
• Spend time getting to know myself
• Strive to make things that I'm a part of better

• Make good decisions
• Be prudent
• Be just
• Be discerning
• Be confident
• Be open (and humble)
• Be willing to change your mind
• Treat each day as a new opportunity to [... what? be great? be effective? act on my values? bring what I want into the world? ah...] live with purpose.

• Keep my life interesting!

I was looking to buy a few albums from Amazon today, when I found out that they only provide their mp3 downloader in a 32-bit package. Moreover, it requires other packages that are not available in the Lucid repositories.

Why do I want the Amazon downloader? Well, you can download individual songs/files from Amazon without the downloader, but in order to get whole albums at once (as well as some goodies that are sometimes packaged only with albums), you need the downloader.

I found a post that addresses this issue in a slightly earlier version of Ubuntu, but it didn't work immediately for me.  So, here's what I did:

1. First, I backed up my /etc/apt/sources.list
2. Then, in the original /etc/apt/sources.list file, I replaced all instances of "lucid" with "karmic". There are packages that the downloader needed that are available in the Karmic repositories, but not Lucid. Specifically, these are the 1.34 versions of Boost libraries:
   
   
   • libboost-filesystem
   • libboost-date_time
   • libboost-iostreams
   • libboost-regex
   • libboost-signals
   • libboost-thread
   I got the idea to use the Karmic repositories from http://www.hilltopyodeler.com/blog/?p=294.
3. Use synaptic to refresh your package list, and to install the above packages as well as libglademm-2.4-1c2a (you can do it from the command line, but I used synaptic).
4. From here, follow the instructions at http://www.ensode.net/roller/dheffelfinger/entry/installing_amazon_mp3_downloader_under (look in the comments for the current location of getlibs.deb). While installing amazonmp3.deb, if it tells you that there are other dependencies missing, go ahead and install those as well.
5. After you've finished, don't forget to restore your /etc/apt/sources.list from the backup you made in step #1.

I'm just a little frustrated at Facebook right now. When I created my event, I checked the little box that said "Anyone can view and RSVP (public event)". Then, on my event page, under Event Information, it says:

This is a public event. Anyone can see the event, RSVP, or invite others to this event.

Only people with a Facebook account, or people that were explicitly invited, can view your event. And there's no way to open it up. Because the only public that we care about is the select portion of the public that chooses to use Facebook. We will alienate anyone else. Sorry.

I hate marketing.  I guess the marketing I'm referring to is the sending of notices through email, twitter, facebook, and every other communication channel you can access. It makes my palms sweat. Makes my heart beat fast. I feel like I'm bothering people. I feel like I'm inundating a lot of already over-inundated people with extra information.

Part of this is that I have to get better at using different communication channels. For example, I sent out a tweet about the Phillyware meeting this thursday. It said:

hey folks, i'm holding a brainstorming meeting for ideas on building civic software community in philly. come by:http://tinyurl.com/24n8g3s

Interested in building civic software tools for Philly citizens? Join @mjumbewu at @indyhall this Thurs http://bit.ly/9QOg1p

So I was sitting here yet again enjoying my Gnome desktop experience, and I was so moved by my enjoyment that I had to write a post about it. I'm really loving my Gnome these days.

Whenever I do a fresh install of Ubuntu (my distro of choice for a few years now), there are just a few things that I do to get my desktop just how I like it:

• I add a keyboard shortcut for the system monitor (usually Shift+Ctrl+Esc).  When something's dogging my system, I don't want to have to click through menus to find out.  I want it right at my fingertips.
• I install Do and Docky. I tried out Do at some point when I was looking for a bit of eye-candy, but it turned out to be so much more. I'm just a Super+Space away from my most used tools. It's like the quick efficiency of a terminal, but with all the glamour of the graphical desktop.  It takes the graphical command runner far beyond Alt+F2 (or Win+R for that matter).  I use Docky at the bottom of the screen.  It took a bit of getting used to, but it just makes me feel good to look at it now.  Maybe it's because I remember back to all those panels and desktop widgets whose main and only objective, it seemed, was to look nice (i.e. look like the Mac OSX bar), eschewing functionality.  Docky covers aesthetics and functionality, and does both swimmingly.
• I download Epiphany and set it as my default browser. Epiphany is nice to use in part because it's the most light-weight browser around.  Its beauty lies in its simplicity.  If I want to open a link or an HTML file, I click and BAM, there it is. Moreover, it's nice to have a browser that just blends into my desktop. Don't get me wrong: Epiphany's not without its issues (on the top of my list are 151943, 598291, and 598584). I do suppliment it with Chrome as, for the time being, some web applications just work better there. If, say, I'm going to be editing several Google Docs, I'll use Chome; it's a less frustrating experience. Also, Epiphany can be slow at rendering some pages (for example, try scrolling down pages that use the experimental -webkit-box-shadow CSS style). However, by and large Epiphany provides a pleasant experience.  With a few extentions, it's my casual-browsing browser of choice.
• I install Hamster and set the notifications to every 15 minutes.  As I endeavor to improve my productivity, it's helpful to know where I spend my time.  Also, being notified that I'm working on "Distractions" is a pretty good motivator to get back to work.

• Information Architecture
  
  This is classification of information within context. It's clumping and separating. It's apples and oranges, but to a botanist or a chef. The information architectural design is going to determine the structure of your menus and links (and probably other stuff).
  
  Start with a detailed breakdown of the domain (context) you're working within, from a users point of view. Classify them according to importance and according to how well they can be achieved through software, and then clump them into separable categories (experiential goals -- things the user wants to do with the software).
  
  The presenter recommended a book: Mental Models: Aligning Design Strategy with Human Behavior by Indi Young.  Might be worth checking out. I should also get around to reading those other books like The Design of Everyday Things.
  
• Back-end Coding
  
  This, in the case of HappyCog and the Greater Philadelphia Tourism Marketing Corporation (GPTMC), was construction of the system that the site maintainers will interact with.  It was choosing and setting up a CMS.  It was also deciding how to transfer all the old data into the new system.
  
• Front-end Coding
  
  This was basically taking the design and coding it.  She gave some good best-practices advice, like making landing pages for all top-level menu links (for folks without javascript), and making :hover = :focus and onMouseOver = onFocus for those people browsing with their keyboards (I do that!).  Also, I like their "megamenus".
  
• Design
  
  I would sum-up design as a word in one of the slides: reinterpreting. It was taking a new [conceptual] image of the city and putting it into a visual statement.  They came up with some emotive themes that they wanted Philadelphia tourism to embody (contemporary, fashionable, and a couple others I don't remember), and evaluated each of their design possibilities against those themes.

(I also posted this over on the OpenPlans blog)

A few weeks back, I sent out a solicitation to a bunch of city clerks and other people connected to City Council bodies around the country that use Granicus's legislative management software, Legistar. I was requesting signatures on a publically available letter as a statement of the importance of opening the legislation data that Granicus collects through Legistar. This type of data has been a persistent interest of mine ever since I helped start Councilmatic a couple years ago.

The Method

To find the cities that use the software I searched Google for all sites on the domain legistar.com (i.e., with the search query "site:legistar.com"). Then I wrote a little scraper to step through the pages of that search and collect the subdomains. I took the results and started collecting contact information for the City Council contacts in those cities (see those results here — it's a far-from-complete list). I then contacted the people in that list with a little bit of background about myself, my projects, and the letter's purpose:

The letter is meant to let Granicus know that their clients think that it is important to have their data open and available to the public, specifically through an API. The letter is directed at Granicus because, as the letter states, they are in a unique position to open this data for a number of bodies all over the country at one time. The developers at Granicus are in favor of this type of action, but we believe that hearing from their clients that this is important would help them to prioritize it further.

...Many opportunities for business enterprise, journalistic storytelling, public advocacy, and citizen experimentation are lost when there is no access to quality data sources.

For demonstration of the latter point, see amazing work done by the likes of Open City folks in Chicago, or AxisPhilly in Philadelphia, or work coming out of the Code for America fellowship. There are many more examples of applications built with open data (here's a list of seven more apps covered by Mashable), but the data has to be out there in a way for developers and tinkerers to work with first.

An additional, unstated purpose was just to raise awareness about the value of opening this data, particularly with so many different parties interested in local legislative data recently.

The Response

I received responses from 5 of the people on the list, two from cities that agreed to sign. Both of the cities with favorable responses were from people that I had some prior connection to, at least through some common acquaintance, underscoring the importance of personal relationships and trust when making changes. I think that it is reasonable to expect a low return rate for any cold-call communication. The person on the receiving end is even less likely to response, I think, when they don't understand the purpose of the solicitation. We're still far from a point where an open data vocabulary is common in government. For example, one of the contacts that got in touch with had this response:

Please note that any documents pertaining to the [city represented] that would be available through Granicus/Legistar are already available to any member of the public through our website. I am fairly confident that is true of any city that utilizes their services. Have a good day and a wonderful new year.

The benefits of making data open and available in raw formats is something that I have found difficult to communicate outside of one-on-one conversations. It is definitely a big step to make data available in a human readable format, but computers aren't good at getting information in the same ways that humans are. Enabling people to build new things, tell new stories, and otherwise make the data useful to more people in more ways requires access to the data in a more computationally malleable form.

What's most frustrating is that, often, data is already sitting in a repository somewhere ripe for developers to use. This is evidenced by the fact that many documents, such as every Legistar legislation record, are generated from raw data. That data is simply inaccessible in its raw form. Access to the data is mediated by the format that the document is rendered into. Scraping is an option in this case, but it is an overly complicated, inherently fragile solution, and there are better solutions to be had.

Data openness initiatives like these have a higher chance of success when there are local supporters who understand the issue and can coach city contacts through what it all means. This is the approach that the OpenCity group is taking in Chicago, with some success. If I did this over, I would start with cities that have a local Code for America brigade to help with the outreach.

I spoke on a panel at a conference a couple of months ago: Reinventing Older Communities, Building Resilient Cities.  The panel I was on was titled "Innovation with New Information":

Governments are using technology to better understand challenges and create solutions to meet them. Residents are using technology to report problems to local government for quicker resolution. In this session, we explore how technology is changing the relationship between governments and their constituents and offering greater efficiency, transparency, and accountability in the process.

I've been thinking recently about how to enhance city institutions using tech skills. I'm still relatively new to really diving into it, but this represents some of my current thinking on the topic.

All I know so far is that this is a two-way street: people need to build up their institutions, and institutions need to support their communities in building them up.

The Gist...

For institutions:

• Let your community help you.
  (Corollary: Avoid, when possible, legal agreements that restrict you from receiving help from your community)
• Empower your community to help you.
• Ask your community to help you.
• Your community is you.

• Just do it.
• Have patience.
• You are your institutions.

A few weeks ago I attended the Supernova Forum at the University of Pennsylvania. It was great — I got to hear from and talk with so-called thought leaders in business, government, education, and media on topics related to the "network age". While there, I was plagued by a feeling that there was some phenomenon absent from the conversation, but I couldn't identify what it was. Since then, I've reflected and named this phenomenon "The tyranny of the tech elite". This tyranny is a reframing of the idea of digital inclusion. It is somewhat in contrast to the idea of the internet as a democratizer. I'm still struggling with this concept, so take this as a work-in-progress.

(I was going to write this post myself, and have been agonizing over how to do it right. After some research, I found that Joshua Breitbart already wrote much of it for me four years ago. So, in summary...)

The future of us...
The Web is a pretty neat technology. With it, people can do and have done some amazing things.
Generally, we (the educated, technologically savvy, modern-day bourgeoisie...yes, probably you too!) think it's so great that we want everyone to take part in it. So, we talk about technology vis-a-vis marginalized groups like "how do we get them to where we are?" We talk about reducing digital division, and increasing digital inclusion. There are a few assumptions in this: (1) Where we are is good (nay, best, at least for now), (2) They will want/need to be where we are.

Joshua Breitbart has this to say about digital inclusion:

[The phrase "digital inclusion"] carries an implication that people who are offline are being brought into a perfect world. That’s clearly not the case.

What we see in the online world is the result of a land rush where English speaking white men had first crack at the virtual real estate. Digital inclusion is like saying poor people, people of color, and non-English speakers are allowed to shop in white neighborhoods.

We want to do more than just include people in the online world as it currently exists. We want that new involvement to transorm that world. This is what I hope to imply with the phrase digital expansion. It’s also what I want to imply when I talk about “open internet.”

• Make the internet universally available. This stuff should be a public utility. Or, better yet, a human right. I don't think that access (or lack thereof) to capital should be a determinant in whether one can access the internet or not (capitalism is one of the most frustratingly unacknowledged forms of sanctioned discrimination).
• Keep (make?) the internet open/neutral. Though, as Breitbart points out, openness on its own is not enough. "Open does not mean equal, it doesn't mean that usage or usefulness is the same for everyone. Access does not equal justice, but it is necessary for justice. Therein lies our work."
• Be aware of and concerned with social justice issues. They don't go away once you move online. In some ways, they're exacerbated.
• Invest in people. Digital literacy and fluency are forms of human capital and require public investment. Digital proficiency must be promoted at neighborhood based locations, especially community technology centers, community based organizations and libraries, to strengthen resident understanding of new technologies. And remember that access at an early age is a key factor in raising people who don't just know technology, but are comfortable with it.
• Build local infrastructure (social and technological). “Price points are one barrier to entry for the poorest community members' use of a network, but so are software design, literacy levels, and misinterpretation of what a community needs from a network,” says Hannah Sassaman, Program Director of the Prometheus Radio Project. Community networks need to be about more than Internet Service Provision — they need to build community-wide Local Area Networks to house information, services, and multimedia on the network itself. Emphasize being a part of a local community and building relationships within that context. Yup, create divides — but divides that are respectful of one another. These will allow diversity to flourish on the Web.

Talk of the death of the web has gotten a fair bit of play since the Wired article on August 17. Notwithstanding its grandiose title, there's actually a fair bit of useful content and food for thought in the article (it's actually two articles—one by Chris Anderson and one by Michael Wolff—explaining the same phenomenon from different perspectives).

Fact of the matter
The Web continues to grow. And so does the internet. The Anderson article says that the web — "largely HTML data delivered via the http protocol on port 80 — accounts for less than a quarter of the traffic on the Internet...and it’s shrinking." However, Internet traffic as a whole is growing exponentially each year (interestingly, United States traffic is only growing roughly linearly — check out Cisco's Visual Network Index forecasting tool).

So, while the Wired articles paints this picture:

The Web is, after all, just one of many applications that exist on the Internet, which uses the IP and TCP protocols to move packets around. This architecture — not the specific applications built on top of it — is the revolution.

• Transport Equality — As far as the Internet is concerned, all nodes and data should be equal. No prioritization based on arbitrary distinctions.
• Collaboration and Transparency — Development proceeds under shared global ownership and is based on open standards.
• Accessibility and Openness — Anyone should have access the content on the Web, and anyone should be able to create content on the Web.

1. Independence of applications
2. New applications can be added anytime that’s a core value
3. Permissionless innovation
4. Open standards
5. Accessible and globally inclusive—anyone can use it
6. User choice—I can choose what applications I use and where I go to with them
7. Ease of use—I can use it in my language, I can use it in a device I’m familiar with
8. Freedom of expression
9. The ability to change rapidly
10. Trustworthy and reliable is one we have to work on; it’s got to be a core value.

In April 2010, OneWebDay.org announced that it would merge with Drumbeat, "a new initiative of the Mozilla Foundation that shares all of OneWebDay's goals and values." Additionally they say that they "will retire [the OneWebDay] brand and invite all of its people to join forces with Drumbeat’s growing community."

I think the merging with Mozilla Drumbeat is a good thing. It is useful and important to have avenues for people who are willing to get more involved. However, does it justify dissolution of the One Web Day brand? Institutionalized celebratory events can play a useful societal role. Nathanial James, former executive director of OneWebDay, says:

I’ve spent most of my time with OneWebDay listening to as many of you as I can reach. Here’s the number one thing I hear from you: “Our OneWebDay was amazing! Let’s do more! What’s next?

Last OneWebDay, Sanjay Patel shared some surprising data comparing "what it costs [him] on a montly basis to get cable speed internet here in the United States along with 3 different quotes [they have] received for installing internet connectivity to a school that is a stone’s throw away from one of Tanzania’s largest cities."

Check out the full post here: OneWebDay | The Epic Change Blog

I'm designing a survey to administer at recreation centers and libraries to get a sense of what people get out of the city's public computing infrastructure, what people seek to get that isn't available, and what they assume isn't available but would like to see. I would expect the survey to be filled out when a user is done with their session on the computer, so that they'll have a sense of whether they had achieved what they came to achieve on the computers.

I'd probably be violating all sorts of human subject research rules by just going in and asking people to take the survey, but if I can't get explicit permission, that's what'll happen.

The survey is at http://bit.ly/8XwlRn. It is still evolving at this point. Most importantly, I have to get people to look over it so that I can get it into some more normal language.

Update: So, I've gotten a couple of pretty good pieces of feedback since I posted this yesterday. First, from @digitallogic. Noting the fact that people are often unaware of the realities of various available technologies and so are unable to determine certian changes that would provide them with a meaningful impact, he says:

The ideal approach would be to follow someone through their whole usage and observe where improvements could be made, though this may be a better in a business environment due to the nature of tasks be performed and obvious privacy issues.

Barring that, hopefully these might go more in that direction:

• Is there anything you would have liked to do on the computer today which you were unable to?
  
• Other than your primary goal (check email, do homework, read websites etc), what did you spend most of your time on? (wording on this is weird, trying to see if there's some major hurdle that could be over come, ie - 5 mins to check email but 15 mins to log in)

Here's an excellent talk by Nalini Kitamraju, assistant professor at the University of Twente in the Netherlands, for Harvard's Berkman Center for Internet & Society on the tension in creating e-government services using user-centered design principles.

This one's highly recommended for anyone interested in how governments can better interact with citizens in the digital age. To be honest, the lessons from the talk are not limited to e-government; they can be applied to government in general. However, as we're fumbling through figuring out how to best use the tools we've developed over the last half-century or so, the video provides some great food for thought.

Nalini Kotamraju on the Tension between User-centered Design and E-government Services

Individuals and institutions are slower to adopt e-government services due to a lack of user centricity in design and development. Work with PortNL, an integrated e-government service for expatriates in the Netherlands, suggests the core of governments' difficulty in creating user-centered services lies in a fundamental tension between the needs of users and those of governments. In this talk, Nalini Kotamraju — an Assistant Professor at the University of Twente in the Netherlands — explains how the purposes of e-government services can be met through a user-centered design approach, and how site builders can put the needs of users ahead of the ideas of governmental clients.

Okay, I'm submitting my Gigabit bid. If you like it, vote for it on the Gigabit Genius Award Google moderator page. It should be one of the most recent ones. This was adapted from a post a few days ago. For more information on the grant, go here.

Summary:

Set up a system where residents could access their desktop and files over the internet, from any computer. Offer a service that rents out netbooks that come configured to connect to the system. For visitors, it would serve as their portable guide to the city. For residents, it would serve as their city dashboard. And for students, it would enhance their classroom materials.

The deadline for submission of ideas to the Gigabit Genius Grant is nearly here (it was extended, at some point, to June 30). I've looked through the submissions (all 158 of them, so far) and there are a few that I like.

Some are good ideas but very vague, some of the ideas are very science fiction, and some have very little to do with high-speed communications at all, but all's good in the name of generating initial ideas.

One thing I notice is that so few of the ideas link to any place where they provide more information, which is a shame; I'd like to see more depth to some of them. So, for some of the ones I liked that didn't have much further explanation, here's my take.


Cloud Computing
So, this was probably my favorite suggestion. Allow for very low-cost computer/netbook rental with "cloud"-based desktops and application access. Actually, there are three or four submissions that propose this in some form. For example:

a truly ONLINE "COMPUTER" where you can access and use all your files and software/application from any connected device. Soft/Apps would be running on the server side.

Create a redbox type rental system for netbooks ($1/day) to provide low income areas an affordable computer solution. Operating System processing and software access would happen in the cloud removing the need for the latest and greatest hardware.

Utilize thin client computing on a residential level to give low income housing an affordable solution for PC's. Operating System processing and graphics rendering would happen in the cloud, negating a need for the latest and greatest hardware.

I would create a "city online" that means: online where and how much is the article you wanna buy + see traffic online + school clases online + trafic cameras online + etc etc : that means from home you can see ALL that happens in the city services

Make every city device a data gathering node. Traffic lights, garbage cans, surveillance cameras, public spaces, metro stations, buses, etc. Then, use statistical analysis on the data to create a better city according to people behavior and needs

Real Time Music Collaboration with Remote Viewing Parties... Artists throughout the city can participate in a real-time concert from the comfort of their own homes, while listening parties allow friends to listen, cheer and sing along. From home.

Redesign all of the existing libraries into a multi-sensory experience on all possible topics.

So, before I forget what happened, over the next couple of days I have to record my thoughts on the brainstorming session that took place tonight at IndyHall.  I'd normally post this kind of stuff on a more personal blog, but this was, after all, a public meeting.  Forgive me if the thoughts are a little stream-of-consciousness.

In my final analysis, I think that the meeting was valuable. I think some useful ideas came out of it. However, I think the next one won't be quite like it. The next thing that comes should be more action oriented. Maybe for a project. I have to contact the other people who were present and see what they think, and get some other input too.

#1

1. Firstly, I have about zero experience in open meeting facilitation. I just want to put that out there. Many issues below were exacerbated, if not caused, by this fact.

I want to consider what we'd need to create a community that encourages engagement with the wider community, fosters developer and designer innovation around software for citizens, and participates in ensuring that the software can get made and that people—all people, or close to it—can use it.

In defense of cities ... I think the responsibility is ours to show the more entrenched factions of the cities that are not letting go of the data where the use is, that there's a huge hacker community right now that's interested in cities.  Some subset of them are interested in it because it's just a brand new set of data, not because they're interested in cities per se.  And I'm worried that we need to move beyond the sense that just having the data solves the problem.  Carol Coletta of CEOs for cities made a comment a couple months ago that "if you listen, you'd think that you could fill a pothole with data.  Just pour it in, smooth it out, and fix your problem."  Basically, what I'm saying is the tougher nuts to crack in city government, I think, need to see tangible examples of the benefits, savings, efficiency ... So, part of the responsibility is ours.

- John Tolva (at about 1:45)

1. Choose a couple of simply defined problems,
2. Determine what is needed for solutions, and
3. Go from there

Issue: Center City residents don't know about CCRA events
Posted By: The Center City Residents Association
More Information:

• Current process -- CCRA plans meetings and events
• How it breaks (undesireable outcome) -- neighborhood residents don't know about events
• What should happen (desired outcome) -- neighborhood residents should be informed of (and maybe even RSVP for) events
• Proposed solution -- A CCRA community calendar online

• Community Engagement: Philadelphia organizations are provided a place to go and discuss technical problems with professionals/volunteers. Also, the happy-hour thing reinforces the connection between tech and the city.
• Developer Innovation: It provides a way for developers (be they full-time employed or not) to get involved with community issues. The issue-tracker format allows a structured, but open-ended approach to finding solutions.
• Political Advocacy: I'm not sure it necessarily covers any technology advocacy areas. Whether it would require any opening up of technology depends on the nature of the issues presented. It's possible that an issue might require (or benefit from) some as-yet unopened data, or extending tech access to an underserved group, but I don't know. This also doesn't prescribe a way of dealing with those issues if they were to come up. Have to think on that one further.

• Community Engagement: Hopefully it would prompt interaction between developers/designers and maybe community business associations or something.
• Developer Innovation: As a specific project, it would require developer innovation to implement. However, it wouldn't in-and-of-itself foster an increased capacity for developer innovation within the creative community. It would be a hell of a thing at show-and-tell though. It might encourage people to get involved with other projects.
• Political Advocacy: Things like Kiva work to shift the power relations in society. I like that. In terms of technology advocacy, however, I'm not sure. I would hope, though, that it would at least further the conversation about how access to technology is often access to opportunity.

Issue: Small West Philly businesses are unable to obtain micro loans
Posted By: The Enterprise Center
More Information:

• Situation: West Philly businesses look for loans
• Undesireable outcome: They cannot qualify for large loans, and don't have access to smaller ones
• Desired outcome: They have access to loans in the amount they need
• Proposed solution: Phiva

I'd like to see a community in Philadelphia that is concerned with the effective creation and distribution of software that allows city residents/citizens to better use and affect their city.  This partly came out of a report that I put together for a class that I took in the Spring.  The report was called "Phillyware: Taking Philadelphia into the 21st Century through Software Developer Civic Involvement".

Anyway, I'd like to get more people in on generating ideas for what this should look like.  So, I'm planning a brainstorming session.

The flyer:

Event Page: http://www.facebook.com/event.php?eid=124590897575130&ref=mf

Please come and join Philadelphia software developers, designers, and technology advocates in discussion about creating a place for developers to innovate around software tools that can better enable residents to use and affect their city.  Let's brainstorm what this could look like, some potential roadblocks, logistical considerations, etc.

Consider three aspects: community engagement + developer innovation + political advocacy = Software for Citizens*.

What: Philly Software for Citizens Brainstorming Session

Where: Independents Hall, 20 North 3rd Street, Philadelphia, PA

When: June 24th, 6:30 PM

Why:

• To keep local government and authorities accountable in the data they release and in how they release data
• To allow Philadelphians to think ahead of the curve about what their city can be
• Because we ARE citizens
• Because it'd be fun (you know it would!)

Some possibilities for discussion:

• Periodic [un]conferences around those three areas
• A community around a project repository (like Gnome.org, but around OpenPlans.org, for example)
• Meetings with Philadelphia community groups
• Camps/Hackathons (like in Ottawa: http://opendataottawa.ca/)
• Bring your own ideas...

---

* "Citizens" in this case isn't political; it just refers to participating members of society. See the Facebook group (http://www.facebook.com/group.php?gid=131677170182058) for more information.

I enjoyed reading this article[1] about increasing instances of using computers to help analyze ones own personal life. This is the kind of world I envision eventually, and that I'd hope to help create. Sometimes I wonder what I'm getting myself into.

While I count myself among those the author describes as data-driven, I don't agree with his portrayl. It seems like he's saying, in the data-driven life people let their data explain them -- praise or condem them. If you gain a pound, if you jump around through tasks, if you have too many drinks, the data is all there to prove it. But really, the data does nothing on its own. I would venture to think that few "data-driven" people think of their data (let alone the machines/tools that they use to record it) as separate from themselves. The details are just more of you exposed, and machines put it within easier reach. The tools and the data simply are you formulating ideas about yourself in different ways. They are an extension of you.

At the end of the article the author notes that his own personal tracking experience was only useful to him as a "source of critical perspective." My point exactly.

So, it turns out[1] that five cities have finally been selected to participate in the Code For America[2] program for 2011: Boston, Boulder, Philadelphia, Washington, D.C., and Seattle. Each of these cities will identify the need for some city government project that can leverage modern web technologies, be assigned a group of five developers[3], and will work with them to develop that project over 11 months, starting in January.

I am on pins and needles to see what comes out of this. I'm also curious how the city governments plan to pick the projects.

Over on GOOD.is, there's a video posted that demonstrates Keiichi Matsuda's conception of what augmented reality might look like. As we've seen in some previous posts, this type of reality may not be [as] far off [as some would like].

But what's with all the ads? Adds a touch of realism, I suppose.

Augmented (hyper)Reality: Domestic Robocop from Keiichi Matsuda on Vimeo.

A few posts ago[1], I referenced the Sixth Sense TED presentation. Here's[2] another technology along the same lines. It's a prototype video for an Android app that retrieves information on a person using facial recognition. They call it Recognizr, an "augmented ID" concept.



Also, for those eye-tracking augmented reality glasses that I mentioned in the other post...the eye-tracker just seems like a slight modification of this[3].

For those of us surrounded by the minutiae of computers all day, it’s easy to forget there’s a world of people out there who just don’t get it. And it’s not their fault. It’s ours.

Computers shouldn't make people feel like idiots [1]

The Real Work is not formatting the margins, installing the printer driver, uploading the document, finishing the PowerPoint slides, running the software update or reinstalling the OS.

The Real Work is teaching the child, healing the patient, selling the house, logging the road defects, fixing the car at the roadside, capturing the table’s order, designing the house and organising the party.

Fraser Speirs, Future Shock [2]

PIM practices become easier if [an] organization provides some infrastructure to alleviate the difficulty of these activities. But a larger value is that the organization can leverage these personal practices to improve the effectiveness of others and to capture that elusive corporate knowledge in an easy way.

Situating Personal Information Management[1]

The conference for Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA) has a new name and overall mission. It's now Systems, Programming, Languages, and Applications: Software for Humanity (SPLASH). I'm liking the name.

Seems like they're attempting to reconcile the inclusion of the Onward! track of the OOPSLA conference. I approve.

In 2002, Onward! was created as a special track within OOPSLA to be a venue for bigger ideas than normally are accepted by mainstream computer science conferences, but within the scope of OOPSLA’s focus. "Bigger ideas" included new approaches to programming, software, and software development; new paradigms; and even new ways to present ideas.

Beginning in 2003, Onward! papers were included in the OOPSLA proceedings, and in 2005, Essays and films were added to Onward!. As the track grew, it became clear that there was a need for Onward! in a larger context than object-oriented programming, and in 2009, Onward! spun off from OOPSLA to become a stand-alone conference focusing more broadly on software and programming in all their manifestations, and including not just the pure technology but also processes, methods, and philosophy.

From 2010, we plan that Onward! will be co-located with SPLASH (the evolution of OOPSLA), but in the future, the sky’s the limit.