Category Archives: BangBangCon

Expanding the !!Con Aesthetic

TL;DR: There’s a cool new project to put on plays about computing. In addition to traditional talks, we’re inviting people to submit more artistic 10-minute plays or other “non-talks” to !!Con as well as The Art of Python.

Sumana Harihareswara and I had lunch sometime late last year and talked for a while about her newest brainchild, The Art of Python. To steal her own description, The Art of Python is a miniature arts festival at PyCon North America 2019, focusing on narrative, performance, and visual art. The Art of Python was recently accepted into the PyCon Hatchery Program, an incubator for PyCon-related ideas. It will be a series of short plays, 5-20 minutes each, held on the evening of Friday, May 3.

Sumana has worked with the !!Con team before and wrote one of my favorite articles about the conference, Toward a !!Con Aesthetic, which I re-read every year to re-center myself on what !!Con can (and in my opinion, should) be. One of its sections is titled Spectacle and play, and explores how some of the talks cross from being a “talk” into being a comedy or sketch show. We as audiences remember these moments where the magician shows us that something more than meets the eye can be done – and in our case, presents a new kind of magic to an audience of magicians.

That’s why we’re inviting people to open up the idea of what constitutes a “talk” at !!Con this year. We would love to see drama, comedy, and magic. If you are interested, we invite you to submit a play to both !!Con and The Art of Python, theme and length (10 minutes for !!Con) allowing. Note that The Art of Python’s CFP specifically says that the play does not actually have to be about Python, just the emotional experiences of programming.

Our CFP will open Feb 6 Feb 8, and The Art of Python’s CFP is already open. Bring us your explorations and reenactments. Show us how your computing is spectacle and emotional. Explore with comedy and tragedy the feelings that surround this field that has only been around for one generation, two or three if we’re really generous.

Some caveats: !!Con is still going to be looking for “the joyous, exciting, and surprising moments in computing”, and we’re not going to have a separate track or approval process for non-talks. But, based on the amazing work that we’ve seen in years past, and ideas we’ve been pitched, I don’t doubt that you the reader have something worth submitting to !!Con and The Art of Python.

Of course, I encourage you to submit your proposals for 5-20 minute plays to The Art of Python regardless of whether you go on to submit them to !!Con! You should also check out the Hatchery program for next year, if you have an idea for a conference track that you’d like to try out in the context (and with the backing of) PyCon.

We talk at !!Con a lot about the intersections of humans and computers, since indeed the things that computers do are ultimately created and consumed by humans (well, or their pets). The lecture format is often ill-suited to explore this emotional side of computing, and especially poor at exploring the relationships between people who make the magic lightning rocks go fast.

I will admit that watching a play about programming sounded awkward at first, but then I reminded myself that 1) the spectacles that Sumana mentioned from previous !!Cons have only been amazing, and 2) when I go to plays about far more uncomfortably introspective topics, I generally end up learning something about myself and/or the world. It’s sometimes not even something that can be put into words – but that’s exactly the point. Some things are very hard to describe by speaking at a room with a slideshow behind you. Perhaps one of these non-talks will inspire, delight, and amaze in a way that’s uniquely deep.

We are still exuberant about accepting traditional-format talks and are looking forward to seeing all of the cool, fun, scary things that you have done with computers. Polish up your demos, dust off and re-submit your !!Con talks that didn’t make it in previous years, think about the fun bug that you defeated, and come tell us about it. If you want some more information before we post the CFP: Lindsey Kuper, organizer emeritus now leading the charge on the west coast, has a great article about the kinds of talks we’re not looking for. I’ve written previously about how we select the talks.

I’m optimistic about this widening of our focus. Similar to our inclusion efforts, there’s a difference between not-not-permitting and inviting. If people had submitted drama/magic/spectacle to previous !!Con we would have judged it seriously, but to openly request it alongside the traditional talks breaks down the assumption that a 10-minute !!Con slot is only for a “talk”, which, to be honest, is just because that’s what other conferences do. I like to think of us as more avant-garde than that.

The !!Con Talk Anonymization and Selection Process

Another conference organizer recently asked the !!Con team what our anonymization process was, and we realized that we didn’t have a good source for them. While we tout our anonymization as a major component of our quest to eliminate implicit bias during the talk selection process, we haven’t written what our anonymization and talk selection process actually looks like in a while.

Julia Evans wrote a great post about our anonymization outcome after the first !!Con in 2014, and Lindsey Kuper has an update from a year later that goes into a lot more detail. We’ve changed some processes since (we no longer use github issues to track the proposals, for example).

One thing I want to point out is that if you’re submitting a talk proposal to !!Con, you shouldn’t worry about how your proposal will be anonymized! Just submit your talk like you want it to be listed, and we’ll take care of the anonymization process.

Here’s the blurb about anonymization from this year’s Call for Talk Proposals! page:

Proposals will be anonymized to avoid bias. Although we ask for your name, email address, and so on in the proposal submission form, only one or two organizers who serve as anonymizers will actually see this information, and they won’t review your proposal. The rest of the organizing team will review your proposal without knowing who you are.

I also think we have a pretty cool way of doing the talk selection, so I’ll cover that as well in this post.

Here’s the secret sauce:

  1. Each year, before we open the Call for Proposals, we decide who is going to be on the anonymization team, and who is going to be on the talk selection team. Generally the anonymization team is one or two organizers plus potentially one or two outside helpers who we recruit. The talk selection team is the remaining four or five organizers. This year, Alex Clemmer and I are the anonymizers, and the rest of the organizing team is the talk selection team.
  2. The anonymization team is the only team with access to the google form (and resulting google spreadsheet) that we use to collect talk submissions. Then we open the Call for Proposals!
  3. It’s tradition that the anonymizer who is supposed to close the google form falls asleep before midnight and forgets to close the form until the next morning. This year it was me and I didn’t close the form until 9am. Whoops!
  4. The “chief anonymizer” saves a copy of the original spreadsheet and begins going through it to anonymize it. Each talk is given an ID (a random unique integer, so that we don’t give away when the talk was submitted), so that we can track and re-aggregate our data at the end of the review process.

Let’s talk about the anonymization process. Here’s an example talk proposal (by me right now for demonstration, this probably wouldn’t be accepted):

I work at FiniteAutomata Corp, so my talk is about how knowing how finite automata work gives you regex superpowers! People will look at me and say, “wow he is so cool with his regex” and I’m like “heck yes I am good because I know how finite automata work let me teach you”. Here’s a link to me giving a longer version of this talk at FiniteAutomataCon 2011:

The anonymizer takes this proposal and does a couple of things:

  1. Remove/Obscure any names of people or companies. We generally leave in product names unless the talk submitter is the main author of that product, in which case we will remove the product name as well. For example, “FiniteAutomata Corp” in the example above would become “[Large Tech Firm]”, in square brackets (or similar).
  2. Remove/Obscure any pronouns or other identifiers. For example, my proposal above says “he” and “his”, which gives away my gender. We would replace these pronouns with “[they]” or “[author]”.
  3. We also try our best to watch / read any associated material and give a quick synopsis of it. For example, the link in the proposal above would become “[Video of the author giving a 30-minute version of this talk at a well-known conference. The talk is well received by the audience.]”

During anonymization, we try not to remove aspects of the speaker’s identity that are essential to the message of the talk. For example, the abstract for Sina Bahram’s 2016 talk about screen readers begins, “I use a computer very differently than most people, because I’m blind.” It wouldn’t have made sense for the anonymizer to remove this fact, because it is Sina’s personal experience that makes the talk proposal especially compelling. We try to use our best judgement here (and probably get it wrong sometimes!)

Once we’ve created the anonymized version of all the talks, we create a copy of the anonymized spreadsheet for each of our reviewers. If I were on the reviewing team, I would get a spreadsheet called “!!Con 2018 – Talk Proposals [Erty]”. This is my personal copy of the anonymized spreadsheet.

The next step is for each reviewer to give each talk a grade. !!Con co-organizer Lindsey Kuper introduced this grading scheme, based on the Identify the Champion process (Specifically Make Champions Explicit):

A – I really like this talk and will advocate for it to get into the conference
B – I like this talk, but not enough to advocate for it
C – I don’t like this talk, but not enough to advocate against it
D – I don’t like this talk and will advocate against it being included in the conference
X – I think I know who submitted this talk, and abstain from giving it a grade

The chief anonymizer then creates a spreadsheet that aggregates these grades into a single place (did you know that you can include cells from one spreadsheet in another using google spreadsheets? It’s pretty nifty!). We look at the best grade and the worst grade for each talk.

Some examples:

A talk that got grades A B B C would have a final grade of AC (A is the best and C is the worst)
A talk that got grades X A A X would have a final grade of AA
A talk that got grades X D C A would have a final grade of AD

If a talk got 0 or 1 grades due to abstain votes, we would handle that as a special case. I’m pretty sure that hasn’t happened in the 5 years we’ve been running !!Con.

We all get on a conference call at this point and start going through the talks! This is really fun! AA talks generally get in immediately. AB talks are usually also admitted as well, if we still have room. AC talks we usually discuss, but they’re generally not admitted, and AD talks are really interesting, since it means someone will advocate for it and someone else will advocate against it! Generally, someone misunderstood the proposal, or two people have differing ideas about what kind of talks should get into !!Con. It’s usually a no, but these talks will sometimes get in (especially if they got AAAD).

More generally, this means that only talks that get at least one A are considered. Your talk needs to convince one of our reviewers to become a champion for it. As Lindsey noted in her blog post Scaling !!Con, last year (2017) we had 86 talks with an A out of 215 total, which we then had to narrow down to just 30!

Remember that all of this is still being done with anonymized talks. The anonymizer(s) will usually sit in on this call and answer any questions about the proposals that the reviewers have (usually around things like the descriptions of videos that were with the proposal). But never speaking up to sway a decision in one way or another. This is difficult! Last year I spoke up because I was really excited about a talk proposal (that ended up not getting in, anyway) but I definitely shouldn’t have done that, since I knew who the speaker was and therefore had implicit bias.

Once we’ve decided which talks we’ll approve for the slots we have available, we take our next favorites (generally about five of them, but it varies) and hold those as backups in case our acceptances aren’t accepted (it happens!).

The anonymizer then goes through and makes sure that we didn’t select the same speaker twice (some people have submitted as many as six talks in a single year!) If we did, we bring some more people off the backup list, then replace them to keep our backup list full. Usually if someone submitted two talks that were accepted, we let them choose which one they’d rather give. Although, if there’s a clear favorite, we might suggest that they choose that one 🙂

At this point we do the dramatic reveal and show the review team who got in! Then we email these people. This is really fun 🙂

One important thing to note: If you didn’t get accepted, then your name is never revealed, not even to our review team. The anonymizer emails out our famous rejection letter after we’ve figured out our lineup. This is not so fun 🙁

That’s it! I hope this is useful information, especially if you’re starting a conference yourself.