Just a quickie to say that I’ve had a small flurry of work on my tiny winshell module. A combination of doc updates, cookbook examples, and some more general-purpose shortcut handling. It’s up to v0.5.1 on PyPI, github, and readthedocs.

Three times, within the space of a few weeks, I’ve had complete strangers drop me an email to tell me that they’ve been making fruitful use of some module of mine and that they’ve got it to work on Python3. They’ve either attached an updated version or indicated what would need to happen.

Spurred on by this, I got my act together and shifted a bunch of my code over to github where it is a little more public and where it’s easier for others to contribute. I’ve also made moves to incorporate the changes & patches supplied. At the same time I’ve done a bit of tidying up around the projects and have made use of the wonderful readthedocs.org.

There’s still work to do, but now winshell — one of those tiny helps-you-a-bit modules I wrote eight years ago and which remains basically unchanged — has just seen a 0.4.1 release which has docs and tests and which runs against everything between Python 2.4 and 3.2 (and possibly more besides). The code is hosted at github, the docs are hosted at readthedocs.org and it’s available on PyPI and pip-installable. Thanks to Steve Peck and Roman Luks for patches & suggestions.

My software development tends to go in bursts when the available time, energy & motivation coincide, so lets hope that I can keep this particular ball rolling. My active_directory module is also available on github and has a py3k branch kindly supplied by Ken Gillett. The complication there is that it’s (naturally) against the last released version, which is a few versions behind my private version. If you’re interested, you can fetch it from the py3k branch. My aim is to re-apply the coresponding diffs from the main branch and then look at merging or splitting or something.

I mean to write an entry on the requirement tensions between github, PyPI, pip and the various recommended practices, especially with respect to the README files, but that’ll have to wait for another moment.

[Note to self: blog about things other than the London Dojo…]

Nicholas “@ntoll” Tollervey has been the London Python Dojo’s parent for all its young life. Once it achieved toddler status, he felt confident enough to start letting other people look after his baby, so over the past few Dojos various other people have run things on the day, always with @ntoll in attendance. Yesterday was the first day on which he felt confident enough not to be there, leaving things in the hands of @tomviner (who could be seen consulting a trusty checklist throughout the evening).

As I re-read, I realise that it looks as though I’m accusing @ntoll of being over-possessive, which I most definitely am not. He’s done — and continues to do — a fantastic job at organising the Dojo and making it happen even when he’s not the evening’s MC. We’re just delighted, as Tom said last night, that he doesn’t feel that he needs to attend it every first Thursday for the rest of his life. (A little secret: the day before this month’s Dojo he was still sending mother-like emails to the rest of the organisers: don’t forget to … remember that … have you …?)

Last night’s Dojo was fun as usual: we were doing the famous Game of Life — probably a text book example of a text book example! Uniquely in my experience, every team had a working version to show after just an hour and a half. The team I was in managed to get something working while @john_chandler and I were still chatting in the background. We fiddled about with it a bit, adding a few preset forms to seed the board etc.

An unexpected visitor was @JohnPinner (of PyConUK fame). He was in London for a meeting and timed things so he could come along for the start of the Dojo, altho’ he had to dash after about an hour to catch his train home. He gave a lightning talk at the beginning outlining various Python-related conferences and training sessions which in the offing. Including this year’s PyConUK, once again in Coventry.

@tomviner’s novelty for this Dojo was the favourite-module question on the sign-up form, which was also used as part of the introduction session. Perhaps unsurprisingly, Kenneth Reitz’s requests module was the clear winner (the only one with more than one vote!). Other unsurprising entries included itertools and collections, but there was a variety of others. I was chatting with John Pinner about the line-up for PyConUK this year, and he pointed out that there’s some mileage for simple talks about a particular module, eg logging or itertools. I’m thinking of proposing such a thing for future Dojos…

There’s also been talk on the python-uk mailing list of a second London-based Dojo, or other Python event, on a Sunday. That might suit some people who can’t make a weekday evening in London but who could manage a weekend. And the more Python events in London the better!

(Yes, that is a deliberately ambiguous title offering two possible interpretations: as a description of what the programming challenge was at last night’s Dojo; or as an imperative to be awed at the might and wonder that is the London Python Dojo. You choose).

Last night was the first time I’ve actually run the London Dojo. For the two and more years since its inception, Nicholas Tollervey (@ntoll to his Twitter friends & acquaintances) has indefatigably turned up every first Thursday to clear up the Fry-IT offices, order the pizza, buy the beer, put up signs, leave out sticky labels, request free books from O’Reilly and then drum up support, keep everyone happy and actually run the show, finishing off by organising everyone to clear up, move chairs, dump the rubbish outside, and finally catch the last train home to Sticksville, Northants. where his wife and children have long ago fallen asleep over their sheet music, having gone da Capo al Segno one time too many waiting for him to return home.

A few months ago, Nicholas asked for volunteers to help out, and a small group of us got together to share the burden. Since then, Bruce, Jonathan, Tom and finally I have taken a turn at organising. And of course it’s not until you have to do it yourself that you realise how much work is involved… I was fortunate because Gautier (who actually works there) and Nicholas himself were both at Fry-IT for the day and were able to do some of the less proximate preparation, including ordering a dozen pizzas and buying three dozen bottles of beer. I was able to make a small contribution in the shape of a pack of sticky labels.

The Dojo itself was very slightly quieter than usual: just under 20 rather than just under 30. That’s not a bad thing in Fry’s offices which are not huge. There was a bit of an introduce-yourself session (which was made even more primary-school-like by the presence of big sticky labels on everyone’s chest with their names or cognomens). And then we had a lightning talk from Martin who has a sort of cut-down Nagios for app developers. (I hope I haven’t done it an injustice). And then a surprisingly straightforward vote on the evening’s programming challenge which gave us… Boggle. (Word game; 8×8 grid of random letters; form words by moving like a chess king).

Four teams; four solutions, all more or less different. Only one team ended up with a working solution at the end of 90 minutes (and they appeared to be optimising by removing all vertical whitespace; or maybe that was an aesthetic choice - who knows?). Our team had visible activity (which is more than Team 1 managed!) but no solution. It’s up to each team how they want to manage their collaboration. We’d gone for the split-team approach, dividing the problem into its eminently decoupled parts: a mechanism to read in a dictionary of words and provide efficient searchability (using a Trie, in case you’re interested); and a structure to hold the board (a dictionary, keyed on coordinates), generate the letters into the grid, and search for all possible words, relying on the dictionary code to indicate success, failure, partial success, or success with more possibilities.

The tweets were still flying this morning as people tried to tweak their solution (or, indeed, get it to work at all) on the train, on their phone or at home overnight. A friend of mine who’s a C++ coder came along mainly because I’d talked so much about the Dojo. He’s not really into Python - in fact he’s not really a programmer: he does medical image analysis. But he enjoyed the atmosphere and made a few small suggestions before simply sitting back and watching the teams get to work.

Look out for the next Dojo at the beginning of January. It’ll be announced on the python-uk mailing list and we’ll tweet about it when we’ve fixed a date.

UPDATE Dirk’s added a blog post of his own:
http://elazungu.wordpress.com/2011/12/02/solving-boggle-with-python/

Last night, the London Python Dojo were the guests of Mendeley on the Clerkenwell Road. I was pleased to discover while chatting over the pizza and beer that I wasn’t the only one to have wandered round the area a few times before hitting on the right spot. Although I had a map of some sort, I’d forgotten just how cluttered central London is: how many passageways, tucked-away building and unlabelled streets.

But it was worth it. Mendeley have larger offices than our usual host, Fry-IT. And while there were slightly fewer of us than normal (low 20s as opposed to the usual high 20s) it was still great to have a bit of breathing space. Thanks very much to the guys at Mendeley for hosting us (and providing pizza, drinks & snacks).

Jonathan Hartley was making his debut as compere and managed very effectively. Well… fairly effectively. (I shouldn’t laugh: it’s my turn next month). He’d lined up three “lightning” talks to kick off with. First up was Ian — who’d arranged for us to use the place. He explained what Mendeley does (a sort of social network for research students) and how they use Python (mostly as a scripting API) and demo-ed some fairly nifty visualisations and tools which people had built on top of their product. Jonathan himself spoke last to ask for help with a Django concurrency issue. Which he promptly got. In between was Robert Rees who showed-and-told very effectively the recently-added Heroku support for Python. This enlarged into a wider discussion of Heroku itself and of its competitors in the Django/Python world.

Then the Dojo itself. As usual, we had a whiteboard available beforehand for people to propose ideas which were then voted on. The Roman Numeral Calculator remained top of the list of unchosen ideas, but the surprising winner was Robert’s suggestion of an ASCII Art Streetfighter clone. (Chosen only after a second round of voting with a Multiple Transferrable Vote). Once this was settled, it was a simple matter of dividing into teams and hitting the editor.

Or almost. We initially failed to be able to count up to 5 in order to divide into teams. Having finally achieved this intellectual feat, we encountered the opposite problem to the one we normally face at Fry IT. There, the office is so small that you’re squeezing into space. At Mendeley, there’s so much space that you’re wandering around for ages trying to find the best spot. And then you’ve got to find the WiFi (which Ian had considerately explained about). And then you’ve got to manually set your DNS Servers to something (as the DHCP wasn’t handing out DNS). Slightly geekily, the WiFi password is mathematical making it easy to remember but still quite long.

And then, in our case, you had to find the Pygame curses emulation which someone knows exists but can’t quite remember the name of. Having got there (with about 20 minutes left now to do the actual coding) you basically scramble your way through a stunted version of Streetfighter (whatever that is; I have no idea), getting a basic solution on which you layer colour and fonts in the manner of lipstick and pigs :)

Finally, the endgame; and it’s the usual hilarious collection of imaginative approaches, stylishly-designed code, and desperate hackery. We saw: elegant ASCII art; flying bullets; gratuitous use of decorators; and many entertaining attempts to achieve an equilibrium between using classy and best-practice code and actually coming up with a solution within the timeframe!

I don’t know where we’ll be next month, but stay tuned to the python-uk list where stuff is announced.

Well, the West Midlands Python team have been busy… (at least, I assume it’s them). This year’s UK Python conference is taking place over the weekend of the 24th - 25th September in Coventry. (Which is fairly central in England, for those of you reading this in a foreign language). I’ve just booked my place and a night in the hotel. I hope can rustle up a talk or workshop to make it even more worthwhile.

It looks to be a bit more casually organised than other conferences and I’m very much hoping that the always friendly atmosphere will be even more friendly… See you there, I hope!

I blogged previously that I was having a look at the Python Testing Cookbook courtesy of the eager marketeers at Packt Publishing. Well I’ve finished reading it — in installments on the Tube — and I think I’ve come to a conclusion.

The short version is: I’m not mad about the layout; it’s not entirely what I expected from a Cookbook; but I think it does what it does very well.

Since I’m overall positive, I’ll just pick up on the very slight negatives first. In a previous Packt review I commented (adversely) on the style of the layout and so on, and I’m afraid that it hasn’t really changed. Obviously, this is somewhat subjective, but I can’t be the only person who’s subconsciously affected by the font and layout choice. (Bear in mind that I was reading it via PDF which may make a difference). To be clear: it’s not awful; it’s not even bad; it’s just suboptimal.

The other slight negative is hardly a negative at all: that the “recipes” in this cookbook are far broader — in some cases, chapter-length — than I’d expected. Subjective expectation, to be sure. But I was a very little bit surprised.

Ok, that’s the downsides. Now the upsides:

* Nice division of chapters
* The right pacing
* Pretty much one example used throughout
* “Why didn’t you do this?” question sections
* Good re-use or alternative use of previous examples or approaches
* Interesting spread and combination of toolsets

The chapter divisions are: unittest, nose, doctest, BDD, UAT, CI, coverage, load-testing, good habits. In broad terms, each chapter builds on previous ones, but thanks to the reuse of the a simple shopping cart example, repeated in nearly every chapter, you can take a chapter on its own without too much difficulty. In addition, different chapters sometimes offer alternative approaches to the same problem when it makes sense to illustrate one point rather than another. One chapter, for example, walks you through creating a nose plugin to show how to do it; the next chapter introduces you to a ready-made one since the focus of that chapter is not nose plugins.

And that brings me to the variety of toolkits & modules mentioned throughout the text. This is not a book which simply uses stdlib code to perform various tasks (which I had rather thought it might be when I started). It introduces the ideas mentioned in the paragraph above. And puts forward Python modules (such as nose or mock) or external tools (such as Hudson / Jenkins or TeamCity) as best-of-breed or useful tools.

One final point which really put the icing on the cake is the occasional introduction of pull-out sections answering the very question which was occurring to me as a reader: “Why didn’t you use this technique here?” or “How is this different from that?”. Perhaps it’s just me, but I find this smoothes my progress through a book considerably which otherwise would be (mentally) jarred by my wondering “Now why did he do that?”.

I can’t but recommend this book. There’s a sample chapter here if you want to have sniff. As a final note, it’s very slightly unfortunate that the timing of its production precluded the author from picking up on Michael Foord’s magnificent work in producing unittest2 / unittest for 2.7 & 3.2. Perhaps there’s a “missing chapter” opportunity in there for someone…

Thanks to the energetic folks in the Packt Publishing Marketing department, I’m currently looking at the Python Testing Cookbook with Python Web Development waiting in the wings. The copies I have are PDFs which are never my favourite, so I end up printing chunks out to read on the train or elsewhere, but it’s great to get to read other people’s insights and ideas. I’m about halfway through PTC so look out for a review soonish.

Brian Jones, co-author of the upcoming edition of the Python Cookbook has just tweeted, asking why anyone would release a new module now without testing for Python 3.

I sympathise, but for me the simple answer would be: because Python 3 support doesn’t solve a problem I have. It’s one more hurdle to leap over in terms of developing, documenting and testing software. And if you’re still using 2.x for production purposes then you don’t need 3.x. Other users might, but there’s little enough time to scratch one’s own itches let alone scratch other people’s for them.

On the plus side, I’m trying to keep my own modules Python3-friendly, running Python 2.x with the -3 flag, making a few decisions which I know will help 2to3 out (or will remove the need to use it altogether). But at the end of the day, the things I write need to work for the environment I have — which at the moment is 2.7 — and not for the environment I’d like other people to be running.

[UPDATE: Thanks to Mike Driscoll for pointing out how to link to a tweet].

(or: release late, release seldom).

I don’t know about you, but I have sort of one-man sprints on some of my modules when for days on end I take my laptop on trains and squeeze in time at lunch or whenever to refactor or enhance modules. The only trouble is that I’m very (read: too) particular about the quality of the code I release to the public. I want to make a good impression. I want it to have good test coverage. (Or at least some test coverage). I want it to have good docs, with an overview, an API guide, and a cookbook. I want it to have a roadmap so people can see what to expect. (And what not to expect).

But of course all that takes time and energy from the limited pool of time and energy at my disposal. And I have several projects, all competing for that same time and energy, not all of them related to Python or programming at all. Like most developers, I suspect, I’d far rather be writing new code, juicing up existing code, refactoring code for the new generation of language features, breaking code out into nicely layered, suitably decoupled modules. It’s not that I don’t value docs, test, etc. I do. But someone’s got to write them. And that’s me.

Also I don’t want to go public with an underwhelming feature-set. I don’t want people saying, “Oh good… Oh no! Where’s….?” So I go on extending the code forever and never actually releasing. The Perfect is the proverbial enemy of the Good.

The case in point at the moment is the rewrite of my active_directory module (cunningly entitled active_directory2). It’s more or less of a ground-up rewrite with a very similar API but more structure and layering etc. etc. The rewrite’s been on the go for well over a year in bits & pieces. The trouble is that the original module works fine for most of what I want. Very occasionally I need something more sophisticated and I can usually cobble something suitable together for the occasion. So there’s only a low incentive level.

Testing it’s a bit of a ‘mare as I need an test AD rig. Obviously. So, courtesy of the PSF MSDN Subscription — which, it seems to me, was created for just this kind of situation — I’ve installed a VM containing Windows 2003 Small Business Server. This is about the smallest modern server which will host Active Directory. This setup has the slight advantage that it forces me to test AD authentication since, in an inversion of the norm, I’ve already logged onto my laptop before firing up the AD server so it can’t be the authenticating agent for my account.

I’ve got the top two layers working and I’m looking at tests — at the very least, exercise tests: ie those which simply call every function just to make sure it doesn’t completely fall over. Then I’ll probably release it as a work in progress while working on the more sophisticated extra layer which does some transparent transformations to and from Python types based on the schema definition of the AD object and which handles things like memberOf entries in a more Python way.

Watch this space…