Wednesday, September 2, 2009

MooseX::AttributeHelpers is dead, long live Moose::Meta::Attribute::Native

Version 0.89_01 of Moose is out, featuring the well-loved MooseX::AttributeHelpers module under a new name. See Moose::Manual::Delta for a summary of changes; the functionality should be almost identical, but some API changes were made to bring the helpers more into line with the rest of Moose (e.g. handles instead of provides).

If you're using MXAH, please grab this dev release and see how your code works when you port it over to the Native traits. Please let us know if there are any problems!

We started work on this at the YAPC Moose Hackathon. We'd talked forever about how we'd move a MooseX module into the Moose core if everyone agreed it was the right way to do things, but it hadn't ever happened. Rob Buels (rbuels) and Jay Hannah (jhannah) helped out with some tedious but necessary work at YAPC -- converting examples and tests -- and Jesse Luehrs (doy), Chris Prather (perigrin), and Shawn Moore (Sartak) have worked on the code and generally helped keep momentum going since then (reviewing commits, doc fixes, and so on).

Cheers to you all, and thanks for your great work.

Tuesday, July 28, 2009

webmachine: intriguing

A few days ago, rjbs pointed me at Webmachine, "a REST-based system for building web applications".

Its central concept is a pretty mind-blowing inversion of control compared to I'm used to in Catalyst-Action-REST. Webmachine says that there's only one way to handle a request to a particular resource. That means that you don't write any of the logic to decide what status code to return -- you write a bunch of hooks that let Webmachine decide it for you. Similarly with content type (both request and response), etag, etc. -- here's the full list.

Webmachine's still sinking into my brain, so I haven't gotten my mind around how it might be done in Perl, let alone how it'd fit into something like Catalyst -- Webmachine also has a URL dispatching mechanism, but it seems to be almost entirely separable from the resource dispatching mechanism (the HTTP graph, above). I'm looking forward to playing around with these ideas more, though, because Webmachine seems to solve a few of the architectural problems I was having with C-A-REST.

Friday, July 24, 2009

back in the saddle

Almost exactly a month after being laid off, I've accepted a job with Grant Street Group. Schwern pointed me in their direction when we were both at YAPC, and I'm glad he did; everything I've experienced with them so far has been positive, and I'm excited to be starting with them (on Monday!).

This also means that we'll be moving back to the East Coast, this time to Pittsburgh. I'll be telecommuting, but while our time in Portland has been fun and Portland is a very enjoyable city, the fact is that we've also really missed being able to visit family without spending a thousand dollars and two days in a plane each time.

The next month is going to be crazy -- our goal is to be in Pittsburgh by September 1st, and between moving and starting a new job, I'm not sure how much time I'll have for blogging or hacking on personal projects. I promise I'll try to pick everything back up as soon as possible!

Thursday, July 9, 2009

YAPC wrapup

I know this post is a bit late, but I was busy after YAPC dealing with my unexpected unemployment. Sorry!

YAPC is always great for reconnecting with people who've been just words on a screen for the last year. rjbs and I managed to get a room together, so much hilarity ensued; the first thing he greeted me with when I arrived was "I've been avoiding working on slides by reading Wikipedia articles about Spider-Man's Clone Saga. Let me tell you all about it!" (Since I also needed to avoid working on my own slides, this was a welcome diversion.)

I always feel a little skeptical about the arrival dinner. It's always loud and expensive and the food is often merely adequate. Rik and I showed up, but didn't stick around for very long, since we'd almost exhausted our store of slide-avoidance time.

Monday I gave two talks, about CPAN packaging and deployment and a quick overview of Dist::Zilla and its workings. I wasn't sure how they'd be received, since this was my first time speaking at YAPC, and it's a little more intimidating than talking in front of a local Perl Mongers group. Both went well, though, and it was gratifying to hear people saying "I'm totally going to use that!"

It was between those two talks that I got the call from OpenSourcery: "This is Brian, with Josh and Kris; can you please call me back as soon as possible?" My first thought was, "Well, I don't have a job anymore. I guess I need to call them back to find out why." It's hard to imagine a better time and place to be laid off than the first day of YAPC, though, with all the networking possibilities (and fellow hackers to buy me consolatory beers).

Monday night I felt feverish and lethargic, and Tuesday morning's talk on Moose roles and type constraints was difficult to get through. I ended 10 minutes early, partly because I didn't have the energy to explain all the examples as much as I had expected to. I went straight back to the dorm afterwards and slept for an hour and a half. This helped immensely.

Tuesday evening's auction and dinner was one of the better ones I can remember. First, I happened to sit at a table full of Canadians, several of whom I found I already knew (Adam Prime, Michael Graham), and we had a fun conversation about all sorts of random things the whole evening. Second, unlike many years, the auction did not drag on interminably, so we had plenty of time for said fun conversation without the mental overhead of having to tune out Uri's haranguing voice.

I barely remember Wednesday, except that Karen called me out during her talk for something I said on Twitter (and a followup from Rik), which caught me by surprise for some reason. Maybe I'm not really used to the idea that people actually read the things I write in public places.

Talks I really enjoyed:

Thursday was the Moose Hackathon, which was a great time that I will address in more detail in another post. I think the highlight may have been watching Rob fall off his chair laughing not once but twice.

My summary from twitter, still true:

despite everything going on with my (lack of) work, probably one of my favorite years

Friday, July 3, 2009


Last Monday, I and 6 or 7 others were laid off from OpenSourcery. I'd just given my first talk at YAPC, and I wasn't really expecting to find out that I didn't have a job anymore. Whoops! If anyone wants to hire an experienced Perl programmer, take a look at my resume.

I'm going to try to fill most of my unemployed time with work on Moose and Catalyst. I'd been working on CPANPLUS::Dist::Deb, but that's much less compelling without a work environment to target -- luckily, I think Josh ben Jore is going to pick up a lot of my slack.

Meanwhile, I'm rebooting my blog here. I'll fill in old posts from OpenSourcery and try to get back into the Iron Man challenge.

Friday, June 5, 2009

Packaging CPAN Modules for Debian

I'm giving a talk on CPAN at this year's YAPC in Pittsburgh. One of the things I plan to mention is the cpan2dist program from CPANPLUS. I use cpan2dist (with CPANPLUS::Dist::Deb) to build Debian packages for our internal use.

I had a conversation on IRC this morning with someone who asked, roughly, "What's the difference between cpan2dist and dh-make-perl?" Until recently I don't know that I would have been able to answer this, and I think the answer reveals a distinction that is important to keep in mind whenever you're trying to bridge between CPAN and your operating system's package manager.

Specifically, I think dh-make-perl is for maintaining a Debian package that happens to be a Perl module, and cpan2dist is for wrapping modules from CPAN in a package manager-friendly way. Let me expand on both of those:

dh-make-perl was written by Debian people, and seems aimed towards people who would like to maintain a module in full Debian style, filling in all the metadata like a changelog and description and so on.

  • It leaves you a source tree with a debian/ directory, so it's easy to take the autogenerated files and go from there on your own.
  • It lets you easily tweak an individual package's Debian package information.
  • It does not recurse into dependencies at all, it just notes them in the generated debian/control file, though I think by default it will warn and ignore any dependencies that don't have a Debian package.

If there are one or two modules you need to install that aren't in Debian -- maybe because they're internal code -- and you don't mind maintaining the Debian metadata, you probably want dh-make-perl (or to submit a request for packaging bug to the Debian Perl Group).

cpan2dist was written by Perl people, and CPANPLUS::Dist::Deb really only pays lip service to Debian's packaging conventions.

  • It assumes that modules' licenses permit redistribution, and fills the bare minimum metadata needed to install the package in many places.
  • It does recursively build debs of dependencies, and gives you some control over how to build all of those packages -- a common prefix to use in the package name, build tool options, etc.
  • It doesn't know anything about some of the weird historical decisions that have gone into Debian's perl (e.g. libcgi-perl is not but some distribution named cvswebedit), and does not (currently) give you quite enough control to make the packages it generates fit in with Debian's perl in every case.

I treat cpan2dist as a way of making it easier on my sysadmin to manage Perl modules that I would otherwise just install with the CPAN shell -- installing into /usr/local instead of /usr, for example -- and I don't try to match up with any of those historical edge cases. If you frequently need newer versions of modules than Debian provides, or if you want to install a lot of internal Perl code as debs (especially if you have an internal CPAN mirror that your code is injected into, e.g. with CPAN::Mini::Inject), or if you don't really feel like interacting with Debian's perl much at all, you probably want dh-make-perl.

(Disclaimer: I do not use dh-make-perl with any regularity, so my assessment of it may be insufficiently deep. Please correct me if I've said anything wrong.)

Long term, I'd like to be distributing my own perl package, at which point I'd be using apt-get and dpkg as a convenient distribution and management mechanism. cpan2dist is a much better fit for that goal -- I don't want to have to spend the time munging each Perl module's Debian package's control files for use with my own perl.

If you aren't using Debian, there may be something like dh-make-perl written specifically for your packaging system (I'm aware of g-cpan and g-cpanp for Gentoo, and I'd be amazed if RPM didn't have something too). cpan2dist should work the same everywhere; it has backends for Arch, Gentoo, Deb, RPM for Fedora and Mandriva, and finally PAR, which is itself cross-platform.

In every case there will probably be this same tradeoff between tighter integration with the native package manager's assumptions and ease of use for installing arbitrary CPAN modules, so be prepared to choose one or the other.

(originally posted at OpenSourcery)

Friday, May 29, 2009

Catalyst-Action-REST update

A while ago, I talked about my plans for Catalyst-Action-REST, and since then I've made some progress on refactoring to use roles instead of classes. Unfortunately, I've been stalled for the last few weeks, and I could use a nudge to get back on track.

My brain is stuck, basically, on naming issues. This feels a little silly and trivial, but it's something I've run aground on twice now, so I need to fix it somehow. The biggest problem is "serialize" -- Catalyst-Action-REST uses it both as a generic name for "things that either serialize or deserialize" and specifically for "things that serialize" (that is, "convert data to formatted text"). This means I end up wanting to use it to describe a bunch of things:

  • Serialization formats, like YAML or JSON
  • Common code shared by all serialization formats
  • Actions implementing specific serialization formats
  • The role for actions that do either serialization or deserialization
  • The specific action role that does serialization
You can see the results of this sort of naming confusion in the current code on github: Catalyst::ActionRole::SerializeFormat is a pretty goofy namespace.

In my previous post I threatened to split serialization out into its own distribution. I'm not sure how this would overlap with something like Data::Serializer, which is currently used by some of C-A-REST's serialization formats. Maybe it'd be simpler to just punt to Data::Serializer for all the serialization and deserialization, but even then I'd have to keep some class names around for backwards compatibility.


(originally posted at OpenSourcery)

Friday, May 22, 2009

local::lib followup: now with easy bootstrapping

Last week I posted about local::lib, some of the work I was doing on it, and what I wanted to get done.

Well, it's done (for now). As of 1.004001, local::lib comes with an example script that completely bootstraps local::lib into an arbitrary directory. It installs cleanly on a fresh install of Perl 5.8.8, just like this:

wget -O- \ | TARGET=./local perl (I copied the script to my server so the URL is shorter -- it's the same as what's on CPAN, though.)

Future enhancements:

  • A shorter url (t0m suggested, which would be pretty cute)
  • Better shell/environment integration -- right now, you have to Just Know how to get at the directory that local::lib created for you
  • Build tool integration, e.g. make locallib, make localinstalldeps (that's a bit of a mouthful)
Anyone have other suggestions for useful features?

(originally posted at OpenSourcery)

Friday, May 15, 2009

working on local::lib

local::lib is really, really fantastic. It's a simple module: it bundles up all of the settings and environment variables needed to install Perl modules into a private directory. By default it uses ~/perl5, but you can give it some other directory for an argument, so it's perfect for bundling dependencies with your application or installing things in a scratch directory for testing. (It also depends on the correct versions of toolchain modules needed to make things behave consistently, which is a sweet bonus.)

The Makefile.PL has a bootstrap mode, which means that you can configure and install local::lib into one of these private directories without having it pre-installed at all and without touching your @INC.

This is really great, but I want one step better; I want to be able to bootstrap local::lib into a target directory without having to find the latest version, unpack it, etc. I've been working on a script to do this, something like

wget -O- $some_url/local-lib | TARGET=./local perl

(possibly with some sort of integration with David Golden's -Mylib), but there are a few tangles I still have to work out.

I did make some progress yesterday. I found a bug in Module::AutoInstall that was breaking the normal installation of local::lib under some circumstances, and I fixed the bootstrapping code in Makefile.PL so that the latest version should install cleanly on Perl 5.8.8 (still the default on many operating systems). Hopefully I'll figure the rest of my problems out this weekend, and local::lib will become even easier to use.

(originally posted at OpenSourcery)

Tuesday, May 5, 2009

Ticket tracking with SD

I have a few longer posts that I want to write, but none of them are coming together in my head right now, so I'll put together a quick note about Prophet and SD, by Jesse Vincent and Chia-liang Kao. Prophet is a replicated, peer-to-peer database (I'm cutting its sales pitch short, see the website for more), and SD is a bug tracking system written on top of it.

I had looked at SD a year or so ago, and then forgot about it until I read a post by gugod describing how to use SD to interact with, through SD's helper script git-sd. This caught my interest, of course, because I use for all the CPAN distributions I maintain.

What makes SD interesting to me, at least more so than other local-storage ticketing systems like ticgit, is that it has adaptors ("foreign replica" classes, in Prophet terms) for several other bug tracking systems, including (as mentioned above) RT and Trac, both of which we use at work.

I like the idea of having tickets available right there with my repository, and I especially love not to have to switch contexts, fiddle with my web browser, etc., every time I finish one bug and I'm ready to move on to the next, but I don't have the luxury of telling all my coworkers that we're switching to some new ticketing system. The fact that I also get free disconnected operation is just icing on the cake. SD feels like git in this regard -- distributed, customizable, and able to play well with existing systems instead of needing a complete change-over.

Since reading gugod's post and getting excited about SD, I've tweaked git-sd a little, and pushed my patches back upstream. In particular, git-sd now defaults to using GIT_DIR/.git/sd for its replica, instead of requiring that you manually configure one, which streamlines the process of setting it up. I have plans for a few more things that I want to do before I use SD heavily:

  • make it easy to store username and password per-replica
  • give replicas a short name to refer to for push and pull, like git remote

SD and Prophet are relatively young, but so far almost all the problems I've run into have been minor interface issues rather than big showstopping bugs. Here's the only thing I'm aware of that might fall into that category:

<obra> soooo close to trac roundtrip too
<confound> which would be great for me
<confound> since we use trac at work
<obra> I have one crippling bug.
<obra> I just need a 4 hour block of time
<confound> what is it?
<obra> when you create a ticket in sd, push to trac and pull from trac, it doesn't properly map the uuid
<obra> oh.
<obra> I know what it is now
<obra> (explaining problems)++
And now that Jesse has achieved enlightenment through explaining the problem to me, I expect it'll be fixed quickly. So glad I could help! :)

(originally posted at OpenSourcery)

Tuesday, April 28, 2009

Bugzilla on Catalyst

One of the talks I submitted for this year's YAPC::NA is on converting legacy CGI and mod_perl applications to run on Catalyst instead, focusing on ways to get Catalyst in between existing code and the webserver so that incremental refactoring is a possibility, rather than "maintain as-is" or "rewrite entirely". I've done this successfully for elementalClinic, and I wanted to find a few more applications, maybe with a little more complexity (the nice way of saying "insanity", in this context), to try my hand at.

Unfortunately, I failed. Instead of a crazy application that would test the limits of my ability to get Catalyst into those hard-to-reach places, I found Bugzilla. (Yes, Bugzilla has some very strange internals, as someone who has hacked on it has told me, but it's not a mixture of three separate templating engines/application systems spread across your webserver's document root.) The end result was anticlimactic -- I got Bugzilla running on Catalyst's standalone http server after only a few hours and some patches to Catalyst::Controller::CGIBin and HTTP::Request::AsCGI. I ended up spending as much time on making it easy for other people to duplicate my results as I did on actual Bugzilla-specific code:

around wrap_perl_cgi => sub { my $next = shift; my $code = shift->$next(@_); sub { $Bugzilla::_request_cache = {}; $code->(); Bugzilla::_cleanup() }; };

This is just a quick hack, and for normal users of Bugzilla it's probably not a big deal. If the Bugzilla maintainers wanted to start converting to Catalyst, though, it would be a great place to start; for example, it'd be trivial now to set up some Catalyst actions to clean up Bugzilla's urls from e.g. /show_bug.cgi?id=17 to /show_bug/17. That could start out as simple as this:

sub show_bug :Local :Args(1) { my ($self, $c, $id) = @_; $c->request->params->{id} = $id; $c->go('/CGI_show_bug_cgi'); }

Later, more logic could be brought into Catalyst controllers, and eventually the old show_bug.cgi path could exist only for backwards compatibility. (I don't know that the Bugzilla team cares about this at all, but it is the kind of thing that people often would like to do with their own applications.)

(originally posted at OpenSourcery)

Thursday, April 2, 2009

Plans for Catalyst-Action-REST

A while ago, I told Jay Shirley (in a moment of weakness) that I'd co-maintain the Catalyst-Action-REST distribution. Over the past week, I've closed several bugs, and I feel comfortable enough with the codebase to start making more sweeping changes.

C-A-REST is certainly useful in its current state, but it has its warts. At the same time, I've been thinking about the fact that Catalyst 5.80 (svn) is built on Moose, and about how Moose could possibly help with some of C-A-REST's rough spots.

Here's a list of things I'd like to improve in the near future, in rough order of how much I've thought them through:

  • Using roles instead of classes for every type of object (Action, Request, and Controller) will make it much easier for C-A-REST's modules to play nicely with other Catalyst extensions. The old classes will stay around for backcompat, but new code will be able to use with 'Catalyst::Controller::Role::REST' instead. (See also Catalyst::Controller::ActionRole, thanks to Florian Ragwitz.)
  • Serialize/Deserialize plugins are currently only loaded from under Catalyst::Action:: -- there's no way to look under MyApp::Action:: or any other namespace. There's also no good way to pass configuration to plugins (a common request). Finally, they all share a lot of code, like looking up data in the stash or reading the body, that needs to be refactored.
  • Serialize/Deserialize are symmetrical -- that is, they're forced to use the same Content-Type. This is OK for basic use, but it puts a pretty hard barrier on extending the basic de/serialization mechanism for application-specific content types, and I'd like to make it easier to specify that certain content types are only used in one direction or another.
  • REST status helpers have clunky syntax and don't cover enough HTTP status codes. Calling $self->method($c, ...) makes me sad; we've talked about using a REST view and/or a REST response role instead. Whatever the solution, the REST component of Catalyst::Controller::REST needs to feel less bolted-on than it currently does.
  • REST and Serialize/Deserialize are unnecessarily conflated. I'm not entirely sure they should even be in the same distribution, but they definitely need to be easier to use separately. Switching to roles may be enough to make me happy, here.

Check out Catalyst-Action-REST on github, and let me know what you think.

(originally posted at OpenSourcery)