Work with old, large systems enough and you start to get a persistent feeling of Deja Vu. Get large enough and you always seem to grow a set of features that is almost like every other large system, just implemented in a completely different way so that code sharing is simply impossible.

How many file-system abstraction mechanisms have we collectively written? How many inter-thread communications mechanisms? How often have we re-implemented configuration management, object-relational mappers, caching, indexing, page-layout, and hundreds of other big sub-systems?

When you're starting out, it's just too much work to figure out the "other guy's" byzantine system for handling almost every possible corner case of the problem domain. You only have to do this one thing. You'll never need all that. You can just code up what you need and be done in a few hours instead of spending days or weeks learning how to debug the huge mass of meta-programming in someone else's system.

So you roll your own and it works. You're an elite hacker. Maybe you even pat yourself on the back for having avoided a dependency. Then the customers ask for one more feature. No problem, a day at most. Another, a bit bigger, you refactor the codebase and now it's elegant and ever so much more general. A few years along, you are the (proud?) owner of a byzantine system that handles every possible case except the 5 that the other system handles. Unfortunately, with completely different substrates and totally alien structures, they can't be used together or even talk to each other, so some of the people use yours, some the other, and the potential pool of devlopers and users is halved.

I'm wondering (more throwing out a straw-man) if the problem really is one of documentation? If every large system came with two pieces of documentation:

Documentation that let another meta-progammer (the type who could create such a system themselves) sit down and figure out all the corner cases from a set of principles and rules, something that let you understand the totality of the codebase and walked you through the exceptions and caveats until you could predict what would be in the code base as easily as you would your own code.

Documentation that exhaustively linked to (runnable, tested) code examples for accomplishing every common (and uncommon) task. It could be backed by some sort of Trac-like system so that every bug report, every mailing-list post, every question that came up was turned into a runnable test/example, with people able to push new examples in...

Would it be enough? Would anyone care? I know the second would be very useful, most programmers seem to start off with copying code samples and altering them to work for their purposes, so maybe it would be enough. Maybe the first document is really a red herring, something you pour your work into, but which isn't necessary, because the people who really want to hack on the code have already got some investment and want to know how to make that one needed effect happen. Maybe it's the code library that needs to be front-and-center to make your library pull people in and avoid having the other systems split the community.

Or maybe the Romans had the right idea, and the beauty of each thing is the thing itself, not the Greek ideal shining through from some platonic realm. Maybe each large system's implementation of the dozens of similar sub-systems is what makes the whole magical.