Production for use

Some languages are made for large teams, and some are made for the individual. It's not just the language, but the supporting framework and tools that are associated with the language.

For example, C# (with the the .NET framework) is a language designed for large teams. The framework is large, much larger than the previous generations of MFC and ATL. The .NET framework has hundreds of namespaces, thousands of classes and tens of thousands of methods. It is much larger than any one person can consume. When building a non-trival application in C# and .NET, the natural response is to divide the work among several people. Not just hire multiple people to work on the project, but to specify different areas for different individuals. The traditional big areas are database design, user interface, and business logic.

The C# language supports this division of labor. So does the .NET framework and the tools offered by Microsoft (Visual Studio and Team Foundation Server). Microsoft recently added the "partial class" feature to C#, allowing multiple individuals to work on separate parts of the same class.

A language designed for a large team leads to large teams. And once you have large teams, you need managers.

On the other end of the scale, languages for individuals work well for a single person but poorly for a team. I think of the languages Forth, Perl, and Ruby as languages for individuals. The languages (and their supporting libraries) are small enough for a single person to understand. Individuals can create and maintain capable applications without the division of labor among multiple workers, and without the oversight of managers.

It seems that languages (with their libraries and frameworks) cannot be both large and small. They must be one or the other. A language -- for example C -- may start small and grow complex -- into something like Visual C++ with MFC -- over time. But those are two different implementations.

Managers have an interest in the selection of language. Languages that lead to large teams are better for managers -- at least better when the manager has an incentive to build an empire. Languages that encourage large teams let managers build those empires. They also let managers partition the work among their staff, and support the manager in that partitioning. Languages for individuals do not encourage large teams, nor do they support the manager in partitioning the work. With languages for individuals, the work of partitioning work among multiple people is harder -- the manager must do it himself, and there is no guarantee that one person will remain within his (arbitrary) limits.

So we have the effect of language design on team organization. Languages for large teams will be used by large teams, but will also grow small teams into large teams. Languages for individuals will be used by individuals, and discarded by large (or even medium-sized) teams.

What does this mean for the future? Will we see a bifurcation of the development world? Will we see one "world" of development for corporations with large development efforts and a second "world" of individual developers? If individuals are not working on corporate projects, what will they work on? (iPhone apps, perhaps?)

Random meetings

I met a fellow IT worker on the train ride home yesterday. He is a project manager, based in Miami. We chatted for a bit, talking about the technical and personnel challenges of the industry. He was unfamiliar with "The Mythical Man-Month" and "Peopleware" so we exchanged cards and I agreed to send him a list of my favorite books.

The list will include Fred Brooks and Demarco and Lister. I will also include Weinberg's "Psychology of Computer Programming" and possibly Knuth's "Things a Computer Scientist Rarely Talks About".

Performance measurements

I've been using Micro-Focus DevPartner Studio for Visual Studio C++ these past few days. (Weeks?) I'm pretty impressed.

It took some time for me to get used to it. Configuring DevPartner Studio is not obvious, although that may be a fault of Visual Studio and its arrangement for plug-ins. But once you configure your projects and build them with the right settings, DevPartner Studio collects run-time information for you. You can collect errors (the old Bounds-Checker type over-runs and illegal references), performance information, or code coverage data. I've been using the performance information, which can be used to improve program run-time performance.

I've made several changes to the code based on the reports. In a few cases, the data showed me programming "goofs" -- things I was doing in C++ that were legal but not quite what I intended. (Yet C++ accepted them, quietly using the default type conversions.) DevPartner's reports allowed me to fix those problems.

From slightly broken to really broken

I upgraded //grendel today. It was running Ubuntu Linux 9.04, and I used the built-in upgrade for Ubuntu Linux 9.10. The upgrade is a complex process, starting with the download and execution of the upgrade programs, preparation for the upgrade, the download and installation of new packages, the removal of obsolete packages, and a final restart.

The whole process took seven hours. In the end, I have a broken PC. Linux boots, almost. The ROM BIOS starts Linux but Linux never switches to graphics mode, so I assume that X is not running. The system responds to ping but I cannot SSH or TELNET to the system.

Phooey.

ACM

I attended the local ACM career day for college students today. I was hoping for some network opportunities, but they were not there. The turn-out was light but the discussions of life in the IT world were quite good.

Geek networking

I attended the Columbia Area Lunix User Group (CALUG) meeting this evening. CALUG is an informal group that meets once a month to discuss things Linux. Attending these meetings requires some planning, as I must drive to the Halethorpe station and park there, staging my car for the trip to the evening meeting. Travel to the meeting will be easier when the Light Rail "Yellow" line is extended from BWI to Columbia. That branch will run right past the Tenable offices, where the group holds its meetings.

Today's speaker was Daniel T. Chen, a contributor to the Ubuntu sound project. He explained quite a bit, and gave us a good understanding of the challenges of the sound project. The team must deal with various hardware (some of it not easily identified or distinguished from similar hardware) and an epidemic of software.

The Ubuntu team has selected a subset of software, simplifying the task. It seems a reasonable approach, although some folks will be upset that their pet project or favorite sound processor is omitted.

After the meetings, I chatted with a few folks. We talked about calendars, compilers, and file formats.

Performance is not magic

I and a few other folks spent a bunch of time examining programs today. Our goal was to identify performance problems. We have several C++ programs that are running but slower than we would like. And they are running slower than the older generation of C++ programs, in some areas significantly slower.

It turns out that the performance issues are caused by STL. The previous generation of programs used "straight" C++ without STL. Our new programs make heavy use of the library. We like STL, since it makes programming much easier. The STL allows for easy use of strings and collections, and we don't have to worry about memory allocation...

And that's the cause of our problems.

The STL collections, from simple strings (which can be viewed as collections of 'char' elements) to vectors and maps requires overhead. It's not much, and the implementors have done a lot of work to make the STL work efficiently, but the difference remains. With our data set (hundreds of thousands of items) the small differences add up to noticeable performance problems.

It's like we wrote the program in Java or C#, with their built-in memory management and garbage collection. Actually, it's a bit worse, since STL uses local copies for everything and doesn't take advantage of copy-on-write or multiple references. It instantiates and copies data with wild abandon. (OK, perhaps not wild abandon. But it makes lots of copies.)

Our previous generation of programs was optimized for memory usage, at the cost of manually tracking every object in memory. The programs are also significantly larger, running at five times the number of lines of code. They were fast to run but slow to write and slow to maintain. (Read "slow" as "expensive".)

The realization of STL performance problems raised and lowered our spirits. It raised them, since we finally understood the cause of the problem. It lowered them, as we also understood the solution.

As Heinlein said in "The Moon is a Harsh Mistress": There Ain't No Such Thing as Free Lunch. If we want shorter development cycles and understandable code, we can use STL but the price is lower run-time performance. If we want fast programs, we have to avoid STL (or perhaps reduce our use of it) and write the traditional hand-optimized code.

Do we want to drink our poison from the large cup or the small cup?

Adjusting

I've been at the new gig since October. That's almost six months, and the "change your password" prompt has arrived again. That's a pretty good clock -- not too frequent, but frequent enough to remind me of the duration.

I'm adjusting to the new situation. The new commute is longer, and I am still working out some of the changes, but things are working. I'm working with a different set of people, and that is going well. I like to think that I am doing good work. My co-workers tell me that I am, yet I have the feeling that I am not doing enough. Is this feeling correct? Or am I using a metric from the former position to measure my current performance? Who's to say that the metric is correct? Perhaps I need to listen more to my co-workers and less to my inner guilt.

Most of my work has been with C++. In some ways, this is good: I'm working with C++ and STL, which is new to me. In other ways, I find it worrisome: the "new stuff" in the market is C# and .NET, Java, Python, and PHP. At least, that's what I see in the advertisements and job postings. (I can't negotiate with folks -- not even in a broad sense -- but I can look at the market and observe the desired skills.)

The dress code is a bit of a puzzlement. Most folks on the train and in the building wear suits and ties. Fitting in with them requires at least a button=down shirt and tie. Yet folks in the office, especially on Friday, are more casual. What to do? Fit in with the immediate office crowd, or go with the larger group?