Open-Source Community and The (lack-of) Developers

[Technologov]

Hi All !

I'm looking at several Open-Source projects, and I wonder why some projects have big communities, and some others don't ?
Why some projects have community developers and others have only community users ?

For example this project, VirtualBox, has very wide OSS community of users and supporters, but don't have even 1 community developer. Why is this ?

-Technologov

[TerryE]

I can only give a personal perspective here. I have done a lot of development in my time, but that last time that I did code cutting in anger at work was over 15 years ago. I have done "hobby" development projects to keep my hand in, including getting to the bowels of OOo which is a larger and more complex project. I do use the VBox source actively, but only as a substitute for decent design documentation and debugging errors. The problem for me is how the Sun team have lobotomised the OSE version compared to the PEUL version. To go to the FLOSS version, I just have to loose too much valuable functionality, so it's just not worth configuring test builds. If I could build the full PEUL version then I would.

[mpack]

IMHO, VBox is not really a community project, hence there are no community developers. In other words, publishing (some of) the source code is not the only requirement of a community project. You also need better design/technical docs, an open roadmap, some kind of source management system, some kind of committment regarding future IP rights etc.

This is not necessarily a bad thing. Community projects can suffer from the problem of "design by committee", leading to bloat. IMHO the pnglib, zlib and ffmpeg community projects all show signs of this.

I believe the OSE version of VBox is important not because it creates a community project, but because it offers the conforting knowledge that someone else could potentially take over the reins in the unlikely event that Sun went belly up, or just decided to abandon VBox.

[h1d]

But what does a project get with more community developers?

Just like what mpack said, getting more hand doesn't simply mean better. Code gets mixed up, someone forgets to document a function or writes it wrong, no one knows those errors and code gets bloated, gets patched to revert part of a patch and update part of a patch, no one will have a clear vision of how to fix, no one has right to dictate the road map, so many useless discussion out of difference in preference out of 100's of developers and good developers leave, while big mouth developers might stay.

It's easy to tell when a project gets bloated and has a hard time moving forward. Just find something that hasn't changed for a while and refuses to fix even the obvious bugs, even if it looks active. Windows (just as an example), Firefox, Thunderbird, Linux desktop (this one is moving forward slowly...), they haven't changed a bit in years, frankly.

I think current development is so far good for VBox, plenty source commits by those people dedicated to it communicating with each others, hard to see the code is bloated or got in jumble by the forum comments.

VBox wasn't open source to start with, and they take the development model just like it was, and while the product is good, no one has any complaint. If it sucked or the development came to a halt, then I'm sure some people might've done something to get a community up around the code.

While having the ability to fork a project is handy to see the insurance that project does not suddenly disappear, I say the better benefit of it is that it can tell it doesn't include anything nasty hidden inside. (Although, without a proper audit, it's hard to find but when widely used application like Skype has been said to have backdoors with very hard to disassemble binary, I like the open source applications.)

[lama]

Politics, pretty much close to the definition of "democracy" & "dictatorship" but in software terms "Open Source" and "Close Source"

lets see where this thread goes...

[TerryE]

Technologov, I've been thinking about this topic. I think that are three key aspect here:

• The first is the relationship between the core project team and the open community. This tends to work in one of two modes: open and collaborative or dysfunctional. In the case where the project is funded and resourced from multiple companies with a diverse team, you tend to the latter. The VBox team is a tight project team based in a single geographic location. I suspect that the core is still the group of guys that worked for Innotek who still hold an isolationist perspective. So in the case of VBox, I think that we very much tend to this latter dysfunctional mode. A case in point is the topic We need a roadmap. Here is a genuine attempt from some of us to engage in a sensible open debate about the future of VBox and by implication where we can help. The VBox team response as articulated by Sander is to ignore the substance of this discussion and make comments such as

  There is a roadmap and it's very detailed. We do not publish it, but we do share it with important customers.

  In other words Sun has a policy of keeping the community supporters in the dark, though we do get random tit-bits such as

  3D acceleration has a higher priority than some might think. Wait for the next major release (no promises though)

  .
• The second is that the FLOSS version of VBox, is pretty functionally cobbled. If as a developer you want to start tweaking and extending the VBox functionality then you first have to give up an awful lot as extremely usable functionalty. So most don't bother.
  
  
• The last factor is the structure of the overall project itself. If you take a functionally larger FLOSS effort such as OpenOffice.org, this is split into a number of subsystems and the architecture also support an "extensions" concept. It is therefore relatively easy to focus on one particular subsystem and make a real contribution there. VBox is a far more tightly integrated architecture which in its PEUL form is closed and none-extensible. Since as a developer you are entirely blind to what the VBox project team is doing, you have no way of knowing if your work will ultimately enhance the product or be rendered redundant by VBox project team developments. So if you take my feature list that I gave in the Roadmap post, as I developer if you knew that the team were not working on X, but that you had some good ideas here, then you could be reasonably confident that you work might ultimately get integrated into the core product, but you just don't know.

The one area that I have been considering working on is a standalone tool / utilities for the manipulation of VDIs, but is it worth it?

[sandervl]

In responds to TerryE:

a) we don't have the time to form a committee to openly discuss roadmaps, features etc. Such a development model would only hurt progress. We are currently happy with the kind and amount of contributions we receive.

b) the OSS version isn't crippled beyond us. Parallel port support was e.g. contributed by an external developer. Lack of USB didn't seem to bother him, regardless of your opinion about it.

c) if you had bothered to look at the project in detail, you would have noticed it is actually extremely modular and extensible. There's a plugin architecture for virtual hardware and host services (host-guest communication) for instance.

[TerryE]

Sander, I didn't suggest forming a committee. I just wanted to see some willingness to have open debate. I didn't say that the OSS version is crippled beyond use, but that it was cobbled. According to my dictionary there is a significant difference in these two statements.

As you also know from previous posts, I do spend quite a lot of time through the source and getting to grips with the product and trying to share that understanding with our user community. I agree that the PDM and HGCM designs are excellent and that they architecturally extensible enabling developers to extend the system. However, AFAIK, the PUEL edition will only load the DD and DD2 modules; you can't add your own; to do that you need to tweak the OSE edition -- and forgo iSCSI, USB, RDP, SATA, ...

What I am trying to do is to have a sensible discussion and you respond with a rebuttal. Doesn't this just underline my point about your team's relationship with the community being dysfunctional?

[sej7278]

What I am trying to do is to have a sensible discussion and you respond with a rebuttal. Doesn't this just underline my point about your team's relationship with the community being dysfunctional?

gotta agree with you there, i love the product, but its being run by sun a bit like "when we took it over it was gpl so we have to have an ose edition, but we'll let it rot".

i was a bit shocked at another thread where someone from sun said something along the lines of "3d acceleration might be on the agenda, you'll have to wait and see", the first thing that i thought was surely its in the svn logs or the project roadmap - but then i realised that the ose edition won't have the 3d stuff, and there is no *public* roadmap!

come on sun, if you really want community support, you have to support the community, not snub us. even if you don't want/get code from us, there are some very good howto's that have been written that have probably meant the difference between users ditching the product after a few hours [of trying to understand the useless manual] and sticking with it longterm.

[Phobos]

nobody's letting the OSE rot... it's just as it has been since january 2007... exactly the same as the binary version, minus a couple of things... same thing as it was pre-Sun innotek

about that couple of things, you really need to think about relevance... common desktop users won't care about something like the iSCSI initiator, Gbit Ethernet controller or RDP... (USB may be another tale to tell... but it's only one of the binary-only features..)

so, for the intended target (desktop users), OSE is 100% functional... not crippled nor cobbled... a couple of extra features don't make it any less functional

I don't really know where your conclusion of a 3D-less OSE came from (you may or may not be right), but AFAIK, there was some initial code for this on the SVN...

Sun doesn't really need to share their roadmap... virtualbox is open source software... nobody's really stopping the community from establishing it's own roadmap and work over it to get the features they need... I have seen very capable people around here, able to do such things... (with the latest being Brendan's experiment of a "baremetal" VBox)

Also, you have to put yourselves in Sun's position... if you can't deliver the things proposed on a roadmap, users will complain (even if they don't pay)... the only option left is to say what is on the works, but not when it's planned to be done... and that's pretty much what sander has done so far... Terry's reversed engineered and date-less roadmap looks very reasonable and I think what's left is for vbox' team to confirm or deny it... but anyways, does knowing those things change anything? (this is a real question I'm asking, I'm not being sarcastic here)

I do agree there seems to be some lack of communication between VBox team and the community, but I don't see it as a complete show stopper, Terry... with the exception of USB, I think most developers (just as end-users) will find that the OSE version fits their needs... lately, I have been reading this in a somewhat greedy position from the community... not everything must absolutely be in the OSE version... but lack of communication seems to be one of the problems leading to this... IMHO, maybe a good reason for USB and the others modules not being in OSE will help convince people that OSE is not an incomplete version of VBox

and to lama's comment, if for you "democracy and dictatorship" equals "open source and proprietary" in software, then explain the linux kernel development model, that, while being open source, it's pretty much a dictatorship...

[sandervl]

Sander, I didn't suggest forming a committee. I just wanted to see some willingness to have open debate. I didn't say that the OSS version is crippled beyond use, but that it was cobbled. According to my dictionary there is a significant difference in these two statements.

Well, from this and many other comments you have made about the OSE version, it's hard to conclude anything else.

As you also know from previous posts, I do spend quite a lot of time through the source and getting to grips with the product and trying to share that understanding with our user community. I agree that the PDM and HGCM designs are excellent and that they architecturally extensible enabling developers to extend the system. However, AFAIK, the PUEL edition will only load the DD and DD2 modules; you can't add your own; to do that you need to tweak the OSE edition -- and forgo iSCSI, USB, RDP, SATA, ...

What's your point? The OSE version allows you to add anything you want. If it's useful and you contribute it back to us, it will appear in the PUEL version too. Otherwise build your own version and distribute it.

And it is possible to load additional modules in the PUEL version. See pdmR3DrvInit in src\VBox\VMM\pdmdriver.cpp.

What I am trying to do is to have a sensible discussion and you respond with a rebuttal. Doesn't this just underline my point about your team's relationship with the community being dysfunctional?

So correcting false statements is dysfunctional communication?

We have two goals with VirtualBox:
1) meet our product goals and continue to generate significant revenue
2) maintain the open source presence/community

The first is obviously the most important. Without revenue there won't be a product. Without our contributions to VirtualBox it's pretty much a dead product, which won't do the community any good.

We are investing significant time and money into maintaining the open source version and an on-line presence. If there were no benefits, we wouldn't do this obviously, but I'm getting quite tired of the demands of certain people for more and more. An open debate is no problem if it is done properly and with a bit of respect. I don't think that's too much to ask for. (this is a generic statement about this topic)

[TerryE]

[Before I go any further, I do have an apology. I used the adjective cobbled to describe the OSE version. What I meant to say was hobbled. My mind fart, sorry. (For those who are not familiar with the term, a hobble was a device put on an animal's forelegs to limit its movement.) My intent was to qualify the scope of the OSE version compared to the PEUL version alone, not its quality.]

Phobos, I agree that if you are a typical end-user that doesn't want virtualised USB support, then the OSE version is a fine VM product for you to use (especially if you use Ubuntu or one of the other Linux distros that provided a pre-built virtual-box OSE distribution). Sun provides an extremely valuable service to the community by providing the VirtualBox OSE version. I also agree that Sun clearly has the freedom to do what its wishes with VirtualBox, and can also free to engage with the open community as it so wishes. However, all of this has little relevance to the thrust of Alexey's original post in this discussion:

VirtualBox, has very wide OSS community of users and supporters, but don't have even 1 community developer. Why is this ?

and it was this point that I was addressing. What I was trying to explore is why developers are not engaging in the VBox OSE project. Such developers are the very sort of people who want to play with headless configurations, advanced peripherals, iSCSI, SATA, etc. AFAIK, it is quite difficult to install multiple copies of VBox on a single system and you can't run it inside a VM, so unless you have multiple sandpit H/W you have to choose between OSE or PEUL: flexibility to tweak or lots of advanced goodies.

The vast majority of my posts in this forum support Sun, its VirtualBox product and its user community. I let this body of contribution stand a sign of my respect of the team's work and product. I feel that I have proposed an argument in this topic that Sun's current stance is a significant factor in this lack of developer engagement, and that Sander's responses simply underline my point.

[sandervl]

I think there are three fundamental problems here:

• VirtualBox is non-trivial software. The learning curve is very steep.
• Some people dislike the commercial aspect of the project. That aspect is perhaps a bit more obvious than with other OSS projects. (closed features)
• We have a highly skilled team that keeps the pace very high: hard to keep track.

The closed roadmap might scare off people, but what ever happened to asking and starting communication? If we are aware of somebody working on an feature that we will soon start on, then we have no intention to waste their time. If you have a good idea, just let us know. There's a developer's mailinglist for such things.

The product has a very high code quality and an extremely modular architecture. There's even an SDK with a developer's manual. Developing extensions for the PUEL version is very easy. There's a defined API for everything. And yes, there are people actually using it.

The assumption that there are zero contributions is plain and simply false. As I said above, parallel port support and some linux opengl work was contributed by an external developer. Why haven't you heard more from him you might wonder. The answer is simple: we hired him.

A lot (most actually) of the translations for the GUI are also contributed. We also received smaller patches from time to time and very detailed bug reports (made possible by the product being open source).

By the way; excuse me if have forgotten any contributions here.

The assumption that we'll receive many more contributions if we only open source everything is very questionable. The people that complain the loudest about this imho just want everything for free and have no intention of contributing.

PS: Mr. Biscuit, I've deleted your postings as they are completely off-topic.

[TerryE]

Sander, re your "fundamental problems", the first two are to do with the nature of VirtualBox: its complexity and the tightness of its team. Yes, some developers have an almost religious dislike of commercial project, but these won't sign the Sun Contributor's Agreement and so their contributions would not be available for integration into the mainline product. ((I signed the earlier Sun JCA years ago). Where I disagree is that I see these as environmental constraints rather than fundamental problems. Other OSS projects have similar constraints but nonetheless have a healthy independent developer community. Sun’s own OpenOffice.org is a good Sun example here. I also think that all of the posters on this topic understand commercial realities, so that isn't core to this debate either.

AFAIK, no one so far in this discussion has assumed "zero contributions." (Your example of your relationship with the NL communities is a good example). We are not talking about all or nothing, we are discussing degree and how we increase it. The irony is that we both agree about a core issue here: Sun is a commercial organisation, and for VBox to have a future it must be commercially successful and profitable. What I am suggest is that you and I have a different mind-set in how you can approach this goal. I am concerned about Sun’s lost opportunity here: for the Vbox team to broaden its base and strengthen its product by embracing its own Public Policy. I believe that Sun can do this without compromising and in fact can enhance its commercial position.

In terms of the PEUL edition we are still talking at cross purposes. Nowhere in this discussion or any other related topic have I suggested or even asked that Sun release the closed code under GPL or an equivalent licence. What I have discussed is how to allow developers the ability to develop or to debug without effectively denying them access to PEUL functionality under PUEL terms. There are many options doing this which do not involve GPLing the PEUL code, including:

• Facilitating dual host installation (PEUL and OSE versions)
  
• Releasing the source under a restricted developers licence; that is the PEUL source itself and any builds which include such source cannot be distributed. IBM has done this for 40 years. MS does this. Sun does this on other projects. So what’s the issue here for Vbox — apart from the fact that Innotek didn’t do this historically?
  
• Only releasing the PEUL components as shareable modules or object libraries, again under a developers licence.
  
• Improving documentation of the APIs and extension techniques.

Sun can either choose to reach out to the development community or it can choose to adopt a combative stance. Yes, it is entirely free to choose which, but labelling people who are advocating alternative polices to the current dogma as wanting everything for free and having no intention of contributing is an example of the latter which adds little value to any debate.

[Technologov]

Well, I think we can try to find a middle ground between the commercial interests and the OSS community.

One wild idea on how-to get both hares at once:
1. Keep the closed-source modules closed, but make the C++ headers open.
2. PUEL license will allow those developers use binary DLLs (and Unix equivalents) for Open-Source developer to use the closed source functionality within their private development environment.

There is still no guarantee that it will bring more OSS developers.

-Technologov