More on bin sizes and what is a "good" Certification Level
In an earlier post about bin sizes we gave some explanation about how our system works in terms of relative value of a particular score (always keeping in mind that this is the 1.0 version of the site and updates to the algorithms are always "just around the corner").
Suffice it to say: if you got a 5 or above you should feel really proud because you're among the best contributors to the project. And of course, as always, if you got a 1 you should be exited about that to (and hopefully excited to improve your score). We consider everyone with an account to be a Rock Star ;)
Today we're happy to share some more insight.
Count of Rockers by Certification Level
For the visual folks, a nice graph:
And now, a table of the rockers by their certification level.
| Count | Score |
| 2421 | 1 |
| 1133 | 2 |
| 674 | 3 |
| 1318 | 4 |
| 446 | 5 |
| 80 | 6 |
| 50 | 7 |
| 4 | 8 |
| 7 | 9 |
| 1 | 10 |
| 2 | 11 |
You can see now when we say that "a 5 is a good score" we really mean it. It means that if you're a 5 or above then you're in the top 8% of the users in the system. Given that we prepopulated the system with 6136 users who were likely to be high score, being a 5 or above means you are well above the 92nd percentile of all drupal.org users.
By the way, scores of 11 are Dries and merlinofchaos (Earl Miles) while Karen Stevenson has a 10.
- Log in to post comments
Comments
Live data?
Is the data on which the score is based live?
For example, if I help a thousand people, make a thousand commits, and go to the 10 past drupalcons by tomorrow (pretend I have a time machine), does it get reflected immediately? If not, when?
Btw, I'm a happy 5.
not live (not yet, anyway)
No, the data requires us to crawl drupal.org (and do some other stuff) so it is not "live".
We are only going to add new users, crawl data for everyone, and publish that new data every few months for now. We obviously plan to make that happen more often. Follow @certifiedtorock or the blog feed to get updates.
Good to know. IMHO the actual
Good to know. IMHO the actual certification pages should have a "What this score means" link that goes to this page.
definitely!
That's in our plan already, just need some more time.
There are 812052 Drupal users
There are 812052 Drupal users as of this writing. There are 590 users with a 5 or above. That makes anyone with a 5 or above in the top 0.073% of Drupal users.
Doesn't quite seem right to me to give those users a score of 5 of 11... 5/11 and 99.827th percentile does not mean the same thing to clients.
somewhat off
The most recent UID might be 812,052 but for a while now drupal.org has been skipping UIDs as part of it's master/slave replication plan. So, the number of users is closer to 600,000. There's also the chance that we didn't include from the top down so I don't agree with your math for multiple reasons.
We are still figuring out where the users should be on the scale and a 5 does feel low. That's part of the fun of building/refining a system.
Community involvement
Certified to Rock is about involvement in the community. There might be 600.000 users on drupal.org, but most of those might have registered to post a question in the support forum or ask for help in one of the issue queues.
Having trouble with the term "certification"
While I can appreciate self-promotion, and the folks most active on drupal.org who give back certainly deserve recognition - what about competent developers who are too busy working on sites to do much on drupal.org? What about veteran LAMP developers who just haven't been at Drupal long enough to have a high score, but who do expert level Drupal development? I don't see how this can be useful as certification, if it effectively excludes competent newcomers. Should we be expected to spend hours a day on drupal.org for a couple of years before we have professional recognition, even if we're doing high quality work right now? Other certifications do not have these problems, because they're based on knowledge, rather than how long you've been contributing. These are gold stars for achievement, not industry wide certification, as far as I can tell. Also, the 5-6 times I've tried to check my score, I wasn't able to, which just adds insult to injury ;)
Also, the 5-6 times I've
When you submit a person to be scored the scoring doesn't happen right away, we have to come along and turn a few gears and have unfortunately, but also fortunately, been busy. Sorry for the confusion about this.
I'm still not on either.
I'm still not on either. Can't wait for a few more people to be added. Not sure why the crawler can't add a few more each day until its done. Is there any timescale for when more users will be added?
it's not a requirement, but an extra bonus
It's really not particularly about self promotion. It's about promoting drupal.org and contributing to Drupal.
And keep in mind that being CertifiedToRock is not a requirement to be involved with Drupal - it's a bonus for people who want it.
That said, it sounds like your complaint is that becoming CertifiedToRock takes too much time from people because they have to spend time contributing to drupal.org. And that is certainly a valid criticism. However, consider the alternative...
We expect other certifications to pop up in the future and those certifications are very likely to
So, either way people are going to have to spend time and in the case of traditional certifications they are also going to have to spend money. Wouldn't it be nice if the "exam" were free and if much of the time that goes into passing the exam instead went into improving the Drupal project.
What I'm saying is: Your complaints are valid, but they are really about certifications in general and not specifically Certified To Rock.
My concerns are not about certifications in general.
Let me reiterate my issue since I don't think it was understood. I don't think it's fair to call this a certification. Certifications are based on a standard of knowledge, and not on your personal history. That's just a basic requirement for an industry standard... Now with that said, I agree that certifications have problems (sometimes useless, often expensive, etc) but this is NOT a certification in any traditional sense of the word, and it's certainly not what management think of, who are the folks who tend to care about such things. Calling this a certification is an injustice to folks getting into the Drupal field, because if it WERE a certification, it would put them at a significant disadvantage to anyone who had been active on drupal.org for quite a few years in spite of their competence and value as a professional, and that's not what certifications are about. While we in the drupal community may care about giving back more than some communities, it's still not cool to discourage newcomers by making it seem they've got a vertical wall to climb to get any professional recognition. That's what this is doing, by calling itself a certification, IMHO. That said, the general idea of giving recognition to contributors is awesome... just not marketed in this way.
You are right, sir
Definitely not! We made the site look non-traditional, we are pretty clear in most blog posts, and the title of the site alone "Certified To Rock" should help make it clear that this is not traditional.
I'm not sure I agree there. Certainly account age plays a role in the certification level, but it's by no means a heavily weighted factor. One of the things we really look forward to highlighting in the next year are people who are brand new to Drupal and have a really high score. Frankly, we plan on using that list to identify people to hire! I hope that when we publish that first list you will check back and consider the system/situation.
I disagree
I would much rather hire a developer with 3 years' Drupal experience, 5000 drupal.org commits, 7 modules on drupal.org and a long tracker history of answering support questions than I would a developer with 10 years of PHP experience and 3 months of Drupal experience. I would also much rather hire a developer with a drupal.org history than one with equivalent skill and experience but no drupal.org history. In other words, yes -- normal certifications are tests of skill, and this one isn't as much -- but it's a strong (albeit imperfect) indicator of Drupal knowledge nonetheless. And yes, you do -- and should -- have a wall to climb to gain professional recognition if you are just recently entering the Drupal community.
You should take in account
You should take in account IRC karma so I can get higher than my current 4 :P
Show the score on our site
Hi,
Have you think about a way to let people include their score in their site by pulling the data from certifiedtorock.com?
Best regards,
Nestor
Usernames may change over time
Which means that in changing my username to something easier for people to cope with, I've become an interesting test case :D
Just flagging that this is something users are permitted to do on D.O, and the CTR system doesn't currently seem to cope with it.
updating usernames here to match drupal.org
Yep, that's a bug/feature we're going to fix...probably for our next release.
Don't look at numbers only
If this system only looks at numbers then it is flawed. For instance, someone may have thousands of commits to a single project, whereas someone else may have only a handful of commits to many different projects. Personally, I think someone who is making a lot of commits to one project should lose karma because it means they are not doing things correctly from the start. The measure of someone's skill should be based on their ability to solve problems in an efficient manner. Social interaction and community involvement is not a measure of tech skills. Soft skills are important but should not be a factor in the algorithm.
Rewarding sloppy sams
I would much rather value and reward contributing community members that are "Sloppy Sams" rather than "Perfectionist Petes" (read http://www.webchick.net/embrace-the-chaos). merlinofchaos has 2300+ commits to three separate projects. Does that mean he doesn't know what he's doing?
I would also much rather value a contributor who's not only active in CVS, but also in the issue queue, responding to users, and posting patches for peer review before committing.
Sloppy sams, but not gaming the system
I agree we should award sloppy sams, but we should take care not to let people commit a tiny change 5,000 times and then get a lot of credit for it...
There's the balancing act we maintain and there's part of the reason we keep the metrics/algorithms secret.
Skewed toward engineers and back end developers
This system seems skewed toward engineers and back end developers. I am a Drupal front end developer / themer and probably won't actually write a module, a patch or contribute to Drupal core code.
However, I have contributed and given back to Drupal in so many other ways (and countless hours) which I am guessing are not tangible or trackable by the algorithm used here.
The many ways I have contributed to Drupal include donating to and sponsoring a module, reporting bugs in issue queues (which were considered valid and then patches were written), helping numerous people out in the forums, blogging about Drupal on my website including how to's with screencasts (http://highrockmedia.com/drupal), spreading the gospel of Drupal on Twitter, developed well tuned elegant Drupal websites, the list kind of goes on...
My fear here is that there will be a Drupal elite class and I think that is just wrong. This article sums up some of my sentiments: http://thinkshout.com/blog/2010/09/sean/leave-no-drupal-dev-behind
thanks for the feedback
We welcome all feedback, but this is not particularly actionable. You identify things that help Drupal, but it's impossible for CTR to measure them.
Our strategy is based on:
We appreciate your contributions, but need ways to incorporate them into our algorithm, which likely means incorporating them into drupal.org. Can we find ways to do that? For example, you mentioned your videos as contributions and there is a screencasts section of the site and I don't see any of your videos in there. Adding them in would be a great way to help get more viewers for your content, help end users to find the videos more quickly, and allow CTR to give you credit for the work (I'm not necessarily saying doing documentation helps in CTR, but if it does, it's only documentation that lives on drupal.org).
If you have more ideas on how we can include coverage for the kinds of things you do that both help the Drupal project and prove proficiency with Drupal then we'd love to hear about it.
Updating statistics
Are there any plans to provide an up-to-date version of the data in this post, or the graph at Explanation of Scores? Not necessarily "live" bin counts, but at least the most recent.
Historical scores (for individual users and the community distribution) would also be nice, and perhaps useful. Especially if there was a guitar hero sound effect ;-)
My score began as 4, and went down to 3. Now It's back up to 4. Phew! Rock and roll, eh?