Greg's blog

CertifiedToRock is dead, long live CTR

Greg — Tue, 26 Nov 2013 17:37:07 +0000

Hello,

If you are in the USA, Happy Thanksgiving. If you are not in the USA, Happy Thanksgiving.

You may have noticed that there hasn't been a site update in a long time. I think the overwhelming feeling among the groupies is sadness that we don't have time to put in the energy to the site that we want to. As the french say: Le bummer.

Now what?

Do you have great hope for CertifiedToRock? Do you have time to put into it? Great!

We are looking to sell the site. You would get: domain name, code, architecture docs and resources, and all artwork in the source format (a mix of svg, xcf and psd). We'll also gladly consult with anyone who makes a serious offer to discuss ways to make money from the site.
We still believe in the mission of CTR so preference is given to inquiries that align with CTR's mission. We'll consider the monetary value, the reputation of the buyer, and their plans for how to use/build CTR.

We (Ben and Greg) have incurred nominal hosting and github expenses in running the site since GVS stopped running it. We plan to pay that back and then let the groupies decide how to invest the money back into the Drupal community. Chances are strong that we would give it as tips to members of the Drupal community on gittip.

Why should you buy CTR?

We think anyone who hires a lot of people (recruiters, large Drupal companies) can make CTR a very valuable part of their recruiting process. When we would run the analysis on the community we would look for people who shot up in ranking and consider hiring them. It's a great way to identify rising stars in the community! There's also room to build out small additional features and sell ads or premium listings alongside those new features. This could easily become a part-time job for someone and maybe a full time job (especially if you already do work related to training or recruiting or marketing Drupal firms). And that's just about monetary motivations.

The site has thousands of links from around the internet. It has become a "go-to" resource for quickly guaging the skils of a Drupal developer. It gets a fair amount of traffic even though we have not updated the metrics or evangelized the site in over 2 years.

When we launched CTR we got a ton of attention (mostly favorable, even). If you sponsor Drupalcon and Drupalcamps, consider that being the entity behind CTR is like having an evergreen sponsorship to get attention from potential clients, other Drupal shops, potential employees, etcetera, etcetera, etcetera

Why are we selling it?

Since we believe in CTR's mission we think a good way to ensure that the next owner is responsible with the site (for whatever missions they have) is to sell it. Someone willing to pay for the site is unlikely to turn around and throw it out (at least that's our theory). We also want to repay our wallets for the small amount of hosting and github service fees we've personally incurred since the GVS days.

Make an offer

Go and do it.. If your offer isn't for a lot of money, please offset that by being exceptionally entertaining.

Certified to Rock August 2011 Score Update for Drupalcon London

Greg — Thu, 25 Aug 2011 14:12:54 +0000

Hello Drupal fans,

Hot on the heels of the announcement of our groupies program designed to expand Certified to Rock and make it more community driven, we are releasing a new set of scores. We are excited to announce the first new groupies though we are still solidifying the group. Look for an announcement on that in the coming weeks and remember that if you would like to join we are looking for candidates (see the post for details on how to join).

This update is a "resource refresh" meaning that we didn't add any new metrics or change weights or calculation functions in the algorithm. Quick guide: resources are information we scrape from drupal.org. metrics are individual elements of those resources, a specific set of weights and calculations are used to tabulate the final 1-to-11 score in an algorithm.

Congratulations to Peter Wolanin who jumped up two spots to become an 11 and to Jesse Beach who moved the most of any user in this update. Without revealing too many secrets, we can say that those two changes were both caused by some changes to underlying data that happened as part of the migration to use Git on drupal.org. Our next improvement to metrics and algorithms will be tweaked a bit to take better advantage of this information. And of course progress cannot be stopped. Even as we work to adjust for current changes folks are busy planning for even more fine-grained attribution of credit for work. We encourage everyone to use the "--author" attribution mechanism available now if that seems appropriate on an issue and to weigh in on the discussion about how to give even more credit.

The usual caveats apply, of course: scores may go down in the future, while the system includes 82,000 of the best contributors to Drupal it's missing a few folks, don't ask what elements go into determining scores, please suggest ways we can improve the system, Drupalers of the world are rockstars and we are your humble groupies.

Tags:

Certified to Rock's Drupal Groupies - August 2011

Greg — Tue, 16 Aug 2011 16:38:29 +0000

With the recent acquisition of GVS by Acquia many people are curious what will happen to Certified to Rock. Acquia has long been thinking about certifications and how to do them well in the Drupal community. But there was a clear sense by those involved in Certified to Rock that CTR would best be served by remaining an independent project. You may have noticed that the site design changed a little bit. We removed all GVS branding from the site and are instituting a program of "Groupies" to support the site.

Our philosophy is strong: the entire Drupal community is rock-stars and the folks working on this site are your groupies. At the same time as providing a means of certification, this site is meant to encourage and promote the best elements of the Drupal community.

So, who are these groupies?

Currently they are:

The oath of the groupies

All groupies agree that they will:

Keep the secret part secret for the rest of your life—the secrecy is a feature, not a bug. If you ask a groupie about the CTR secrets, they will try not to reveal information. If you catch them revealing teh secrets, let us know. That'll earn them bus-bathroom-cleaning duty for a month!
Improve CTR in ways that support and improve the Drupal project.
Once a groupie, always a groupie - even if their interest (and vote, it's a do-ocracy after all) diminishes folks remain CTR groupies forever.

Seeking additional groupies

The current set of groupies has a mix of development, design, certification, and of course, Drupal community experience. We need more of that. In particular, if you want to work on building an embeddable widget, expanding the algorithm to be more accurate, work on scalable scraping and data processing, and you are willing to agree to the oath then let us know.

New groupies will be admitted based on very important scientific process that mixes tea leaf analysis, Turkish coffee readings, and propensity towards getting shit done.

Do you want to become a CTR groupie? Are you familiar with the Drupal community, issues around certification, and have some free time to contribute? Contact us.

Tags:

groupies

Certified to Rock's third algorithm update - Spring 2011

Greg — Mon, 16 May 2011 19:34:28 +0000

Certified to Rock has recently finished re-calculating ratings, with fresher data and a new algorithm. This is the second update of 2011, after the January 2011 refresh that added 82,000 people in the system. It is the third algorithm update since the April 2010 launch at Drupalcon San Francisco.

The data for this update was collected earlier this year, so won't reflect your most recent work. But, it does include twice as many metrics from across the drupal.org universe as the previous algorithm (which was itself an expansion over the previous number). This means CTR levels should be more accurate for a broader group of contributors.

Distribution of Certified to Rock Levels

The distribution of ratings is still skewed heavily toward the lower end, though folks are creeping on up the ladder.

Count of rockstars	CTR Level
8	11
9	10
13	9
46	8
140	7
326	6
333	5
801	4
12203	3
41556	2
26577	1

Newest Drupal Rock Stars

The most amazing new contributor is Danpros (Danang Probo Sayekti) who received a level 7! Two users who both are at the 5 level are dczepierga (Damian Czepierga) (dczepierga on CTR) and amateescu (Andrei Mateescu) (amateescu on CTR). These three had accounts for about a year or less when the resources were gathered for this edition of Certified to Rock.

It's interesting to note the country of residence for these three individuals. Danpros is form Indonesia, Damian is from Poland and Andrei is from Romania. According to Wikipedia, the gross annual wage in each of those countries is $2,103, $8,753 and $4,170 respectively. In my research for my talk about Certifications at Drupalcon Chicago I learned that some traditional certifications like the Oracle program can cost $2,000 to become fully certified, and test takers are prohibited from taking the test if they live in certain countries. The cost of that Oracle certification is roughly 100%, 25%, or 50% of a year's wage for Indonesia, Poland and Romania. Can you imagine spending an entire year's wages to become certified? And that's just the test fee. It doesn't include books, and it assumes you pass every test the first time.

We are very proud to say that anyone who contributes to Drupal can become Certified To Rock, regardless of what country you are from or how much money you make.

New Certified to Rock Field on Drupal.org

In the last update, we publicly announced the availability of json output for a user. As usual, the Drupal community was fast to respond and incorporate the new technology. There's now a Certified To Rock Field for Drupal 7 which takes a drupal.org UID and looks up their CTR level. Great work by Steven Jones (CTR level 7) of Computer Minds.

As always, we hope you enjoy the site, that you'll include a link to your CTR profile on your drupal.org page or blog, and that you'll always keep on rocking in contributions to drupal.org.

Certified to Rock Drupal's 10th Birthday - Level and site updates

Greg — Sun, 16 Jan 2011 20:44:13 +0000

Our latest score update saw some interesting changes. We updated the site and we refreshed the data. We added 58,000 people to the system. It's now at 82,003 rockstars!

Some highlights:

Because this was just a resource refresh, 90% of the people who had a score stayed the same and about 10% went up
Dave Reid and dww are the newest 11s, joining the ranks of Dries, moshe weitzman, webchick, merlinofchaos, and Sun
The biggest score increases were dstol, tristanoneil and chia who all went up 4 levels!

Running on Drupal 7

We upgraded certifiedtorock.com to the brand new Drupal 7. We were previously running on an early alpha version. It feels great to be on solid footing and get all the Drupal 7 features and bugfixes that happened since we first launched the site.

Resource Refresh

We had the system collect updated data that the score levels are based on. We didn't change any of the metrics nor their weightings in the final algorithm &emdash; just the data that is input into the system. The result is that these new scores are a direct comparison to how you were doing in our last release in September of 2010. This data was collected between the last days of 2010 and right before the release of Drupal 7.0.

Interested in incorporating Certified To Rock into your site or application? We're currently providing json data for users. You can see it by going to http://certifiedtorock.com/u/1/json - basically tack "/json" on the end of any user ID. So far we don't know of anyone using it, but look forward to seeing who is first ;)

Tags:

resource refresh

Newest Drupal Rockers - Fall 2010

Greg — Sun, 03 Oct 2010 17:35:43 +0000

In his great keynote presentation at the Pacific Northwest Drupal Summit Josh Koenig (CTR level 6) had a lot of great quotes, but one that really struck me was that

the smartest, most creative person in the room may be the guy who just walked in.

His point is that if we only look to current members and contributors to lead us then we will lose out on the people who will really grow the project in the next 5 years.

One thing that I think is really interesting about the Certified to Rock Scores is looking at people who have a high score and are new to the project.

Certified to Rock and New to the Drupal Community

This list shows people with a score above 4 who have been a member of drupal.org for less than 2 years when the score was calculated. There are lots of great people on this list. I immediately recognize Lisa Rex who is a member of the Growing Venture Solutions team. Everett Zufelt who quickly became Accessibility Maintainer for Drupal 7 after giving a great presentation at the Drupalcon San Francisco Core Dev Summit. mr.baileys is a member of the Drupal Security Team, so I know him from there, but has also contributed in other ways.

And of course the profiles of the other folks speak for themselves...

Certifications don't have to be exclusive of newbies

Sean Larkin wrote a great blog post on his thinkshout.com about some of the problem he perceives with Certified to Rock. One of them is about how the scores are used and whether the system might serve to exclude people rather than pull them in:

3. We are flooded with ourselves.
We are largely a self-taught community. Part of why Drupal is so popular is that it's an easy framework to pickup and learn without familiarity with other development tools. If we focus on hiring developers strictly based upon Drupal experience, rather than plain-old software engineering experience (or God help us, rather than with folks that have been working in Rails or other equally-to-more robust development frameworks), we will miss learning opportunities and we'll become the stereotypical Royal Family - and a little too close to close to kissing cousins.

This echoes a concern that Bill Fitzgerald raised and I addressed: Certifications are mostly useful if your time of engaging with the individual is very short. If your engagement is likely to be long (like with an employee) then you are better off hiring the person with the right personality and teaching them Drupal.

That said, Certified To Rock is a scalable automated system based on publicly available data and someone who gets active will quickly find their Certified to Rock score is sky high.

We plan to keep releasing data about new Drupal rockstars as we update our algorithm and hope that this will help encourage new contributors to get involved quickly.

State of Drupal Certifications in 2010

Greg — Mon, 27 Sep 2010 15:04:35 +0000

If you ask most folks they'll say that there is no certification for Drupal. However, the truth is far from that. There are at least 3 launched programs and 1 planned certification.

Acquia Certified Engineer (joke name, Certification may really be coming)

Acquia has been talking for a few years about their certification. More recently "Certification" on Acquia.com is about their services to help with the United States Goverment Certification and Accreditation process, which is more about security than stating an individual's specific level of skills.

The name Acquia Certified Engineer was a bit of a joke started by @jhibbet on Twitter and propagated by Heather James when she retweeted it.

Though @hjames quickly recanted the naming and pointed out the irony in the initialism (ACE being UK slang).

Nobleprog Drupal Certification

NOTE: they updated their site and removed this certification, the screenshot below shows the details as does this forum post on drupal.org

The folks at NobleProg created a certification and prominently displayed the completely-unrelated-but-official-looking Drupal Association Badge at the top of the page describing the certification. If we overlook that somewhat shady practice what can we say about the program?

It's broken down into three certifications:

NobleProg Drupal Certified Administrator
NobleProg Drupal Certified Developer
NobleProg Drupal Certified Architect

It's hard to say how many people are certified under this program. Based on Quantcast data showing the site gets unmeasurably low traffic and Compete and based on the fact that nobody puts this as part of their online profiles/resumes (a search has zero results) I guess that not many people use or respect the Nobleprog certification.

Gloscon Drupal Certification

The folks at Gloscon.

While there is a bit of a mixed history with BPOCanada/Gloscon, they play an important role in the Indian Drupal market leading camps and they also play some role as an offshore/outsourcing development company serving other locales.

However, again, nobody seems to be saying they have passed the gloscon drupal certification nor a gloscon certified drupal professional.

oDesk Certified Drupal Administrator

oDesk has also been offering certifications via their tests for a few years. They provide an embeddable image for people who have taken the test and several people are showing it on their sites. The test was launched in 2009 along with their Drupal Group.

I took the "Drupal 6.14" test and in 15 minutes got a "4.6 out of 5" which put me in the top 10% of people who have taken the test. The quality of the test is relatively horrible. Their question about how to block spam mentions Captcha or Spam modules, but ignores Akismet, Mollom, reCaptcha, and the many other popular methods. The question about page caching has two possibly correct answers - admin settings or the config file - but setting a variable in the settings.php (aka config file) is a relatively uncommon and underused feature that I imagine the test writer was not aware of.

Given those criticisms - is the oDesk certification test useful in identifying people who have skills in Drupal? Probably at the lower end it does a good job of figuring out whether a person has some basic knowledge. It won't help you find someone who can write a theme or a module. And it probably won't help you find a great site architect. But if you need someone with basic skills to do some basic things on a site...it would probably work OK.

I'd love to hear feedback from anyone who has hired an oDesk Certified Drupal Administrator.

Certified to Rock

On the other hand...we have Certified to Rock which has seen some strong uptake.

Open Source Catholic is proud to be Certified to Rock level 4.
indytechcook is proud to be Certified to Rock level 5.
Bojan Zivanovic is proud to be Certified To Rock level 5.
Brian Gilbert is Certified to Rock level 4.
Jay Wolf is proud to be Certified to Rock level 6 (though it seems his site is offline...).
claudiu.cristea is Certified to Rock level 6.
Fleet Thought is Certified To Rock level 5.

And a ton of people on twitter (many of whom we bribed with awesome hats) are Certified to Rock at various levels... (see a google search for the site).

In announcing their job openings, the fine firm Lullabot asked for people to show a Certified to Rock score of 5 or spend a little more time describing their skills and achievements. Which seems to us like a great way to handle a certification - it should help open doors but not guarantee a job nor exclude anyone.

This FreelanceSwitch Job asks for a CeritifiedToRock score. Riff.org asks for a score of 2 or higher in their job post in France. Similarly, Grammy.com's job opening asks for a Certified To Rock score. This might just catch on ;)

The future of getting certified with Drupal is still largely unwritten. We hope that Certified to Rock will play a role in shaping it as a process that has low barriers to entry, practical, and which reinforces the community orientation that Drupal has always held so strongly.

Update: I forgot to include the Lullabot Drupal Certified post which mentions that they give every attendee of their training a certificate of completion that makes no guarantee of the individual's skills or retention of the information. So, even they are clear that what they offer isn't a certification beyond "he took the course."

Tags:

drupal certification

certifications

More on bin sizes and what is a "good" Certification Level

Greg — Tue, 18 May 2010 22:26:23 +0000

In an earlier post about bin sizes we gave some explanation about how our system works in terms of relative value of a particular score (always keeping in mind that this is the 1.0 version of the site and updates to the algorithms are always "just around the corner").

Suffice it to say: if you got a 5 or above you should feel really proud because you're among the best contributors to the project. And of course, as always, if you got a 1 you should be exited about that to (and hopefully excited to improve your score). We consider everyone with an account to be a Rock Star ;)

Today we're happy to share some more insight.

Count of Rockers by Certification Level

For the visual folks, a nice graph:

And now, a table of the rockers by their certification level.

Count	Score
2421	1
1133	2
674	3
1318	4
446	5
80	6
50	7
4	8
7	9
1	10
2	11

You can see now when we say that "a 5 is a good score" we really mean it. It means that if you're a 5 or above then you're in the top 8% of the users in the system. Given that we prepopulated the system with 6136 users who were likely to be high score, being a 5 or above means you are well above the 92nd percentile of all drupal.org users.

By the way, scores of 11 are Dries and merlinofchaos (Earl Miles) while Karen Stevenson has a 10.

Not all bins are created equal

Greg — Wed, 21 Apr 2010 07:16:37 +0000

If you're a nerdy data analysis person you like to talk about "bins" of users.

CertifiedToRock.com has a scale from 1 to 11, but there are more 1s than there are 11s. The "bins" are not equal.

We really want to promote involvement in the project with this certification system. So, it's easy to move from a 1 to a 2. It's a little harder to move from a 2 to a 3 etc. This is about giving people success early so they become addicted to it.

But then going up the scale at the high end our goal was to make the scale into a useful distinction between levels of contribution.

The other thing about this is that we preloaded the site with a few thousand accounts, but nowhere near the number we hope to certify in total. And when you only have a handful of the number that you hope to eventually include there's no way to split them into exact sized groupings of folks. So, this first cut is a lot rougher than future cuts will be. In the future we will publish a distribution of how many people there are at each level, but not yet...

Suffice it to say: if you got a 5 or above you should feel really proud because you're among the best contributors to the project. And of course, as always, if you got a 1 you should be exited about that to (and hopefully excited to improve your score). We consider everyone with an account to be a Rock Star ;)

How can I improve my Certified to Rock level?

Greg — Tue, 20 Apr 2010 16:20:40 +0000

One of the first questions we are getting from folks who use the site (yay!) is "how can I increase my certification level?"

Improving your Certified to Rock certification level

Well, the answer is both simple and extremely complex...

Simple answer is just "get more involved in contributing to the Drupal project." Our goal is to measure community involvement and develop the system to provide scores to people who have contributed strongly to the community in meaningful ways that approximate how well they know the Drupal software.

So...the complex answer is one that has been covered in dozens of places, but one decent guide is contribute page on drupal.org.

Do that, and you will become even more of a RockStar.

What kinds of things are included in the system?

Lots of people want to know what's in the algorithm. We're not revealing specifics. But we do want ideas on how to improve the accuracy of our measurements and it's important to consider the impact a new metric might have on the algorithm and the Drupal community.

Here are a couple of the criteria we use when including a new metric into an algorithm.

It must be easy to automate. There are about 500,000 people on drupal.org and more every day. We can't add something to the system that is going to require much manual work. Certainly nothing that requires us to manually do a task more than a few dozen times. So, we could make a list of everyone who organized a DrupalCon and have the algorithm system use that because that's only a few dozen people (we don't currently do that and may never). But we aren't going to go through github, launchpad, and gitorious and associate people's ID on those systems with their drupal.org user ID and then have the system give credit to people who send lots of e-mails. That's not scalable. Pro tip: keep your work on drupal.org itself - if you don't like the way it looks/functions/whatever then help the redesign.
It must not encourage anything that is harmful to the community. This is somewhat tricky. If, for example, the system gave points to people who have a lot of projects on drupal.org then that would encourage the creation of lots of projects including a lot of really bad ones. That makes it harder for new users to find projects. Which is really bad. So, we don't use any metrics like that!
It must be balanced. One of the things we're really trying hard to do is measure skill and knowledge of Drupal in a way that is fair to people with different skill sets. Someone who is an awesome site builder (can't code much, can't design, but really can choose modules and configure them) should be on equal footing with someone who can design or who can code. This is...hard. In particular it is hard to measure the skill/contributions of designers/ux/ia people and site builders. So, if you have ideas on metrics that measure their skills, please share those ideas!