Criticism of certification

The typical certification program is expensive and outdated. Tests are expensive to write and administer so they usually are expensive to take. Traditional certifications are not always available in all parts of the world shutting out people who may benefit from them the most. They often require training from the certifying body prior to receiving certification. The tests take time to write so they frequently cover software that is no longer particularly relevant. And, they are really a test of whether the person studied to pass the test instead of being a test of how well the person can apply those skills in the real world.

This page helps address how CertifiedToRock deals with the common complaints about certification programs.

Brian Vuyk - Certifications are a burdensome barrier to entry

From an e-mail to the consulting @ drupal.org mailing list Brian raised these points:

1. Clients being what they are, if a certification becomes somewhat
common, it will become much more difficult to get freelancing work
without it.
2. For that matter, every IT / tech recruiter posting a job
description on any job sites will now require it.

This may be true, but it's not a problem in itself. This is only a problem when combined with the next items:

3. Certificate exam would need to be retaken every time a new version
of Drupal comes out.

Because our rating is based on publicly available information like contributions to the Drupal project it is always up to date as long as the RockStar gets engaged in the project.

4. Unless cost is very low, this is an added barrier to entry for
small shops and freelancers.

A CertifiedToRock.com certification doesn't cost anything beyond involvement with the Drupal project. If someone can afford the tools to do Drupal work (a computer, internet connection) then the additional cost to them is only their time.

What confuses me most, is what problem are we trying to solve with a
certificate program?

With CertifiedToRock.com we are trying to solve two problems. First, people claim to be Drupal experts who are not. They do a disservice to their clients by building bad sites and they do a disservice to the Drupal community because their clients then think that Drupal is a bad system. Second, we hope that this system will motivate people to help make Drupal better, possibly preempting a traditional certification program that only serves to enrich the test administrators and not directly improve the Drupal project.

A strong portfolio and contributions history tell far more than any certification ever could.

We couldn't agree more! Unfortunately that contribution history is opaque to most people outside of the Drupal world. This site tries to distill that complex history down to a simple number that anyone can understand.

A certification says nothing about a developers ability to manage a project, work ethic, or
creativity, all of which vastly impact the development process far more
than a set of Drupal-related trivia ever could.

In short, a certificate doesn't remove from the client the
responsibility to do their homework about a developers experience, and
meet with them.

Yes, people still need to their own evaluation of these softer skills.

Bill Fitzgerald - Certification is just part of a bigger picture

Bill has made several posts on this topic as well like this and this.

Most of his criticism boils down to a long term or short term decision. If you are hiring for the short term (a consultant or freelancer or a project on a short deadline) then Drupal skill might be a more important factor. If you are hiring for the long term then other factors weigh more heavily and those are the kinds of factors which can't be measured by most certifications.

That's just sound advice and inside of that is a recognition that certifications can be useful to the hiring decision as part of a complete evaluation.

Chris Johnson - Certifications are an expensive waste of time

Chris Johnson points out that they are expensive and a waste of time:

As a hiring manager,
I've always wondered exactly why the person spent the time and money
to get the certificate, and if that time and effort might not have
been better spent practicing the skills or otherwise gaining practical
experience.

Fortunately, CertifiedToRock.com's scoring is free and doesn't require any time beyond normal interaction with the Drupal project.

Liam McDermott - Write a module as a practical test

Liam McDeremott says:

Maybe a certification could be based upon completing a practical
project. The exam might have only one question: 'Write a contributed
module to do x' (it could require examinees to write documentation too).
This also has the added advantage of not needing to be re-written every
time the Drupal API changes.

Seems like a great system, but grading it would be hard and this only measures development and not theming, module selection ability, nor other skills. CertifiedToRock.com's score includes the ability to write a module as one of dozens of different measures of skill. The rating of those projects is, further, based on a variety of different measures.

Evan Leibovitch - Useless and expensive to create certifications without a business model

Evan registers several problems from the perspective of someone who has written a certification program.

- Usually people get certified as a way to help them get employed. Right
now demand for Drupal developers appears to exceed supply, so people
don't need certifications to help them get work and the actual numbers
of people willing to pay to be certified will be fairly small.

Even 2+ years after his post this is still relatively true, but increasingly there is a supply of developers who don't have real skills. Various companies and individuals claim to know Drupal in addition to Plone, Joomla, Sharepoint, ASP, Java, JBoss, Flash, Flex, Silverlight, iPhone, Android and of course Ruby! Certifications provide some value even in a lopsided market.

- There's no obvious corporate backers to sponsor such an effort, in the
way Sun sponsors Java certification or Zend sponsors the PHP one.

The Drupal Association could. Acquia (created since his post) is another obvious choice. But this criticism is based on the concept that certifications are really expensive to develop. Our solution lets us take an alternate perspective.

- Is the certification for developers or administrators? If it's for
developers, how do you test the creative component of programming?

What about the creativity of administrators in selecting modules? But seriously, there are things no certification can test and for those everyone must realize that a certification is only a component. When we recognize and include their imperfections in part of our analysis certifications can still be useful.

- Drupal is still evolving, and backwards compatibility is not a
priority. That means that the "life" of any specific Drupal
certification exam could be very short before the things it tests are
obsolete. By the time you're ready to deliver an exam, the APIs it tests
will probably not be in use anymore.

Yep, fortunately CertifiedToRock.com is compatible with Drupal's "drop is always moving" philosophy.

- Also consider that by some industry analysis, the demand for IT
certification in general is dropping

Perhaps so. All the same, this has value outside of the traditional "certification" purpose so we're happy to invest in it.

I helped create the LPI certification for Linux. Each of the three skill
levels it tests cost more than $400,000 to develop properly. Less
ambitious, more community-driven efforts such as the BSD
certification, can be many years in the making before a single exam is
delivered -- and STILL be without a usable business model.

Our certification has been a bit cheaper to build, probably because we didn't follow the traditional methods ;) As to our business model, that's simple:

1. Collect RockStar data
2. ?
3. Profit

See, simple!

Later on Evan pointed out in response to a comment from Karoly Negyesi:

> I agree. Would you ask Earl Miles or me some piece of paper? Your code
> speaks better than a piece of paper...
>
That theory works only so long as Drupal remains niche rather than
mainstream.

If/when Drupal people need to get hired by large organizations, the
initial job for finding employees is made by human resources staff.
These people wouldn't know the name "Richard Stallman", let alone "Earl
Miles"; today they need a programmer, tomorrow they will be hiring for
the warehouse. Their role is to perform the first level of determining
qualifications; it is at this initial stage where certifications, rather
than peer respect, are useful.

Exactly. One goal with CertifiedToRock.com is to make it dead simple to figure out what kind of a RockStar Drupal community member they're talking to without having to learn the ins and outs of the community first.

Dries Buytaert - We need more reputation tools on drupal.org itself, not opaque data anlyzers

I talked this idea over with Dries at Do It With Drupal. He urged me to consider adding more reputation features to Drupal.org instead, and that's a fair argument. One reason we didn't decide to do that is that all code on drupal.org has to be open source and my involvement with the project has shown that when people are told what hurdles they need to receive inclusion on the Drupal Services page people immediately treated it like a checkbox system and started doing 3 modules, 10 handbook edits, and 5 forum posts and nothing after that. And they would do so in an extremely low-quality way. The modules would be horrible copies of existing code, the handbook edits would be trivial, and the forum posts would be pointless. Sometimes it's necessary to keep a scoring system secret (right?).

That said...we do plan to improve our rating system and know that to do so we will need to expose more data on the profiles of users on Drupal.org. If that improves the identity system on drupal.org then it's a win-win and we'll certainly do our best to choose that option when possible.