Technical debt in Drupal 8 (or, when will it be ready?)
This is the first in a series of posts on Drupal 8's progress toward release.
We're coming up fast on Drupal 8's API freeze deadline of July 1, when the Drupal community will stop changing Drupal 8 APIs, start porting contributed modules, and focus on polishing Drupal 8 (or, in other words, making it releasable). But what does that mean, exactly?
We'll create our first release candidate for Drupal 8 when there are no known critical issues remaining. Based on that information, and on the rate at which we've been fixing criticals, drupalreleasedate.com estimates that the release date for Drupal 8 will be... ready for it?...
Did I scare you yet?
Well, okay. Before you fire all your developers and migrate your sites to Joomla, let's take a closer look. Since the Drupal 8 API isn't stable yet, many developers are still -- for three more days -- focusing on making API changes. Typically, some of these changes will accidentally introduce bugs that we don't discover until later. After July 1, everyone's focus should shift to fixing our outstanding bugs and release blockers.
Still, the number of critical issues has been steadily increasing over the past six months. This wasn't the case earlier in the release cycle, and without lots of work, it's going to affect how soon (or not) Drupal 8's myriad improvements are available to the world.
Technical debt in Drupal 8
In Drupal core, we estimate our technical debt by looking at the number of major and critical issues that are currently open. (Critical issues are issues that need to be fixed before we can release, as mentioned above. Major issues have a significant impact, but Drupal can still be released before they are fixed.) These major and critical issues include outstanding bugs, but also important tasks like refactoring, improving DX and performance, and documenting our API changes for contributed module authors.
As of right now, Drupal 8 has 44 critical bugs, 77 critical tasks, 144 major bugs, and 167 major tasks. This is significantly more issues than we had at the end of 2012. (For most of the Drupal 8 release cycle, these counts hovered near our issue thresholds of 15 criticals and 100 majors in each category.)
So why is the number increasing, and how many issues are we fixing? Are we adding new technical debt? Uncovering problems we didn't know about last fall? Wrestling the same sticky issues all that time?
Last week, we ran a series of queries against the Drupal.org database to get some metrics on our majors and criticals. Rather than simply looking at issue counts, we compared the total issues posted to the total issues fixed in the 8.x branch over time. The graph below shows the cumulative work done for major and critical issues over the Drupal 8 release cycle.
So let's take a stroll through Drupal 8's history. We started the Drupal 8 release cycle with 170 outstanding major and critical issues that we inherited from Drupal 7 (the red line). Over the first six months of the Drupal 8 cycle, Drupal 8 was mostly the same as Drupal 7, so new issues posted during that time were generally additional Drupal 7 technical debt being discovered. By the time catch became the Drupal 8 release manager in the end of September 2011, we'd already fixed over 150 majors and criticals in the 8.x branch (the green line). Only 100 new major and critical issues were posted during that same time, so the technical debt (blue line) decreased.
For about a year, the 8.x technical debt was more or less constant, and new issues were filed and fixed at about the same rate. Then, in September 2012, the pace of core development picked up when Dries announced the upcoming feature freeze. Since then, our known technical debt has gradually increased, as bit by bit, more new issues are filed than are fixed.
That doesn't mean Drupal 8 development has been stagnant -- quite the opposite. In fact, we're fixing three times as many issues per week as we were a year ago. The graph below shows the weekly new technical debt and issue fix rate:
This graph is interesting because it helps us visualize three different parts of the Drupal 8 release cycle. During the development phase, the rate at which majors and criticals are fixed in 8.x mostly matches the rate at which they're discovered. When feature freeze is announced, though, the technical debt shoots way up, with as many as 20 new majors or criticals per week, and only 10-12 majors or criticals being fixed per week during that time period. Then, when the clean-up phase begins, the technical debt does continue to increase, but at a lower rate. (When the change in technical debt (the blue line) is positive, the technical debt increased during that week. When it's negative, the technical debt was reduced.)
The take-home here is that the decision to encourage the development of new features (or not) has a noticeable impact on our technical debt.
Let's take a closer look at the exact data for clean-up phase of the release cycle, which ends July 1:
The good news is, since DrupalCon Portland, we've finally managed to decrease our technical debt again. The current pace of development is truly impressive. The bad news is, even if we keep fixing 6 more major or critical issues than we add every week, it would take 32 weeks to get back down to the current issue thresholds -- and the plan has been to actually reduce the thresholds for criticals after API freeze. Even if we somehow managed to not introduce a single new major or critical, and also managed to sustain the current pace of development, we wouldn't be at thresholds until nearly September.
That seems pretty dire, so what do we do?
Frankly, we're in a tough spot, and we need help. We need to fix more issues, and introduce fewer. Furthermore, as Dries has posted, every commit we make needs to move us closer to release.
- Core maintainers will reject more patches and move them to Drupal 9. Every nice-to-have will introduce unforeseen issues, no matter how self-contained it is.
- Reviewers should be more careful and more critical of the patches they review. Well-tested, well-documented, clean code is less likely to introduce unexpected technical debt.
- Patch contributors should shift their focus to resolving existing issues, especially majors and criticals.
- Even new contributors can improve the momentum on our major and critical issues. See YesCT's blog post for straightforward ways you can help!
So how can you help? A quick guide to getting a patch committed after API freeze
- Do not propose a change to a public API unless you've discussed it with a core committer.
- Ensure the issue summary is complete and up to date.
- Supply evidence of test coverage. If existing tests already provide test coverage, link them in the issue summary. If there's a bugfix, include a test-only patch to demonstrate the coverage.
- Supply evidence of manual testing, especially for UI, CSS, and JS changes.
- Make sure the change is complete and requires no followups unless a core committer has agreed.
- Do not make unrelated changes in your patches. Scope creep is not acceptable at this point.
- Always focus on moving Drupal 8 closer to release.
This week's criticals
Five critical issues were fixed this week, thanks to hard work from: tim.plunkett, YesCT, Berdir, plach, amateescu, yched, dawehner, andypost, swentel, sun, amateescu, dagmar, kkaefer, and chx.
xatoo, linclark, Gábor Hojtsy, tstoeckler, fubhy, David Rothstein, and Crell also helped document API changes introduced by recent fixes.
Extra thanks to Alex Pott and Gábor Hojtsy for their help on this post.
Raw query results
For anyone who wants to do their own analyses. :)
- Major and critical issue posted dates
- Major and critical issue fix dates (non-backports)
- Major and critical issue fix dates (backports)