Academic Study: Don't Bother To Refactor Code for Quality

When I'm flailing about in code, I often quickly slap something together just to get it working and then go back to clean it up with simpler structures.

Real developers commonly refactor their code in this way to improve quality, readability, reusability and so on.

From Sri Lanka comes an academic report saying you probably shouldn't even bother -- if code quality is your primary concern.

The researchers defined refactoring as "the process of improving the design of existing code by changing its internal structure without affecting its external behavior, with the main aims of improving the quality of software product."

"This study [indicates] that refactoring does not improve the code quality," concluded Sandeepa Harshanganie Kannagara and Dr. W. M. Janaka I. Wijayanayake in a report published in January, titled "An Empirical Evaluation of Impact of Refactoring on Internal and External Measures of Code Quality."

One measurement, however, indicated an improvement in a metric called the "maintainability index," which logically follows since refactored code should be easier to analyze and understand by humans and thus be easier to maintain.

It's as if I "refactored" the study's title to change

"An Empirical Evaluation of Impact of Refactoring on Internal and External Measures of Code Quality"

to

"The Impact of Refactoring on Code Quality."

The new title is easier to read but doesn't improve much on the actual purpose of indicating the study's content.

The Sri Lankan researchers were trying to dig deeper into the nuts and bolts of the code to see if actual quality improvements could be proven. They noted that previous studies have concluded code refactoring improves quality, but most provided no quantitative evidence. "Although, the refactoring is by definition supposed to improve the maintainability of a software product, its effect on other quality aspects is unclear," the study said. "Therefore, there are hot and controversial issues in refactoring."

From where I sit, the study itself might be in for some hot and controversial commentary. It looks like there might be some concerns about sample size and quantitative analysis methodology. The researchers themselves admitted as much and called for further study on the subject.

"There are several arguments that may come against this study," they said, including the fact that more expert developers well versed in system design could serve as better evaluators of the code, even though the undergraduate students apparently used for analysis "have comparable assessment ability compared to a professional software developer." I personally have doubts about that -- there's no substitute for real-world experience.

So, basically, the study -- published in the International Journal of Software Engineering & Applications -- indicates the need for further study. "The results of this study indicate that there is further need of addressing the impact of refactoring," the researchers concluded. "Refactoring techniques used in this study were selected from the ranking done by previous study. Therefore, in the future it is better to conduct a study to find refactoring techniques which are commonly used in industry by a survey."

That makes sense to me. Developers and teams spend a lot of time on refactoring, and as this site's "Agile Architect" columnist Mark J. Balbes, Ph.D. said, it's instrumental to Agile practices and test-driven development. "If you are doing your work test-driven, then you also have the opportunity to think while you are writing the code," Balbes said. "The small refactorings you make along the way drive your system to a good design."

That's the common knowledge, but this study might give enterprise developers food for thought and perhaps incite them to study their own engineering practices. With release cadences quickening and dev cycles condensing, where should the devs expend their effort? If readability is improved but not code quality, is the time spent on refactoring worth it? What's the opportunity cost?

Posted by David Ramel on March 3, 2015