Discussions continue behind closed doors on replacement plug-in active install growth figures – WP Tavern

Earlier this month, WordPress.org meta-contributors removed the active install growth graph from the plugin, sending plugin developers who rely on this data into a state of frustration and anger. The submission cited “insufficient data obfuscation,” but there was no clear communication about when and where this decision was made. Developers have asked for more transparency about chart removals, but have not received a clear answer.

As speculation grew, multiple opportunities to communicate the details behind the decision were deliberately passed up. Some contributors not directly involved in the conversation have prematurely insisted that it had been removed due to security or privacy concerns, but Samuel Otto Wood has explicitly confirmed that it was neither.

In a recent appearance on the WPwatercooler podcast, Wood elaborated on the decision, which he said was made in a discussion initiated by Matt Mullenweg via private channels via Slack DM in May.

“The reason is really simple,” Wood said. “It was removed because by and large, no one used them. No one used the diagram itself. In general, the diagram was not useful to most people, and it didn’t really fit what we had in mind when we implemented it the goal of.”

According to Wood, the active growth chart is designed to show the growth or decline of plugins on a weekly basis, but the data is not working as expected:

People want feedback on whether the plugin is growing, shrinking, etc. This is valuable information for developers and valuable information for users to know. But it really doesn’t work that way.

The data it provides is percent based, and very weakly percent based.So in general most of the usage of this data is people scraping the data and using it to go back to the exact quote, the exact number

This is exactly the problem, people mostly use it to get those numbers. Now, that’s fine by itself, but the reverse math doesn’t work. This is wrong for a number of reasons, mostly because we’re obfuscating the data in this way, making that number wrong.

Second, actually, it’s kind of funny. It actually always gives the number too high, so it gives the wrong impression. Third, indeed, people believe it’s an active number, an active cell number, that they rely on it to make decisions and things like that. This is not a good idea.

Although Otto was not involved in the work on the project at the time, he participated in the discussions and conveyed some details:

I read through all the discussions, we worked, they worked for a long time, Scott and a few people tried various things before deleting it. They tweaked the numbers, tweaked the numbers. They, they went through ridiculous iterations and none of them worked in the end. People still use it even though it basically gives them crap. So finally removing it is the only thing to do. We do have plans to replace it. We just have no immediate plans to replace it. However, we believe that providing them with false active install counts is more detrimental to the interests of users and developers than not providing them at all. So that’s why it was removed.

The concern highlighted by podcast host Sé Reed and guest Matt Cromwell was that the decision was communicated in a way that suggested it was safety-related. Since this isn’t a sensitive security or privacy issue, Reed asked why it was handled in private chats rather than meta-channels when the decision had such a profound impact on the trajectory of developers being able to track their plugins.

Since the chart’s inaccuracy is well known to those more familiar with the problem, Wood said its removal “wasn’t a big deal,” and others ultimately assumed it was. They didn’t expect the removal of the charts to spark a storm of tracking tickets from developers pleading to restore the charts.

“The physical visual diagram itself is not that important to the way I operate things,” says GiveWP founder Matt Cromwell. “But that’s the act of removing it without any dialogue.

“What does this mean for long-term data on plugins on.org and the viability of our continuing to have them? That’s the real question. It’s an indicator that the underlying problem isn’t getting better.”

The incident sparked a discussion about what kind of partnerships plugin developers should expect from WordPress.org, and whether they should look to each other rather than the platform for support, as Eric Karkovack suggested in WP Minute. With plugin developers losing more valuable data that hasn’t been replaced, Alex Denning, managing director of digital marketing agency Ellipsis, raised the case for WordPress.org being ineffective for plugin distribution in 2022. He argues that new WordPress plugins don’t pass the 100k, 500k or 1m+ install threshold and that the directory doesn’t provide organic coverage for plugins.

The focus of this ticket has shifted from calling for WordPress.org to bring back active growth charts to more useful plugin stats for brainstorming and insights plugin developers would like to see. It still receives angry and frustrated comments from developers who believe the data should belong to the community.

“I can’t stress enough that conversations about what to replace active growth graphs should take place on public Slack channels or Trac tickets,” said Amber Hinds, CEO of Equalize Digital. “This data should belong to the community, and the community should be able to Participate in deciding how (or not) to display it.”

The reason for the purported obfuscation has not been clearly explained, but many participants in the discussion urged WordPress.org to simply publish the raw data so that it can be accessed and processed independently of the platform. @Starbuck suggested that the community will be able to create websites that present data in meaningful and interesting ways.

WordPress developers need more data than was previously available. Hinds asks for a variety of possible or impossible data points:

Things that tell us if the README and other ranking factors are OK:

• The number of searches (or impressions) for the target keyword
• Average position (months) for the target keyword in the time frame
• Impression-to-install conversion rate for target keywords

Things that tell us if the plugin is buggy:

• Number of deactivations per time period (month, preferably one week)
• Number of deletions per time period (month, preferably one week)
• Average time from activation to deactivation or deletion

Better test version things:

• Top 20 plugins are also active
• Top 20 topics are also active
• PHP version (percent)
• WordPress version (percent)

Atarim CEO Vito Peleg Suggest Some other tools for monitoring growth/decline, Matt Mullenweg responded that some of these ideas are “very feasible”:

• Time to churn (deactivate) signals good/bad onboarding, UI/UX
• Repeat installs – how many users (anonymous) install on multiple sites for community opp and promotion
• Time to Result: The developer can choose 1 single hook that fires as a “result” and calculate the time the check takes from installation to getting there. By changing the location of the hooks, developers can optimize the entire process.
• Internal page tracker: which/how many internal plugin pages users visit
• PHP version distribution, general installs by country, active installs to comment ratio

Wood confirmed that the active install growth graph is not returning to its previous form, and the endpoints that people were scraping before will remain disabled. Those involved in the private discussions are monitoring the Trac ticket for feedback, he said.

“What’s going to happen is that the active install count is going to change instead of being rounded to the nearest number,” Wood said. “I don’t know the exact breakpoint cutoff, but for example, show individuals up to 50, then round to the nearest 10 up to the thousand, and the nearest hundred up to 10,000. So we’ve made the active install count more granular than ever. So in that sense, yes, we’re giving you data. It’s not going to be an exact number, but it’s going to be a lot better than it was before. We’re still working on that.”