Blog

phpBB Numerology – All customisation downloads (Part 4)

Posted by battye in Extensions, Modifications, Styles, Uncategorized with the tags , , , , , , on November 16th, 2016

MODs, extensions and styles are what immediately come to mind when thinking of the phpBB.com customisation database, but there are other types of customisations too.

This part of phpBB Numerology takes a look at every type of customisation.

Overview of all customisation downloads:

part_4_1

There are some obvious anomalies in the data (like the Viewtopic Birthday MOD) which haven’t been removed from this raw analysis. With that said, this data still gives a fairly accurate representation of how people use the customisation database.

The first figure that sticks out in this table is the the 52.02 views per download for bbCode. This is disingenuous. Taking the fairly popular bbCode “Open URL in new window” as an example, it’s had 8026 views and 0 downloads. Why is this? Because there is nothing to download – the instructions are on the main contribution page: https://www.phpbb.com/customise/db/bbcode/open_url_in_new_window/

MODs, due to their prevalence in the phpBB2 and phpBB 3.0 era, are still the undisputed king of the customisation database and will take some catching up. Styles may catch up in the medium term though, as it’s a forever expanding customisation type in phpBB 3.1.

It’s not surprising to see phpBB tools well represented on the list. AutoMOD and UMIL alone represented 208757 and 151820 downloads respectively – very popular tools. Converters and Bridges round out the top 8, being quite niche categories.

As noted earlier, the figures may be a little high due to anomalies but after taking into account off-site downloads through development topics and the like the numbers may actually be conservative. It’s therefore very pleasing for everyone involved in the phpBB community, and something to be very proud of, that phpBB customisations have exceed 15 million downloads.

For the fifth and final part of phpBB Numerology we’ll go back to where it all started – extensions – and take a closer look at the categories of extensions that are most popular.

phpBB Numerology – Language packs (Part 3)

Posted by battye in Extensions, Modifications, Uncategorized with the tags , , , on November 9th, 2016

The third part of this series has a look the most popular language packs.

Most downloaded language packs (phpBB 3.x):

Most downloaded language packs (phpBB 3.x):

The most interesting observation, bar an anomaly with Brazilian Portuguese, is that the number of downloads corresponds quite closely with the number of total pageviews. This implies that when people download language packs, they know in advance what they want and there is little hesitation in following through to actually download it.

The average views per download for language packs is 2.29. By comparison, for phpBB 3.1 extensions that figure is a little higher at 3.53 – which suggests perhaps people casually browse more for extensions than language packs – which makes sense given for region specific forums a language pack is a necessity.

That European languages dominate the top positions in the list is not surprising either. It is evident from the make up of phpBB.com’s own community that phpBB is very popular in Europe.

In part four of phpBB Numerology, we’ll look at some of the lesser known customisations.

Codename for phpBB’s next version

Posted by Marc in Development on November 5th, 2016

After two weeks of voting and a bit of back and forth between the four candidates, we now have a final winner for the codename of our next version:

Proteus

With Proteus we’ll be moving on from Saturn and continue our deep space journey to Neptune’s second largest moon (based on the name Proteus, the shape-changing sea god of Greek mythology).

We’ll be updating the release process page in the next few weeks to reflect this as well as to update current maintenance periods and target release timeframes.

Thanks to everyone for participating in this vote!

phpBB Numerology – MODs (Part 2)

Posted by battye in Extensions, Modifications with the tags , , on November 2nd, 2016

For the second blog post in this series I thought it would be interesting to do a direct comparison with the data presented in the 2011 post.

Most downloaded phpBB3 MODs:

part_2_1_updated

There are a few figures which are unrealistic, which have been struck out accordingly. For example, “User recent activity” apparently has nearly 80,000 downloads while its customisation page has only had 5300 page views.

The most significant movers and shakers were the Advanced BBCode Box 3 MOD, which had a huge spike in downloads in moved up 18 positions from 2011. The ReIMG Image Resizer also experienced a spike, but due to its favourable position in 2011 only moved up 7 places. None of the top 10 from 2011 experienced large drops.

Most downloaded MOD/extension authors (primary authors for phpBB2 and phpBB3):

Most downloaded MOD/extension authors (primary authors for phpBB2 and phpBB3)

Due to the overwhelming successes of the Attachment MOD and Cash MOD for phpBB2, both Acyd Burn and Xore remain among the most downloaded phpBB MOD/extension authors of all time.

nickvergessen is the third most downloaded author of all time, despite not having any entries in the top 10 downloads for an individual customisation. This is because two of his submissions are/were highly popular and fall just outside the top 10 of their respective lists; phpBB Gallery for 3.1.x has 57,070 downloads while NV advanced last topic titles for 3.0.x has 49,372 downloads.

In part three of this series, we’ll look at the most downloaded language packs for phpBB which will also offer some insights into which regions phpBB is most popular.

phpBB Numerology – Extensions (Part 1)

Posted by battye in Extensions, Modifications with the tags , , on October 26th, 2016

Five years have now passed since our first statistical journey through the archives of the customisations database (see MOD Statistics: The most downloaded MODs from the MOD Database). Armed with just one table from our database, I’m setting myself the challenge of creating five blog posts worth of in-depth (and hopefully interesting) statistical data about the way users download extensions from phpBB.com.

The obvious starting point is to revisit the original blog post, and see what the state of play is in 2016. Then, we looked at the most downloaded MOD authors and well as the most downloaded MODs of 2011 and of all time. We’ll repeat that exercise with extensions now.

Starting with the most downloaded extensions overall, we can see it’s a tight race for first place between Board Announcements, Advanced BBCode Box and Pages. Little over a thousand downloads separate the three extensions which average close to 30 downloads a day.

Most downloaded phpBB3 extensions:

Most downloaded phpBB3 extensions

Just after the half way point of 2016, Application Form and Scroll Page are favourites of the phpBB community with close to a thousand downloads each.

Most downloaded phpBB3 extensions submitted in 2016:

Most downloaded phpBB3 extensions submitted in 2016

While dmzx has submitted more extensions than any other author with 43 extensions, it’s VSE who has been associated with the most downloads by far – 55875 in total.

Most downloaded phpBB3 extension authors (primary authors):

part_1_3_updated

Stay tuned for part two of Numerology in next weeks blog post, where we’ll take a stroll down memory lane and look at the movers and shakers of phpBB 3.0.x between 2011 and 2016.

* Data accurate as of June 28, 2016; click on each image to view at full resolution

Vote For phpBB’s Next Codename

Posted by MattF in Development with the tags , , , , , on October 16th, 2016

At phpBB we love codenames as much as we love Bertie.

Previously we have travelled to Mars and Saturn to find the most prominent features to name our versions. Now it’s time to come up with another codename for the version of phpBB.

In line with having an open mind for new features, looks, and changes in phpBB’s next major version we’re also opening up the decision on the codename to the community.

The phpBB Team has already pre-selected four possibilities from a rather long list.

Now it’s time for you, the community, to have the final say:
[Vote] Codename of next phpBB version

The vote will end in two weeks on October 30th, 2016.

Happy voting!


 beer_bertie

phpBB not vulnerable to ImageMagick exploit

Posted by Marc in Development, Security, Support on May 5th, 2016

As some of you might have noticed, a security issue has recently been uncovered in ImageMagick, a widely used software suite for displaying and modifying images. You can read about the full details of the exploit on news outlets like Ars Technica (article on ImageMagick exploit) and the ImageMagick community forum.

It has come to our attention that admins are unsure whether this affects their phpBB installation and/or what they might have to do to secure their installation.
First and foremost let me make it clear that, based on the currently available information, phpBB is not vulnerable to the exploits.
phpBB already verifies the supplied image types and therefore already contains one of the described mitigations for the ImageMagick vulnerabilities.

phpBB supports using ImageMagick for creating thumbnails only. Other parts of phpBB, such as image or general attachment uploads, are not using ImageMagick at all. The described exploits rely on directly passing uploads to ImageMagick and/or passing “clear text” image formats such as SVG or MVG to it. phpBB itself does not support these image types for creating thumbnails and therefore will not pass them onto ImageMagick. Additionally, supported image types are checked for integrity before passing it onto ImageMagick for thumbnail creation. Passing malicious files as described and disguising them as other image types is therefore not possible.

We hope this will help with clearing up any confusion or uncertainty on this subject.

Extension Skeleton Development Tool

Posted by MattF in Extensions on March 29th, 2016

Hello extension writers!

At phpBB’s conference in Prague last year, nickvergessen created a tool that makes getting an extension project started super easy!

ToolKit phpBB Extension Skeleton

The Extension Skeleton is an extension that you install in your development phpBB board. Once installed, you can use it to generate the starter files and directories for your own extensions.

The Extension Skeleton will ask you for some basic information about the extension you want to create, such as the name, author information, requirements, and what type of components it will use (listeners, controllers, styles, template events, unit testing, etc.). It will then generate an extension package based on your needs. It will be a working “skeleton” of the simple Acme Demo extension, and will be ready for you to begin to edit and update with your own code.

It can be accessed via Graphic User Interface through a link in your forum’s navbar

Extension Skeleton Tool

It can also be accessed from the Command Line Interface with the command:

$ bin/phpbbcli.php extension:create

The Extension Skeleton is a great little tool that every extension developer should have in their toolkit. It simplifies and expedites the mundane task of starting a new extension, and ensures you have a correctly configured set of composer, PHP and style files with which to begin coding your extension.

Happy coding developers!

phpBB 3.2 Rhea is near – help us test it now!

Posted by naderman in Development with the tags , , on March 26th, 2016

On October 28th, 2014, we published phpBB 3.1 Ascraeus, the culmination of nearly 8 years of development since phpBB 3.0—an eternity when it comes to web development. We learned our lesson from trying to build a major feature release over a timespan that saw major changes in web technologies; while our roadmap had to change frequently, none of the progress was made available to you—our users. When we finally released phpBB 3.1, I announced that phpBB would from now on see feature releases on an annual basis.

It has now been one year and 6 months since I made this statement. Our most recent release has been phpBB 3.2 Rhea 3.2.0 Beta2, on March 7th 2016. So we missed our goal, but we’re well on the way to reaching a new stable feature release before summer this year. We’ve been making great progress with tweaking our workflows to more strictly adhere to the schedule in the future.

As the Development Team Leader, I updated you much too infrequently on development progress, partly due to having many other tasks to also focus on. Unless you follow our development forums at Area51, or our social media accounts on Facebook or Twitter, you are unlikely to have heard of our recent 3.2 Beta releases. As these responsibilities exceed what a single person can do well, Marc Alexander stepped up in February to take over the Development Team Lead position. From now on, I will be responsible for more frequently informing you of all developments regarding phpBB, as its new Product Manager.

We are looking forward to your feedback on the Beta releases and upcoming final release candidates of phpBB 3.2 Rhea.We cannot produce a stable final product without your testing and bug reporting. Download Beta packages from our archive at https://download.phpbb.com/pub/release/3.2/unstable/3.2.0-b2/. Please keep in mind that you should not run this version of phpBB on your live sites yet, and no support will be offered until the RC phase.

If you’d like to get involved with phpBB development to help us finish new features faster, check out the information available on Area51. And lastly, if you’d like to stay up to date on phpBB development progress, follow this blog!

The return of the database

Posted by MichaelC in Website on August 9th, 2015

Any of you who have tried to access phpBB over the past few days might have experienced a variety of different errors, most notably one about being unable to find the database `phpbb`. This is the full, slightly more technical post mortem of the issues highlighted in the announcement here.

Infrastructure

Firstly, some background on our infrastructure. It is hosted very generously and free of charge by Oregon State University Open Source Lab [OSUOSL or OSL] which is funded by donations and grants from individuals and large organisations (such as Google). We run our systems across a number of virtual machines (VMs), some of which are hosted on our own dedicated machines. Others were moved onto the main OSL cluster back in January as our old dedicated machines became more unreliable and unstable. We also run a number of services on centralised and function dedicated machines by the OSL such as their database servers, mailing list instance etc. which are used by a number of projects. They also very generously manage many of our VMs (using Chef), something which we’d like to make a blog post about in the future.

We now host all our databases on OSL’s main database cluster of two virtual machines served by a virtual ip which will use server SQL1 when possible or failover to SQL2 if SQL1 is down (which has happened a few times). SQL1 is their master (read & write) and SQL2 is their slave (Read-only) and SQL1 replicated to SQL2.

Replication Errors

On July 15th a number of issues with replication from SQL1 to SQL2  were noticed with some session tables which caused replication to be paused and a large number of statements were skipped. OSL then restarted replication but on the 16th we began to experience even more issues in far greater numbers than on the previous day and on the 20th a decision was made to entirely reload SQL2. Anticipating the only effect would be a bit of a slowdown on SQL1 due to a large number of reads, they went to do the maintenance at 00:00 on the 31st of July. The standard procedure for that is break replication, delete the databases off SQL2 one-by-one, and resync from SQL1.

 SQL1 Issues

At 00:20 SQL1 started experiencing errors and investigations began immediately. Despite the fact that it was believed to be a master-slave configuration between the two servers, it was in fact a master-master replication meaning although SQL1->SQL2 replication had halted, SQL2->SQL1 had not and SQL2 replicated a lack of data back to SQL1 causing the dropping of databases to occur on both machines. Immediately database dropping was halted and therefore some other project databases were unaffected but phpBB databases had already been dropped. As we had no valid failover to a read-only server (Normally this would be SQL2), OSL were then left to restore backups. However, the backups server had quite slow I/O and there were a lot of databases to restore (Just one of our databases is ~33GB and we have a number of databases and there are a number of other projects on the cluster). The backup restoration finished at 10:49 on the 1st August and took such a long time due to the aforementioned reasons and issues with problematic database structures. Once the backup restoration was complete the binlogs (essentially a log of all sql queries executed) were replayed to catch the backups up to just before the maintenance. The binlog replaying finished at around 04:22 and production databases then began to be moved back into production. More details on the SQL1 and replication issues can be read about on OSL’s own postmortem.

Maintenance Page on .com

Unfortunately, throughout most of this time we were just displaying an error saying that the `phpbb` database did not exist. Due to the time of year most of our team who would normally work on putting up (and then taking down once things were fixed) a maintenance page were away on holiday or leave (without internet or without their login credentials that they’d need such as ssh keys or sudo logins or on restricted internet connections not allowing SSH) and team members who did have access to the internet didn’t have the necessary access to repositories or our servers. Therefore, only information available about the downtime was from our twitter account and facebook page. For this we do sincerely apologise as we are aware many of you were unaware of why our site was down.

Missing Data

On Wednesday we began to realise there was some data missing after we discovered some posts had disappeared. Replication from SQL1 to SQL2 broke on the 15th July and we take backups from SQL2. The backup OSL had restored therefore was the one from the 15th July. OSL found the binlogs (/var/lib/mysql) on SQL2 and then replayed those in order to bring the state of the database back to the present. Unfortunately the command used to replay the binlogs only replayed the sql commands from the 17th July and this gap was only realised after OSL had blown away /var/lib/mysql on SQL2 in order to restart replication from SQL1 to SQL2 once we were back up and in production.

This means all actions done between the 15th and 17th July on any *.phpbb.com sites will have been lost.

Notifications

Also due to some unusual behaviour, everyone’s notifications settings on phpBB.com have been reset to never send out emails for notifications. You can still change your settings here but in order to ensure people don’t miss notifications, we’ve set everyone’s settings to now email for the following but you can of course revert this change if you so wish:

  • Someone replies to a topic to which you are subscribed
  • Someone quotes you in a post
  • Someone creates a topic in a forum to which you are subscribed
  • Someone sends you a private message

Into the future

OSL are implementing a number of policy changes and changes to their backup procedures (Read more here) in order to prevent this sort of thing from happening again, as are we. We understand many of you depend on phpbb.com for support as well as the resources it provides (downloads, documentation etc.). We will be looking at provisions to ensure that our infrastructure is less interdependent and removing single points of failure. This has also highlighted some rare edge-case bugs in phpBB which we are looking to patch as a matter of priority. We’ll also look at how we can better communicate downtime in the future, make appropriate maintenance pages easier to display, and ensure we always have people around who can deal with such situations.

Often when these kinds of situations arise, we receive questions about how our community can help us. While we appreciate the gesture of making a donation to us, the phpBB project does not accept financial donations. If you would still like to make a financial gift, you can support us indirectly by donating to OSUOSL. You can also help us directly by being active in our support forums, IRC, or by submitting source code patches. We are only here because of the community behind us, so anything you do to help the rest of the community helps us.

We do apologise once again for any problems this might have caused you.

As a note, all times are UTC+1 (British Time).