phpBB • Free and Open Source Forum Software

This week we saw many smaller bugfixes and improvements. One of these changes was to ensure timezones in phpBB 3.1 can be translated correctly. So I picked an explanation of our timezone handling for this week’s blog post.

In phpBB Olympus 3.0 we provide the user with a list of UTC offsets and a corresponding timezone name. In addition to the timezone a user can select whether DST (Daylight Saving Time) or Summer Time, as it is called in other places, currently affects them. If the DST option is selected the UTC offset is increased by one hour. I live in Germany so I select “[UTC + 1] Central European Time, West African Time”. In the spring I have to check the DST box and in the fall I deselect it. This results in UTC+2 in the summer and UTC+1 in the winter.

Clearly it is annoying for users to have to switch their timezone settings twice every year. Most users will probably not bother to change DST if they are even aware of the option to begin with. Users may also be unfamiliar with the name of their timezone or the corresponding UTC offset. So back in 2010 Chris Smith (ToonArmy) proposed an alternative approach to handling timezones in the Switch to PHP timezone handling RFC on our Area51 development community.

In this example my current timezone is set to Europe – Berlin where the current time is 1:40 am. However I am currently in New York City so phpBB suggests I update the timezone to match my local time which is UTC-5.

In phpBB Ascraeus 3.1 we now offer a much longer list of timezones grouped by continent and identified by cities and countries in these continents. Each timezone selection includes information on when DST applies in the respective location so users no longer have to manually select whether DST applies to them. As this list contains around 500 different options we allow the user to first select the current local time and then reduce the options to all those timezones which actually match the user’s current time. So user’s really only have to pick what time it is at their current location anymore. We use JavaScript to guess which timezone the user is most likely in, based on the browser’s reported timezone. On upgrading from Olympus the UTC offset is retained until the user selects a different timezone.

Further pull request 1812 added a new type of notification which informs administrators of newly registered users if the community requires admin activation. This makes use of our new notification system, so the notification will be appear in the forum’s header and be available via email and/or XMPP.

Thank you all for your feedback on the responsive version of prosilver which we published on Area51 last week. Your feedback has lead to a large number of improvements and corrections over the last days, the last of which will show up on Area51 later this week. Following the responsive version of prosilver, Arty has also finished a pull request for a responsive version of the Administration Control Panel. So you will be able to test the responsive ACP on your handheld devices in Alpha2 which should be released within the next 2 weeks.

We integrated a pull request which enables CodeSniffer in our automated test suite. CodeSniffer is a tool which can automatically check our code for coding standard violations. A human readable version of the coding guidelines for Ascraeus is available at area51.phpbb.com/docs/31x/coding-guidelines.html. This change will ensure that every pull request and commit is automatically checked for misformatted code as part of our Travis CI builds.

Some MOD authors have started to test the new extension system but we are still looking for more feedback and help with improving the extension documentation on the wiki. One of the improvements implemented this week expands the routing configuration for extensions to allow a definition of routes across multiple files. Routing is the process of matching a URL to a particular piece of code within an extension which is handled with the Symfony Routing Component in Ascraeus.

Since the release of phpBB 3.1 Alpha1, the development and styles team have been busy working on further improvements to the version codenamed Ascraeus. Thanks to Arty finishing up his responsive design for prosilver pull request, phpBB’s default style now dynamically adapts to the screen size of the device used to browse the forum. We’ve updated the Area51 forum to the latest development version so go visit it with your favourite handheld device and check out the differences to the desktop version!

We’re still working out some kinks and trying to improve it, and the improved version of prosilver will ship with Alpha2 soon!

In other news, our Travis CI build matrix has been updated to run our tests on MariaDB in addition to MySQL and PostgreSQL. Tests are automatically executed for every pull request sent to the GitHub phpBB repository. When they run through successfully, this is noted in green on GitHub so the developer reviewing the changes can be certain it did not unintentionally break other phpBB features.

Today, we are pleased and excited to announce the first preview release of the upcoming phpBB 3.1, codenamed Ascraeus.

This preview release is targeted at phpBB developers and advanced users who wish to assist us with this very important testing phase. We do not recommend, and will not support, running this version in a live production environment.

Additionally, this marks the beginning of a number of significant improvements to the phpBB development process. Our new extension system replaces phpBB MODs with a flexible plugin system that does not require direct modification of code. Consequently, updating phpBB forums will be easier than ever with potential for conflicts reduced to a minimum. This change also allows us to transition to a scheduled release cycle, with more drastic changes to the core now having less impact on the backwards compatibility of MODs.

With this alpha release, the new extension system will be put through its paces. We wish phpBB 3.1 to offer a rich and smooth extension environment for developers, right from the start. MOD authors are asked to convert their MODs into 3.1 extensions and thereby assist us with identifying and reporting any shortcomings, no matter how small. It is particularly important that we provide all necessary events for extensions to offer the same level of flexibility that has been seen with MODs. Please help us complete the documentation on developing extensions on the wiki if you run into any poorly documented areas.

You can review a continually evolving list of feature highlights on our wiki. This list will be completed as we approach the final release. Apart from the aforementioned extension system, we have worked on modernizing the phpBB experience for end users. The user interface now makes use of AJAX functionality. A new notification system, similar to those found on many modern web services, has been added. Gravatar integration for avatars is available by default. Login via third party services such as Facebook and Google is available. The new soft delete feature adds a “recycle bin” functionality to prevent accidental data loss.

Behind the scenes, we have been making major changes to phpBB’s architecture. Over the last few years you may have seen us discussing plans for phpBB4 – a rewrite of phpBB based on the Symfony2 PHP web framework. In the time since, we have been gradually integrating Symfony components into phpBB 3.1. After much discussion and deliberation, we have determined that a full rewrite of the phpBB codebase is not the ideal process for achieving these major improvements. Instead, an iterative process of replacing individual subsystems within phpBB will allow users to see these improvements realized in shorter periods of time. We have already started this process, and most recently replaced phpBB’s old template engine with twig. The increased utilization of third party libraries allows us to dedicate more resources on improving the forum components while simultaneously benefiting from the improvements that third-party developers make to their components. You will therefore be seeing improvements we had originally planned for phpBB4 much sooner, in 3.1, 3.2, 3.3, and beyond.

Once again, please use this release for testing purposes only. As always, we will provide no support for upgrading until the RC stage and urge users not to use testing releases in a production site. Please help us by reporting any bugs you find via a ticket on our tracker.

Unstable packages are available on Area51 at area51.phpbb.com/downloads/. Information on moving from 3.0 to 3.1 (currently only for testing purposes) is available at wiki.phpbb.com/PhpBB3.1#Moving_from_3.0_to_3.1

Please note that phpBB 3.1 has some requirements that differ from phpBB 3.0:

• PHP version 5.3.3 or greater
• PHP’s json extension
• Only PostgreSQL versions 8.3 and above are supported

My phpBB Summer of Code project involves the implementation of a Post Edit History, that is Post Revisions, system. This is a completely optional feature that keeps track of changes made to posts (each edit is called a “revision”). I have tried to write up a brief but informative post detailing what all is being done in regards to this feature.

As I mentioned, each time a post is edited, a new revision, containing the content of the post prior to being edited, is created and stored in the database. The text is stored in full, rather than as diff instructions, so that we can circumvent any potential issues that could arise (for instance, if a revision is deleted, the diff instructions between the remaining revisions would have to be recalculated, resulting in a potential for unresolveable conflicts).

One issue, however, with storing the entire content of the post each time it is edited is that there is potential for a significant increase in database usage and a decrease in performance for large boards with lots of posts (especially long posts) and active users. To combat this, administrators can configure the maximum number of revisions to save per post, as well as the maximum length of time that any given revision should be stored. After all, how often will you need to view a post 6 revisions ago or see what it said 10 months ago? Old and excess revisions are automatically, periodically pruned. Additionally, the entire post revision tracking system can be disabled, and a new option has been added to immediately purge all revisions from the database. While that may be drastic and ideally will not be used very often, it can help decrease the amount of load put on the server, as well as reduce the volume of content stored in the database.

A new page, revisions.php, is being added to allow authorized members to view and manage any given post’s revisions. By default, when viewing this page, you will see a list of the revisions made to a post, in order, detailing who made each revision, when it was made, and for what reason (if provided). Each revision will have a checkbox next to it, allowing you to mark it for deletion or protection (see below). Underneath the revision list you will see a rendered diff (i.e. a visual representation of the changes made between two given texts) between the oldest available revision and the current post, as it exists on the *_posts table. You will be able to constrain the comparison by selecting a starting point and an ending point. You are also able to view an individual revision, by itself. Lastly, the post can also be reverted to any revision in the list.

In the previous paragraph, I mentioned the ability to mark revisions as protected. In some cases, you may wish to save a previous version of a post. Therefore, revisions can be marked as “protected”, and will not be included in the automatic revision pruning cron tasks, and will only be deleted if done so manually either via the post revisions view page or the mass revision purge option in the ACP.

One application of the Post Revisions system has also been implemented: wiki-style posts. Any user with the “f_wiki_create” permission can mark a post they have made to be a “wiki post”. Once this is done, the post can be publicly edited by anyone with the “f_wiki_edit” permission. When paired with the revisions system, changes can be tracked and reverted as needed. Note that there are separate settings for wiki posts to regulate the total allowable number of revisions per wiki post and the age of wiki post revisions; this way, you can keep track of more history of a wiki post for a shorter or longer period of time than for normal posts.

Well, I’m not done yet, and here is a list of some of the things I have yet to do
– Attachment versioning – I still have to figure out the best way to do this (if at all)
– Stress testing with a large board/userbase
– Figure out what Diff engine to use (I’m currently using PHP FineDiff, but I’m really looking for a UTF-8 compatible one, so if anyone has suggestions, feel free to let me know!)
– Further integration with the viewtopic screen – Currently, most of the revision-related front-end is contained in revisions.php; the only way to get there is from the edit count on viewtopic. So, for instance, I might display a compact list of revisions on a post, or maybe incorporate AJAX in some ways, etc.
– SubSilver2 changes (currently I have only done proSilver)
– More? I’d love to hear suggestions and comments.

For those interested, you may test what I have done so far, follow progressing development, and provide input by viewing the GitHub Pull Request and the RFC Topic (on the phpBB Area51 Development Board).

Thank you for your time!

During Google Summer of Code, I have been working on refactoring the authentication system (login and registration) to allow for login via OpenID providers and Facebook Connect [code]. The system’s old authentication plugins (db, ldap, apache) have all been ported to the new authentication provider system and can be enabled/disabled via the administration control panel. Currently, OpenID is implemented using the Zend Framework 2 OpenID package. The Zend Framework 2 OpenID code, currently does not allow login via Google (a popular OpenID provider), however, support for login via Google should be available by the end of Google Summer of Code. The user interface has been nominally changed from its previous look, however, additional changes to the interface will be made to make logging in via various popular providers more convenient for users.

The current (temporary) login box design.

The current (temporary) registration design.

As can be seen from the images, not much work has been done on styling the interface. It currently is focused on functionality for use in tests. The design of the interface will undoubtedly change before the end of Google Summer of Code.

For the remainder of the summer, I will be implementing login via Facebook Connect and potentially (time permitting) adding general support for OAuth2.0 providers. I will also be writing a wrapper class that will handle old authentication plugins so that they will continue to work while they are being transferred over to the new provider system.

I look forward to continue working with my mentor, Nils Adermann, on the remainder of the project. It has been a great experience so far contributing to phpBB, and although I’m am uncertain of how much time I will be able to commit after this summer, I look forward to continuing to work with phpBB and its community.

During Google Summer of Code, I have been working on Search Backend Refactoring. The project involves minimizing code duplications by improving class abstraction in search backends and integrating PostgreSQL fulltext search [Code] and Sphinx search [Code] into phpBB core. Up till now, the progress has been good with PostgreSQL fulltext search backend already integrated into phpBB 3.1 and Sphinx backend nearing completion too.

The project has been a great learning experience. Apart from learning the various features of Git, I have been able to understand the basics of PostgreSQL and MySQL Databases more clearly. I was introduced to Sphinx during the course of project and honestly I was quite impressed by its performance. All boards running on MySQL and PgSQL databases will soon be able to switch to Sphinx search backend for faster indexing and searching.

Next I intend to start code refactoring and fixing bugs in multiple search backends to provide better search functionality to both users and administrators (by exposing more options in ACP). I will also be writing automated tests side by side to ensure search functionality continues to work correctly.

All this has been possible only because of the timely help I have received from mentors and other team members. They have been patient and appropriately guided me at all times. The overall development experience in the phpBB community has been fantastic, I would love to remain involved and contribute even after GSoC.

During the Google Summer of Code I’ve been working on improving the current attachment system to have a bit more of a modern feel to it. I’m integrating Plupload which has a variety of features such as drag-and-drop, multiple file uploads and an upload queue.

The work has been pretty straightforward and I’ve been getting a lot of help from mentors and other team members so I’ve made considerable progress in a short time. The following screenshot shows Plupload integrated with phpBB and you can also look at the code itself in my pull request:

By the end of it, phpBB users (with modern browsers) should be able to enjoy a much smoother attachment experience without a bunch of page refreshes interrupting their posting or waiting ages for a file to upload only to find out it was too big.

It’s not my first time contributing to an open source project but, surprisingly, one of the most useful things I’ve learnt is actually how to use git properly. There are a huge number of really useful features that I’ve never had to use until I started contributing to a project of this size. It’s given me more than my fair share of headaches and some of my activity history on github is a bit embarrassing now but everyone’s been really patient and helpful and I like to think I’m starting to get the hang of things now.

I’m having a lot of fun and intend to keep contributing to the project long after the GSoC has finished.

Automated Testing is a vital part of the development cycle, as it decreases the amount of time developers must spend making sure certain functionality works, and prevents new patches from breaking areas of functionality that are covered by the test suite. As you may have guessed, Automated Tests are simply sets of functionality tests that are automatically run as new features are developed, bugs are fixed, and changes are made to the codebase. Nothing is merged into the core phpBB repository without ensuring that whatever changes are being made do not break any existing functionality. If any tests fail, the developer is able to save time and energy that would have been spent later on, when a bug report would have been filed, by finding the problem and fixing it up front. Not only does this help by increasing the efficiency of the development cycle, but it also improves the quality of the software as a whole, because we can be certain that all features work as intended without having to manually test each one every time something is changed.

That sounds great, doesn’t it? Well, it is. The only problem is, only a very small portion of the code is actually covered by tests. One reason for this is because the automated tests have not been around for as long as the phpBB3 codebase, so they have had to be written long after the features they are testing have been merged, rather than in conjunction with the features themselves. Another reason is that there are two types of tests that need to be written: unit tests and functional tests.

Unit tests are simply tests that attempt to run certain peices of code that do not require a webserver or phpBB installation to work. These tests are generally easier and faster because all of the information they need is readily available and hardcoded into the test suite.

On the other hand, Functional tests are needed to run functionality that requires a phpBB installation. For instance, if you want to test whether an authentication plugin works, you have to actually be able to test logging into a board. For that, you need a database with a user account. You also need to be able to test that a cookie is sent. Unit tests cannot handle that, but Functional tests, are able to simulate browsing and using the forum almost just like a normal user.

However, functional tests cover even less of the code and functionality than unit tests do, and time that is spent writing functional tests is time that is not spent writing new functionality or expanding on existing functionality. So while we need functional tests to reap the benefits I explained above, it does not make sense to devote all of our time to writing tests and therefore have a significant “dry period”, during which no new or changed functionality is incorporated into the software.

This is where you come in. I, like many of you, was kind of intimidated when I first started to contribute to the core of phpBB. I felt much safer writing MODs because I knew my work would not be scrutinized as much as if I were to submit a patch to the core (that is not to say that the MOD team does not carefully review each and every MOD, but rather, that my code would likely not be viewed by as many people and would not be held to as high a standard). However, I found that by writing functional tests, I not only was able to contribute to the betterment of the software, but I also was able to better learn how things worked and how I could therefore use the things I learned in not only my MODs but also my core patches.

So all of that to say, there is plenty of work to be done and we would love to see contributions to our test framework from our vast, talented phpBB community. As I said, these contributions help not only the software and phpBB community as a whole, but also can give you the knowledge and skills you need to improve your MODs and, should you decide to, your patches to the phpBB software.

There is a wealth of information in the phpBB Development Wiki (see Automated Tests) to get you started with writing and running tests. And if you have a question or need help understanding how to do something, you are welcome to ask me or another developer in the #phpbb-dev IRC channel.

Since we opened up development and moved our code onto GitHub, we’ve received several contributions. It allowed a number of non-development team members to contribute. We have been listing these contributors in the 3.0 release announcements, and you can also see them on the area51 contributors page.

However, I think we can do better. And this is why I want to reach out to the modding community. We have limited ressources and we need your help.

Contribute patches

The bug tracker is filled with bug reports. You may have reported a few bugs yourself. There’s so many, it’s likely that a large amount will never get fixed.

This is your chance. If you ran into a bug that really bugs you (pun absolutely intended), now is the time to fix it. If you’ve interacted with the MODding community, maybe by submitting some MODs to the cusomisation database, you should already be somewhat familiar with the phpBB code base and its coding guidelines.

Creating a patch is like creating a MOD. Except you don’t have to write a MODX file. Instead you can have a diff generated for you or just use git to manage the changes.

Once you are done with your patch, you can either attach a patch file to the ticket, or you can be awesome and create a pull request on GitHub. Pull requests make it easier for us to read, discuss, and test your patch. For this reason it will likely be merged faster.

Review pull requests

Pull requests are patches awaiting review. Most of the time they will be from developers. That’s right: every change to the phpBB code base needs to be reviewed by another developer. This allows us to catch problems early, and distribute the blame when something goes wrong.

There is a list of open pull requests that are waiting to be merged. Some of them already have comments, while others have not been reviewed yet.

Reviewing these is a lot of work. But you can help! If you know the phpBB code base well, you can help by checking the diffs and making sure no new problems are being introduced. Be sure to comment with your opinion; even simply saying “looks good” is helpful because it shows that someone has reviewed the patch.

The other thing you can do is test pull requests. Reviewing is nice and dandy, but testing allows you to see if things are really working. And this is very similar to testing MODs. So if you are a MOD author or maybe even a junior MOD validator, this process should be fairly familiar to you. The only difference is that everyone will be watching you.

Git

We use git for version control. This may be intimidating to lots of you, but for development there is really no way around it. It is a great tool, and I encourage anyone who writes code to learn how to use it.

You will first need to fork the phpbb3 repository. Then you will need it to create a branch on your fork for your change. When you have finished your patch, push to your fork on GitHub and submit a pull request. You can also checkout someone else’s branch with that person’s proposed changes, in order to test it.

I’ll be honest with you: it’s tough.

But there are lots of great resources out there! One of my favorites is the “Getting Git” talk by Scott Chacon. He explains the basics of how git works. This really helps you to get an initial understanding, so that you can move on to learn how to actually use it. Some great guides that can get you started using Git are the Git Reference or Pro Git. And we also have the Working with Git article on the phpBB wiki.

And fear not, we are here to help. If you run into any problems, just come onto IRC and ask.

In conclusion

So there you have it, secure your place on the area51 contributors page!

If you have questions or just want to see what we are up to, join us in the #phpbb-dev IRC channel on irc.freenode.net.