Blog

Archive for the ‘Development’ Category

Development Update: Responsive Design

Posted by naderman in Development with the tags on October 20th, 2013

Screenshot of Mobile Viewtopic

Since the release of phpBB 3.1 Alpha1, the development and styles team have been busy working on further improvements to the version codenamed Ascraeus. Thanks to Arty finishing up his responsive design for prosilver pull request, phpBB’s default style now dynamically adapts to the screen size of the device used to browse the forum. We’ve updated the Area51 forum to the latest development version so go visit it with your favourite handheld device and check out the differences to the desktop version!

We’re still working out some kinks and trying to improve it, and the improved version of prosilver will ship with Alpha2 soon!

In other news, our Travis CI build matrix has been updated to run our tests on MariaDB in addition to MySQL and PostgreSQL. Tests are automatically executed for every pull request sent to the GitHub phpBB repository. When they run through successfully, this is noted in green on GitHub so the developer reviewing the changes can be certain it did not unintentionally break other phpBB features.

phpBB 3.1 Alpha1 Preview Release

Posted by naderman in Development with the tags , on October 15th, 2013

Today, we are pleased and excited to announce the first preview release of the upcoming phpBB 3.1, codenamed Ascraeus.

This preview release is targeted at phpBB developers and advanced users who wish to assist us with this very important testing phase. We do not recommend, and will not support, running this version in a live production environment.

Additionally, this marks the beginning of a number of significant improvements to the phpBB development process. Our new extension system replaces phpBB MODs with a flexible plugin system that does not require direct modification of code. Consequently, updating phpBB forums will be easier than ever with potential for conflicts reduced to a minimum. This change also allows us to transition to a scheduled release cycle, with more drastic changes to the core now having less impact on the backwards compatibility of MODs.

With this alpha release, the new extension system will be put through its paces. We wish phpBB 3.1 to offer a rich and smooth extension environment for developers, right from the start. MOD authors are asked to convert their MODs into 3.1 extensions and thereby assist us with identifying and reporting any shortcomings, no matter how small. It is particularly important that we provide all necessary events for extensions to offer the same level of flexibility that has been seen with MODs. Please help us complete the documentation on developing extensions on the wiki if you run into any poorly documented areas.

You can review a continually evolving list of feature highlights on our wiki. This list will be completed as we approach the final release. Apart from the aforementioned extension system, we have worked on modernizing the phpBB experience for end users. The user interface now makes use of AJAX functionality. A new notification system, similar to those found on many modern web services, has been added. Gravatar integration for avatars is available by default. Login via third party services such as Facebook and Google is available. The new soft delete feature adds a “recycle bin” functionality to prevent accidental data loss.

Behind the scenes, we have been making major changes to phpBB’s architecture. Over the last few years you may have seen us discussing plans for phpBB4 – a rewrite of phpBB based on the Symfony2 PHP web framework. In the time since, we have been gradually integrating Symfony components into phpBB 3.1. After much discussion and deliberation, we have determined that a full rewrite of the phpBB codebase is not the ideal process for achieving these major improvements. Instead, an iterative process of replacing individual subsystems within phpBB will allow users to see these improvements realized in shorter periods of time. We have already started this process, and most recently replaced phpBB’s old template engine with twig. The increased utilization of third party libraries allows us to dedicate more resources on improving the forum components while simultaneously benefiting from the improvements that third-party developers make to their components. You will therefore be seeing improvements we had originally planned for phpBB4 much sooner, in 3.1, 3.2, 3.3, and beyond.

Once again, please use this release for testing purposes only. As always, we will provide no support for upgrading until the RC stage and urge users not to use testing releases in a production site. Please help us by reporting any bugs you find via a ticket on our tracker.

Unstable packages are available on Area51 at area51.phpbb.com/downloads/. Information on moving from 3.0 to 3.1 (currently only for testing purposes) is available at wiki.phpbb.com/PhpBB3.1#Moving_from_3.0_to_3.1

Please note that phpBB 3.1 has some requirements that differ from phpBB 3.0:

  • PHP version 5.3.3 or greater
  • PHP’s json extension
  • Only PostgreSQL versions 8.3 and above are supported

phpBB Summer of Code 2012: Post Revision Tracking

Posted by imkingdavid in Development on July 23rd, 2012

My phpBB Summer of Code project involves the implementation of a Post Edit History, that is Post Revisions, system. This is a completely optional feature that keeps track of changes made to posts (each edit is called a “revision”). I have tried to write up a brief but informative post detailing what all is being done in regards to this feature.

As I mentioned, each time a post is edited, a new revision, containing the content of the post prior to being edited, is created and stored in the database. The text is stored in full, rather than as diff instructions, so that we can circumvent any potential issues that could arise (for instance, if a revision is deleted, the diff instructions between the remaining revisions would have to be recalculated, resulting in a potential for unresolveable conflicts).

One issue, however, with storing the entire content of the post each time it is edited is that there is potential for a significant increase in database usage and a decrease in performance for large boards with lots of posts (especially long posts) and active users. To combat this, administrators can configure the maximum number of revisions to save per post, as well as the maximum length of time that any given revision should be stored. After all, how often will you need to view a post 6 revisions ago or see what it said 10 months ago? Old and excess revisions are automatically, periodically pruned. Additionally, the entire post revision tracking system can be disabled, and a new option has been added to immediately purge all revisions from the database. While that may be drastic and ideally will not be used very often, it can help decrease the amount of load put on the server, as well as reduce the volume of content stored in the database.

A new page, revisions.php, is being added to allow authorized members to view and manage any given post’s revisions. By default, when viewing this page, you will see a list of the revisions made to a post, in order, detailing who made each revision, when it was made, and for what reason (if provided). Each revision will have a checkbox next to it, allowing you to mark it for deletion or protection (see below). Underneath the revision list you will see a rendered diff (i.e. a visual representation of the changes made between two given texts) between the oldest available revision and the current post, as it exists on the *_posts table. You will be able to constrain the comparison by selecting a starting point and an ending point. You are also able to view an individual revision, by itself. Lastly, the post can also be reverted to any revision in the list.

In the previous paragraph, I mentioned the ability to mark revisions as protected. In some cases, you may wish to save a previous version of a post. Therefore, revisions can be marked as “protected”, and will not be included in the automatic revision pruning cron tasks, and will only be deleted if done so manually either via the post revisions view page or the mass revision purge option in the ACP.

One application of the Post Revisions system has also been implemented: wiki-style posts. Any user with the “f_wiki_create” permission can mark a post they have made to be a “wiki post”. Once this is done, the post can be publicly edited by anyone with the “f_wiki_edit” permission. When paired with the revisions system, changes can be tracked and reverted as needed. Note that there are separate settings for wiki posts to regulate the total allowable number of revisions per wiki post and the age of wiki post revisions; this way, you can keep track of more history of a wiki post for a shorter or longer period of time than for normal posts.

Well, I’m not done yet, and here is a list of some of the things I have yet to do
– Attachment versioning – I still have to figure out the best way to do this (if at all)
– Stress testing with a large board/userbase
– Figure out what Diff engine to use (I’m currently using PHP FineDiff, but I’m really looking for a UTF-8 compatible one, so if anyone has suggestions, feel free to let me know!)
– Further integration with the viewtopic screen – Currently, most of the revision-related front-end is contained in revisions.php; the only way to get there is from the edit count on viewtopic. So, for instance, I might display a compact list of revisions on a post, or maybe incorporate AJAX in some ways, etc.
– SubSilver2 changes (currently I have only done proSilver)
– More? I’d love to hear suggestions and comments.

For those interested, you may test what I have done so far, follow progressing development, and provide input by viewing the GitHub Pull Request and the RFC Topic (on the phpBB Area51 Development Board).

Thank you for your time!

Google Summer of Code 2012: Authentication Plugin Refactoring and User Integration

Posted by Hardolaf in Development with the tags , , on July 16th, 2012

During Google Summer of Code, I have been working on refactoring the authentication system (login and registration) to allow for login via OpenID providers and Facebook Connect [code]. The system’s old authentication plugins (db, ldap, apache) have all been ported to the new authentication provider system and can be enabled/disabled via the administration control panel. Currently, OpenID is implemented using the Zend Framework 2 OpenID package. The Zend Framework 2 OpenID code, currently does not allow login via Google (a popular OpenID provider), however, support for login via Google should be available by the end of Google Summer of Code. The user interface has been nominally changed from its previous look, however, additional changes to the interface will be made to make logging in via various popular providers more convenient for users.

The current (temporary) login box design.

The current (temporary) registration design.

As can be seen from the images, not much work has been done on styling the interface. It currently is focused on functionality for use in tests. The design of the interface will undoubtedly change before the end of Google Summer of Code.

For the remainder of the summer, I will be implementing login via Facebook Connect and potentially (time permitting) adding general support for OAuth2.0 providers. I will also be writing a wrapper class that will handle old authentication plugins so that they will continue to work while they are being transferred over to the new provider system.

I look forward to continue working with my mentor, Nils Adermann, on the remainder of the project. It has been a great experience so far contributing to phpBB, and although I’m am uncertain of how much time I will be able to commit after this summer, I look forward to continuing to work with phpBB and its community.

Google Summer of Code 2012: Search Backend Refactoring

Posted by dhruv.goel92 in Development with the tags , , , , on July 14th, 2012

During Google Summer of Code, I have been working on Search Backend Refactoring. The project involves minimizing code duplications by improving class abstraction in search backends and integrating PostgreSQL fulltext search [Code] and Sphinx search [Code] into phpBB core. Up till now, the progress has been good with PostgreSQL fulltext search backend already integrated into phpBB 3.1 and Sphinx backend nearing completion too.

The project has been a great learning experience. Apart from learning the various features of Git, I have been able to understand the basics of PostgreSQL and MySQL Databases more clearly. I was introduced to Sphinx during the course of project and honestly I was quite impressed by its performance. All boards running on MySQL and PgSQL databases will soon be able to switch to Sphinx search backend for faster indexing and searching.

Next I intend to start code refactoring and fixing bugs in multiple search backends to provide better search functionality to both users and administrators (by exposing more options in ACP). I will also be writing automated tests side by side to ensure search functionality continues to work correctly.

All this has been possible only because of the timely help I have received from mentors and other team members. They have been patient and appropriately guided me at all times. The overall development experience in the phpBB community has been fantastic, I would love to remain involved and contribute even after GSoC.

Google Summer of Code 2012: Attachment Improvements

Posted by Fyorl in Development with the tags , , , , on July 4th, 2012

During the Google Summer of Code I’ve been working on improving the current attachment system to have a bit more of a modern feel to it. I’m integrating Plupload which has a variety of features such as drag-and-drop, multiple file uploads and an upload queue.

The work has been pretty straightforward and I’ve been getting a lot of help from mentors and other team members so I’ve made considerable progress in a short time. The following screenshot shows Plupload integrated with phpBB and you can also look at the code itself in my pull request:

Plupload integration

By the end of it, phpBB users (with modern browsers) should be able to enjoy a much smoother attachment experience without a bunch of page refreshes interrupting their posting or waiting ages for a file to upload only to find out it was too big.

It’s not my first time contributing to an open source project but, surprisingly, one of the most useful things I’ve learnt is actually how to use git properly. There are a huge number of really useful features that I’ve never had to use until I started contributing to a project of this size. It’s given me more than my fair share of headaches and some of my activity history on github is a bit embarrassing now but everyone’s been really patient and helpful and I like to think I’m starting to get the hang of things now.

I’m having a lot of fun and intend to keep contributing to the project long after the GSoC has finished.

Help Us Automate phpBB Testing

Posted by imkingdavid in Development with the tags , , , on May 22nd, 2012

Automated Testing is a vital part of the development cycle, as it decreases the amount of time developers must spend making sure certain functionality works, and prevents new patches from breaking areas of functionality that are covered by the test suite. As you may have guessed, Automated Tests are simply sets of functionality tests that are automatically run as new features are developed, bugs are fixed, and changes are made to the codebase. Nothing is merged into the core phpBB repository without ensuring that whatever changes are being made do not break any existing functionality. If any tests fail, the developer is able to save time and energy that would have been spent later on, when a bug report would have been filed, by finding the problem and fixing it up front. Not only does this help by increasing the efficiency of the development cycle, but it also improves the quality of the software as a whole, because we can be certain that all features work as intended without having to manually test each one every time something is changed.

That sounds great, doesn’t it? Well, it is. The only problem is, only a very small portion of the code is actually covered by tests. One reason for this is because the automated tests have not been around for as long as the phpBB3 codebase, so they have had to be written long after the features they are testing have been merged, rather than in conjunction with the features themselves. Another reason is that there are two types of tests that need to be written: unit tests and functional tests.

Unit tests are simply tests that attempt to run certain peices of code that do not require a webserver or phpBB installation to work. These tests are generally easier and faster because all of the information they need is readily available and hardcoded into the test suite.

On the other hand, Functional tests are needed to run functionality that requires a phpBB installation. For instance, if you want to test whether an authentication plugin works, you have to actually be able to test logging into a board. For that, you need a database with a user account. You also need to be able to test that a cookie is sent. Unit tests cannot handle that, but Functional tests, are able to simulate browsing and using the forum almost just like a normal user.

However, functional tests cover even less of the code and functionality than unit tests do, and time that is spent writing functional tests is time that is not spent writing new functionality or expanding on existing functionality. So while we need functional tests to reap the benefits I explained above, it does not make sense to devote all of our time to writing tests and therefore have a significant “dry period”, during which no new or changed functionality is incorporated into the software.

This is where you come in. I, like many of you, was kind of intimidated when I first started to contribute to the core of phpBB. I felt much safer writing MODs because I knew my work would not be scrutinized as much as if I were to submit a patch to the core (that is not to say that the MOD team does not carefully review each and every MOD, but rather, that my code would likely not be viewed by as many people and would not be held to as high a standard). However, I found that by writing functional tests, I not only was able to contribute to the betterment of the software, but I also was able to better learn how things worked and how I could therefore use the things I learned in not only my MODs but also my core patches.

So all of that to say, there is plenty of work to be done and we would love to see contributions to our test framework from our vast, talented phpBB community. As I said, these contributions help not only the software and phpBB community as a whole, but also can give you the knowledge and skills you need to improve your MODs and, should you decide to, your patches to the phpBB software.

There is a wealth of information in the phpBB Development Wiki (see Automated Tests) to get you started with writing and running tests. And if you have a question or need help understanding how to do something, you are welcome to ask me or another developer in the #phpbb-dev IRC channel.

How you can aid development

Posted by igorw in Development on January 11th, 2012

Since we opened up development and moved our code onto GitHub, we’ve received several contributions. It allowed a number of non-development team members to contribute. We have been listing these contributors in the 3.0 release announcements, and you can also see them on the area51 contributors page.

However, I think we can do better. And this is why I want to reach out to the modding community. We have limited ressources and we need your help.

Contribute patches

The bug tracker is filled with bug reports. You may have reported a few bugs yourself. There’s so many, it’s likely that a large amount will never get fixed.

This is your chance. If you ran into a bug that really bugs you (pun absolutely intended), now is the time to fix it. If you’ve interacted with the MODding community, maybe by submitting some MODs to the cusomisation database, you should already be somewhat familiar with the phpBB code base and its coding guidelines.

Creating a patch is like creating a MOD. Except you don’t have to write a MODX file. Instead you can have a diff generated for you or just use git to manage the changes.

Once you are done with your patch, you can either attach a patch file to the ticket, or you can be awesome and create a pull request on GitHub. Pull requests make it easier for us to read, discuss, and test your patch. For this reason it will likely be merged faster.

Review pull requests

Pull requests are patches awaiting review. Most of the time they will be from developers. That’s right: every change to the phpBB code base needs to be reviewed by another developer. This allows us to catch problems early, and distribute the blame when something goes wrong.

There is a list of open pull requests that are waiting to be merged. Some of them already have comments, while others have not been reviewed yet.

Reviewing these is a lot of work. But you can help! If you know the phpBB code base well, you can help by checking the diffs and making sure no new problems are being introduced. Be sure to comment with your opinion; even simply saying “looks good” is helpful because it shows that someone has reviewed the patch.

The other thing you can do is test pull requests. Reviewing is nice and dandy, but testing allows you to see if things are really working. And this is very similar to testing MODs. So if you are a MOD author or maybe even a junior MOD validator, this process should be fairly familiar to you. The only difference is that everyone will be watching you.

Git

We use git for version control. This may be intimidating to lots of you, but for development there is really no way around it. It is a great tool, and I encourage anyone who writes code to learn how to use it.

You will first need to fork the phpbb3 repository. Then you will need it to create a branch on your fork for your change. When you have finished your patch, push to your fork on GitHub and submit a pull request. You can also checkout someone else’s branch with that person’s proposed changes, in order to test it.

I’ll be honest with you: it’s tough.

But there are lots of great resources out there! One of my favorites is the “Getting Git” talk by Scott Chacon. He explains the basics of how git works. This really helps you to get an initial understanding, so that you can move on to learn how to actually use it. Some great guides that can get you started using Git are the Git Reference or Pro Git. And we also have the Working with Git article on the phpBB wiki.

And fear not, we are here to help. If you run into any problems, just come onto IRC and ask.

In conclusion

So there you have it, secure your place on the area51 contributors page!

If you have questions or just want to see what we are up to, join us in the #phpbb-dev IRC channel on irc.freenode.net.

New Development IRC Channel: #phpbb-dev

Posted by igorw in Development on February 28th, 2011

One of the most important tools in open source software development is IRC. For those who are not familiar with it, IRC stands for Internet Relay Chat and is a protocol for real-time online communication — chat.

For historical reasons the development team was using a private channel until recently. So while discussions on the area51 development board and the tracker were public, the IRC discussions were not. It turns out that a lot of development discussion happens on IRC. Quick discussions between developers are easily achievable on IRC and often beneficial for reviewing patches and making low-impact decisions.

We wish to open up all aspects of phpBB development and encourage all users to participate.

So whether you’re just interested in how we work or if you want to make a contribution to the project…

Join #phpbb-dev on irc.freenode.net

We’re waiting for you.

Invading GitHub

Posted by igorw in Development, Modifications, Styles with the tags , , on November 13th, 2010

After the phpBB core migrated to GitHub earlier this year, most of the sub-projects have now been moved as well.

The initiative begun during the Libertyvasion conference, where we moved the UMIL, AutoMOD and MPV repositories. After that the customisation database and the support toolkit followed. Last night we moved the remaining MOD team projects: MODX, and the MODX Editor, Generator, Creator and finally QuickInstall.

I’m really excited about this relocation because of the huge contribution potential Git and GitHub have to offer. We have already seen a few translators take the hurdle to learn Git, and it makes the whole translation submission process a lot more open. There have also been a few patch contributions, and all I can say is keep those patches coming!

Now with all these public repositories accepting contributions, we need some guidelines. For this reason we came up with a set of Sub-Project Contribution Guidelines. We tried to keep them as short as possible, here is the substantial part:

  • Projects should be hosted on GitHub under the phpBB organization.
  • All work by team members should take place in the main repo.
  • Use of topic branches is very strongly encouraged.
  • Contributors _must_ use topic branches, foreign main branches are never merged.
  • Version tags must be fully lowercase and prepended by a ‘v’, for example ‘v1.0.0-rc1’.
  • Branch names should simply describe the topic and use hyphens, for example ‘quick-reply’ or ‘issue1234’.
  • Translations should have a branch named ‘intl-<lang>’, for example ‘intl-de’.
  • Projects with many core contributors can use forks.

Please do read the full guidelines, as they include some additional information.

I am happy to announce that we have completed our GitHub invasion and are feeling very much at home. Fork away!