Blog

Posts Tagged ‘ascraeus’

Google Summer of Code 2012: Search Backend Refactoring

Posted by dhruv.goel92 in Development with the tags , , , , on July 14th, 2012

During Google Summer of Code, I have been working on Search Backend Refactoring. The project involves minimizing code duplications by improving class abstraction in search backends and integrating PostgreSQL fulltext search [Code] and Sphinx search [Code] into phpBB core. Up till now, the progress has been good with PostgreSQL fulltext search backend already integrated into phpBB 3.1 and Sphinx backend nearing completion too.

The project has been a great learning experience. Apart from learning the various features of Git, I have been able to understand the basics of PostgreSQL and MySQL Databases more clearly. I was introduced to Sphinx during the course of project and honestly I was quite impressed by its performance. All boards running on MySQL and PgSQL databases will soon be able to switch to Sphinx search backend for faster indexing and searching.

Next I intend to start code refactoring and fixing bugs in multiple search backends to provide better search functionality to both users and administrators (by exposing more options in ACP). I will also be writing automated tests side by side to ensure search functionality continues to work correctly.

All this has been possible only because of the timely help I have received from mentors and other team members. They have been patient and appropriately guided me at all times. The overall development experience in the phpBB community has been fantastic, I would love to remain involved and contribute even after GSoC.

Google Summer of Code 2012: Attachment Improvements

Posted by Fyorl in Development with the tags , , , , on July 4th, 2012

During the Google Summer of Code I’ve been working on improving the current attachment system to have a bit more of a modern feel to it. I’m integrating Plupload which has a variety of features such as drag-and-drop, multiple file uploads and an upload queue.

The work has been pretty straightforward and I’ve been getting a lot of help from mentors and other team members so I’ve made considerable progress in a short time. The following screenshot shows Plupload integrated with phpBB and you can also look at the code itself in my pull request:

Plupload integration

By the end of it, phpBB users (with modern browsers) should be able to enjoy a much smoother attachment experience without a bunch of page refreshes interrupting their posting or waiting ages for a file to upload only to find out it was too big.

It’s not my first time contributing to an open source project but, surprisingly, one of the most useful things I’ve learnt is actually how to use git properly. There are a huge number of really useful features that I’ve never had to use until I started contributing to a project of this size. It’s given me more than my fair share of headaches and some of my activity history on github is a bit embarrassing now but everyone’s been really patient and helpful and I like to think I’m starting to get the hang of things now.

I’m having a lot of fun and intend to keep contributing to the project long after the GSoC has finished.

Help Us Automate phpBB Testing

Posted by imkingdavid in Development with the tags , , , on May 22nd, 2012

Automated Testing is a vital part of the development cycle, as it decreases the amount of time developers must spend making sure certain functionality works, and prevents new patches from breaking areas of functionality that are covered by the test suite. As you may have guessed, Automated Tests are simply sets of functionality tests that are automatically run as new features are developed, bugs are fixed, and changes are made to the codebase. Nothing is merged into the core phpBB repository without ensuring that whatever changes are being made do not break any existing functionality. If any tests fail, the developer is able to save time and energy that would have been spent later on, when a bug report would have been filed, by finding the problem and fixing it up front. Not only does this help by increasing the efficiency of the development cycle, but it also improves the quality of the software as a whole, because we can be certain that all features work as intended without having to manually test each one every time something is changed.

That sounds great, doesn’t it? Well, it is. The only problem is, only a very small portion of the code is actually covered by tests. One reason for this is because the automated tests have not been around for as long as the phpBB3 codebase, so they have had to be written long after the features they are testing have been merged, rather than in conjunction with the features themselves. Another reason is that there are two types of tests that need to be written: unit tests and functional tests.

Unit tests are simply tests that attempt to run certain peices of code that do not require a webserver or phpBB installation to work. These tests are generally easier and faster because all of the information they need is readily available and hardcoded into the test suite.

On the other hand, Functional tests are needed to run functionality that requires a phpBB installation. For instance, if you want to test whether an authentication plugin works, you have to actually be able to test logging into a board. For that, you need a database with a user account. You also need to be able to test that a cookie is sent. Unit tests cannot handle that, but Functional tests, are able to simulate browsing and using the forum almost just like a normal user.

However, functional tests cover even less of the code and functionality than unit tests do, and time that is spent writing functional tests is time that is not spent writing new functionality or expanding on existing functionality. So while we need functional tests to reap the benefits I explained above, it does not make sense to devote all of our time to writing tests and therefore have a significant “dry period”, during which no new or changed functionality is incorporated into the software.

This is where you come in. I, like many of you, was kind of intimidated when I first started to contribute to the core of phpBB. I felt much safer writing MODs because I knew my work would not be scrutinized as much as if I were to submit a patch to the core (that is not to say that the MOD team does not carefully review each and every MOD, but rather, that my code would likely not be viewed by as many people and would not be held to as high a standard). However, I found that by writing functional tests, I not only was able to contribute to the betterment of the software, but I also was able to better learn how things worked and how I could therefore use the things I learned in not only my MODs but also my core patches.

So all of that to say, there is plenty of work to be done and we would love to see contributions to our test framework from our vast, talented phpBB community. As I said, these contributions help not only the software and phpBB community as a whole, but also can give you the knowledge and skills you need to improve your MODs and, should you decide to, your patches to the phpBB software.

There is a wealth of information in the phpBB Development Wiki (see Automated Tests) to get you started with writing and running tests. And if you have a question or need help understanding how to do something, you are welcome to ask me or another developer in the #phpbb-dev IRC channel.

Explaining Hooks

Posted by igorw in Development with the tags , , , on October 9th, 2010

Most of you have probably already seen the blog post regarding the RFC process for hook locations to be added to Ascraeus (phpBB 3.1). And many of you may not know what a hook system actually is, so I will take the time to explain it.

In Olympus (phpBB 3.0) all new additions to the core must be made by editing or patching the core files. The MODification is either installed manually or by using AutoMOD. What a MOD usually does is add some specific code at a specific location, this ranges from database queries, template assignments and loading of language files to any custom PHP code. Now, this approach worked for 2.0 and it works for 3.0, but frankly it produces a horrible mess. Installing MODifications is a great pain, uninstalling them is almost impossible in many cases.

To solve this issue we are introducing a hook system. The hook system defines code injection points, giving each one of them a unique identifier. The hooks controller allows MODifications to bind to a specific code injection point or hook location. When that point is reached in execution, the hooks controller will call all of the hooks that registered, passing in the context, so that local variables can be read and/or modified by the hook, thus “injecting” the hook into that location. Technical details can be found within the General Hook Architecture RFC.

Automagic

To remove the need for any core edits the files defining and registering hooks will be automatically loaded. They will be in a certain folder that phpBB will scan regularly. This will hopefully make the whole MOD installation procedure automagic. You drop in the files and are ready to go.

To make the registering of hooks more convenient, MOD authors will be able to define a class of which each method represents a hook.

This allows all hooks belonging to a certain MOD to be grouped within one file that can be automatically loaded and registered.

Example

Here is an example with code. Note that this API may change since it’s not final yet. The hook location will be in user_add as described in this RFC topic ([RFC] user_add hook).

We will create a file called hook/logging.php (convention will change) containing the following class, which will be loaded automatically by phpBB.

Code: Select all
class phpbb_mod_logging
{
public function user_add(&$sql_ary, &$cp_data)
{
file_put_contents('registered_users.txt', time() . ": " . $sql_ary['username'] . "\n", FILE_APPEND);
}
}

Since we want to hook into user_add and the RFC describes the input arguments as $sql_ary, $cp_data (by reference), our hook will have to take those arguments. In this example we will write the unix timestamp and the username of every new user into a log file. This happens right before a user is created.

The user_add() function will execute all ‘user_add’ hooks before inserting the user. While we are only reading the arguments here, it is also possible for hooks to modify them.

Under the hood

So, how does this work? Let’s take a look at the engines that power this. These are mostly internals of the hook system that you will not have to deal with, because it is all taken care of by phpBB.

Before we can do anything hook related, we need to create a hooks controller. The hooks controller manages the hooks and makes sure all of them are called. All hook-related actions go through this class.

Code: Select all
// create the hooks manager
$hook_handler = new phpbb_hooks_controller();

In phpBB this controller is automatically loaded, and there is also code that will load all the hook files (hook/*.php). It registers the hook containers of those files with the hooks controller.

Moving on to the actual hook invocation. Within the actual user_add function, you will find the following:

Code: Select all
// call hooks
$hook_handler->invoke_hook('user_add', array($sql_ary, $cp_data));

This calls all hooks that bound to ‘user_add’, passing the two arguments into them. There will be quite a few of these hook locations to take advantage of, and they will always provide a context that you can work with.

As a MODification author you will also be able to provide your own hook locations to other MOD writers. All it takes is a call to invoke_hook, which will allow others to hook into your code.

There are some more advanced features about hooks, such as (un)installing, enabling/disabling, prioritising hooks. But I’ll save that for later.

In summary

  • class with methods representing hooks
  • invoke_hook will call all hooks that registered
  • hooks are automagic

So now that you understand the basics regarding hooks, I hope you can look forward to this great addition in ascraeus and help us by contributing Hook Location Requests.

phpBB “Ascraeus” 3.1 Feature Freeze

Posted by naderman in Development with the tags , , , , on July 20th, 2010

Yesterday marked the announced feature freeze date for phpBB Ascraeus 3.1. We have postponed this date once before and we are still a bit behind schedule, but we will stick to it this time. We have picked a number of features with unfinished implementations or still in RFC phase which we believe to be very important and which can be realistically finished in time. We are still learning to work with our new development model and I’m sure we will improve until the phpBB Arsia 3.2 release. All features that are implemented or suggested after today are going to have to wait for phpBB Arsia 3.2 to become part of phpBB (small exceptions may be made).

As you can see from the list below this release is going to focus on improving a number of existing features as well as improving MOD installation & MOD writing. The following is the list of features and improvements for 3.1 in no particular order:

Merged:

Patch in progress:

RFC only:

  • Migrations (UMIL) RFC Ticket Data Providers RFC
  • Changing IP banning to use ‘longest prefix matching’ RFC Ticket
  • Ability to delete auto login keys RFC Ticket
  • Authentication Plugin Refactoring, User Integration & OpenID RFC Ticket
  • Session Backend Abstraction (memcache support) RFC Ticket
  • Search Backend Refactoring RFC Ticket
  • Pre-Compile Template Includes RFC
  • Overridable imageset and theme paths for CDNs RFC

I would like to thank the rest of the development team for their work so far, and am looking forward to to finishing the first release of Ascraeus together with them. A special thank you goes to all the other volunteers who have brought great ideas to the proposal process and have provided us with high quality patches. They have really made me appreciate our active community and I’m looking forward to more collaboration during the next steps toward 3.1. You can find a list of all the contributors who have written code that was already merged at github This list will gain in length over the coming months as more of the proposed changes are merged.

Please discuss this blog post on the forum.

Moving on – the future of phpBB development

Posted by naderman in Development with the tags , , , , , , , , , , on February 18th, 2010

About a month ago I was promoted to phpBB’s lead developer position. When I was given that role I was asked to present to the management team what my plans for phpBB are. But since they concern all of you, the phpBB community, I believe they belong here as well.

We are currently in the process of releasing phpBB 3.0.7, a smaller bug fix release of Olympus. After that development will split into multiple branches. Meik has already explained our new versioning system in a previous announcement. What this means is that after 3.0.7 is released we will start working on two branches: Ascraeus – the future phpBB 3.1, and Rhea – the future phpBB4. At the same time we will continue maintanance releases of Olympus.

Over the last few weeks we have made a few important decisions, that we believe can positively impact phpBB development. These include changes to development infrastructure as well as our development model. I will briefly outline these changes here, more details will be posted on the blog or as an announcement soon. Read the rest of this entry »

Londonvasion Re-Cap – phpBB Ascraeus

Posted by Acyd Burn in Development with the tags , , , on September 19th, 2008

Hi,

I think it is time to write about the presentations we (the developers) gave at Londonvasion revealing the planned features in the upcoming version of phpBB – codenamed Ascraeus.

Please do not take anything mentioned within this blog post as guaranteed. There may be changes, some things may not evolve at all, some may be changed completely. But yeah, this is the current state of planning.

Read the rest of this entry »