Enable the use of temporary queue tables when sending campaigns

Enable the use of temporary queue tables when sending campaigns

The main issue we have with huge AppMail  apps is that the campaign_delivery_log table becomes huge (I’ve seen installs with hundred GB’s of data in that table alone) and we use that table to join against when we send the campaigns so that we can see to which subscriber we have to send the emails, filtering the ones that already received the emails.
Joining SQL tables is an expensive operation in itself, joining on a table with million records is even more expensive, which leads in the end to slower sending speeds for campaigns.

In order to fix this problem, starting with version 1.3.7.9 we have introduced an option that if enabled, when a campaign starts sending, instead of joining on the campaign_delivery_log table, it will actually create a new database table, it will copy all the subscriber ids that it has to send into this new table, then it will start processing based on this new table.
Each time an email is sent, a record from this new table is removed and, in the end, when delivery finishes, the table is removed for good.

Before enabling this method, you have to make sure that all your campaigns have finished sending, otherwise you’ll end up sending duplicate emails. Once all the campaigns have finished sending, you can enable the feature by adding following configuration param in your configuration file:

'send.campaigns.command.useTempQueueTables' => true,

After this, start sending a new campaign and watch your database, you should see a new database table added during sending, it will be named: tmp_cmp_queue_X (where X is the campaign ID), and it will be removed when sending finishes.

    • Related Articles

    • Debug send-campaigns command

      Starting with version 1.3.5, the cron job command that sends campaigns (send-campaigns) has been modified so that it will help debugging sending speed. In order to make use of this functionality, you should have ssh access to your server so that you ...

    • Recurring Campaigns

      One great feature of AppMail is recurring campaigns, which means you can set it in a way that every 10th of each month you could send an RSS FEED campaign with latest blog posts. But this is not all, you can even set it to send in various days of the ...

    • My campaigns don’t send, they’re stuck in “Pending-Sending” status

      When this happens, most likely the cron jobs that must run in order to pick up the campaigns and send them are not running, and no, it’s not something wrong with the application, but you simply need to feed the application with the right data in ...

    • Sending Groups

      Historically, until version 2.0, when you wanted to send a campaign to more than one list/segment, AppMail would merge the selected lists/segments into a new list and would then send the campaign to that new list. Performance wise, this is a very bad ...

    • Enable campaign geolocation tracking for opens

      In order to enable geolocation tracking for opens, to show you a breakdown of the countries that open your emails, something like: You have to follow a few steps. 1. Go to Backend > Extend > Extensions and enable the Location extension. 2. Once the ...