heartwordpress1I was inspired by our CTO’s eloquent post on how our development process has evolved over the years and I thought I would add how we start the dev process with a written specification for the specific feature or product we want to develop.  We call these FRD’s or Feature Request Documents.  There a number of steps in any software specification but all good specifications should start with the history of what specifically is driving the feature.

The reason we start with the history of the feature is that it gives the eventual software developer some idea of how this feature evolved and why it is needed.  For the most part software developers are experts on, well developing software.  They usually have no deep domain expertise and rely on the documentation to supply the context of the feature set.  Also my software guru, Joel Spolsky insists that specs get written before code.

So during my forced haitus I developed a serious addiction to the WordPress CMS/Blog/(soon to be social) platform.  As I realized the depth and breadth of WordPress I started thinking about ways of using it to automate some of the processes at D-Tools, specifically automating our monthly email newsletter.  As I dove deeper into this I was surprised that I could not find anything like this, I mean it seemed like a natural thing to do.  So I decided to build a WordPress-to-email newsletter plugin.

Since I am not a software developer I had to write a software specification for this feature that I could give to our dev team here.  Normally this would be an internal document but I decided to share it for three reasons:

  1. It helps me write better when I publish for the world
  2. It helps other people in the world learn and enhance their business processes
  3. I can show the world that I am the smartest person in the room by publicly putting my stake in the ground on this feature

So without further ado, I am pleased to introduce the history of the WordPress-to-email newsletter plugin.

We have had a monthly email newsletter using the I Make News (IMN) service since 2001.  The content of this was usually custom created and then stuffed into a crusty template and sent out to our subscriber list that was managed by the IMN opt-in process.

This was a decent solution but the following problems became evident:

  1. More and more the relevant content was being created on our blog and then being re-purposed (read cut and paste) to the newsletter.
  2. The IMN newsletter looked like an ass hat and needed a serious makeover.  You can see it here.
  3. The read more links to each article in the newsletter went to the web version of the newsletter on the IMN servers.  This destroyed any good link juice and search results.
  4. The archive was on their servers, not ours and once we moved it was impossible to keep them without a massive cut and past effort.  I hate losing our content and data.
  5. No way for the readers to comment on each article, it was a one way communication.  You will read this and you will like it, mister.
  6. Duplicate work that took the marketing team 10-15 hours a month to create, compile and publish.
  7. Any article that was published in the newsletter was pretty much lost forever whereas the same article that was published on our blog would live forever

As I got more and more into the WordPress platform I realized that a lot of these problems could be solved by using WordPress as the publishing platform and re-purposing the content to the monthly newsletter.

The first part of the journey was a search for “WordPress newsletter plugin”.  I wanted something that would perform all of the list management features as well as allow me to create, manage and send newsletter templates from within WordPress  This is what I found:

  1. Shift This WordPress Newsletter plugin looked OK but it was not supported anymore.  No point in investing time into an unsupported platform.   Next…
  2. A bulk email service provider SendBlaster has a WordPress opt in email subscription Widget but it looked to be a desktop program for mass email with a WordPress plugin, and as far as I could tell  no way to automatically generate content directly from WordPress.
  3. I bought the Tribulant WordPress mailing list plugin for $44.99.   It looked liked it had all the features I was looking for but once I actually started using it I realized that is was  (in my professional opinion) a P.O.S., and if the number of unanswered posts in the support forum is any indication I am not the only person that has this opinion.  I wrote it off and continued my search.
  4. I found MailPress, a free plugin available here.  Seems to be feature complete but there are a lot of modules and code.  Also I did not want free.  I would rather pay for the product and support.   Anyway, I did the full install, realized that this was too big and bulky, came with limited documentation and no support.  It was not worth my time to try and figure out all the subtle nuances.

In any case I realized that any newsletter plugin that used WordPress will have a fatal flaw in that you will be using your ISP to send the emails.  ISP’s DO NOT like to send bulk emails.  After some checking with Dreamhost (the ISP of this blog), I found that their bulk email for any account is throttled to 200 emails per hour,  At that time our list was over 5000.  Also they will terminate your account if you violate their anti spam TOS.

So no matter what, any plugin that uses your ISP to send bulk email is eventually going to get you thrown off the island, not worth it.  I stopped looking for internal ways to do this and started looking in other directions.

As I got more into the RSS feed specification I started to wonder if I could create a category-specific feed from our blog and send that to some sort of bulk email service provider who could take the feed and create a newsletter out of it.  That seemed like a more logical approach.

I found that you can create category-specific RSS feeds from any blog that uses categories by getting the feed for the blog and then adding the word “category” and the actual category name or ID before the feed. For example, here is my friend Chris Roth’s Visio Guy blog.  http://www.visguy.com/ The normal feed for the entire blog looks like this http://www.visguy.com/feed/ and “shape” category feed looks like this:  http://www.visguy.com/category/shapes/feed/ In our case I created a category called “newsletter” in the D-Tools blog so all of the content in that category could be easily exported into RSS/XML.

Here is some more information on creating custom RSS feeds:

Some additional RSS info before I forget.  There are different flavors of RSS and choosing the correct one will have an impact on your automated newsletter generation.  This feed http://www.visguy.com/feed/ will generate a RSS 2.0 feed WITH a publish date.  This feed http://www.visguy.com/feed/rss/ will generate a RSS .92 feed WITHOUT a pub date.  Trust me, you want the publish date.  You can also burn a custom feed using Feedburner and get this valid RSS feed http://feeds.feedburner.com/visguy/Quxt to track your feeds.

So I could get category specific feeds but they looked pretty bland since WordPress only offers a choice of the full post will all the images or a text only excerpt of the post with no images.  What I wanted was the first image in the post to be automatically converted to a thumbnail, left aligned with the an excerpt of the test along with a link back to the origonal post.  After taking a closer look at other feeds I realized that some feeds have pictures and some don’t; some have the full article and some don’t, some have a just a thumbnail of the first image and an excerpt.  Some images were left aligned, some were right aligned.  It was all over the place.  Here are some sample feeds:

I am jumping ahead but this is what I was able to do with some custom PHP code as a WordPress plugin.  Here is our blog link http://www.d-toolsblog.com/ and here is the feed for the blog http://feeds.feedburner.com/D-toolsBlog Notice how clean and tidy everything looks?  That is exactly what I wanted and is totally automatic from here on out.

Here is the newsletter specific feed.  http://www.d-toolsblog.com/category/newsletter/feed/ and here is the actual newsletter that was automatically created from this feed.  http://campaign-archive.com/?u=9a33d51fe9c048578a898aa66&id=1ec8c6efa4

Keep in mind that a custom RSS feed has to be generated at the source.  For the most part all any RSS reader software does is display the feed, it does not attempt to add or format the feed So all format changes have to be done before the feed is generated.

Stay tuned for Part II: UI’s