Mid week update
August 17, 2011 § Leave a comment
After the big update last week not a lot has happened. We had to clean up a few smaller issues but nothing out of the ordinary (and thankfully no down time). The next few weeks will be slow because we’re all traveling.
Different code, different problems
The old code only ever used the live feeds — they were downloaded, processed to generate the pages and then discarded. This had some major drawbacks, especially for the stats since most feeds are too short to represent a whole month of posts.
Now that we’re temporarily storing information, we have to deal with other issues. For example, wordpress users mostly have rss feeds which do not offer a timestamp for the last update. So if something changes in a post (say, somebody adds a PlanetMO tag), the feed does not easily reflect this and our database does not update because checking more than the timestamp is rather cpu heavy. There’s not much we can do except appeal to bloggers to use the atom protocol, but since we’re only keeping information for a month, we hope it’s not a huge issue. After all, most posts never see an update.
After last weeks huge update only four new feeds found us and were added today.