Newsbeuter 2.1 will contain support for 256-color terminals

May 6, 2009

Recently, I found and fixed a bug that prevented the awesome STFL to function correctly on 256-color terminals such as xterm with TERM=xterm-256color. In addition, I added support to both STFL and newsbeuter to specify colors from the whole 256-color palette. Currently, if you want to try out this feature, you need the latest SVN version of STFL (can be found here) and the latest git master branch version (can be found here). To specify one of the “extended” colors, simply use “color<n>” (e.g. “color123”) instead of using the color name (“blue”, “black, “green”, etc.) in the color-related configuration. A chart of color numbers and how they look like can be found here. Finally, here’s a screenshot of the whole thing:


As you can probably recognize, these are not the usual colors from the usual 16-color palette. But, since I’m really bad at designing colorful themes, I ask everyone who has an interesting newsbeuter color configuration to submit it to me so that I can include it to newsbeuter’s contrib directory that already contains a few theme examples.

Newsbeuter 2.0 released

April 21, 2009

Yep, you read correctly, newsbeuter 2.0 is out. In the last 4 months of work, I added so many more features than I had originally planned, so I though, “there will probably never be a release with more new features, this is the time to make a major release”.

One reason for all the new features is probably a tragic one: a few days after moving to Berlin, I had to be brought to the hospital, where I had to spend two weeks, and thus had a lot of time to work on new features. That’s also why I call this release the “Virchow release”: the hospital where I was brought was the Virchow hospital in Berlin (CharitĂ© Campus Virchow, to be exact). Thanks also go out to newsbeuter’s Debian maintainer, Nico Golde, and his girlfriend, who visited me in the hospital a few times and made the time there a lot less boring, and to text tool rockstar Sven Guckes, who also visited me and gave me a lot of great feedback regarding newsbeuter.

As usual, you can grab the latest newsbeuter release from newsbeuter’s download page.

As already mentioned, a lot of new features have been added, and so I will describe them to give a better understanding of what has changed and improved.

First of all, the whole dialog management was made much more flexible. Before this release, all dialogs (i.e. feed list, article list, article view, …) were internally stored as a stack: opening e.g. a feed pushed a new article list on this stack, and quitting this dialog removed it from the stack again. In this release, this functionality has been extended: it is now not only possible to push new dialogs to the stack, you can now jump to other dialogs that are open but not on the top of the stack, and you can even remove open dialogs. Just press the ‘v’ key, and you’re in the “dialogs” view from where you can do all these operations. So, what’s the advantage? It e.g. allows you to open more than one feed or article at the same time. By pressing Ctrl-V or Ctrl-G, you can go to the next or previous dialog in the list, making it possible to quickly switch between different dialogs.

The second new big feature is the replacement of libmrss and libnxml with an own RSS/Atom parser which uses libxml2. Not only were about 6000 lines of C code replaced with about 600 lines of C++ code, newsbeuter now also builds upon on a proven and flexible XML parser. This change also led to a nice side effect: due to the new code, it was pretty easy to implement proper support for the HTTP/1.1 “If-Modified-Since” and “If-None-Match” conditional download features. libmrss had similar abilities, but had to do two requests. This also means that the new code downloads feeds much faster and with less resource usage. Basic support for Yahoo’s Media RSS extension has also been added: Media RSS content is handled like enclosures. And it’s now even possible to configure local files as RSS feeds, by using the “file://” prefix before the local file path.

Another improvement is that newsbeuter is now able to handle more than one OPML files when using OPML URLs as URL source: this allows it to subscribe to several OPMLs and read all of the contained feeds.

Another convenience feature is the ability to configure for how many days articles shall be kept (coniguration option “keep-articles-days”). Until now it was only possible to configure the maximum number of articles per feed that shall be kept. In addition to this, it is now also possible to configure how long articles shall be kept since they were initially downloaded. Both features can be combined, of course.

A security improvement has also been added to newsbeuter 2.0: URLs that contain authentication information (e.g. http://username:password@hostname.tld/feed.rss) are now “censored”, i.e. username and password are replaced by ‘*:*’, making it impossible to see the authentication information anywhere on the screen.

HTML rendering has also been improved: text within links is now underlined, text between <u> tags is underlined, too, and text between <strong> or <b> tags is now rendered as bold. This makes HTML rendering much nicer than before.

The commandline has been improved, too: newsbeuter now contains support for commandline completion using the TAB key: type 0 or more characters of the command you want to execute, and press TAB to cycle through all the matching commands. The “set” command even allows completion of the available variable names. Another extension is that entered commands and search queries are now persisted to history files in the ~/.newsbeuter configuration directory. This allows to enter commands, quit newsbeuter, reopen it, and to have all the previously entered commands and search queries available in the commandline resp. search history. Command and search history persistence can of course be disabled.

Other nice new features are the “source” commandline command to reload configuration files, the “pipe-to” key to pipe the current article to an external command, a new “between” operator for the internal query language, a new “age” attribute for articles to filter for articles for relative age (in days) and the ability to sort feed list and article list interactively.

And even the search function was improved: it is now possible to search for text from the article view (found text now stands out in the text), and when opening articles from the search result dialog, the search phrase is also marked in the text.

And last but not least, a kind of “funny” feature was added too: when pressing Ctrl-K, newsbeuter now goes to a random unread article. This makes reading articles with newsbeuter a lot less predictable. This wasn’t my idea, but is based on input from a user, and since I usually listen to the newsbeuter users, I implemented this feature. Even though it’s quite unusual to have such a non-deterministic feature in a newsreader, there seems to be a demand for such functionality.

And another achievement during development was that I finally found the reason for the occasional crashes that some of you probably experienced: it was a subtle but really stupid programming error where I forgot that C++’s std::vector class is not thread-safe, and concurrent read/write accesses from several threads led to occasional crashes. This is fixed now, and newsbeuter has been rock-stable in my tests so far.

So, I hope you like this new release, download it, use it, and if you find any bugs or have any feature requests or wishes, don’t hesitate to report them to me. Enjoy!

A new RSS/Atom Parser

December 31, 2008

From its very beginning until now, newsbeuter has always used libmrss to parse RSS and Atom feeds. But there were a few problems: libmrss (and libnxml, the XML library it builds upon) are both written in pure C. Also, they are quite big (more than 6000 lines of code), and licensed under the LGPL, a license that I don’t really prefer.

So I decided to replace libmrss and libnxml with a self-developed RSS/Atom parser, using libxml2 as its base XML library. While the development of a new RSS/Atom parser isn’t too difficult by itself, it is still not that trivial due to the vast amount of variations of RSS. I already tested it with more than 100 feeds, but I urge everyone to check out the git master branch from newsbeuter’s git repository, compile it, and test it with the feeds that you read. Don’t hesitate to report any issues or irregularities back to me. Every feedback helps with the stability of newsbeuter, and I want to make the new parser as stable as possible before releasing the next version of newsbeuter.

Migration to GitHub

December 20, 2008

Earlier today, I migrated newsbeuter’s Subversion repository to a Git repository that is hosted on GitHub. You can find information about the new GitHub repository (including how to access it) here.

Why git? Why GitHub? Well, the primary reason to migrate the source repository away from its old location was that I simply wanted to move it away from my own infrastructure. One property of git, namely being a distributed version control system, made the decision easy in favor of it, i.e. wherever the source is hosted, it can be easily migrated to other locations. The choice for GitHub fell when I saw the kind of features that it provides. Not only does it make its service available for free for open-source project, but it also makes it easy to start own forks, including the ability for developers (like me) to view and (optionally) merge changes and improvements that others produced. And it also forces me to learn git better. 😉

newsbeuter 1.3 released!

December 6, 2008

Finally, after 3 months of developing, I released newsbeuter 1.3. You can download it from the usual location.

So, what’s new in this release? First of all, the most fundamental change is an internal change: I improved the internal architecture of newsbeuter to make all the random crashes disappear that many people had: most datastructures within newsbeuter are now using smart pointers, using the C++ TR1 std::tr1::shared_ptr<> implementation. This also means that a g++ 4.0 or later is required to build newsbeuter.

Besides this change, a number of new features have been added:

  • Podbeuter now keeps finished downloads in the queue until they’ve been played.
  • Placeholders can now be used in the download-path so that podbeuter downloads are automatically sorted into a directory structure and/or get meaningful names.
  • The list of subscriptions can be edited directly from newsbeuter’s feedlist by pressing “E”. The content of the $EDITOR environment variable will be used to edit the ~/.newsbeuter/urls file.
  • A file format to exchange information about read/unread  articles between newsbeuter installations has been added. That makes it possible to e.g. dump the GUIDs of read articles at work, send them home, and import them there. This avoids the situation that a user is bothered twice with an article he has already read.
  • A new configuration option feed-sort-order has been added that allows the user to sort the feedlist by the first tag that has been assigned to every feed.
  • It is now possible to toggle the read flag from the article view. This feature was implemented by Isaac Good and sent to me as a patch.
  • It is now also possible to configure the number of parallel download threads to enhance the download experience for users with lots of subscribed RSS feeds.

In additon, I would like announce the newsbeuter group on Google Groups. I created this group to provide a discussion platform for newsbeuter users that aren’t comfortable with using the existing IRC channel (#newsbeuter on

I hope that the next release of newsbeuter won’t take 3 more months, but don’t expect it too soon, either, as I’m going to relocate from Linz, Austria to Berlin, Germany where I accepted a job as software developer (doing C on Unix) for a big German webhosting company. The relocation will happen in January, and during that time, the Subversion repository will also be offline and probably be relocated to another server.

newsbeuter 1.2 released: fixes incomplete security fix, please upgrade

September 2, 2008

OK, this is kind of embarrassing… please upgrade to newsbeuter 1.2, as the security fix in 1.1 was incomplete. In addition, you even get new features, such as two new configuration options “download-retries” and “download-timeout” to make newsbeuter more reliable over unreliable connections such as TOR proxies, and improved HTML rendering for <pre> environments. And I hope this was the last release until a month or so…

Newsbeuter 1.1 released: contains security fix, please upgrade

September 1, 2008

Newsbeuter 1.1 was just released. Grab it from here. It contains a security fix for an issue where opening specially crafted URLs with the browser could lead to shell command execution, so an upgrade is highly recommended.