New Release

The Taskwarrior team would like to announce the simultaneous release of Taskwarrior 2.3.0 and Taskserver 1.0.0.

Taskwarrior 2.3.0

Taskserver 1.0.0

We would like to thank everyone for the questions, complaints, bugs, feature requests, code patches, doc patches, test platforms, ideas, arguments, discussions, persistence and patience.

We especially want to thank those who donated continuous integration build server time.

Advertisements

Improving Filters

Filters support the idea of restricting a set of tasks according to the attributes of that task. The simplest filter is an empty one, which performs no restriction, resulting in a list of all tasks:

    task all

The ‘all’ report is used here because it has no intrinsic filter. Other reports have filters that obstruct the illustration. A simple filter is to restrict to only ‘pending’ tasks, like this:

    task status:pending all

This match can be inverted by inserting an attribute modifier:

    task status.not:pending all

Multiple matches can be combined, and the result is a conjunction, further restricting the results:

    task status:pending project:Home all

These attribute modifiers have been used for a long time, but they are limited.  Recent changes have added support for better filtering, and opening the syntax up for more interesting capabilities. For example, the ‘:’ can be replaced by an ‘=’ character, allowing this:

    task status=pending all

While this appears to be a simple substitution, it is actually something more.  The ‘name:value’ syntax is recognized by the command line parser, then converted to ‘name=value’ internally and then the evaluator *only* supports ‘=’ as an exact match operator. This is true of almost all the syntax for filters; the command is parsed, converted, and the internal format is evaluated. The above command is equivalent to:

    task status = pending all

Suppose you wanted to filter on tasks containing ‘the’. There are several ways to do this, the simplest is:

    task the all

This is just a shorthand way of specifying:

    task description.contains:the

But you can use regular expressions too:

    task /the/ all

Here the // characters delimit the regular expression. Internally, all of the above are treated the same:

    task description \~ the all
    task 'description ~ the' all

You can see here that either the backslash \ or the quotes ” are needed to hide the ~ character from the shell, which will it expand to your home directory.  There are many special characters that the shell will act upon.  The above can be inverted (and abbreviated) with this:

    task 'desc !~ the' all

Similarly, filters containing multiple restrictions, such as this:

    task status:pending description.contains:the all

Are using conjunctions, which means the above is equivalent to inserting an implicit ‘and’ operator between the terms like this:

    task status:pending and description.contains:the all

Rewriting this using operators instead of attribute modifiers, we get:

    task 'status = pending and description ~ the' all

In addition to ‘and’, you can use ‘or’:

    task 'status = pending or description ~ the' all

You can use parentheses to control precedence:

    task '( status = pending or status = completed ) and desc ~ the' all

For a description of current filter support see Filters.

Where Are Filters Heading?

Filters are being simplified. That means less of the syntactic sugar (/the/), with more operators (~, !~ …) and standardization. The attribute modifiers are too limited and will not be further enhanced. Soon they will be deprecated,
ultimately removed.

Filters are simply algebraic expressions, evaluated in the context of a single task. Taskwarrior is getting a new command line parser and expression evaluator for 2.4.0, and this means new operators and greater use of expressions.

Next…

The next article will cover the new functionality coming in 2.4.0 that takes the new filter syntax much further.

Codebase 101 – Color

Taskwarrior supports color in the terminal, and does this by implementing a color model, which is a nice abstraction for handling colorized text.  The model allows us to specify color such as “red”, instead of using…

Terminal Escape Sequences

Terminal programs such as XTerm display the data received from an application in the window or on the screen.  But not all of it.  If the data contains an escape sequence, which is a known sequence of data, usually indicated by some leading unprintable character, then XTerm does not display it.  Not directly at least, because it must first decode what is being sent, and act accordingly.  There are hundreds of different XTerm escape sequences, but just a few of them handle color.

It can help to think of HTML as an analogy.  To display a word in boldface, you might use this HTML:

    regular <b>bold</b> regular

Here the word ‘bold’ is surrounded by a Bold tag.  XTerm color works in a similar fashion, but is a lot less readable.

Typically a program such as Taskwarrior will look at the $TERM environment variable, use that value to look up the supported escape sequences in termcap for the given terminal, and use only the supported subset.

Support For Different Terminals

Does Taskwarrior look at $TERM?  No.  Does Taskwarrior use termcap at all?  No.  Does it use NCurses to manage all this?  No.  It simply uses Xterm escape sequences, ignores other terminal types, and so far (going on seven years now) no one has asked for any other terminal support.  There just doesn’t appear to be a need.

How Are Escape Sequences Used?

Here is an example of the escape sequence for the foreground color red, used directly in the terminal:

The echo command simply sends the data to the terminal, and the first example contains only the word ‘Hello’.  The second example inserts the escape sequence ^[[31m to turn on the red foreground, and ^[[0m to turn off all color.  In order to enter the second example “red” sequence in the terminal, using Bash, type the following keys in sequence:

    Ctrl-V   <Escape>    [   3   1   m

To be specific, type Ctrl-V (both ‘Control’ and ‘V’ at the same time), then hit the ‘Escape’ key, then ‘[‘, then ‘3’, then ‘1’ then ‘m’.  Note that when you enter a literal escape character (with Ctrl-V, Escape) it shows up as ^[.  This is expected, as escape sequences do strange things to your terminal.  Also note that if you do this, and mess up the terminal (it’s possible), all you need to do is close and reopen the terminal window.

It should be noted that the second example echoes 14 characters, but only 5 are displayed.  The others are interpreted as color codes and consumed by the terminal.  The important aspect of this is the ’31’, which is the code for foreground color red.

There are also escape sequences for bold, underline, inverse.  The special escape sequence ^[[0m resets all the color, so that makes it possible to combined colored and non-colored text, like this:

Here the ‘4’ means underline.  In addition to foreground color, there is a background color, and these may be combined like this:

See how the ’31’ (foreground red) is combined (using ‘;’) with ’44’ (background blue) to specify both foreground and background color.  Using what we have seen so far, colorful “graphics” (they are not really graphics, just colored text cells), can be produced, for example this text:

Does not display any text, just spaces on colored backgrounds ’44’ (blue) and ’47’ (white), and it looks like this:

Taking this further, it becomes possible to construct complex sequences of color codes to produce, for example, a monthly burndown chart for the Taskwarrior 2.3.0 project:

All this is simply text, spaces and escape sequences.  Here is another example of the Taskwarrior calendar, using a purple theme, combining text and escape sequences:

The Taskwarrior color model has three responsibilities, which are:

[1] Naming Convention

This is simply a way to map the XTerm escape sequences into some more readable and usable set of names for colors.  There are four ways to specify colors, the first is by simple name:

red, blue, black, green, cyan, magenta, yellow, white

These are the simple 8-color names.  Then there is an RGB model:

rgb000 – rgb555

This ranges from black (rgb000) to white (rgb555) through a color cube, where the red, green and blue components each have six possible values (0, 1, 2, 3, 4, 5).  The model maps these coordinates to the right escape sequence, which, as one would hope, follows an orderly convention.

There there are the 24 grey colors:

gray0 – gray23

These follow a monochrome scale, and are used in the demo “task logo” command (go ahead, try it).

Finally there is a direct specification of all 256 colors using:

color0 – color255

This sequences ranges from the 8 standard colors, the brighter equivalents, through the color cube, to the gray-scale.

[2] Composition

The color model provides a small language to allow composition.  For example, there is the trivial case of just specifying a foreground color:

    red

But to distinguish foreground from background, the syntax used the word “on”:

    red on blue

This can include other attributes, such as:

    bold red underline on bright blue

Any of the naming conventions can be used here.

[3] Blending

This is really a stretch, given the simplistic nature of the terminal color, but there is support for combined two colors into one.  It is rudimentary and can only blend easily combined attributes, such as foreground with background, color with underline.  Future improvements will allow combinations like red + blue –> magenta.

Here are all the color combinations supported by Taskwarrior, as seen in the command “task color”:

You might ask “Why does ‘on bright green’ look like two kinds of grey?”.  This is because the terminal used to generate these screenshots is using a modified Solarized theme, which remapped color8 – color14.

Advanced Topics

The standard 8 colors cannot be combined with the 256-color equivalents.  When this is attempted (red on gray15), the red is upgraded to color1.  Similarly, ‘bold red’ is upgraded to color9.  Similarly, ‘on bright red’ is upgraded into the 256-color system.  This is handled automatically.

There is a missing advanced feature – optimization.  Sometimes it is possible to examine colorized text and remove some redundant color codes.  Consider this pseudo-HTML:

    <red>one</red><bold-red>two</bold-red><red>three</red>

This could be reduced to:

    <red>one<bold>two</bold>three</red>

Although this optimization is not implemented, the redundancy does exist, and should be addressed, because it can greatly reduce the I/O of Taskwarrior.

Codebase 101 – Grid View

Most Taskwarrior data is displayed in a rectangular grid, with rows and columns, header, footer and messages. Displaying data in that grid is the result of some complex text formatting. This formatting will be described in detail, but first a simple example to illustrate the issues at hand:

Shown are two columns of data, the first is a single word, the second shows multiple words. This is a typical layout problem, although a simple one.

Taskwarrior implements two similar view objects, ‘ViewTask’ which displays task data, and ‘ViewText’ which is a more general grid of strings. The ‘ViewTask’ object will be described here, but the ideas apply to both.

It should be pointed out that only ViewText stores a copy of the data. ViewTask does not need a data copy, as the task database is readily available, and there is a large performance cost to copying that data, which is best avoided.

What is a row?

A row in a report corresponds directly to a single task, which is itself a set of attributes (description, project, priority …), arranged horizontally in the output. A row may occupy one or more lines of text, depending on how the data is formatted.

What is a column?

A column is a task attribute, formatted in a specific way. There are many different column formats supported, as an example, the ‘priority’ attribute may be shown in ‘long’ or ‘short’ form:

    priority.long Pending
    priority.short P

(See ‘task columns’ for a full list, with examples).

Custom Reports

A custom report contains two major elements: the column list and the filter. The column list determines the quantity, type, formatting and ordering of the columns, and this indirectly determines the width of the report.

The filter is an algebraic expression that is applied to the set of all tasks, which reduces the set to only those matching the specified criteria, and determines the length, or rows of the report.

But Does It Fit?

Before formatting output, Taskwarrior determines the dimensions of the terminal window, in characters. All output must fit within the width of the terminal. The height is a different matter, because of vertical scrolling. Note that it is possible to control the height of a report using something like ‘limit:page’ or ‘limit:5’ to restrict the number of rows of data shown.

Once the terminal width is known, it must be determined how the columns will be formatted. Each task attribute is represented by a code object, for example the description attribute is represented by ColDescription. This object, which inherits from the base Columns object, has the ability to measure the data, and render the data.

Measuring Data

In most cases this is straightforward. Using the ‘priority’ column as an example, we know that if a report uses this, then the corresponding column has two formats to support:

    priority.short (default)
    priority.long

The short format use a single-character representation (‘H’, ‘M’ or ‘L’). The long format uses words (‘High’, ‘Medium’, ‘Low’), with the longest word (‘Medium’) needing 6 characters.

Thus the ColPriority object, when asked to measure itself, will yield either a width of 1 or 6 characters, according to the format.

But there is also the column label to consider. In this case, the label is ‘Pri’, and needs 3 characters. Therefore, the final result is that the priority columns needs 3 or 6 characters. Priority is a fixed-width column.

This measurement is performed for all tasks that will be shown in the report. This means the ViewTask will know how much space each column needs, for the set of tasks it will display.

Variable Width Columns

The description column requires more complex formatting. When asked to measure itself, it determines two measurements – the ideal width and the minimum width. The ideal width is the length of the description field. This assumes infinite terminal width, with no word wrapping. Here is an example:

  |Words of varying length used.|
   1                          29

The ideal width is 29 characters. The minimum width is calculated to be the length of the longest word in the description. This corresponds to the ideal width, if only one word were displayed per line:

  |Words  |
  |of     |
  |varying|
  |length |
  |used.  |
   1     7

The minimum width is 7 characters. This means that the description shown can be comfortably rendered using a width of 7 characters or more, up to a maximum width of 29 characters. Not discussed here is hyphenation, which is a way to reduce the minimum width down to an arbitrary size, if hyphenation is used. Taskwarrior supports hyphenation.

Using Terminal Width Wisely

Fixed width columns have specific width requirements, and variable width columns are flexible. Provided the terminal width is large enough to contain the sum of the column widths, then the report can be shown.

If the terminal is wider than the minimum needed, the variable-width columns in the report can all benefit from the extra space. The excess width is then distributed among the variable-width columns, until they reach their ideal width, or there is no more space.

If the terminal width exceeds the sum of all the ideal widths, then the whole terminal width is not even used, because it would have no effect.

Then It Gets Complicated

Taskwarrior uses UTF8 only. This has an impact on ViewTask, because some Unicode characters occupy two character cells.

The width usage of a character is determined by using Markus Kuhn’s free implementation of IEEE Std 1002.1-2001:

    http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c

Further complications arise when color is considered. The color escape sequences have zero display width, but occupy space in the text. Great care must be taken when determining the length of text; there are important differences between character count, display character count, and string length.

Other ViewTask features complicate matters further. There is the notion of a blank character in between each displayed column (rc.column.padding=3):

Finally there is report indentation (rc.indent.report=8):

While there are always improvements that can be made, the ViewTask grid meets all the immediate needs for formatting.

A Personal Task Server

This article will explain how to set up and sync with your own task server.

Why Do I Need A Task Server?

You may want to sync tasks between several machines, or you might anticipate doing that. You may also want to have a backup copy of your tasks.

Why Do I Need A Personal Task Server?

Setting up your own Task Server means having complete control and privacy.  There will be secure hosted server accounts soon, but for now you will need to set up your own server.

First Things First

This is new territory for Taskwarrior, which means using beta software, and that means we’re going to make a backup as step 1. Here is a very easy way to backup your data:

$ cd ~/.task
$ tar czf task-backup-$(date +'%Y%m%d').tar.gz *

Now put that file somewhere for safe keeping, because beta software contains bugs. The truth is that all software contains bugs, so if you aren’t making a backup of all your data, then, well, why not?

Satisfy the Dependencies

Before building the software, you will need to satisfy the dependencies by installing the following:

• GnuTLS
• libuuid
• libreadline
• cmake
• make
• A C++ Compiler

Note that some OSes (Darwin, FreeBSD …) include libuuid functionality in libc.

Build the Code

The next step is to obtain the code and build it. This means getting the Taskwarrior 2.3.0 and Task Server 1.0.0 beta2 tarballs. Here are the steps, and notice that the code is built in the home directory. Naturally you can build the code anywhere you wish, but the later in this article we will be referring to this location. Adjust accordingly.

$ cd
$ curl -O http://taskwarrior.org/download/taskd-1.0.0.beta2.tar.gz
$ tar xzf taskd-1.0.0.beta2.tar.gz
$ cd taskd-1.0.0.beta2
$ cmake .
$ make
$ sudo make install
$ cd
$ curl -O http://taskwarrior.org/download/task-2.3.0.beta2.tar.gz
$ tar xzf task-2.3.0.beta2.tar.gz
$ cd task-2.3.0.beta2
$ cmake .
$ make
$ sudo make install

Note that you may choose to run the server on one machine, and the client on another. Bear this in mind as you obtain and build the code.

Server Setup

On the server machine, a location for the data must be chosen and created. The TASKDDATA environment variable will be used to indicate that location, and is also used by the server itself. In fact, most server commands require either that the TASKDDATA variable is set, or the “–data …” option is fully specified. In this example, we are using ~/server_data.

$ export TASKDDATA=~/server_data
$ mkdir -p $TASKDDATA

Everything the server does will be confined to that directory. Now we let the server initialize that directory:

$ taskd init
You must specify the 'server' variable, for example:
taskd config server localhost:53589
Created ~/server_data/config

Next we create an organization (‘Public’) and a user (‘Your Name’ is used here). Substitute your name in the following commands:

$ taskd add org Public
Created organization 'Public'
$ taskd add user 'Public' 'Your Name'
New user key: cf31f287-ee9e-43a8-843e-e8bbd5de4294
Created user 'Your Name' for organization 'Public'

The organization is a grouping of users, and will become essential in later server releases.

Note that you will get a different ‘New user key’, than shown here, and you will need to retain that, to be used later. Note that the key is just a unique id, because your name alone is not necessarily unique.

Now we create certificates and keys. The command below will generate all the certs and keys for the server, but this includes self-signed certificates, and this is not recommended for production use. This is for personal use, and this may be acceptable for you, but if not, you will need to purchase a proper certificate and key, backed by a proper certificate authority.

$ cd ~/taskd-1.0.0.beta2/pki
$ ./generate
...

$ cp client.cert.pem $TASKDDATA
$ cp client.key.pem $TASKDDATA
$ cp server.cert.pem $TASKDDATA
$ cp server.key.pem $TASKDDATA
$ cp server.crl.pem $TASKDDATA
$ cp ca.cert.pem $TASKDDATA

$ taskd config --force client.cert $ROOT/client.cert.pem
$ taskd config --force client.key $ROOT/client.key.pem
$ taskd config --force server.cert $ROOT/server.cert.pem
$ taskd config --force server.key $ROOT/server.key.pem
$ taskd config --force server.crl $ROOT/server.crl.pem
$ taskd config --force ca.cert $ROOT/ca.cert.pem

Now we configure some basic details for the server. The chosen port is 53589.   Note that we allow Taskwarrior clients specifically.

$ cd $TASKDDATA/..
$ taskd config --force log $PWD/taskd.log
$ taskd config --force pid.file $PWD/taskd.pid
$ taskd config --force server localhost:53589
$ taskd config --force client.allow '^task [2-9]'

You can look at all the configuration settings:

$ taskd config

It is time now to launch the server:

$ taskdctl start

This command launched the server as a daemon process. Note that you use ‘taskdctl stop’ to terminate the server. This command requires the TASKDDATA variable. Your server is now running, and ready for syncing.

Client Setup

On your client machine it is assumed that you are already using Taskwarrior, and already have local tasks to sync. If this is not the case, setup and add one.  We need to create a cert and key for you, based on the server cert:

$ cd ~/taskd-1.0.0.beta2/pki
$ ./generate.client your_name
...

Copy the cert, key and ca to your ~/.task directory. The reason we are copying the CA cert is because this is a self-signed cert, and we need the CA to validate against. Alternately we could force Taskwarrior to trust all certs, but that is not recommended.

$ cp your_name.cert.pem ~/.task
$ cp your_name.key.pem ~/.task
$ cp ca.cert.pem ~/.task

Now we need to configure Taskwarrior to be aware of these certs:

$ task config taskd.certificate ~/.task/your_name.cert.pem
$ task config taskd.key ~/.task/your_name.key.pem
$ task config taskd.ca ~/.task/ca.cert.pem

Now set the server info and provide the credentials, which include the ID key generated when the user was added to the server:

$ task config taskd.server localhost:53589
$ task config taskd.credentials 'Public/Your Name/cf31f287-ee9e-43a8-843e-e8bbd5de4294'

Now your client is ready to sync.

First Time Sync

The first time you sync is special – the client sends all your pending tasks to the server. This is something you should only do once. Run this:

$ task sync init
Please confirm that you wish to upload all your pending tasks to the Task Server (yes/no) yes
Syncing with localhost:53589
Sync successful. 2 changes uploaded.

You should get an indication that tasks were uploaded, in this case 2 of them.

General Sync

After the first time sync, you switch and just use this command:

$ task sync
Syncing with localhost:53589
Sync successful. No changes.

This will give you feedback about what happened. Please note that it is perfectly safe to run this command as often as you wish. Syncing is safe and does not consume great system resources.

But it does require network connectivity, and if there is no connectivity you will be notified. It is not a problem if a sync fails because of this, because the next sync that works will catch up with all the changes, and do the right merging.

The general usage pattern is that after you modify data locally, Taskwarrior will start notifying you that you need to sync, after commands, like this:

$ task project:foo list
No matches.
There are local changes. Sync required.

This is just a reminder to sync. Respond with a sync, and the reminder goes away:

$ task sync
Syncing with <server>:<port>
Sync successful. 1 changes uploaded.

Please remember this is beta software, and if you find bugs or have unanswered questions, please contact us at support@taskwarrior.org.

For the latest version of this document, with corrections and added info see: http://taskwarrior.org/projects/taskwarrior/wiki/Server_setup

Document Object Model – DOM

Any program that manages data has a responsibility to make that data accessible. Furthermore, any open, extensible system needs to make that data readily accessible to third party scripts and programs in different ways.

There is a entire zoo of different species of scripts out there that need to be supported, and the future form of these animals is unknown:

http://taskwarrior.org/projects/taskwarrior/wiki/ExternalScripts

Will the next script be in Bash or Python? Will it consume data? Will it produce data? Does it need access to an individual datum? Will it maintain an open connection to the data or simply make one change?

For most purposes, the ability to import and export tasks in JSON format is the most complete method. All data can be accessed, but this requires parsing JSON for the whole task, and it is possible that a script needs only to access, for example, the task description.

Document Object Model

This is where a Document Object Model (DOM) can help. A DOM provides an addressing mechanism to reference individual data items. While Taskwarrior has supported DOM addressing for a while now, it is only with the upcoming 2.3.0 release that is has become more complete and useful for extensions.

In its simplest form, the DOM provides references to attributes in a task. For example, suppose you were adding a new task:

    $ task add Take out the trash due:friday wait:friday

This is a task that has a due date, but remains hidden until that due date. See how ‘friday’ is duplicated in that command? The DOM can be used to just reference that due date instead:

    $ task add Take out the trash due:friday wait:due

That ‘due’ is a DOM reference, and it means ‘the due date of the task in hand’. This is not much help now, but one arbitrary date math is supported (perhaps in 2.4.0), it will make the following possible:

    $ task add Mothers Day due:2014-05-11 wait:due-1wk

Furthermore, this combination of DOM reference and date math will allow default wait rules to be implemented.  A more complex DOM example is when data from a specific task is referenced:

    $ task add Buy the ingredients due:21st
    Task 123 added
    $ task add Bake the cake due:123.due

This example shows a reference ‘123.due’ being used when creating task 124. DOM references can also refer to completed and deleted tasks, using the UUID, which means DOM addresses may be one of the following forms:

    <attribute>
    <id>.<attribute>
    <uuid>.<attribute>

Additionally, configuration settings can be accessed:

    rc.<name>

And finally a few program values:

    system.os
    system.version
    context.width
    context.height
    context.program
    context.args

DOM addresses can be accessed using a helper command, new in 2.3.0, named ‘_get’
which can be used to evaluate DOM addresses:

    $ task _get system.version
    2.3.0.beta1

    $ task _get 213.project
    tw.230

    $ task _get 208.tags 208.urgency
    bug 4.838356

    $ task _get 1.uuid
    8ad2e3db-914d-4832-b0e6-72fa04f6e331

    $ task _get 8ad2e3db-914d-4832-b0e6-72fa04f6e331.description
    Support listing breaks due, hour, day, week, month

    $ task _get $(task _get 1.uuid).description
    Support listing breaks due, hour, day, week, month

Note that when querying configuration values, you need to distinguish between the DOM address and a runtime override, by shutting off parsing like this:

    $ task _get -- rc.dateformat YYYY-MM-DD

The DOM is useful for both third-party scripts and user command-line use.  New features will make more use of DOM references, and you’ll start to see them being used in example documentation.

Virtual Tags

Virtual Tags

The tag syntax is very powerful and simple to use.  There are two ways to use this, shown here:

These two commands illustrate the complete tag interface. The first command is a filter that lists only tasks that have the HOME tag. The second command is a filter that lists only tasks that _do not_ have the WORK tag. The + and – syntax means presence and absence of a tag. This is simple to use, and can be combined like this:

This shows tasks that have the HOME tag, but do not have the WORK tag. This is a very simple and easy to use mechanism, but it does require that your tasks are properly tagged. In other words, it is based directly on task metadata.

Complex Filters

Some Taskwarrior filters are simple in concept, but the syntax is not that straightforward.  For example, to determine whether a task has a due date that falls on the current day, you need to use this filter:

This filters tasks with a due date during the narrow time window of ‘today’. Note that is is not sufficient to just specify the date, because due dates all have associated times (defaulting to 0:00), and if you want to match the date, you need to consider the time. So for example, this command _does not_ list tasks due today:

Instead, this filter matches tasks with a due date of today, and a time of 0:00.  In order to see all tasks due today, you need to provide proper range bracketing.

Simplification

Here is where virtual tags can help, by providing a simple tag interface to more complex state conditions of the task.  There is a virtual tag, named TODAY that can be used in filters, and it means that instead of this filter:

We can now use this:

Which is a much simpler way of filtering tasks due today. Because this is a tag interface, we can also invert it:

This shows only tasks that are not due today.

Virtual tags are built in to Taskwarrior. They are evaluated at run time, which means they do not require direct metadata, and therefore do not occupy space in the data files, but are determined according to the state of the task in the same way that the complex filter example above is determined.

Thus virtual tags combine the ease of use of the tag interface with more complex defined conditions, for convenience.

Supported Virtual Tags

Since version 2.2.0, Taskwarrior has supported virtual tags, and the list will continue to grow.  Here are the supported virtual tags as of version 2.3.0:

• BLOCKED – Is the task dependent on another incomplete task?
• UNBLOCKED – The opposite of BLOCKED, for convenience. Note +BLOCKED == -UNBLOCKED and vice versa.
• BLOCKING – Does another task depend on this incomplete task?
• DUE – Is this task due within 7 days? Determined by rc.due
• DUETODAY – Is this task due sometime today?
• TODAY – Is this task due sometime today?
• OVERDUE – Is this task past it’s due date?
• WEEK – Is this task due this week?
• MONTH – Is this task due this month?
• YEAR – Is this task due this year?
• ACTIVE – Is the task active, ie does it have a start date?
• SCHEDULED – Is the task scheduled, ie does it have a scheduled date?
• PARENT – Is the task a hidden parent recurring task?
• CHILD – Is the task a recurring child task?
• UNTIL – Does the task expire, ie does it have an until date?
• WAITING – Is the task hidden, ie does it have a wait date?
• ANNOTATED – Does the task have any annotations?

Perhaps you have a suggestion for more virtual tags?  Please let us know!

See http://taskwarrior.org/projects/taskwarrior/wiki/Feature_virtual_tags for the full list that will be updated over time.

The Multiple Client Challenge

There is an issue that we need to very careful about.  We need to make sure different task server clients don’t corrupt data through the selective implementation of features.

Each client of the task server will implement different functionality.  Even if feature parity is attempted, one will likely be released ahead of the other, so essentially there will always be the possibility of different functionality between clients.

But the data is shared.  That’s the tricky part.  Fortunately this can be solved with a rule.  The rule is this:

A client MUST preserve intact all unrecognized task attributes, for all I/O operations.

That’s a fairly simple rule, but one that is essential to make different clients work together.  Let’s look at an example.

Suppose there is a client that doesn’t support tags.  By not supporting tags, it would not show tags, or allow them to be added, removed, or modified.  But as a client to the task server, tags are sure to be found in the data.  By obeying the rule, any task that does have tags will not lose those tags when handled by the client.

To look at this at a lower level, the format of the tasks to/from the server is JSON, and the rule says that the JSON structure and contents must be preserved, and only recognized constructs may be manipulated.

This may sound restrictive, but is in fact very permissive.  It allows for clients that are incredibly simple.  An example of this might be a Top Ten task app that only supports a completion action on the ten most urgent tasks, and furthermore only displays the descriptions.  Such a simple app, by obeying the rule, and implementing only one feature, remains compliant, and does not harm user data, even though it doesn’t understand all the data.

This opens the door for lightweight clients.  Small apps that might simply capture new tasks, or an app that just spell-checks task descriptions.  Not every client of the task server needs a UI…

Patterns and Back-References

The original intention of aliases was to allow alternate command names.  An alias is created like this:

But as so often happens, a feature is released, and some clever person finds a way to use it that was not intended.  The ability to alias one word to an arbitrary string was requested.  That led to this:

Now that’s a nice feature, and can be thought of as a form of macro.  But taskwarrior supports regular expressions, so why not allow a pattern for the alias name?  It seems like an unnecessary feature.  But suppose I wanted to add a reference to a bug.  I might want to type this:

And have that automatically expanded, via alias, to this:

You might think that because all that needs to happen is for ‘BUG’ to be replaced by http://taskwarrior.org/issues and you’d be right, but BUG1299 is the whole word, and aliases only work on whole words.

Instead, if the alias allowed a pattern, like this:

Now this is not a supported feature, and this command would not work, but you can see that the alias itself is a pattern.  Furthermore, that pattern captures the bug number, and in the expanded value of the alias, there is a \\1 back-reference.

Thanks to Vivenzio and Ben for the ideas of back references and enhanced aliases.  It will be a nice addition.

Flod Testing

 What is Flod?

Continuous Integration is a best practice, and Flod is a tool that manages that for us.  While there are many such tools available (and they are very good), this one is incredibly simple.  We like simple products.

There is a development repository, and commits there trigger the need for test builds, recorded by a central server, and serviced by a satellite node.  As a developer, you don’t notice this happening, because the software runs (locally or remotely) as a daemon, so it’s all very much behind the scenes.

Why do we need Flod?

There were 27 or so releases of Taskwarrior before we started using Flod, so this is not a small data sample.  Of those, more than half were bug fix releases.  That is to say releases we were obligated to make, to improve stability and quality, rather than releases that were a balance of minor bug fixes and new features.  Compounding that, Taskwarrior runs on multiple platforms, and in the past that meant there was a lot of last-minute “does it work on this platform?” experimentation.

Since using Flod for the last year, we have not needed to make any emergency releases.  It’s still a little early to tell, but it appears to be working well for us.

What are the benefits?

The main benefit is instant feedback about test failures across multiple platforms, allowing us to respond quickly.  This means that the quality of the product is higher during early stage development, and the testing is a good indicator towards the end of the cycle of whether the release is ready.  We have higher confidence of the code quality on platforms that we do not use day to day.

Naturally, this depends on other factors, such as having good test coverage, but that’s another discussion.

The setup we are using looks like this:

While that diagram shows three satellites (and therefore three platforms tested), we have about 10 such platforms that run tests as soon as they are able.  These satellites are running on a variety of machines and VMs.  For example Amazon AWS, Rackspace, Linode, personal machines, VirtualBox and VMWare instances are all running satellites.

Not all of the above are available all the time.  The VMs in particular, can spend weeks in an “off” or hibernated state, and when awoken, they process the backlog and go back to sleep.  This is common with distributed systems where connectivity is not only not guaranteed, but can be very sporadic.

Here is an example of a live test report, covering several platforms, for the development branch of Taskwarrior.

How is the community participating?

We have been very lucky, with several contributors running Flod satellites.  Bryan is one example (see taskwarrior-2.3.0) providing an Arch 64-bit platform for testing.  This satellite connects daily and runs a limited number of tests to minimize impact, and the result is invaluable – we know that Taskwarrior 2.3.0 runs properly on Arch.

Where is it going?

Flod will be generalized to be able to handle more revision control systems.  The current Git support will be extended to include at least Mercurial.  In addition to CMake, we would like to support Autoconf.

It might also be wise to include other testing setups.  Currently TAP is the only supported model, but TAP13 specifically is on the list, which would make for richer reports.

Is it available?

Yes and no.  While the code is open source, and freely available (Flod), it is not generalized yet, so will only work on the kind of projects we operate (git, cmake, make, TAP).  So while Flod is great for us, it needs to mature before it is useful to others.  But we still wanted to tell you about it, because it is doing a good job of making Taskwarrior more reliable.