Improving todo.txt & Todoist plugin

The GSoC coding period just ended. I would first like to apologize for not updating about my work. I am working on improving Todo.txt and Todoist integration to GNOME To Do. During the coding period, a lot of improvements were added to Todo.txt and Todoist and in this blog post I write about my journey and describing the implementation details.

Todo.txt & Todoist Updates

For those not familiar with todo.txt, it is a text-based format for storing and Todoist is quite a popular task manager application that can be used on smartphones as well as Personal Computers.

My aim for the Todo.txt was to improve the current code, make the plugin feature similar to To Do and document the syntax of todo.txt. I was able to complete all the task.

  1. The first task was to implement the support for notes in the todo.txt plugin. Notes are basically a more lengthy description of the task. Since todo.txt format allows adding custom key: value pairs for task description, I introduced a new key “note” and the value follows the key in quotes “”.  The only major challenge was handling the quotes and special characters like newlines, quotes etc in the description. This was done storing the notes in escaped form (i.e adding an extra ‘/’ before the special character and removing it during parsing phase.
  2. Adding support for creation and completion date –  The completion date occurs just after priority and creation date occurs after the completion date as per the todo.txt format. i.e x (priority) completion-date creation-date. Parsing completion-date & creation was easy using the last seen token. I had to subclass GtdTask into GtdTodoTxtTask since we need support for setting creation date. As the task are stored as text, the creation date needs to be cached and set at first load. This was done by subclassing GtdTask and adding a setter for the completion date.
  3. Adding support for list background – I noticed that list background color was not being cached into todo.txt and hence the color was lost on exit and start of To Do. After discussion with my mentor we decided to add two hidden custom lines in todo.txt, here hidden just means a marker “h:1” at the start of the line in todo.txt and are used to denote ToDo specific features. 1 custom line for storing list background color and other for storing Lists. Below is an example. For this task, I had to change the parser a bit. It was slightly confusing at first because I didn’t have a clear understanding of function pointers (needed for parser vtable implementation) but feaneron helped me with it.
      h:1 Lists @test @test2
      h:1 Colors test:#729fcf test2:#5c3566
  4. And now the most important task was to implement the subtask feature. The subtask was supported in the initial stage of the plugin but was removed because of instability. It wasn’t very clear to me as to how to go about but eventually, we decided to use indentation to denote parent-child relation between task described in immediate lines. An indentation of 4 spaces means the task is a child of the previous task. The algorithm works by creating all the task and maintaining it in a hastable with key as list of tasks. Once all the tasks are created the subtask relation is determine using the indentation property.
  5. Finally I documented the syntax of todo.txt so that new user find it easier to use the plugin. The documentation is present here.

With these changes todo.txt is much more stable and has all the feature that ToDo supports. Todoist had 3 major work to be done, fixing the network issue, auto sync of tasks and removing GOA and making Todoist plugin handle accounts on it’s own.

  • The network connectivity issue was handled by adding a network monitor and listening the network changes. Incase To Do cannot connect to the Todoist the provider is removed and hence no data loss happens due to user changes. The below screenshot show that wifi connectivity loss/gain is now handled by Todoist.
  • Peek 2018-08-06 23-39
  • The autosync patch is not merged yet but is very close to being merged in master. To reduce the number sync request with Todoist we manage time intervals at which to sync based on user window focus. When user focusses on Todoist for first time, sync request is generated immediately and timeout is set to 30 seconds. But if the focus is not on ToDo, the timeout is set to 10 minutes.
  • The final work is to making Todoist plugin handle it’s own accounts by using Keyring to store tokens. This work is still in progress and unfortunately I wasn’t able to get it merged. I have added patches for changes to preferances panel and implementation of keyring but still to integrate both these changes and also make modifications to plugins and providers. I will continue working on this and finish the implementation.

Finally, I would really like to thank my mentor Feaneron for guiding me and helping me whenever I was stuck, reviewing my code which was filled with code style errors and being patient enough to keep reminding me about my mistakes. I would also like to thank GNOME for giving me this great opportunity.

Advertisements

Working on GNOME To Do this Summer

A brief intro about myself
I am Rohit Kaushik (kaushik on IRC) from Delhi, India. I am currently pursuing B.E Computer Science at BITS Pilani, Goa. I am interested in Software Engineering, Machine Learning and Research. I usually spend my free time playing badminton, cricket or listening to music.
Last year, I worked on implementing Todoist for GNOME To Do and this time again I will be working on GNOME To Do, improving the two plugins that I wrote earlier and implementing newer features. I am grateful to GNOME and my mentor feaneron for giving me this opportunity.

About the Project

The project is aimed at improving todo.txt and Todoist error-free and support all the features of To Do. At the end of the project, we wish to have both the plugins in a usable and bug-free state. I am outlining the current list of task that I have planned to work on but any other suggestions are very much welcomed 🙂

Improvements to todo.txt plugin

  • Improve the parser and make it usable and bug-free
  • Add support for missing features, i.e list background colour, notes, subtask support
  • Handle edge cases in syntax parsing
  • Document the todo.txt syntax

Improvements to Todoist plugin

  • Move it outside GNOME Online Accounts – Currently Todoist uses GOA, for maintaining Todoist accounts and we wish to remove this dependency and have a login panel in To Do which manages user accounts.
  • Automatic synchronization of tasks and lists
  • Handling network loss and other failing cases.

I will keep this blog updated with the work in progress. If you have suggestion, please feel free to add them in comments or send me a mail.

Looking forward to a great Summer.

Cheers!

GSoC: Final Report

The GSoC coding period has come to an end and I am wrapping up my work ,writing reports and descriptions for my project. I still have to fill my Google evaluation and submit the links. This blog is a final status report of my work and possible further improvements.

Status of Todoist Integration

I was able to complete most of my goals , except one related to compressing commands in a single HTTP post request. Two of my current patches are yet to be pushed, so I will be working on implementing command compression and the review’s to my un-pushed patches. The good thing is non of not-pushed patches blocks the usability of Todoist plugin and the plugin can be used to manage Todoist tasks and users account. So, I encourage all those who are using Todoist to try out GNOME To Do from master branch. And rest assured, I will work to my best so that the remaining work and patches are also pushed to master before the 3.26 release.

I have also updated my wiki page , which now includes a detailed overview of the project and links to merged/unmerged commits.

Thank You

I would like to thank every GNOME member, especially my mentor, feaneron, for being so nice and helpful. I have learnt a lot in the past 3 months and it has been a great experience to be a part of this wonderful community. I also like to thank GNOME and Google for giving me this opportunity.

Not the End

Although GSoC has ended but this is not the end of my contributions to GNOME. I would keep contributing to GNOME and try to make it even more awesome than it already is.

Cheers!

GUADEC 2017

I attended my first GUADEC this year, held in beautiful city of Manchester. I am in general not very enthusiastic about social meetings but GUADEC was something I always wished to attend and I am glad I did.

The Conference Days:

The first day started with talks about GNOME and different applications and ended with lightning talks from interns and newcomers. I was a bit nervous about presenting in front of so many people, mainly because I am not very good with presentations but everyone was extremely nice and  encouraged me before the talk and this made it much more easier task. I am glad I had this opportunity and I hope that I improve with my presentations from here on.

I liked hearing people talk about GNOME softwares and being a newcomer it was nice way to get to know more about the projects. I attended most of the talks and liked all of them but I would like to specially mention some of them:

  1. Status of Builder – I use builder for all my development work hence I really appreciate the work of Christian and I loved the demonstration.
  2. The History of GNOME – I got to know so many knew things and how GNOME progressed as a project.
  3. How to get better milage out of Glade – I am glad that there is work going on to improve Glade, since this is one application I would love to use more.

The rest of conference was meeting new people and talking about things related to my project or GNOME and even Non GNOME stuffs in general. I also had chance to meet my mentor feaneron and personally thank him for all the things he has helped me with and helping me become a contributor from a complete newbie.

Unconference Days: 

So Unconference Days was scheduled with workshops at the start of day and ended with social event. I tried to utilize the time to work on my GSoC project and get some reviews and help from my mentor. I also attended the later half of the Builder workshop and liked it. Ah! I should have attended more Workshops/BOFs.

Trek and City Tour:

City Tour was cancelled on Friday due to rain but thankfully that didn’t happen on Sunday. Pic from the tour. More can be found here .IMG_20170730_193530

I went on a trek for the first time in my life and it was a really nice experience. I would like to thank the team for planning the Trek and also our Group Leader Allan Day.

DSCF4564

Acknowledgement

I would like to thank GNOME Foundation for sponsoring me. I also thank the team for planning everything and working so hard to make the conference a success and memorable 🙂 Thank You GNOME!

sponsored-badge-simple

Update: Todoist Integration

In the last blog post, I gave a brief idea as to how Todoist plugin is structured to work and I talked about my work related to loading Todoist tasks inside GNOME To Do. After modifying the previous work and making some improvements, the code has finally been pushed to master. I also worked on implementing functions that will save any changes made to Todoist tasks or lists. The Todoist Api allows applications to send an HTTP Post requests for adding/deleting tasks/lists. To Do makes uses of the Todoist Api v7 and sends HTTP response for fetching and storing the changes.

static void
gtd_provider_todoist_remove_task (GtdProvider *provider,
                                  GtdTask *task)
{
 GtdProviderTodoist *self;
 JsonObject *params;
 g_autofree gchar *command;
 g_autofree gchar *command_uuid;

 self = GTD_PROVIDER_TODOIST (provider);
 command = command_uuid = NULL;

if (!self->access_token)
 {
  emit_access_token_error ();
  return;
 }

 params = json_object_new ();

 command_uuid = g_uuid_string_random ();
 command = g_strdup_printf ("[{\"type\": \"item_delete\", \"uuid\": \"%s\", "
                            "\"args\": {\"ids\": [%s]}}]",
                            command_uuid,
                            gtd_object_get_uid (GTD_OBJECT (task)));

 json_object_set_string_member (params, "token", self->access_token);
 json_object_set_string_member (params, "commands", command);

 post (params, (RestProxyCallAsyncCallback) post_generic_cb, self);
}

The above snippet is to remove a task-list. Other function that work as editing the changes to Todoist, have similar implementation. They send param such as “token” and “command” and make use of post method to send an HTTP Post request. The post method use librest to make a call to Todoist sync endpoint.

What has been completed?

  1. GNOME To Do can load Todoist tasks and lists. So now, users can view their Todoist tasks inside To Do.
  2. Not just view, but one can also modify the tasks and lists and these changes will get saved to user Todoist account.
  3. Deleting tasks and list is also possible now. Although, deleting list won’t work because of a bug in GOA which doesn’t provide correct access rights to Todoist Account.

What is to be done?

  1. We cannot create a todoist task or list from GNOME To Do currently. This will be covered by next week.
  2. To Do cannot sync tasks and lists from Todoist automatically. This is currently in progress and hopefully will be completed by next 2 weeks.
  3. After completing the above tasks, I will work on limiting the number of post requests made by To Do so that we do not exceed the Todoist limit of 50 per minute.

A small video demonstrating the above Changes:

Do checkout the work and feel free to open bugs on bugzilla if you encounter them . Any suggestion for improvement is most welcomed. Don’t forget to drop a mail if you have Todoist specific features suggestions i.e Productivity Graph that can be implemented in GNOME To Do.

GSOC : Post Phase 1 Update

The first phase of my GSoC project has completed. I received an evaluation on my work from my mentor, feaneron. I am happy that the project is moving well. I hope to keep up with the work and continue this awesome journey and learn in the process.

So, let me come back to the project update. We have come a long way and I am happy to tell everyone that To Do can now load Todoist task lists and tasks. Although this work is still in progress and patch related to this change will be pushed probably by this week. So everyone waiting to use Todoist with To Do keep a watch on To Do repo for the update.

The major problem that I faced was synchronizing the tasks and task lists so that they are updated automatically. After discussion with my mentor, we have decided to sync tasks every hour, although this may change. I wanted the sync calls and functions updating the changes to be separate so that there is less friction between the code.

The current flow of Todoist plugin is:

Todoist Plugin WorkFlow (1).jpg

All the phases shown in the flowchart, except the last part which is synchronizing the tasks, has been completed. The major problem with task synchronization is code repetition due to addition of update functions. After some thinking I decided to make the function that initially loads tasks to be good enough so that it can be also be used to update tasks. To achieve this I had to add a check if lists/tasks already exists using the id. So if suppose the list already exists, I just fetch the list from Hash Table using the id as key and make changes to it’s properties. This way the list gets updated and not created again if it already exists. The problem that still exists is with the “list-changed” signal which is not behaving as thought and I need to fix this.

I have a short video demonstration for the work in progress.

Feel free to drop suggestions and give your feedback.

Todoist Preferences Panel for To Do

A lot has happened in the first two weeks of coding period and i am excited to share it with everyone. So, for the first two weekend I worked on adding a Preferences Panel that allows users to add/modify Todoist account that they want to use with To Do. Earlier we had planned to make it work in sandboxed environment, but upon mclasen’s suggestion we are sticking with getting the Goa Portal and Todoist account addition to work on non-sandboxed environment first.

So here’s how the Todoist Preferences Panel has turned out and it’s looking really cool :

out.gif

Since Todoist provider is not yet merged with GOA, we don’t have a todoist account to test with, in which case To Do shows a no accounts page and asks if you wish to add one. Mind you the add button will be invisible if there is todoist provider in your GOA but i have enabled it just for demo.

Incase you have a Todoist Account in GOA added, your preferences panel would this along with todoist accounts in the ListBox. I need to make some changes to preferences including the icon which would done in a later stage.

Screenshot from 2017-06-16 18-41-32

I have also progressed on adding some required functions to GOA and the patches have undergone a few review iterations and we can can expect them to be pushed to master before end of next week.

What additions are in line for GOA?

  1. ListProviders: We can now make a DBus call to GOA and get a list of supported providers. This can be useful if apps want to query if GOA supports a specific provider. To Do will make use of it to decide whether to show a Add Account button or not.
  2. AddAccountForProviders: This is yet another DBus method that takes in a string “provider_type” and spawn the Control Center to allow addition of an account, which apps can make use of.
  3. Todoist Provider: This is being added by Recipe members and with this GOA will support Todoist accounts which To Do, Recipe etc. can use.

All the patches related to GOA are in progress and not pushed to master branch, if you wish to check these out, patches have been uploaded here.

A short video demonstrating these changes :

I am using D-feet to trigger the Dbus calls. As you saw in last video, the AddAccountForProvider method is already being used by To Do.

out1

This is all I had for this blogpost, I will keep updating the changes as it occurs. Feel free to drop suggestions. They are always welcomed 🙂