Developed by Bruno Casteleiro during Google Summer of Code 2019
Mentored by Diogo Cordeiro
This is a technical Google Summer of Code Project Report describing briefly what I did from May 27 to August 19.
The whole code I wrote during this period is available here.
This document helps understanding what was done in a simple and informal way, highlighting the most relevant parts, why some decisions were made and what I've learned with it. Unless explicitly said, the order in which I'll write about my tasks may not exactly correspond to the real order by which I did the work.
GNU social is a social communication software used in federated social networks.
My work for this summer involved, in a general way, working the different network systems of GNU social. Due to an unexpected increase of the workload in some of the tasks, my plan, although still related to network systems, turned to be a lot more focused in the ActivityPub protocol, a communication standard for federated software. During my journey, other systems like the OpenID protocol or the internal url mapper were part of my work.
OpenID is an authentication protocol that allow users to use an unique identifier to login in multiple sites without needing to individual registration in each of them.
In GNU social, OpenID support is provided by a plugin and my task was to perform a revision of it.
During my revision it was necessary to update the external libraries used by the plugin. Since no code changes were required afterwards, this was a trivial update.
The OpenID user interface was also updated. Users had no option to choose whether to sync or not their GNU social accounts when adding an OpenID identifier in the settings. I've extended the interface with such sync option.
Being the first task, reviewing OpenID helped me to better understand how plugins were structured and the back/front-end code interaction.
Internally, the URLMapper class is responsible for the creation of the many routes used in GNU social. This class was known for having a few issues and, having affected some of my work during other tasks, I was responsible to fix some
of those issues in different times during the summer.
Fully reworking the Mapper was considered as an option at a certain time, however, taking into account the many routes spread all over the codebase and my work schedule, I decided that fixing the existent class in the best way possible was the safest and more rentable option I could take.
One of the first changes to the Mapper were result of needing to have two or more routes with exactly the same path to coexist, differentiated only by the accept-headers that should be set in the request.
This necessity was solved by extending the route creation function with an option array of accept-headers. When populated, the function verifies the request accept-headers and in case of any match, it stores/substitutes the new route in the appropriate variables.
In some cases, some paths that would differ from one another only in their regex attributes were being routed. Although fundamentally equal, these paths generated different regex's during route creation. Because of this and because of being stored by order of arrival, path matching was matching the wrong route in some cases.
This problem was solved by simply storing new routes at the head of the according array variable.
The Mapper distinguishes what are dynamic and static routes. Static routes are the ones with regex attributes in their paths. Because of both being stored in the same place, grouped by the action to be executed, some static routes were incorrectly being matched in the place of the dynamic ones, simply because of being found first in the storage array.
It makes sense to look all dynamic routes first, specially when these don't have the problem of wrongly match with a request for a static route. Therefore, this problem was solved by separating both types in different variables and explicitly enforcing the lookup of the dynamic routes first.
Spread all over the codebase, a great deal of the calls to the route creation function were making an improper use of the function parameters. All these calls were appropriately fixed.
Specially because of some undo/redo that these changes required, this task was important for me to be more cautious with core code changes. Verification of the surrounding code and careful analysis of the changes to be done is very important.
GNU social private messaging support is provided with the DirectMessage plugin . By requirement of the changes related to ActivityPub, this plugin end up being totally reworked by me.
The plugin suffered many changes in practically all of its code, making it hard to write about every change. Still, because of their importance, three changes are worth of an highlight spot:
GNU social already had notices, so it didn't make much sense to have a separated table just for private messages. The most important part of this port was ensuring an upgrade function and the correct handling of a newly introduced Notice MESSAGE_SCOPE scope.
Porting to the Notice table automatically enabled text mentioning, so with just a few adjustments the plugin was handling multi-recipient messages.
Of course, the whole reason for this task. Events were added in key points of the private messaging flow for the federation plugins to subscribe and handle in their own way.
Because of its big extent, this task was a test to everything I had learned so far and, therefore, completing it gave me great pleasure and some energy boost for what was to come. In particular, I've learned how to add upgrading logic to plugins.
Support for the ActivityPub protocol in GNU social is given by means of a plugin wrote by my own mentor, during last year's edition of GSoC. Since the last tests with other federated software some things changed and it was known that some interactions weren't properly working, so part of my job was to review and ensure ActivityPub could federate again. Furthermore, some functionality was yet to be implemented and other came along the way, summing up the reasons as for why this task to consume more time than the expected.
Following, the most important changes this plugin seen this summer:
The followers/following endpoints for ActivityPub users weren't retrieving the most accurate information due to accounting invalid users. This was easily fixed by porting and correctly adapting the subscriptions functions from the core to the plugin. Furthermore, I've handled the caching of these collections.
This sub-task involved ensuring the operation of all interactions supported by the plugin, both from the sender and receiver sides. These interactions are: Accept-Follow, Create-Note, Delete-Note, Follow-Person, Like, Undo-Follow, Undo-Like and Announce. In some of the cases the changes were something trivial, others require small function rewrites and bug-fixing. The interactions were tested with remote instances of GNU social, Pleroma, Mastodon and Pixelfed.
Later I've also introduced and ensured the operation of the Delete-Person activity.
The GNU social queue system was added for ActivityPub notice distributions, these were proving to be quite a time consuming experience.
At this point me and my mentor, Diogo, agreed that, despite not being supported in GNU social, we should still properly handle the different non-public types of notes that the inbox could receive, like the unlisted, followers-only or private/direct types.
Handling private notes was fairly easy since I had already reworked the DirectMessage plugin, so it was only needed to introduce a new Create-Direct-Note activity and subscribe the DirectMessage events. It is worth mentioning that this
type of note has (yet) no standard way to be differentiated. After some research me and Diogo decided that making GNU social conforming with the discussed directMessage flag was the best way to handle the activity.
The unlisted and followers-only types of note were handled by storing such notices with a special GATEWAY type flag. This flag was originally introduced to represent notices with origin in services other than the OStatus protocol. Being
that way guarantees that the notice won't show in the public timelines. For the time being, I believe this is the best option for handling such note types: 1. easy to change (if required) in the future when proper support for these types is added
and 2. provides at least the minimum requirements of visibility scope.
With this task I've learned how to use more systems of the program, like the caching and queue systems. Furthermore, I gained further knowledge in different aspects of the ActivityPub standard.
Aiming to provide better user experience and to ensure federation in a general way, this task led to a new RemoteFollow plugin. This plugin is responsible to fully control the remote-follow button that shows on user profiles and to provide
the necessary events for all federation plugins to subscribe and make this button work.
As there isn't no standard way to treat a remote-follow button ActivityPub, the common way to do it is to extend the logic of OStatusSub, the OStatus way of handling such button. That is precisely what I did. I've ported the necessary actions related
to remote-following a user from the OStatus plugin and adapted it to be event-driven.
This task was my first time working in the OStatus plugin, so I learned some details of its implementation.
I've introduced the "ActivityPub way" of fetching remote users and notices, allowing remote-following users without needing to leave the local instance.
Of course, there's always something more to do :)
DirectMessage plugin (not crucial)