XWiki Watch Developers' Guide

• XWiki Watch Developers' Guide
  • What does XWatch do and what does it not do
  • XWiki Watch data model
  • Where is the XWiki Watch code
  • Application flow

What does XWatch do and what does it not do

XWatch is:

• a tool for collaborative competitive intelligence tasks
• focused on data extracted from web sources namely RSS feeds 
• which allows the potential descentralization of the watch activity
• and facilitates the distribution of the results (via web)

XWatch is not:

• (just) a collaborative RSS reader

XWiki Watch data model

XWiki Watch allows users to register various data sources (RSS feeds) and organize them in groups for which it fetches and stores the articles, content which can be enriched by users: flagged, trashed, read, tagged, commented.

XWiki Watch stores all its data in XWiki documents and objects, as follows:

• XWiki.AggregatorURLClass -- stores the RSS feeds, which have
  • a name (unique identifier), 
  • a title (the pretty name of the feed, the one to be displayed to users), 
  • a url (the feed url), 
  • the groups a feed is in (the groups that this feed belongs to, identified by the page names, see below) 
  • other fields to mark the last load date, number of articles, etc
• XWiki.AggregatorGroupClass -- stores the groups to organize feeds in groups, which have
  • a name -- the friendly name for the group, to display to the user
• XWiki.KeywordClass -- stores a keyword to search for in Watch (will be removed in the close feature, to be replaced by a full filter saved), and contains:
  • a name -- the actual keyword
  • a group -- the group within which to perform the search
• XWiki.FeedEntryClass -- stores an article fetched from a feed, and contains:
  • the name, as fetched from the source
  • the 'parent' feed name
  • the 'parent' feed url -- this two are both stored for caching purposes, to be able to get information easier, without having to join two tables to get the feed name
  • read status
  • flag/trash status. Flag and trash are stored in this same field which acts like a 'vote' value: flag is +1 on this position, trash is -1 on this position and 0 on this position is nothing (not flagged, not trashed)
  • tags

Each article is stored in its own document. The comments for the articles are stored as real comments of the XWiki documents where the articles are stored. Article tags are not stored as XWiki tags of the documents the articles are stored in for querying speed reasons.

Most of this data is stored in separate tables using XWiki custom mapping, see platform/core/xwiki-core/src/main/resources/feeds.hbm.xml .

Where is the XWiki Watch code

XWatch is a XWiki application (.xar) that uses the feed plugin in the platform (activated by default) and the xwiki-web-gwt API in web. It is a GWT application so it contains some javascript code generated by GWT and other GWT resources (images, stylesheets) but all these are packed in a zip and attached to a document (WatchCode.GWT) and accessed with the download action and the zip explorer.

The code is in:

• the feed plugin, which reads data from the stored feeds and stores the articles in the article documents (therefore all the article documents & objects are created here and the feed documents are updated to match the new data)
• the xwiki-web-gwt module, which contains GWT Api for the most frequent xwiki actions (documents / objects retrieval and edit), all exposed through the XWikiService (XWikiServiceImpl) interface.
• the Watch GWT client code - which contains the Watch GWT specific code: it is only client code, which means it cannot access data or functions from the server otherwise but through the XWikiService. The GWT code handles the Watch Reader: interface, data fetching, data display, update of data on the server on user actions, etc
• XWiki sheets and documents: used to either execute code for the GWT component in a REST-like manner (the GWT calls an URL, gets the result and uses it as HTML) or to expose the watch data on the server in the wiki: WatchSheets, WatchCode spaces.
• starting with 1.1 M1, there also is a XWikiWatch GWT server-code, which is nothing else but a GWT Servlet implementing a Watch specific service

Application flow

When a feed is added in the XWiki Watch interface, it is stored in a XWiki.AggregatorURLClass object, in the . document. The WatchCode.LoadingStatus document calls the feedplugin's updateFeedsInSpace() method which detects all the objects of the specified class in the space and loads all articles for them.

On the client javascript side, the architecture is as follows (thanks Cati for the great picture):

The XWatchService depicted in the above image only exists from the 1.1 version of watch. Until then, the DataManager communicates with the generic XWikiService

The communication between the client and the server is based on asynchronous calls, which are all handled through the DataManager class in the client side code, most of them with a pass through the controller Watch class, to handle properly all updates. There are usually refreshData() calls after each call to the DataManager which fetches data from server (or alters data on the server) and stores it in the Config class in the model. The refreshData() function of the WatchWidget is responsible with updating the interface to match the fetched model. Also, as a rule, we're trying to take advantage of all opportunities to fetch new data from the server to the client code, so that it permanently matches the data on the server: it is a collaborative tool after all and users should get eachother's changes in real time (an automatic refresh mechanism is planned for future versions of Watch).

Future thoughts and plans for Watch

The most important change we plan is the implementation of a Watch component that would handle all data access and data operations for Watch, keep consistency, etc. See more details about what that means on the thread on the devs list: http://xwiki.markmail.org/search/?q=XWiki+Watch+component#query:XWiki%20Watch%20component+page:1+mid:zohayizwp2jdi6lh+state:results .

How to install Watch?

Keep a constant eye on: XWiki Watch

• for XWiki Watch < 1.0, the xwiki watch xar is enough, or the watch.zip archive (the one attached to the WatchCode.GWT with all the GWT javascript in it) generated when building the watch .xar. When importing it it overrides old code so this is the only thing needed to be done, even for reinstall. Keep in mind that browsers cache js code, and, if you change .js code and you don't see the changes, think about convincing your browser to get new js code (clearing cache for Firefox usually does it).
• for the newer versions (the future 1.1), there will also be a server code that you need to install like a plugin, and also changes in web.xml. But the install manual will be available in the dedicated section in the Installation document.

Resources

• Watch sources on the GitHub repo: https://github.com/xwiki-contrib/xwiki-watch
• the XWiki Developers' mailing list, the XWiki Users mailing list
• the xwiki IRC channel