Monday, January 27, 2014

JSON-LD, Jena, Virtuoso and Named Graphs

After working for a couple of years on the Domeo Annotation Tool I am now working on a couple of projects that focus on the creation of a back-end for saving/searching annotation. I am planning to use the Open Annotation model and some other ontologies such as: PAV (Provenance, Authoring and Versioning) ontology and maybe CO (Collections Ontology).

Named Graphs and JSON-LD

Most importantly I am going to make large use of Named Graphs and their serialization in JSON-LD format, which is the recommended format for Open Annotation. JSON-LD became very recently a W3C Recommendation.
A Named Graph is a collections of Statements that is identified by a URI.
JSON-LD is a lightweight Linked Data format. It is easy for humans to read and write. It is based on the already successful JSON format and provides a way to help JSON data interoperate at Web-scale.
JSON-LD provides a very slick way of representing Named Graphs. Here is an example of Named Graph used for representing a very basic annotation (with Open Annotation):
  
  {
     "@context": {
        ...
     },
     "@id": "http://example.org/graphs/1",
     "@graph":
     [
        {
          "@id": "http://www.example.org/ann/1",
          "@type": "oa:Annotation",
          "hasBody": "http://www.example.org/body/1",
          "hasTarget": "http://www.example.org/target/1"
        }
     ]
  }

  Figure 1 - JSON-LD representation of a Named Graph and Open Annotation data.
  You can find the full @context in the Open Annotation specifications.

Loading JSON-LD in memory with Jena API 

I would like to store the above Named Graph for instance in the triple store Virtuoso Open-Source Edition. For this task I chose the Apache Jena API that makes use of the JSON-LD implementation for Java. 

I will start by loading in memory the above JSON-LD code (figure 1) that is currently in a JSON file:
  
  JenaJSONLD.init(); // Only needed once
  
  Dataset dataset = DatasetFactory.createMem();
  InputStream inputStream = new FileInputStream(annotationFile);
  if(inputStream == null) {
    throw new IllegalArgumentException("File: " + annotationFile + " not found");
  }
  RDFDataMgr.read(dataset, inputStream, "http://example.com/", JenaJSONLD.JSONLD);

  Figure 2 - Jena API code for loading the JSON-LD file in an in-memory Dataset.
The reason why I used a Dataset rather than a Model is because the
Dataset is a collection of named graphs and a background graph (also called the default graph or unnamed graph)
And that fits exactly the needs we have with the code in Figure 1. And the needs of much more complex use cases related to Domeo. Also, this approach works for both the JSON-LD making and not making use of graphs. If the JSON-LD does not contain any graph, the Statements will belong to the default graph.

Note: When I tired to use the Model and not the Dataset for loading the JSON-LD files, I realized that only the files with no @graph declarations were loaded correctly. The ones with the @graph declaration were not generating any statement.

Persist the Named Graphs in Virtuoso 

And these are the few lines of code I use to store the in-memory graphs in the Virtuoso store (I am sure there is a better way of doing this and combining the above step with these lines of code, however, this seems to work the way I want):
  // Default graph
  if(dataset.getDefaultModel()!=null && dataset.getDefaultModel().size()>0) {
    VirtGraph virtGraph = new VirtGraph (
      "jdbc:virtuoso://localhost:1111", "dba", "dba");
    VirtModel virtModel = new VirtModel(virtGraph);
    virtModel.add(dataset.getDefaultModel());
    // Print the triples
    println "graph: *"
    RDFDataMgr.write(System.out, dataset.getDefaultModel(), JenaJSONLD.JSONLD);
  }

  // Named graphs
  Iterator names = dataset.listNames()
  while(names.hasNext()) {
    String name = names.next();
    Model model = dataset.getNamedModel(name)
    VirtGraph virtGraph = new VirtGraph (name, 
      "jdbc:virtuoso://localhost:1111", "dba", "dba");
    VirtModel virtModel = new VirtModel(virtGraph);
    virtModel.add(model);

    // Print the triples
    println "graph: " + name
    RDFDataMgr.write(System.out, model, JenaJSONLD.JSONLD);
  }

  Figure 3 - Saving default and named graphs in Virtuoso

Software versions used in the example above

For the above examples I've used the following libraries/versions:
  • jena-core v. 2.11.0
  • jena-arq v. 2.11.0
  • jsonld-java-jena v. 0.2.99
  • virtjdbc4.jar
  • virt_jena2.jar

Wednesday, August 21, 2013

JSON parsing in JavaScript

Recently I've has some blocking issues with some code I wrote a while ago. A colleague tried to re-use that code with a different back-end and he kept experiencing the exception:
JSON.parse: unexpected character
Using Firebug, I promptly inspected the JSON content that was retrieved from the new service and by copying and pasting it into http://jsonlint.com/ it seemed to be valid JSON. However, the exception was still there and clearly indicated an issue with the format of the incoming content. I therefore inspected the code of both client and server and it turns out that (1) the old JavaScript snippet was using the eval function and (2) the new back-end, mimicking what my old testing code was doing, was generating through a servlet some testing JSON just by concatenating some strings and serializing the results as JSON.

The serialization of the service was including some '\n' (new lines) that, in the old ecosystem, improved the visualization of the JSON content apparently without disrupting the activity of the eval function. The serialized content included also some dates in the format 'MM/dd/yyyy HH:mm:ss Z'.
The eval function invokes the JavaScript compiler. Since JSON is a proper subset of JavaScript, the compiler will correctly parse the text and produce an object structure (read more on json.org).
Strangely the same code that was working fine in my configuration, was failing in the configuration used by my colleague that seemed failing while interpreting the format of the dates.

As I had a similar problem months ago, I've decided therefore to move away from the eval function:
    In the old GWT code, making use of JNI:

    public static native JavaScriptObject parseJson(String jsonStr) /*-{
        return eval('(' + jsonStr + ')');
    }

    The equivalent in pure JavaScript would be (remember the parenthesis 
    as they turn the code into an expression that returns, rather than 
    just code to run):

    function parseJson(jsonStr) {
         return eval('(' + jsonStr + ')');
    }
to use the more recent JSON.parse which provides validation of the JSON content unlike eval that is faster but allows the string being parsed to contain absolutely anything including function calls.
Native JSON support is included in newer browsers and in the newest ECMAScript (JavaScript) standard. Similar features were already available with some JS libraries such as JQuery (http://api.jquery.com/jQuery.parseJSON/).
See http://www.w3schools.com/json/json_eval.asp for browser and software support.
When using the JSON.parse it is however necessary to escape the control characters.
According to section 2.5 of the JSON spec at ietf.org/rfc/rfc4627.txt: "All Unicode characters may be placed within the quotation marks except for the characters that must be escaped: quotation mark, reverse solidus, and the control characters (U+0000 through U+001F)."
In that specific testing case the newline character could have been present in both the JSON values and in between JSON elements.
    // 1) Example of JSON with newline in the value
    [{"name":"Paolo Ciccarese \n"}]

    // 2) Example of JSON with newline between elements
    [\n{"name":"Paolo Ciccarese"}]
The above case 1) can be addressed by escaping the content (for instance replacing '\n' with '\\n'. The case 2) is illegal in JSON and would work only if using the eval function.
In my case the servlets were generating the '\n' because of the use of out.println instead of the harmless out.print
So in my case the parseJSON function became:
    In the old GWT code, making use of JNI:

    public static native JavaScriptObject parseJson(String jsonStr) /*-{
        try {
            var jsonStr = jsonStr      
                .replace(/[\\]/g, '\\\\')
                .replace(/[\/]/g, '\\/')
                .replace(/[\b]/g, '\\b')
                .replace(/[\f]/g, '\\f')
                .replace(/[\n]/g, '\\n')
                .replace(/[\r]/g, '\\r')
                .replace(/[\t]/g, '\\t')
                .replace(/\\'/g, "\\'");
            return JSON.parse(jsonStr);
        } catch (e) {
            alert("Error while parsing the JSON message: " + e);
        }
    }-*/;

    In pure JavaScript would be:

    function parseJson(jsonStr) {
        try {
            var jsonStr = jsonStr      
                .replace(/[\\]/g, '\\\\')
                .replace(/[\/]/g, '\\/')
                .replace(/[\b]/g, '\\b')
                .replace(/[\f]/g, '\\f')
                .replace(/[\n]/g, '\\n')
                .replace(/[\r]/g, '\\r')
                .replace(/[\t]/g, '\\t')
                .replace(/\\'/g, "\\'");
            return JSON.parse(jsonStr);
        } catch (e) {
            alert("Error while parsing the JSON message: " + e);
        }
    }
  
Therefore in the case:
    [{"name":"Paolo Ciccarese \n"}]
the newline is not interpreted as a newline in the JSON source anymore but as a newline in the JSON data, which is perfectly fine.

In conclusion, I am still not sure why the dates interpretation was failing with the eval method. But with the JSON.parse approach the problem is gone.

I also found this: 'A fast and secure JSON parser in JavaScript'. I did not have time to check out yet but it is promising: ' does not attempt to validate the JSON, so may return a surprising result given a syntactically invalid input, but it does not use eval so is deterministic and is guaranteed not to modify any object other than its return value'.

Saturday, July 20, 2013

Domeo, Annotation Framework, Catch Annotation Hub and Grails Plugins architecture

I found organizing big projects in components always a reassuring idea.
Component Oriented Programming? I let you decide if that is what I mean. I’ve read several discussion on the topic Component Oriented Programming vs. Object Oriented Programming and I am personally one of those who believes the two strategies are complementary and not in competition. As I am not interested in debating the theoretical differences, I would stick to what I normally do and not what I think.
That is one of the reasons I’ve always liked - and I still like - OSGi and that is also one of the reasons I’ve been always attracted by the Grails Plugins architecture.
The components oriented approach did not always pay off. Occasionally I just gave up when I found myself fighting with the technology of the moment, which was getting a little on the way. I am sure most of my problems were related to my limited knowledge of that particular technology... still, deadlines are deadlines and I needed to get things done.
I am certainly not the first nor the last developer celebrating the Grails Plugin Oriented Architecture. Here is a blog post that shows how a domain class defined in a plugin can be reused by other components of the architecture.

However, I have been thinking about the OSGi-based Eclipse architecture for a long time and I even tried to develop a lighter Java framework for developing applications along the same lines. Naturally, since I've been using Grails, I’ve been thinking on how to reproduce the same behavior in web applications by using Grails plugins. Basically I am talking about conveniently leveraging plugins to benefit from all the perks of the Grails platform: domain classes, services, controllers and views. I will defer to future posts some of the technical details. Meanwhile I wish to provide a little context.

I am thinking of leveraging the plugin architecture for a project called CATCH that I’ve been working on for a tiny grant awarded by Harvard Library Labs. As the Domeo Annotation Tool already provided some of the features I need for CATCH I've decided to refactor and spin off some of its components. I've  created a new GitHub project called Annotation Framework which will collect all the new improved modules that will be later used by both Domeo and CATCH. 


CATCH Annotation Hub

The goal of CATCH is to provide a hub for collecting/searching and sharing annotation produced by several clients. These includes the Domeo annotation client, HighBrow - an annotation client developed at Harvard by Reinhard Engels - and annotator.js an open-source JavaScript library and tool - developed by Nick Stenning  - that can be added to any webpage to make it annotatable.

Both CATCH and the older sister project Domeo are meant to be installed in several instances that should be able to communicate with each other in a federated architecture. You can think this as a series of Annotation Framework Nodes that are distributed and connected so that when a user performs a search on one of the nodes, it can also find results that have been created and stored in other authorized/linked nodes. All with access control...


Friday, October 12, 2012

It seems all a matter of size

After the new iPhone 5 sporting a bigger screen and a thinner body*, comes the Samsung SIII mini and probably a new smaller iPad mini. Then I see the new Vaio Tap 20 that it is really a new kind of PC. Watching movies on a home-portable bigger screen, playing social games... but still having a full PC available for editing two documents at the same time by typing with a real old-style classic keyboard.

 New Vaio Tap 20 - source Sony

I don't really like the idea of 'touching' the screen of a laptop, which seems way too delicate and wobbly but I kind of like the idea of a bigger home-tablet. I am just wondering what happens when this big screen falls on the ground... because that is going to happen.

* I would have honestly gave that up for a better battery

Thursday, October 11, 2012

Here's To The Crazy Ones, Again

Years ago I cam across the famous Apple Computers most inspiring piece of advertisement - at least for me:
"Here's To The Crazy Ones. The misfits. The rebels. The trouble-makers. The round pegs in the square holes. The ones who see things differently. They're not fond of rules, and they have no respect for the status-quo. You can quote them, disagree with them, glorify, or vilify them. About the only thing you can't do is ignore them. Because they change things. They push the human race forward. And while some may see them as the crazy ones, we see genius. Because the people who are crazy enough to think they can change the world - are the ones who DO !" - Apple Computers
Naturally, I believe many people mentally associated this piece to one person Steve Jobs. At the end, the good times of Apple seem always been happening during Steve's lead - and I believe that is one of the reasons why everything that is going on with iOS6 and iPhone 5 is carefully monitored and inspected by the entire world. Personally, I just liked what the piece conveys. 

Anyhow, after days of reading about the new iPhone 5 problems, and in particular about the fact that the aluminum scratches really easily, I read this: These Laser-Engraved iPhone 5s Look Really Cool (via Gizmodo). 

I like the fact that a problem - Apple inefficiency - has been translated into an opportunity. As the iPhone 5 max customization you can get is the choice between two colors, the engraving would make your phone really unique and different. I remember I ordered my first iPod Nano with the engraving
Dreams change the world - Paolo Ciccarese
That made my iPod Nano unique until it has been recalled by Apple some years later and exchanged for a new anonymous Nano that I basically never used. I feel the iPhone 5 engraving potential is going to make a difference in the 'think different' iWorld.


Friday, September 28, 2012

New Apple maps and Cook's iSorry

It is well known that maps are one of the most useful features of smartphones. I still see a very good amount of unexploited potential in maps. And I believe this is also the reason why Apple decided to develop a version of their own.

But I am surprised to see Apple delivering the new products with such poor and buggy maps. I not surprised to read Cook's official apologies. I am a little surprised by Cook recommending, in the same letter, to use competitors product. I would have expected a link to a different page listing alternatives.

I am sure Apple had very good reasons to deliver the phone in September even if map were clearly not ready. As I am assuming they knew the new maps were not classic Apple-quality, I cannot stop wondering if this would have happened with Jobs in charge.

Anyhow, customers are always lining up for iPhones and I am sure Apple will fix and work on the maps to make them amazing. Still, I find what happened very Apple-unusual… and I guess Cook felt it too if he delivered the official Apple iSorry.

Meanwhile - maybe also because of the new potential competitor? - Google keeps improving Google Maps:
- Google Maps, Now With More High-Res Satellite And 45° Aerial Imagery
- Google Maps Goes Diving, Provides “Seaview” Of Great Barrier Reef, Hawaii and Philippines
- Google Brings Its Indoor Maps To France

Wednesday, May 09, 2012

Running GWT 2.4.0 with STS 2.9.1 and Grails 2.0.3

Alright, if you followed all the steps of the previous post, you should be able to see the results of the GWT code execution at the address:
http://localhost:8080/{projectname}/hello.gsp 
As you might have noticed, I previously asked the plugin to compile the GWT code (compile-gwt-modules). That operation takes some seconds as the entire artifact is recompiled (when the application will grow, the compilation can last minutes).  Luckily there is an alternative, the hosted mode. When an application is running in hosted mode, the Java Virtual Machine (JVM) is actually executing the application code as compiled Java bytecode, using GWT plumbing to automate an embedded browser window. This means (1) we don't need to recompile the code as STS is producing the Java bytecode for us incrementally (2) the debugging facilities of your IDE are available to debug both your client-side GWT code and any server-side Java code as well.

First we start the grails application (run-app), second, in the command prompt we type:
run-gwt-client
This last command starts the hosted mode and you should see appearing a window like the one depicted in figure 1. 

Figure 1: hosted mode window.

Now we can 'copy to clipboard' and open that link in a web browser:
http://localhost:8080/GwtTest/hello.gsp?gwt.codesvr=127.0.0.1:9997
The page should appear in the window and the alert 'hello' should display. Now you can verify the usefulness of the hosted mode by updating the alert - in the class HelloApp.java from 'hello' to 'hello me' - and reloading the page. The alert will display the new string without you having to trigger the GWT code compilation manually. And this will result in a huge save of time. (Read more)

Note. Remember that the manual compilation is still necessary prior to deployment as the deployed version of the page will run the compiled JavaScript.


Friday, May 04, 2012

Setting up STS 2.9.1, Grails 2.0.3 and GWT 2.4.0

If you need to integrate Grails and Google Web Toolkit (GWT) here is a list of tips. If you are interested in Grails you might want to consider the SpringSource Tool Suite or briefly STS. I usually install the STS package in a directory which name includes the version of the STS software to make sure multiple versions of the tool can beautifully coexist on the same machine. The tool is currently in version 2.9.1 and, as the previous versions of STS, it offers a good set of features for integrating with Grails. After the installation is completed I suggest to open the Dashboard and select the tab "Extensions". If you have just installed the tool, it should look more or less like in figure 1.

Figure 1: STS 2.9.1 Dashboard right after installation.

As you can easily verify in figure1, I would suggest to pick two extensions in particular: Grails Support and Google Plugin for Eclipse - which now comes with GWT 2.4.0. Select those and proceed with the installation. I don't usually pick the "Grails (current production release)" as I usually download and install Grails separately. The current release version of Grails is 2.0.3 and you should download it and unzip it in a convenient location.

Figure 2: Setting up Grails in STS.

Once the installation is terminated, it is time to configure the plugins. The Grails Support plugin needs to know where the Grails installation is (figure 2). We can now create a new Grails project that I am going to name GwtTest.

Figure 3: Creating a new Grails project in STS 

The creation process should ask you if you want to change the perspective into the Grails perspective offered by STS. Once the project is created it is time to install the GWT Plugin for Grails.

Figure 4: Opening the Grails command prompt

You can perform this operation through the Grails command prompt offered by STS (figure 4) by typing
install-plugin gwt
And the process should complete displaying the details in the console:

Figure 5: Console after installation of the GWT plugin (0.6.1) for Grails
 
Note: Remember also to set up the GWT_HOME that is used for compilation.

We are now ready to create our first GWT module (learn about GWT modules here). Using once again the Grails command prompt offered by STS, we type:
create-gwt-module org.example.HelloApp
Once the process is completed, if you refresh the project, you will be able to notice that a new directory 'src' has appeared in the project explorer.

Figure 6: Project explorer and Console after the creation of the HelloApp GWT module

In order to declare this folder as a source code folder we can right click on the project and select 'Build Path'>'Configure Build Path' dialog. In the tab 'source' we select 'add folder' and we pick the newly created 'src/gwt' folder.

Figure 7: Declaring the GWT folder as a source code folder

We can now create the page hosting the GWT script by typing in the command prompt:
  create-gwt-page hello.gsp org.example.HelloApp
The new page will include the line of code that is performing the actual import of the GWT artifact:
<script src="${resource(dir:'gwt/org.example.HelloApp',
 file:'org.example.HelloApp.nocache.js')}" type="text/javascript">
</script>

We are now going to modify the HelloApp.java file adding Window.alert("hello"); within the method onModuleLoad() so that when the script is loading the alert 'hello' will fire.

We then compile the GWT code by typing in the command prompt:
compile-gwt-modules
And we are potentially ready to go. After running the Grails project - a simple run-app is good enough - and opening the page:
http://localhost:8080/{projectname}/hello.gsp
you will notice that nothing happens. As a matter of fact, if you check out what happens with Firebugs, you will notice that a resource has not been found. Unfortunately, it seems the Grails Resources Plugin, which is installed by default in the project, is not compatible with the Grails GWT Plugin. As suggested in this post we can modify the Config.groovy file to include:
grails.resources.adhoc.excludes = ['**/gwt/**']
When restarting the server, the hello.gsp page should now work and fire the 'hello' alert.

Saturday, January 14, 2012

Domeo v2 working much faster on Firefox then Chrome

Since a week now, I am deeply involved in developing the v.2 of the Domeo annotation tool. Domeo is a combination of GWT (Google Web Toolkit) and JavaScript. I've been mainly working on the infrastructure and, initially, as it is not much UI work, I was testing only on Chrome (16.0.912.75). Sadly, I've  noticed several times that my code was not running fast enough and I started to study alternative ways of performing various tasks.

Then, out of curiosity, I ran the same tests on Firefox and surprise... what was running slow on Chrome was running almost without any latency in FF (6.0.2).


The test consists in 18 steps and each of them is adding the same amount of complexity. Looking at the above figure (milliseconds on the Y-axis) you will easily detect what I am talking about. The difference seems way too big.

Monday, September 26, 2011

Collections Ontology v.2... list of persons [1]

Back in 2008 I've published a couple of posts explaining my need of creating an ontology for collections:
After some months of work with Silvio Peroni we are almost done with v.2 of Collections Ontology (CO) expressed in OWL2.  Following is a simple example that illustrates some of the features of CO2. Before that here is how to set up the environment for testing the features yourself. First of all I would suggest you to install Protege 4.1 and to make sure to install the Pellet plugin for it.

With Protege up and running, I performed the import of the development version of the Collection Ontology v.2 with URL: http://collections-ontology.googlecode.com/svn/trunk/collections.owl.

Figure 1 - Import of the development version of the Collections Ontology (CO) with Protege

Figure 2 - Collections Ontology (CO) is imported

After that I've created a class person - note that I haven't reused classes such as foaf:Person to keep the example simple - and the instances in figure 3 in order to model a list of persons (you can download the file here).

Figure 3 - Collections Ontology (CO) example instances (ovals)

After triggering the reasoner, it is immediate to notice inferred properties (figure 4 in light yellow).

Figure 4 - Inferred types and properties for item instance itemOne. For instance itemOne has been defined as instance of item, the reasoner infers itemOne (i) is a list item (ii) is follwed by both itemTwo and itemThree (iii) is item of and is first item of persons.

Figure 5 - Explanation of itemOne type 'list item' obtaining by clicking the highlighted button

We can now proceed with some DL Queries.

Query 1
For instance we can ask for all the items that have item content (has item content) persons with name "Paolo Ciccarese":
item and 'has item content' some (person and (name value "Paolo Ciccarese"))
In the Protege tab named 'DL Query' we can enter the query above and, if we select on the right side the option 'Individuals' we are going to  retrieve one item (itemOne):


Query 2
We can ask for all the lists where the first person is named 'Paolo Ciccarese' (answer 'persons'):
list and 'has first item' some (
     item and 'has item content' some (person and (name value "Paolo Ciccarese"))
)
Query 3
Similarly we can ask more complex queries such as: find all the persons lists where the first item points to a person named 'Paolo Ciccarese' and the last item points to a person named 'Silvio Peroni' (answer 'persons'):
list and (
    'has first item' some (item and 'has item content' some
           (person and (name value "Paolo Ciccarese")))
    and
    'has last item' some (item and 'has item content' some
           (person and (name value "Silvio Peroni")))
)
Query 4
Another query can be: give me all the lists where the first person is named 'Paolo Ciccarese' and the second is 'Marco Ocana' (given the transitive nature of the property 'is followed by' the answer is 'persons'):
list and (
      'has first item' some (item and
            'has item content' some (person and (name value "Paolo Ciccarese"))
            and
            'is followed by' some (item  and
                    ('has item content' some(person and(name value "Marco Ocana"))))
      )
)
Query 5
Returns all the lists containing a person named 'Paolo Ciccarese' (answer 'persons'):
list and 'has item' some (
     item and 'has item content' some (person and (name value "Paolo Ciccarese"))
)
Query 6
Returns any list where a person named 'Paolo Ciccarese' is followed by a person named 'Silvio Peroni' (answer 'persons'):
list and (
    'has item' some (item and
            'has item content' some (person and (name value "Paolo Ciccarese"))
            and
            'is followed by' some (item  and
                    ('has item content' some(person and(name value "Silvio Peroni"))))
      )
)
Query 7
Returns all the lists where a person named 'Silvio Peroni' is preceeded by a person named 'Paolo Ciccarese' (answer 'persons'):
list and (
    'has item' some (item and
            'has item content' some (person and (name value "Silvio Peroni"))
            and
            'is preceded by' some (item  and
                    ('has item content' some(person and(name value "Paolo Ciccarese"))))
      )
)

Thursday, September 22, 2011

SWAN, AlzSWAN, HyQue and Nanopublications

While developing the SWAN ontology and the SWAN platform (see AlzSWAN for Alzheimer disease) there have always been two open issues: (i) the use of named graphs and (ii) the translation of the textual discourse elements (claims such as: Intramembranous Aβ could behave as chaperones of other membrane proteins) into a formal representation made of triples.


(i) The use of named graphs is a useful way for wrapping some content and specifying its provenance. Basically the idea is to create an 'onion layers' model where each layer has its own provenance. At the time - back in 2006 - I have been investigating the usage of named graphs - and TriX - for representing SWAN content.  However, we decided not to implement such approach because the technological uncertainty in that uncharted territory - of developing an application like SWAN - was already high enough, named graphs usage was not homogeneous across the community and  their serialization was not standardized. This meant introducing de-facto reification for some of the SWAN relationships in order to be able to attach the appropriate provenance. As Graphs are the topic of one of the task forces of the RDF Working Group for updating the 2004 RDF Recommendations, I was starting to think of resuming the old plans.

(ii) The translation of the textual discourse elements into a formal representation made of triples is, for instance, possible through the HyBrow (now HyQue) approach. Translating narrative into triples is not easy job though. Many already found the SWAN manual creation process of narrative claims very labor intensive. In fact, the SWAN curators have been usually rephrasing each claims/hypothesis to make them simple and self contained (including the minimum necessary context). Translation into triples requires, even more, starting from neat hypothesis and claims. And these are not always that easy to obtain.

These two SWAN-related issues have been in my thoughts since a while when the Nanopublication [1] concept came out.

A Nanopublication is a "set of annotations that refers to the same statement and contains a minumum set of (community) agreed-upon annotations.
The concept itself is simple and in the above linked slideshow you can find a first attempt based on real SWAN data. With respect to the paper, the concept of 'statement' (triple) has to be updated to 'statements' (triples) as one single statement is not always enough to satisfy needs of real use cases.
 Statement --> statements
Starting from the above example, we are now trying to formalize a bit better what a Nanopublication architecture would look like... it is work in progress, but if you look at the slides you will get the drift.

[1] Paul Groth, Andrew Gibson, Jan Velterop. The anatomy of a nanopublication. Information Services and Use (2010). Volume: 30, Issue: 1, Publisher: IOS Press, Pages: 51-56 (on Mendeley)

Wednesday, September 21, 2011

UIBinder, CssResource and CSS (GWT)

Few month ago I blogged about ClientBundle, UIBinder and CSS (GWT). I just realized that an important use case was missing. And here it is.

4) Using CssResource for CSS expressed in the UIBinder. When creating a UI with the UIBinder it might happen to include some CSS rules in the ui.xml file.

<ui:UiBinder
  xmlns:ui='urn:ui:com.google.gwt.uibinder'
  xmlns:g='urn:import:com.google.gwt.user.client.ui'>

    <ui:style>
       .outer {
          width: 100%;
       }
    </ui:style>
    <g:SimplePanel ui:field='sideBar'>
    </g:SimplePanel>
</ui:UiBinder>

It is possible to refer to those CSS rules from the GWT code. For doing so we have to declare a 'type' for ui:style:

<ui:UiBinder
  xmlns:ui='urn:ui:com.google.gwt.uibinder'
  xmlns:g='urn:import:com.google.gwt.user.client.ui'>

    <ui:style type='org.example.Example.ExStyle'>
       .outer {
          width: 100%;
       }
    </ui:style>
   
    <g:SimplePanel ui:field='sideBar'>
    </g:SimplePanel>
</ui:UiBinder>


Now, in the class  org.example.Example, we can reference the CSS rules and use them:

public class Example extends Composite {

    interface Binder extends UiBinder { }    
    private static final Binder binder = GWT.create(Binder.class);
   
    @UiField SimplePanel sideBar;
    @UiField ExStyle style;

    interface ExStyle extends CssResource {
        String outer();
    }

    public Example() {
        initWidget(binder.createAndBindUi(this)); 
        sideBar.setStyleName(style.outer()); 
    }
   ...

Wednesday, September 14, 2011

Grails 1.3.7, Spring Security and OpenID... with exception

Here is the list of steps I performed to set up OpenID authentication with Spring Security:

1) Create Project with Grails 1.3.7
grails create-project UserManagement 
The file application.properties will look something like this:
app.grails.version=1.3.7
app.name=UsersManagement
app.servlet.version=2.4
app.version=0.1
plugins.hibernate=1.3.7
plugins.tomcat=1.3.7 

2) Installing the Spring Security plugin (website and documentation)
grails install-plugin spring-security-core
The file application.properties will now look something like this:
app.grails.version=1.3.7
app.name=UsersManagement
app.servlet.version=2.4
app.version=0.1
plugins.hibernate=1.3.7
plugins.spring-security-core=1.2.1
plugins.tomcat=1.3.7

3) Creating Controller and template classes
grails s2-quickstart org.commonsemantics.scigrails.module.users.security User Role
You will notice the following new files
controllers/LoginController.groovy
controllers/LogoutController.groovy
domain/org.commonsemantics.scigrails.module.users.security.User.groovy
domain/org.commonsemantics.scigrails.module.users.security.Role.groovy
domain/org.commonsemantics.scigrails.module.users.security.UserRole.groovy
views/login/auth.gsp
views/login/denied.gsp
The Config.groovy will be updated with the following lines:
grails.plugins.springsecurity.userLookup.userDomainClassName =
      'org.commonsemantics.scigrails.module.users.security.User'
grails.plugins.springsecurity.userLookup.authorityJoinClassName = 
      'org.commonsemantics.scigrails.module.users.security.UserRole'
grails.plugins.springsecurity.authority.className = 
      'org.commonsemantics.scigrails.module.users.security.Role'
If you are going to change the package of the above classes, just remember to update the above properties.

4) Move controllers to the desired package - org.commonsemantics.scigrails.module.users.security

5) Install the OpenID module (website and documentation)
grails install-plugin spring-security-openid
The file application.properties will now look something like this:
app.grails.version=1.3.7
app.name=UsersManagement
app.servlet.version=2.4
app.version=0.1
plugins.hibernate=1.3.7
plugins.spring-security-core=1.2.1
plugins.spring-security-openid=1.0.3
plugins.tomcat=1.3.7

6) Creates OpenId Controller and templates for it.
grails s2-init-openid
This script adds the following files:
controllers/OpenIdController.groovy
views/openId/auth.gsp
views/openId/createAccount.gsp
views/openId/linkAccount.gsp

7) Move OpenIdController to the desired package - org.commonsemantics.scigrails.module.users.security

8) Add support for the remember-me checkbox
grails s2-create-persistent-token 
     org.commonsemantics.scigrails.module.users.security.PersistentLogin
It adds the file:
domain/org.commonsemantics.scigrails.module.users.security.PersistentLogin.groovy
The Config.groovy is updated with the following lines:
grails.plugins.springsecurity.rememberMe.persistent = true
grails.plugins.springsecurity.rememberMe.persistentToken.domainClassName =
     'org.commonsemantics.scigrails.module.users.security.PersistentLogin'
Once again, if you are going to change the package of the above classe, just remember to update the correspontend property.

9) Creating the OpenID domain class
grails s2-create-openid 
     org.commonsemantics.scigrails.module.users.security.OpenID 
The script creates on file:
domain/org.commonsemantics.scigrails.module.users.security.OpenID.groovy
The Config.groovy is updated with the following line:
grails.plugins.springsecurity.openid.domainClass =       
     'org.commonsemantics.scigrails.module.users.security.OpenID'
Once again, if you are going to change the package of the above classe, just remember to update the correspontend property.

10) Adding OpenIDs to the User domain class

The following line of code has to be added to the existing User.groovy class:
static hasMany = [openIds: OpenID]

11) Creating some test users

As suggested by the documentation we can now create some test users by editing the Bootstrap.groovy file as follows
import org.commonsemantics.scigrails.module.users.security.Role
import org.commonsemantics.scigrails.module.users.security.User
import org.commonsemantics.scigrails.module.users.security.UserRole

class BootStrap {

    def springSecurityService

    def init = { servletContext -> 

        String password = springSecurityService.encodePassword('password')
        
        def roleAdmin = new Role(authority: 'ROLE_ADMIN').save() 
        def roleUser = new Role(authority: 'ROLE_USER').save()

        def user = new User(username: 'user', 
            password: password, enabled: true).save() 
        def admin = new User(username: 'admin', 
            password: password, enabled: true).save()

        UserRole.create user, roleUser 
        UserRole.create admin, roleUser 
        UserRole.create admin, roleAdmin, true 
    } 
}


12) Redirect the login requests

It is now necessary to direct the authentication calls to the new Controller that manages also the OpenIDs. We can add to the UrlMappings.groovy file the following:
"/login/auth" {
         controller = 'openId'
         action = 'auth'
      }
      "/login/openIdCreateAccount" {
         controller = 'openId'
         action = 'createAccount'
      }

13) Running Grails and testing
grails run-app 
Welcome to Grails 1.3.7 - http://grails.org/
Licensed under Apache Standard License 2.0
...
...
Configuring Spring Security ...
Configuring Spring Security OpenID ...
Server running. Browse to http://localhost:8080/UsersManagement

When accessing the page you should see something like this:
I will try it yourself but the authentication did not work for me right away. I logged in with my Google OpenID and I got sent to the screen for creating an account.
I then selected the 'link this OpenID' option.
After introducing the testing credential 'user' and 'password' I got a 'user not found' back.

14) Turning on the logging

First thing I wanted to see if there were any weird things happening by turning on the logging in the Config.groovy file:
debug 'org.springframework.security'
No exceptions emerged when repeating the process.

15) Making sure the User is stored in the DB

We can inspect the default HSQLDB by adding at the end of Bootstrap.groovy init the following:
org.hsqldb.util.DatabaseManager.main()'
this will open the in memory database inspector. Just remember that if you close the inspector that will also shutdown your Grails app. Read more about the inspector here and remember to take out that line once the debugging is done..

The users resulted stored in the database.

16) Making sure the encoding process works correctly

After editing the Bootstrap.groovy file with:
String password = springSecurityService.encodePassword('password')
println 'a ' + password
def admin = new User(username: 'admin', password: password, 
     enabled: true).save(failOnError: true)
println 'b ' + User.findByUsername('admin').password
and a restart, I obtained:
a 5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8
b 113459eb7bb31bddee85ade5230d6ad5d8b2fb52879e00a84ff6ae1067a210d3

It is clear that the login process is not performing correctly. After investigating I found out that the User domain class includes the methods:
def beforeInsert() {
          encodePassword()
     }

     def beforeUpdate() {
          if (isDirty('password')) {
                encodePassword()
          }
     }

17) Updating the Bootstrap.groovy file

As the User domain class is already performing the encoding, we have to edit the Bootstrap.groovy file line:
String password = springSecurityService.encodePassword('password')
Into
String password = 'password'

And now the login process works for me. We can now remove org.hsqldb.util.DatabaseManager.main() from the end of the Bootstrap.groovy and maybe comment out the logging (see point #14).

18) Remove the unused views and controllers?

It is not possible to remove any of the generated controllers. As far as I know the only file that can be removed is
views/login/auth.gsp
as the new auth.gsp file provided by the OpenID plugin is now in use (see point #12).


You can check out the code as follows:
svn checkout 
     https://common-semantics.googlecode.com/svn/tags/UsersManagement20110914  
     UsersManagement

Resources
OpenID website
Spring Security Plugin for Grails website
OpenID Plugin for Spring Security website

Tuesday, August 30, 2011

UIMA, Clerezza, AO and Linked Open Data [1]

More than a year ago I started to work on the integration of text mining services with DOMEO, the tool I am building. In DOMEO we try to focus, as much as possible, on annotations that refers to entities defined in controlled vocabularies, ontologies or knowledge bases. In other words entities identified by URIs. Almost naturally, after writing a few connectors to existing text mining services, we realized how nice would have been to have a common way of exposing such services.

After some investigations, I found Apache Clerezza a service platform based on OSGi (Open Services Gateway initiative) which provides a set of functionality for management of semantically linked data accessible through RESTful Web Services and in a secured way. As I am familiar with the OSGi technology, I contacted the responsible for the UIMA integration - Tommaso Teofili - right away and we started to exchange ideas. After writing to the Clerezza mailing list, I decided to write and contribute some code able to transform UIMA results into Annotation Ontology (AO) RDF format.

Working with Antony Scerry (Elsevier) months ago, I learned right away that the hard part was not the code itself but the fact that the UIMA types are extremely flexible. Therefore, even if most existing NLP tools are not returning URIs, when text mining tools do return URIs it is not trivial to capture the URI of the recognized entity. That info can be encoded in whatever way the service developers decide to. That can be a problem when trying to integrate multiple text mining service under a common interface.

As I did not want to propose something too complicated, Tommaso and I convened that adopting a little convention could make our life easier. After a few discussions, Tommaso introduced a couple of basic entities: ClerezzaBaseAnnotation and ClerezzaBaseEntity. These are both implementing the property 'uri' and ClerezzaBaseEntity is also implementing the property 'label'. When creating new type system descriptors, in order to adopt the proposed convention, it is simply necessary to extend these two types and used them appropriately.  As a result, through a patch I wrote, it is now possible, for instance, to map the results to the Annotation Ontology (AO) RDF format without any effort and, therefore, to display your results on the analyzed document through DOMEO.

Thanks to Tommaso,  my patch has been integrated in the Clerezza code today. The documentation will be made available soon as well as other code that will allow to easily set up web services for publishing UIMA algorithm.

Thursday, May 26, 2011

DOMEO: Linking science and semantics with Annotation Ontology (AO) [1]

In the last few months I've been focusing on the development of the SWAN Annotation Tool (recently renamed DOMEO)*. DOMEO (Document Metadata Exchange Organizer), is an extensible web component enabling users to visually and efficiently create and share ontology-based stand-off annotation metadata on HTML or XML document targets, using the Annotation Ontology RDF model. The tool supports manual, fully automated, and semi-automated annotation with complete provenance records, as well as personal or community annotation with access authorization and control. DOMEO is one of the pieces of a bigger architecture that we internally call Annotation Framework.

The DOMEO interface
The idea itself is pretty simple, DOMEO is basically a little browser inside the browser. It allows the user to type a URL, open the correspondent document and annotate it. It is also possible to pass a URL as a parameter so that the tool opens with the page you want annotate already in the content frame. This option is particularly helpful when integrating the tool with other applications or other sections of the Annotation Framework.

Figure 1: A screenshot of DOMEO. You can notice the address bar where you see the URL of the document displayed below. The document displays the same way it would appear when opened in a new browser window.
The annotation can be performed manually by the user or automatically by text mining or entity recognition services. The two features are available through the two buttons in the DOMEO toolbar and labeled respectively 'Annotate' and 'Text Mining'. When the option 'Text Mining' is selected, the tool lists all the available text mining or entity recognition services. The user can then decide which one or which ones to run on the loaded document.

The button 'save' is saving the produced annotation. DOMEO supports a complex versioning system that basically saves items only when necessary, keeps track of the different versions and for each of them records the full provenance data. I will probably explain the versioning and the provenance models in another post.

* The development of DOMEO is managed and carried out by Dr. Paolo Ciccarese. DOMEO is a product of the MIND Informatics group - Mass General Hospital. The tool is developed in parallel with the Annotation Ontology (AO)

Thursday, March 03, 2011

HTML5: The new 'semantic elements'

One of the news in HTML5 consists in the so called 'semantic elements'. The goal of these elements is provide better way for web authors to define the parts of a document and, potentially, to improve accessibility (i.e. screen readers). Some of these elements are: section, nav, article, aside, hgroup, header, and footer.

Let's say I want to create the main page of a blog. My page is going to have a header and probably a footer. If you look at web pages source code, you will notice that these two elements are present in almost every page in many different variants. The blog you are reading already uses <header> and <footer>.

The elements <header> and <footer> can be used not only for defining the structure of the main page but also for defining headers and footers of sections, articles and asides.


If you look at the structure of the blog you are reading, this is pretty much similar to the one depicted in the above figure. The 'nav' areas is correspondent to the 'Blog Archive' and the 'aside' area is collecting my picture, 'label' and 'links'. Using these new elements - together with the new features brought by CSS3 - is certainly a way to avoid the well known 'Div Mania'.

Once we have a <header>, it is possible to use the element <hgroup>: it represents the heading of a section. The element is used to group a set of h1–h6 elements when the heading has multiple levels, such as subheadings, alternative titles, or taglines. Also, in the specs, given the following example, it is stated that the point is to mask the h2 element (which acts as a secondary title) from the outline algorithm.
<hgroup>
  <h1>The Coolest Application</h1>
  <h2>Alpha Release</h2>
</hgroup>
I have to admit, it seems a bit rigid and I feel my code will end up with one element more than usual. I have to force myself in not using <div> or <span> or <p> for things that look more like a subtitle and do something like:
<header>
  <hgroup>
    <h1>The Coolest Application</h1>
    <h2>Alpha Release</h2>
  </hgroup>
  <p>Description...</p>
</header>
One thought, the new elements seems still structural - with different level of granularity - and not semantic to me. Maybe structure can be seen as a kind of semantic, but I don't think the name is a good idea as it intersect with the pool of the technologies of the Semantic Web. In other words, saying a chunk of the document is a section does not help machine understanding of the content - besides that it knows where it starts and where it ends - , but if I say the section is a 'http://rdfs.org/sioc/types#BlogPost' we are starting to attach meaning we can leverage. And we can certainly do that with RDFa for example.

Monday, February 28, 2011

SWAN Annotation Tool: what is new in build 7

Here is a list of some of the new features that I will deploy probably this week.


The development of the SWAN Annotation Tool is managed and carried out by Dr. Paolo Ciccarese. The fist build of the SWAN Annotation Tool has been developed by Dr. Paolo Ciccarese and Marco Ocana. The SWAN Annotation Tool is a product of the MIND Informatics group - Mass General Hospital - directed by Tim Clark.

Sunday, February 27, 2011

ClientBundle, UIBinder and CSS (GWT)

In a previous post I was discussing all the possible ways for using image resources with GWT. Another thing you might want to do is dealing with CSS.

1) Standard use of CSS. While using GWT you can certainly use the CSS how you would use them for any other application, simply declaring CSS classes in a *.css file and importing it in the webpage of interest. With GWT widgets you will then simply set the style name as follows:
widget.setStyleName("cssClassName");
This approach works, however, if the CSS declaration is missing no exceptions are raised. Also, if you use multiple CSS files as I do, it is always annoying to find the declarations when you need to.

2) Using the UIBinder. If you are already using UIBinder, the easiest way to include CSS declarations is to add them to the binder. It is easy and it is safe as the Eclipse plugin is helping you out in finding missing declarations. I always use Eclipse but if you don't I am assuming you'll still find the problems at compile time.
<ui:UiBinder
  xmlns:ui='urn:ui:com.google.gwt.uibinder'
  xmlns:g='urn:import:com.google.gwt.user.client.ui'>

    <ui:style>
       .outer {
          width: 100%;
       }
    </ui:style>

    <g:VerticalPanel styleName='{style.outer}'>
    </g:VerticalPanel>
</ui:UiBinder>
The downside of this approach is redundancy. Sometimes I want to use CSS declarations multiple times and, with this approach, I have to repeat them for each single Binder.

3) Using CssResource. Another alternative consists in doing something similar to what you can do for icons with ImageResource. First I declare the set of declarations of the stylesheet in a file that I name Commons.css:
.smallIcon {
    height: 16px;
    width: 16px;
}

Then, linking the CSS file, I declare the use of the stylesheet as a resource for the application:
public class Example implements EntryPoint {

  public interface Resources extends ClientBundle
  {
     public static final Resources INSTANCE =  GWT.create(Resources.class);
 
     public interface Resources extends ClientBundle { 
        @Source("org/example/application/client/Commons.css")
        CommonsCss commonsCss();
     }
     
     ...
  }
  
}
Now, where the dots in the above snippet are, I can declare the stylesheet declarations I want to expose to the application:
public interface CommonsCss extends CssResource {
     String smallIcon();
}
As the CSS class is named as the method everything works fine. However, sometimes you might want to change the name of the method. Using Java annotations you address that issue as well:
public interface CommonsCss extends CssResource {
     @ClassName("smallIcon")
     String smallIconClass();
}
Now, we can write something like:
Resources resources = Resources.INSTANCE.factory().create();
Image img = new Image();
img.setStyleName(resource.commonsCss.smallIcon());
...
This approach allows you to collect in one single place CSS declarations you need to use in multiple packages in your application. Also you can leverage a good amount of validation in your Java code. You might argue the process can be a bit tedious but I can assure you, for a big GWT application, it can help you saving lots of time later on especially when refactoring the code.

There are other interesting things to know about the ClientBundles, but for now I'll stop here.

Saturday, February 26, 2011

Dublin Core and PRISM

As I was saying in one of my previous posts, distinguishing the different kinds of contributions is not trivial. However, sometimes is necessary. And this is probably the case of publishers that want to keep track of the exact role of the different contributors to a resource.

This is the case of PRISM (Publishing Requirements for Industry Standard Metadata), a metadata vocabulary for managing, post-processing, multi-purposing and aggregating publishing content for magazine and journal publishing. PRISM allows to distinguish between different creators roles: writer, editor, composer, speaker, photographer... you can find the full list in the The PRISM Controlled Vocabulary Namespace. PRISM is also using parts of Dublin Core Element Set and Dublin Core Terms, the subset of terms is listed in the document named The PRISM Subset of the Dublin Core Namespace.

The combination of DC and PRISM, for instance for a book, will become in XML something like:
<dc:creator prism:role=”writer”>John Doe</dc:creator> 
<dc:creator prism:role=”editor”>Paolo Ciccarese</dc:creator>
<dc:creator prism:role=”graphicDesigner”>Micheal Doe</dc:creator>
In RDF, according to the specifications (paragraph 3.5.2 of the PRISM Subset of the Dublin Core Namespaces: Version 2.1), this would look like:
<dc:creator rdf:resource=”contributorrole.xml#writer”>
     John Doe
</dc:creator>
<dc:creator rdf:resource=”contributorrole.xml#editor”>
     Paolo Ciccarese
</dc:creator>
<dc:creator rdf:resource=”contributorrole.xml#graphicDesigner”>
     Micheal Doe
</dc:creator>
However, this is not valid RDF for a couple of reasons that you can find yourself through the RDF Validator Service. Dublin Core Element Sets properties used by PRISM and by the PRISM aggregator message are: creator, contributor, description, format (PRISM records restrict values of the dc:format element to those in list of Internet Media Types [MIME]), identifier (for instance DOI), publisher, subject, title, type. Other properties are listed but not as items of the PAM format: language, relation, source.

For instance, this is how PRISM can deal with identifiers in RDF:
<dc:identifier>10.1030/03054</dc:identifier>
<prism:doi>http://dx.doi.org/10.1030/03054</prism:doi>
<prism:url rdf:resource=”http://dx.doi.org/10.1030/03054”/>
Basically, besides the usage of dc:identifier, PRISM is using the properties prism:doi - which is declaring more explicitly than dc:creator what the identifier is - and prism:url. Strangely enough, the property prism:doi is actually taking as value the DOI proxy URL and not the DOI string. Therefore, I see prism:doi and prism:url as redundant properties. You can find some more details on this old blog post by Tony Hammond.

Moreover, PRISM PAM is making use also of the Dublin Core Terms dct:hasPart and dct:isPartOf for detecting for instance images that are part of a document:
<dcterms:hasPart rdf:resource= ”http://www.myexamples.com/ExamplePhoto.jpg”/>