Webclient: more user friendly page panel rendering

[jri]

Currently the page panel (right side) lists a lot of information twice. This is because all the topics backed by the selected topic's schema definition, as rendered in the upper area, appear another time in the schema less Associations listing ("what's related?"), as rendered in the lower area. This results in a cluttered page panel with a lot of redundancy and user unfriendly computer jargon appearing in the UI ("Aggregation", "Composition", "Instantiation").

2 measures are proposed:

• Fixed rule: do not render a topic in the Associations listing if the topic appears already in the upper area (schema defined).
• Programmatic configuration: a plugin developer could suppress the entire Associations listing. She could do so by manipulating the page model via the "pre_render_page" event.

Open question:

• Should the "what's related" Associations listing be grouped by Association Type (as it is currently) or by Topic Type? Advantage of the latter would be the computer jargon ("Aggregation", ...) to disappear from the UI. This is because Association Type Names are rather technical while Topic Type Names rather origin in the user domain. However, it seems there is no "right" decision per-se as it depends on the respective application data model.

[jri]

Should the "what's related" Associations listing be grouped by Association Type (as it is currently) or by Topic Type? Advantage of the latter would be the computer jargon ("Aggregation", ...) to disappear from the UI. This is because Association Type Names are rather technical while Topic Type Names rather origin in the user domain.

=> I'll go for grouping by topic type now.

[Jörg Richter]

Webclient: sort listing by topic type (#315).

The "what's related" topic listing in the detail panel is grouped by Topic Type (instead Association Type).
Thus the group headings are more user friendly.

BREAKING CHANGE

Server-side serialization format: a RelatedTopic?'s association is NOT serialized anymore. That is the JSON object has no "assoc" property anymore.
For the moment it is not clear weather the association will be needed in the future. For the moment it reduces network traffic considerably.

See #315.

• Changeset: 0c7423ca497e8da1f9d8395128ccf9c6f20fd234
• URL: ​https://github.com/jri/deepamehta/commit/0c7423ca497e8da1f9d8395128ccf9c6f20fd234

[Jörg Richter]

Webclient: sort listing by topic type (#315).

The "what's related" topic listing in the detail panel is grouped by Topic Type (instead Association Type).
Thus the group headings are more user friendly.

BREAKING CHANGE

Server-side serialization format: a RelatedTopic?'s association is NOT serialized anymore. That is the JSON object has no "assoc" property anymore.
For the moment it is not clear weather the association will be needed in the future. For the moment it reduces network traffic considerably.

See #315.

• Changeset: 0c7423ca497e8da1f9d8395128ccf9c6f20fd234
• URL: ​https://github.com/jri/deepamehta/commit/0c7423ca497e8da1f9d8395128ccf9c6f20fd234

[Jörg Richter]

Webclient: sort listing by topic type (#315).

The "what's related" topic listing in the detail panel is grouped by Topic Type (instead Association Type).
Thus the group headings are more user friendly.

BREAKING CHANGE

Server-side serialization format: a RelatedTopic?'s association is NOT serialized anymore. That is the JSON object has no "assoc" property anymore.
For the moment it is not clear weather the association will be needed in the future. For the moment it reduces network traffic considerably.

See #315.

• Changeset: 0c7423ca497e8da1f9d8395128ccf9c6f20fd234
• URL: ​https://github.com/jri/deepamehta/commit/0c7423ca497e8da1f9d8395128ccf9c6f20fd234

[JuergeN]

+++ STOP! +++

I am not sure if I am getting this right!? Do you mean that the information in the details panel should not show the associated topics based on the association type anymore? If this is only relevant for the technical associations of the topic based on the type modell, I would not totally mind the change.

But if it is also relevant for user defined association types, that it is a no-go! Why? Say a associate a person with a book and the association is of the type author or owner or reviewer that is exactly what I want to know and what makes DeepaMehta so relevant.

I think the different types of associations are really key and very, very relevant for DeepaMehta and one should not (and must not) loose the structured access this information.

Even for the technical types of association from the type model I think the information can be quite relevant to better understand the idea of DeepaMehta, but I agree that this information is desplayed way to prominent compared to its value at the moment.

Please just make sure that all user defined associations and information gets much more focus here! And please do NOT skip the list of associated topics sorted by user defined association types.

[jri]

Topic grouping "by Association Type" vs. "by Topic Type"

[jri]

Thanks, JuergeN, for your dedicated objection!

Yes, association types are important. And yes, we must find a way to reveal the respective association types when displaying the list of related topics in the detail panel. But I think this does not boil down to the question weather to group the listing by association type (as in DM 4.1.2) or by topic type (as currently in 4.1.3-SNAPSHOT). In its rudimentary form both grouping modes have advantages and disadvantages. There is no winnner.

Real world experience with both, our university project, and the Poemspace project revealed that grouping by association type is too difficult to grasp by non-computer users.

For non-computer people e.g. the "Topic Type - Topic Instance" concept is not familiar (let alone the concept of "Association Type - Association Instance"). Lets have a look at the enclosed screenshot. When I explain to a user "to tell the type of a topic look under Instantiation in the listing, there you find e.g. Note, this is the Topic Type of the topic". So they have to learn to differentiate 6 things at once: 1) there are Topic Types and Topic Instances, 2) Note is a Topic Type, 3) my topic is of type Note, 4) types are associated with instances via Instantiation associations, 5) there are Association Types and Association Instances, and 6) Instantiation is an Association Type. According to my real world experience this is much to complicated and almost unacceptable for non-computer users. And even worse, in the listing the most important information -- that Note is the Topic Type -- is not even visible in the listing (see screenshot, left hand side), but the indirect information of Instantiation is.

Similar difficulties arise when the user is supposed to answer the most basic questions about a topic, e.g. what workspace(s) it belongs to, or in which topicmaps it appears. In case of workspace I have to explain: "look in the listing under Aggregation for a yellow-star symbol. The name displayed along with it is the workspace". Huuhh?? Be aware the most basic terms, namely Workspace and Topicmap are not even appearing on the screen at all (but the cryptic terms Aggregation and Topic Mapcontext do). And be further aware that novice users have not internalized that "yellow star" stands for "Workspace" and "green star" for "Topicmap".

Every time I find myself forced explaining things this way it becomes evident to me, that the design of the detail panel is fundamentally flawed. A reliable indication of good design is when it is easy to explain.

New users must be provided with a step-by-step learning process. In this sense Topic Type is the much more basic concept than Association Type. With "Grouping by Topic Type" new users are provided with a rendering which is completely understandable before they even know about Association Types. That is not the case with "Grouping by Association Type".

Lets see how Grouping by Topic Type looks like (see screenshot, right hand side). All basic information is visible: The Topic Type of this topic is Note. The topic is assigned to the Workspace named DeepaMehta and appears in a Topicmap named untitled. All that basic information (including the proper terms) appears on the screen directly.

You might argue that all 3 examples -- Type-assignment, Workspace-assignment, Topicmap-assignment -- involve "technical" association types as opposed to "user defined" association types. But in fact no such categories exist in the DM-model and thus can not be exploited for rendering. (However, each type has an URI and URIs are namespaced. Perhaps we could rely on categorizing the types of the dm4.core.* hierarchy as "technical" and all other as "user defined". As one possible solution path we can further contemplate about how a rendering strategy based on this categorization might look like. But that's not the way I like to propose here.)

To sum up my arguments/findings:

• "Grouping by Association Type" suppresses the most basic information in favor of secondary (indirect) information.
• "Grouping by Association Type" is much harder to explain to non-computer users and thus is an obstacle for DM's acceptance.
• "Grouping by Topic Type" on the other hand makes the most basic information visible on-screen.
• "Grouping by Topic Type" allows for a step-by-step learning curve.

My proposal:

• "Grouping by Topic Type" (as it is in current 4.1.3-SNAPSHOT).
• Enrich the listing by association type (and possibly role type?) information.

Open questions:

• How could this enriched listing look like?
• Besides the Association Type do we want reveal the association's Role Type(s) as well?

[jri]

Thanks, JuergeN, for the constructive phone talk!
I think we found good solutions.

Here I summarize our decisions and pose open questions.

Displaying the association types in the related topics listing

• The related topics listing is grouped by Topic Type (as it is in the current SNAPSHOT)
• Each list item (topic icon + label) is appended with the name of the respective association type in parenthesis:

  icon topiclabel (assoctype)
  

• The association type is a link that reveals the related topic (along with the association) and selects the association.

I'll realize it this weekend.

Dealing with redundancies (see ticket description)

• In the related topics listing suppress topics which represent information displayed in the (structured) upper area already.
• Revealing suppressed topics on the topicmap is still possible by enhancing the upper area: aggregated child topics are links that reveal and select the respective topic. In contrast compositioned child topics are not displayed as links (revealing them is rather pointless).

Open questions:

• How to display the link in case of a composite child, e.g. an "Address". Note: in the (structured) upper area there is no "Address" text per-se which could act as link (only simple values appear as text) but a grey box container (which represents the composite).
• Should only the direct childs be displayed as links in the (structured) upper area? This would be the functional equivalent to the current (redundant) topic listing which lists only the directly related topics. -- Or should nested childs be displayed as links as well? What happens when such a link is clicked? Note that this child is not directly related but there is a number of intermediate topics. Should the entire path (topics + associations) be revealed on the canvas?

This remains to be discussed.

As an alternative solution we could leave the (structured) upper area as it is (no links) and in the related topics listing suppress only the compositioned childs but not the aggregated ones. This would allow revealing the child on the topicmap (as it is now) and still reduce the redundancy to a certain amount.

[Jörg Richter]

Webclient: extend REST API (#315).

Core API:

The Type interface has a new method:

boolean hasAssocDef(String childTypeUri)

Webclient REST API:

A new service call requests a topic's related topics excluding the "property topics":

GET /webclient/topic/{id}/related_topics

"Property topics" are child topics that adhere to a parent topic's "Composition" definition.

In preparation of suppressing redundant topics in the detail panel's related topics listing (see #315).

• Changeset: 54b704fd81717f71fefab75fda6714c60296c484
• URL: ​https://github.com/jri/deepamehta/commit/54b704fd81717f71fefab75fda6714c60296c484

[Jörg Richter]

Suppress redundant topics in detail panel (#315).

Redundant topics are suppressed in the detail panel's related topics listing. A topic is regarded redundant if 1) it is already represented in the detail panel's upper area (the "model-driven box structure"), and 2) there is a "Composition Definition" in the underlying model.

Core API:

The Association interface has a new method:

boolean isPlayer(TopicRoleModel roleModel)

The REST Client has 2 new methods:

dm4c.restc.get_related_topics(topic_id, sort)
dm4c.restc.sort_topics(topics)

See #315.

• Changeset: 262ab4f5d169c0b0085caaaf64714438cb5d8326
• URL: ​https://github.com/jri/deepamehta/commit/262ab4f5d169c0b0085caaaf64714438cb5d8326

[Jörg Richter]

Webclient: show assoc type in detail panel (#315).

In the detail panel's related topics listing for each topic the respective association type is shown beneath the topic.

Bug fix: tooltips in File Browser's "Folder Content" listing.

DEVELOPER NOTES

Core serialization format (server-side):

• Revert change as of Nov 3 (0c7423ca): a RelatedTopic?'s association part *is* serialized (again).

So the serialized JSON object *has* an "assoc" property (again).

Webclient API (client-side):

• The Topic constructor adopts the underlying object's "assoc" property as well

• New convenience method:

  dm4c.association_type_name(type_uri)
  

BREAKING CHANGE

• Webclient convenience method renamed:

  dm4c.type_label(type_uri) -> dm4c.topic_type_name(type_uri)
  

See #315.

• Changeset: c6c0e09274668e5f201a36ed112a224de225a4a0
• URL: ​https://github.com/jri/deepamehta/commit/c6c0e09274668e5f201a36ed112a224de225a4a0

[Jörg Richter]

Webclient: extend REST API (#315).

Core API:

The Type interface has a new method:

boolean hasAssocDef(String childTypeUri)

Webclient REST API:

A new service call requests a topic's related topics excluding the "property topics":

GET /webclient/topic/{id}/related_topics

"Property topics" are child topics that adhere to a parent topic's "Composition" definition.

In preparation of suppressing redundant topics in the detail panel's related topics listing (see #315).

• Changeset: 54b704fd81717f71fefab75fda6714c60296c484
• URL: ​https://github.com/jri/deepamehta/commit/54b704fd81717f71fefab75fda6714c60296c484

[Jörg Richter]

Suppress redundant topics in detail panel (#315).

Redundant topics are suppressed in the detail panel's related topics listing. A topic is regarded redundant if 1) it is already represented in the detail panel's upper area (the "model-driven box structure"), and 2) there is a "Composition Definition" in the underlying model.

Core API:

The Association interface has a new method:

boolean isPlayer(TopicRoleModel roleModel)

The REST Client has 2 new methods:

dm4c.restc.get_related_topics(topic_id, sort)
dm4c.restc.sort_topics(topics)

See #315.

• Changeset: 262ab4f5d169c0b0085caaaf64714438cb5d8326
• URL: ​https://github.com/jri/deepamehta/commit/262ab4f5d169c0b0085caaaf64714438cb5d8326

[Jörg Richter]

Webclient: show assoc type in detail panel (#315).

In the detail panel's related topics listing for each topic the respective association type is shown beneath the topic.

Bug fix: tooltips in File Browser's "Folder Content" listing.

DEVELOPER NOTES

Core serialization format (server-side):

• Revert change as of Nov 3 (0c7423ca): a RelatedTopic?'s association part *is* serialized (again).

So the serialized JSON object *has* an "assoc" property (again).

Webclient API (client-side):

• The Topic constructor adopts the underlying object's "assoc" property as well

• New convenience method:

  dm4c.association_type_name(type_uri)
  

BREAKING CHANGE

• Webclient convenience method renamed:

  dm4c.type_label(type_uri) -> dm4c.topic_type_name(type_uri)
  

See #315.

• Changeset: c6c0e09274668e5f201a36ed112a224de225a4a0
• URL: ​https://github.com/jri/deepamehta/commit/c6c0e09274668e5f201a36ed112a224de225a4a0

[jri]

Replying to Jörg Richter:

Redundant topics are suppressed in the detail panel's related topics listing.

This applies only to the topic-related-topics so far.
It is still missing for the association-related-topics, that is when an association is selected.

The "Topic Mapcontext" associations could be used as a test case.

[Jörg Richter]

Reveal direct childs via page panel (#315).

The direct child topics of the selected topic (which are not appearing anymore in the page panel's related topics listing) can be revealed via the page panel's upper area (which shows the nested boxes): when hovered the corresponding box it is highlighted and a pointer cursor appears. On click the underlying topic is revealed in the map.

See #315.

• Changeset: aa6a30ee47b829aac7b1bdf5a98ed691002fd724
• URL: ​https://github.com/jri/deepamehta/commit/aa6a30ee47b829aac7b1bdf5a98ed691002fd724

[Malte]

The latest feature introduced here, revealing a topic from within the Page Panel, when I do this on a "Search Result"-Box, the server throws an error, at client-side nothing happens (see update below).

Me personally, I find that this new "reveal underlying topic" feature now receives visually/spatially much to much attention/occupies to much space of my Page Panel. One complex topics I can barely click somewhere else and not reveal a new topic. I would favour a solution which just occupies space at the right border of such a box.

When the whole box is highlighted, I think more of "inline editing" just that specific part of a topic.

SCHWERWIEGEND: A DeepaMehta resource method or event listener threw an exception resp. an error occurred. Mapping exception/error to response: 500 (Internal Server Error).
java.lang.RuntimeException: Fetching topic -1 failed
	at de.deepamehta.core.impl.EmbeddedService.getTopic(EmbeddedService.java:102)
	at de.deepamehta.plugins.webservice.WebservicePlugin.getTopic(WebservicePlugin.java:56)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
	at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
	at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
	at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.ResourceObjectRule.accept(ResourceObjectRule.java:100)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1480)
	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1411)
	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1360)
	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1350)
	at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
	at org.apache.felix.http.base.internal.handler.ServletHandler.doHandle(ServletHandler.java:96)
	at org.apache.felix.http.base.internal.handler.ServletHandler.handle(ServletHandler.java:79)
	at org.apache.felix.http.base.internal.dispatch.ServletPipeline.handle(ServletPipeline.java:42)
	at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:49)
	at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:33)
	at org.apache.felix.http.base.internal.dispatch.FilterPipeline.dispatch(FilterPipeline.java:48)
	at org.apache.felix.http.base.internal.dispatch.Dispatcher.dispatch(Dispatcher.java:39)
	at org.apache.felix.http.base.internal.DispatcherServlet.service(DispatcherServlet.java:67)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
	at org.mortbay.jetty.Server.handle(Server.java:326)
	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
	at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:926)
	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
	at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: org.neo4j.graphdb.NotFoundException: Node[-1]
	at org.neo4j.kernel.InternalAbstractGraphDatabase.getNodeById(InternalAbstractGraphDatabase.java:858)
	at de.deepamehta.storage.neo4j.Neo4jStorage.fetchNode(Neo4jStorage.java:969)
	at de.deepamehta.storage.neo4j.Neo4jStorage.fetchTopicNode(Neo4jStorage.java:955)
	at de.deepamehta.storage.neo4j.Neo4jStorage.fetchTopic(Neo4jStorage.java:115)
	at de.deepamehta.core.impl.StorageDecorator.fetchTopic(StorageDecorator.java:48)
	at de.deepamehta.core.impl.EmbeddedService.getTopic(EmbeddedService.java:100)
	... 43 more

Update: On client-side stuff does happen after I accidentially revealed a non-existing topic.

GET http://localhost:8080/core/topic/-1 500 (Internal Server Error) jquery-2.0.3.min.js:6
x.support.cors.e.crossDomain.send jquery-2.0.3.min.js:6
x.extend.ajax jquery-2.0.3.min.js:6
request rest_client.js:312
RESTClient.get_topic_by_id rest_client.js:11
fetch_topic webclient.js:1371
do_reveal_related_topic webclient.js:168
(anonymous function) page_model.js:292
x.event.dispatch jquery-2.0.3.min.js:5
y.handle jquery-2.0.3.min.js:5
Uncaught RESTClientError: GET request failed (error: Internal Server Error) 

resulting in a "frozen" topicmap (one which cannot be panned anymore).

[jri]

Replying to Malte:

The latest feature introduced here, revealing a topic from within the Page Panel, when I do this on a "Search Result"-Box, the server throws an error, at client-side nothing happens (see update below).

Thank you very much for reporting!
This error will be fixed very soon.

Me personally, I find that this new "reveal underlying topic" feature now receives visually/spatially much to much attention/occupies to much space of my Page Panel.

Sorry, but space occupation is exactly zero.

One complex topics I can barely click somewhere else and not reveal a new topic.

Why should you click in the page panel's upper area when you want *not* reveal a topic?
Note: text selection is still possible.

I would favour a solution which just occupies space at the right border of such a box.

How exactly would this look like?

When the whole box is highlighted, I think more of "inline editing" just that specific part of a topic.

I admit the current solution is not perfectly nice. The objective for this new "feature" was to bring back the functionality that was there already before "redundant" topics are suppressed from the page panel's "What's related" listing. In fact DM's web browsing feature (based on revealing a Web Resource's URL topic) was paralyzed by that change. Now it is usable again.

Concrete suggestions for improvement are very welcome.

[Jörg Richter]

Webclient fix: revealing search results (#315).

Revealing topics from a search result works again.
Thanks, Malte, for reporting!

When hovered the Search Result field is not highlighted by a blue frame anymore.

Its more difficult to corrupt Topicmaps. Adding invalid associations (a player topic is missing) fail early. The topicmap remains usable. Client side errors like

Uncaught TypeError: Cannot read property 'x' of undefined

should not occur anymore.
This applies to Topicmaps created from now.
If you have a corrupt topicmap (which does not render anymore) please tell me. I will send you repair instructions.

See #315.

• Changeset: aacf2809a5c625a03485dbf35788e503304bcac4
• URL: ​https://github.com/jri/deepamehta/commit/aacf2809a5c625a03485dbf35788e503304bcac4

[Jörg Richter]

Page Panel: topic labels are not truncated (#315).

Long topic labels in the detail panel's "related topic" lists are not truncated anymore.
Formerly there was a limit of 50 characters.

See #315.

• Changeset: f46f222574719dd2f318c533624b117984e154cc
• URL: ​https://github.com/jri/deepamehta/commit/f46f222574719dd2f318c533624b117984e154cc

[Jörg Richter]

Fix suppression of redundant topics (#315).

Topics are regarded as redundant also in case of an Aggregation Definition (not only Composition Definition).

Semantics of the GET /webclient/topic/{id}/related_topics request has changed accordingly.

So the wording "Property Topic" in conjunction with this request is not applicable anymore.

See #315

• Changeset: 8e67f0ddcd87d8acf204fa0f6363dd76f1557ab2
• URL: ​https://github.com/jri/deepamehta/commit/8e67f0ddcd87d8acf204fa0f6363dd76f1557ab2

[jri]

See follow-up #1070