"By-text" search doesn't work with special characters

[joern]

When trying to search for e.g. latex-code, the webclient returns an error in some cases.

Example query:

$z_{1}^{{2}=\left(2\,\exp\left(i\,\frac{\pi}{4}\right)\right)}{2}=2^{{2}\,\left(\exp\left(i\,\frac{\pi}{4}\right)\right)}{2}=4\,\exp\left(2\, i\,\frac{\pi}{4}\right)=4\,\exp\left(i\,\frac{\pi}{2}\right)$

Error Msg:

ERROR while searching:

"RESTClientError: GET request failed (error: Internal Server Error)"

[jri]

Puh, that's a wild search string ;-)
It distracts the Lucene (the search engine utilized by DM) query parser considerably.

In your search term you have to escape these characters when you want to search for them literally:

+  -  &&  ||  !  (  )  {  }  [  ]  ^  "  ~  *  ?  :  \

These characters have special meanings for Lucene:
​http://lucene.apache.org/core/old_versioned_docs/versions/3_0_1/queryparsersyntax.html

Escaping works via backslash \ character.
So, when you want search for e.g. curly brace { you have to write it as

\{

To search for backslash \ itself:

\\

Also consider using "" when searching for phrases.

NOTE: the DM webclient passes the search term you entered almost literally to Lucene. Thus, you can e.g. work with wildcards and boolean operators. However, there is one exception: DM adds a * to your search term. This is intended to make begin-of-word searches easy.

To avoid the search problem you described DM could automatically escape all special characters. On the other hand this would prohibit the advanced query possibilities offered by Lucene. How to cope with these contradicting conditions and how to improve the Webclient's search GUI remains to be discussed. For the moment resort to escaping.

Thanks for your report!

[Jörg Richter]

Core fix: Search with Lucene query syntax (#302).

In particular phrase search ("..") and escaping (\) are working.

Note: the Webclient's default search is now a whole-word search.
For a begin-of-word search you have to add the asterisk * manually, e.g. Hell*

IMPORTANT for plugin developers: the "wholeword" parameter is globally dropped.

Core Service Java API:

• dms.searchTopics() has no "wholeWord" parameter anymore

Core REST API:

• GET /core/topic expects no "wholeword" query parameter anymore

RESTClient:

• dm4c.restc.search_topics() has no "whole_word" and no "index" parameters anymore

Webclient REST API:

• GET /webclient/search expects no "wholeword" query parameter anymore

RESTClient:

• dm4c.restc.search_topics_and_create_bucket() has no "whole_word" parameter anymore

See ticket 302.

• Changeset: 9bb9ec244a7b6ff8a3fdebfbd19b69af334d7e18
• URL: ​https://github.com/jri/deepamehta/commit/9bb9ec244a7b6ff8a3fdebfbd19b69af334d7e18

[joern]

I found another huge problem with the search. This probably just emerged with the update:

I get Rest API errors when searching for "komplexe zahlen", "komplex" and "zahlen". Even more strange, a query for "komplexezahlen" works fine. I changed the Ticket to critical, because we will need to use the search extensively.

[Jörg Richter]

Deploy Neo4j MehtaGraph? 1.1-SNAPSHOT (#302).

The parent POM is updated to version 4 in order to deploy a fixed Neo4j MehtaGraph?.

This error doesn't appear anymore when searching:

java.lang.RuntimeException: Searching topics failed (searchTerm="komplexe", fieldUri="null", clientState={dm4_topicmap_id=10631, JSESSIONID=17vsf61a7y90guti3xxblerxc, dm4_workspace_id=9676})
	at de.deepamehta.core.impl.service.EmbeddedService.searchTopics(EmbeddedService.java:145)
	at de.deepamehta.plugins.webclient.WebclientPlugin.searchTopics(WebclientPlugin.java:79)
	... 47 more
Caused by: java.lang.IllegalArgumentException: Reference error: ID 76173 refers to a MehtaEdge when the caller expects a MehtaNode
    at de.deepamehta.mehtagraph.impl.Neo4jBase.buildMehtaNode(Neo4jBase.java:71)
    at de.deepamehta.mehtagraph.impl.Neo4jMehtaGraph.queryMehtaNodes(Neo4jMehtaGraph.java:113)
    at de.deepamehta.core.impl.storage.MGStorageBridge.searchTopics(MGStorageBridge.java:143)
    at de.deepamehta.core.impl.service.EmbeddedService.searchTopics(EmbeddedService.java:140)

At client-side this error appears as

ERROR while searching:

"RESTClientError: GET request failed (error: Internal Server Error)"

As a workaround MehtaEdge? results are just dropped.
The underlying problem remains to be investigated.

See ticket 302.

• Changeset: be54394b07843d068f5ff2dd3a4ea3950b1686d2
• URL: ​https://github.com/jri/deepamehta/commit/be54394b07843d068f5ff2dd3a4ea3950b1686d2

[jri]

The Search should now be fixed on the EduZEN installation.
Please check it out.