once geo-indexed dbs cannnot be migrated to 4.5

[Malte]

This is not crucial, since i did not index my production db yet but we need to find a way to prevent a similar error in the future.

INFORMATION: ########## Initializing workspace properties
Dez 10, 2015 8:29:31 PM de.deepamehta.core.impl.PluginImpl installPluginInDB
WARNUNG: ROLLBACK! (plugin "DeepaMehta 4 Workspaces")
Dez 10, 2015 8:29:31 PM de.deepamehta.core.impl.PluginImpl$1 addingService
SCHWERWIEGEND: Adding service de.deepamehta.plugins.facets.service.FacetsService to plugin "DeepaMehta 4 Workspaces" failed
java.lang.RuntimeException: Activation of plugin "DeepaMehta 4 Workspaces" failed
	at de.deepamehta.core.impl.PluginImpl.activate(PluginImpl.java:515)
	at de.deepamehta.core.impl.PluginManager.activatePlugin(PluginManager.java:64)
	at de.deepamehta.core.impl.PluginImpl.checkRequirementsForActivation(PluginImpl.java:481)
	at de.deepamehta.core.impl.PluginImpl.addService(PluginImpl.java:433)
	at de.deepamehta.core.impl.PluginImpl.access$000(PluginImpl.java:47)
	at de.deepamehta.core.impl.PluginImpl$1.addingService(PluginImpl.java:373)
	at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:932)
	at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:864)
	at org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked.java:256)
	at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:229)
	at org.osgi.util.tracker.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:894)
	at org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:943)
	at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:794)
	at org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:544)
	at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4445)
	at org.apache.felix.framework.Felix.registerService(Felix.java:3431)
	at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346)
	at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:320)
	at de.deepamehta.core.impl.PluginImpl.registerPluginService(PluginImpl.java:705)
	at de.deepamehta.core.impl.PluginImpl.activate(PluginImpl.java:505)
	at de.deepamehta.core.impl.PluginManager.activatePlugin(PluginManager.java:64)
	at de.deepamehta.core.impl.PluginImpl.checkRequirementsForActivation(PluginImpl.java:481)
	at de.deepamehta.core.impl.PluginImpl.addService(PluginImpl.java:424)
	at de.deepamehta.core.impl.PluginImpl.access$000(PluginImpl.java:47)
	at de.deepamehta.core.impl.PluginImpl$1.addingService(PluginImpl.java:373)
	at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:932)
	at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:864)
	at org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked.java:256)
	at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:229)
	at org.osgi.util.tracker.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:894)
	at org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:943)
	at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:794)
	at org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:544)
	at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4445)
	at org.apache.felix.framework.Felix.registerService(Felix.java:3431)
	at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346)
	at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:320)
	at de.deepamehta.core.osgi.CoreActivator.checkRequirementsForActivation(CoreActivator.java:153)
	at de.deepamehta.core.osgi.CoreActivator.addService(CoreActivator.java:128)
	at de.deepamehta.core.osgi.CoreActivator.access$000(CoreActivator.java:20)
	at de.deepamehta.core.osgi.CoreActivator$1.addingService(CoreActivator.java:93)
	at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:932)
	at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:864)
	at org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked.java:256)
	at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:229)
	at org.osgi.util.tracker.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:894)
	at org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:943)
	at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:794)
	at org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:544)
	at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4445)
	at org.apache.felix.framework.Felix.registerService(Felix.java:3431)
	at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346)
	at org.ops4j.pax.web.service.internal.Activator.updateController(Activator.java:345)
	at org.ops4j.pax.web.service.internal.Activator$3.run(Activator.java:285)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Installing plugin "DeepaMehta 4 Workspaces" in the database failed
	at de.deepamehta.core.impl.PluginImpl.installPluginInDB(PluginImpl.java:551)
	at de.deepamehta.core.impl.PluginImpl.activate(PluginImpl.java:502)
	... 58 more
Caused by: java.lang.RuntimeException: Running migration 5 of plugin "DeepaMehta 4 Workspaces" failed
	at de.deepamehta.core.impl.MigrationManager.runMigration(MigrationManager.java:147)
	at de.deepamehta.core.impl.MigrationManager.runPluginMigration(MigrationManager.java:91)
	at de.deepamehta.core.impl.MigrationManager.runPluginMigrations(MigrationManager.java:59)
	at de.deepamehta.core.impl.PluginImpl.installPluginInDB(PluginImpl.java:540)
	... 59 more
Caused by: org.neo4j.graphdb.NotFoundException: 'node_type' property not found for NodeImpl#1684424.
	at org.neo4j.kernel.impl.core.Primitive.newPropertyNotFoundException(Primitive.java:184)
	at org.neo4j.kernel.impl.core.Primitive.getProperty(Primitive.java:179)
	at org.neo4j.kernel.impl.core.NodeImpl.getProperty(NodeImpl.java:52)
	at org.neo4j.kernel.impl.core.NodeProxy.getProperty(NodeProxy.java:155)
	at de.deepamehta.storage.neo4j.NodeType.isTypeOf(NodeType.java:50)
	at de.deepamehta.storage.neo4j.ModelIterator.fetchNextNode(ModelIterator.java:92)
	at de.deepamehta.storage.neo4j.ModelIterator.hasNext(ModelIterator.java:65)
	at de.deepamehta.core.impl.ObjectIterator.hasNext(DeepaMehtaObjectIterable.java:110)
	at de.deepamehta.plugins.workspaces.migrations.Migration5.run(Migration5.java:43)
	at de.deepamehta.core.impl.MigrationManager.runMigration(MigrationManager.java:139)
	... 62 more

I think that this node probably shouldn't be fetched but it raises another question: Is this geospatial neo4j node under DeepaMehta authority or not? If so, it might be relevant to assign it to some workspace, no?

Note: I am just assuming everything here i have no evidence about the claim made in the title of the ticket since i could/did not inspect that very neo4jnode which seems wrong here. It could be also a dm4-typed node gone wrong but when updating the same database (non-geo-indexed) no such error occurs at all.

[Malte]

Same exception occurs (now thrown by Migration8 of the DeepaMehta 4 Workspaces plugin) when trying to update a deepamehta-db from 4.7 to 4.8 (which has a geospatial index layer). This is still not crucial but i am sure i will stumble upon this again at some point. It would be great if we could change this and maybe it is better to extend the geospatial index topic about a valid "node_type" value? I think this would also be a legit solution since it indexes "dm4.geomaps.geo_coordinate" topics. What do you think?

[Jörg Richter <jri@…>]

In 4f0e76a6bdaeec664cb34fdde8335dd02babb424/deepamehta:

dm4-storage-neo4j: deal with non-DM nodes (#911).

The CoreService?'s get-all methods doesn't throw if non-DM nodes exist in the DB.
Non-DM nodes are those created by 3rd-party Neo4j components, e.g. Neo4j Spatial.
A node is regarded "non-DM" if it has no "node_type" property.

dm4.getAllTopics()
dm4.getAllAssociations()

dm4.addTopicPropertyIndex()
dm4.addAssociationPropertyIndex()

Thanks to Malte for reporting!

See #911.

[jri]

Hi Malte, can you please test if migration is now working?
That would be very nice :-)

[Malte]

Hi, yes, sure.

So, i initialized a kiezatlas db and populated the extra, spatial index with 4.7. After that i tried to upgrade that db to 4.8.1-SNAPSHOT and .. the workspace migration ran through fine. Thank you.

[jri]

Great!
Thank you very much!