Ticket #391 (closed Enhancement: fixed)

Opened 8 years ago

Last modified 8 years ago

Extract MehtaGraph SPI and provide default implementation

Reported by: jri Owned by: jri
Priority: Major Milestone: Release 4.1
Component: DeepaMehta Standard Distribution Version: 4.0.13
Keywords: Cc: dgf, Malte
Complexity: 8 Area:
Module: deepamehta-core

Description

The MehtaGraph? interfaces should be extracted from Neo4j MehtaGraph? in order to allow 3rd-party implementations.

Change History

comment:1 Changed 8 years ago by jri

  • Status changed from new to accepted

comment:3 Changed 8 years ago by Jörg Richter

comment:4 Changed 8 years ago by Jörg Richter

Remove Neo4j Mehtagraph dependency (#391).

Instead 2 new bundles are included in the Standard Distribution:

  • deepamehta-mehtagraph # The API
  • deepamehta-mehtagraph-neo4j # The default implementation

See ticket 391.

comment:6 Changed 8 years ago by Jörg Richter

comment:7 Changed 8 years ago by Jörg Richter

Remove Neo4j Mehtagraph dependency (#391).

Instead 2 new bundles are included in the Standard Distribution:

  • deepamehta-mehtagraph # The API
  • deepamehta-mehtagraph-neo4j # The default implementation

See ticket 391.

comment:8 Changed 8 years ago by Jörg Richter

Core: dynamic MehtaGraph? instantiation (#391).

The core has no static dependency to the MehtaGraph? implementation.

2 new config properties:

dm4.database.bundle -- symbolic name of the MehtaGraph? implementation bundle
dm4.database.factory -- class name of the MehtaGraph? implementation factory

See ticket 391.

comment:10 Changed 8 years ago by Jörg Richter

Move MehtaGraph? interface to Core (#391).

Module deepamehta-mehtagraph is dropped.

Content of package

de.deepamehta.mehtagraph

is moved to

de.deepamehta.core.storage

Background: in order to reduce the bridging effort the MehtaGraph? API
is about to use de.deepamehta.core.model classes directly.

See ticket 391.

comment:11 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.1 (#391).

The storage bridge is about to be dropped completely.

See ticket 391.

comment:12 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.2 (#391).

Does not yet compile.

API Changes:

  • Class AssociationModel?
    • Add getOtherRoleModel(long id) convenience method.
  • Class ResultSet?
    • Add a default constructor to construct an empty result set.
    • Add addAll() method to combine result sets.

BREAKING CHANGES:

  • Interface AssociationRole?
    • replace getAssociationId() by getPlayerId()
  • Interface TopicRole?
    • replace getTopicId() by getPlayerId()

See ticket 391.

comment:13 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.3 (#391).

Does not yet compile.

BREAKING CHANGES:

See ticket 391.

comment:14 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.4 (#391).

Does not yet compile.

See ticket 391.

comment:15 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.5 (#391).

Deleting a lot of bridging code.
The MehtaGraph? API is about to be dropped.

Does not yet compile.

See ticket 391.

comment:16 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.6 (#389, #391).

Introduce association index.
Does not yet compile.

See ticket 389.
See ticket 391.

comment:17 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.7 (#391).

Does not yet compile.

See ticket 391.

comment:18 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.8 (#391).

Does not yet compile.

See ticket 391.

comment:19 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.9 (#391).

The Core compiles! For the first time since 3 days. Yippie!

Not yet functional.

See ticket 391.

comment:20 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.10 (#391).

The Standard Distro compiles. At least with -D maven.test.skip
Not yet functional.

BREAKING CHANGES

  • Core's 6-argument AssociationModel? constructor got a 7th parameter: "uri". However, this constructor is rarely used by application developers.

See ticket 391.

comment:21 Changed 8 years ago by Jörg Richter

Core: eventually drop MehtaGraph? API (#391).

The concept of a DM-independant MehtaGraph? API is abolished. That intermediate storage layer is now dropped.
Instead DM storage implementations are coded to the DM model API directly.
The remaining "MehtaGraph?"/"MG" wording in class names etc. will be dropped soon.

Neo4j traversal framework is not used anymore as it suffers from the super node performance problem.
Instead further Lucene indexes are deployed.

Not yet functional.

BREAKING CHANGES

Core: interface DeepaMehtaTransaction? has moved.
Application developers must change their imports from

    import de.deepamehta.core.DeepaMehtaTransaction

to

    import de.deepamehta.core.storage.spi.DeepaMehtaTransaction

See ticket 391.

comment:22 Changed 8 years ago by Jörg Richter

Repackage/rename core and storage classes (#391).

  • Combine packages de.deepamehta.core.impl.[service|storage] just into "de.deepamehta.core.impl"

Neo4j storage implementation:

  • Rename "dm4-mehtagraph-neo4j" module to "dm4-storage-neo4j"
  • Rename "deepamehta-mehtagraph-neo4j" artifact to "deepamehta-storage-neo4j"
  • Rename "de.deepamehta.mehtagraph.neo4j" package to "de.deepamehta.storage.neo4j"
  • Rename "Neo4jMehtaGraph" class to "Neo4jStorage"

See ticket 391.

comment:23 Changed 8 years ago by Jörg Richter

Consistent storage method wording (#391).

In the DeepaMehtaStorage? interface the "MehtaGraph/MehtaNode?..." method wording is now retired.

The StorageDecorator? method wording is consistent with DeepaMehtaStorage?.

See ticket 391.

comment:24 Changed 8 years ago by Jörg Richter

Core: bidirectional assoc indexing (#389, #391).

The new association metadata index can be queried "from both sides".
See ticket 389.

First tests of the Neo4j storage implementation are adpapted to the new storage layer API.
See ticket 391.

comment:25 Changed 8 years ago by Jörg Richter

Core: bidirectional assoc indexing (#389, #391).

The new association metadata index can be queried "from both sides".
See ticket 389.

First tests of the Neo4j storage implementation are adpapted to the new storage layer API.
See ticket 391.

comment:26 Changed 8 years ago by Jörg Richter

Core: providing storage as OSGi service (#391).

Instead of instantiating through the factory the Core consumes the storage implementation as an OSGi service. A storage implmentation is expected to register DeepaMehtaStorage? as an OSGi service.
The factory remain to exist in order to run the tests. While the test phase the OSGi environment is not running.

2 recently introduced config properties are dropped:

  • dm4.database.bundle
  • dm4.database.factory

Note: the storage can *not* be hot-deployed. Uninstalling the storage will not stop the Core. This is because the Core has technically no package dependency on the storage. The Core just depends on the storage service (DeepaMehtaStorage?) which is defined in the Core package itself.
While being still a bundle the storage implementation is not a DeepaMehta plugin. It is not derived from PluginActivator? (but from BundleActivator?). The storage implementation provides a basis service required by the Core to run. Once the Core is up and running it acts as container for the various DM plugins.

See ticket 391.

comment:27 follow-up: ↓ 28 Changed 8 years ago by Jörg Richter

Core: move tests to new dm4-test module (#391).

The circular (test scope) dependency between Core and Storage is resolved.

The Neo4j Storage module is removed from the hot-deploy list.
The Neo4j Storage module is included in the reactor build.

A reactor build with a pristine Maven repo works.
Thanks dgf for advise!

See ticket 391.

comment:28 in reply to: ↑ 27 Changed 8 years ago by jri

Replying to Jörg Richter:

The Neo4j Storage module is removed from the hot-deploy list.

Thats outdated. The Storage module IS hot-deployed.

comment:29 Changed 8 years ago by Jörg Richter

Core fix: indexing of HTML fields (#391).

For HTML topics: the HTML tags are stripped before indexing.

BREAKING CHANGES
(for Storage implementers only)

DeepaMehtaStorage? API:

  • storeTopicValue() has additional "indexValue" parameter (optional)
  • storeAssociationValue() has additional "indexValue" parameter (optional)

So, there are separate 2 values provided: the value to be *stored* and the value to be *indexed*.
The latter one is optional. If not specified (null) the former one is used for both operations.

See ticket 391.

comment:31 Changed 8 years ago by Jörg Richter

comment:32 Changed 8 years ago by Jörg Richter

Remove Neo4j Mehtagraph dependency (#391).

Instead 2 new bundles are included in the Standard Distribution:

  • deepamehta-mehtagraph # The API
  • deepamehta-mehtagraph-neo4j # The default implementation

See ticket 391.

comment:33 Changed 8 years ago by Jörg Richter

Core: dynamic MehtaGraph? instantiation (#391).

The core has no static dependency to the MehtaGraph? implementation.

2 new config properties:

dm4.database.bundle -- symbolic name of the MehtaGraph? implementation bundle
dm4.database.factory -- class name of the MehtaGraph? implementation factory

See ticket 391.

comment:35 Changed 8 years ago by Jörg Richter

Move MehtaGraph? interface to Core (#391).

Module deepamehta-mehtagraph is dropped.

Content of package

de.deepamehta.mehtagraph

is moved to

de.deepamehta.core.storage

Background: in order to reduce the bridging effort the MehtaGraph? API
is about to use de.deepamehta.core.model classes directly.

See ticket 391.

comment:36 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.1 (#391).

The storage bridge is about to be dropped completely.

See ticket 391.

comment:37 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.2 (#391).

Does not yet compile.

API Changes:

  • Class AssociationModel?
    • Add getOtherRoleModel(long id) convenience method.
  • Class ResultSet?
    • Add a default constructor to construct an empty result set.
    • Add addAll() method to combine result sets.

BREAKING CHANGES:

  • Interface AssociationRole?
    • replace getAssociationId() by getPlayerId()
  • Interface TopicRole?
    • replace getTopicId() by getPlayerId()

See ticket 391.

comment:38 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.3 (#391).

Does not yet compile.

BREAKING CHANGES:

See ticket 391.

comment:39 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.4 (#391).

Does not yet compile.

See ticket 391.

comment:40 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.5 (#391).

Deleting a lot of bridging code.
The MehtaGraph? API is about to be dropped.

Does not yet compile.

See ticket 391.

comment:41 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.6 (#389, #391).

Introduce association index.
Does not yet compile.

See ticket 389.
See ticket 391.

comment:42 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.7 (#391).

Does not yet compile.

See ticket 391.

comment:43 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.8 (#391).

Does not yet compile.

See ticket 391.

comment:44 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.9 (#391).

The Core compiles! For the first time since 3 days. Yippie!

Not yet functional.

See ticket 391.

comment:45 Changed 8 years ago by Jörg Richter

Core: refactor storage layer, pt.10 (#391).

The Standard Distro compiles. At least with -D maven.test.skip
Not yet functional.

BREAKING CHANGES

  • Core's 6-argument AssociationModel? constructor got a 7th parameter: "uri". However, this constructor is rarely used by application developers.

See ticket 391.

comment:46 Changed 8 years ago by Jörg Richter

Core: eventually drop MehtaGraph? API (#391).

The concept of a DM-independant MehtaGraph? API is abolished. That intermediate storage layer is now dropped.
Instead DM storage implementations are coded to the DM model API directly.
The remaining "MehtaGraph?"/"MG" wording in class names etc. will be dropped soon.

Neo4j traversal framework is not used anymore as it suffers from the super node performance problem.
Instead further Lucene indexes are deployed.

Not yet functional.

BREAKING CHANGES

Core: interface DeepaMehtaTransaction? has moved.
Application developers must change their imports from

    import de.deepamehta.core.DeepaMehtaTransaction

to

    import de.deepamehta.core.storage.spi.DeepaMehtaTransaction

See ticket 391.

comment:47 Changed 8 years ago by Jörg Richter

Repackage/rename core and storage classes (#391).

  • Combine packages de.deepamehta.core.impl.[service|storage] just into "de.deepamehta.core.impl"

Neo4j storage implementation:

  • Rename "dm4-mehtagraph-neo4j" module to "dm4-storage-neo4j"
  • Rename "deepamehta-mehtagraph-neo4j" artifact to "deepamehta-storage-neo4j"
  • Rename "de.deepamehta.mehtagraph.neo4j" package to "de.deepamehta.storage.neo4j"
  • Rename "Neo4jMehtaGraph" class to "Neo4jStorage"

See ticket 391.

comment:48 Changed 8 years ago by Jörg Richter

Consistent storage method wording (#391).

In the DeepaMehtaStorage? interface the "MehtaGraph/MehtaNode?..." method wording is now retired.

The StorageDecorator? method wording is consistent with DeepaMehtaStorage?.

See ticket 391.

comment:49 Changed 8 years ago by Jörg Richter

Core: bidirectional assoc indexing (#389, #391).

The new association metadata index can be queried "from both sides".
See ticket 389.

First tests of the Neo4j storage implementation are adpapted to the new storage layer API.
See ticket 391.

comment:50 Changed 8 years ago by Jörg Richter

Core: providing storage as OSGi service (#391).

Instead of instantiating through the factory the Core consumes the storage implementation as an OSGi service. A storage implmentation is expected to register DeepaMehtaStorage? as an OSGi service.
The factory remain to exist in order to run the tests. While the test phase the OSGi environment is not running.

2 recently introduced config properties are dropped:

  • dm4.database.bundle
  • dm4.database.factory

Note: the storage can *not* be hot-deployed. Uninstalling the storage will not stop the Core. This is because the Core has technically no package dependency on the storage. The Core just depends on the storage service (DeepaMehtaStorage?) which is defined in the Core package itself.
While being still a bundle the storage implementation is not a DeepaMehta plugin. It is not derived from PluginActivator? (but from BundleActivator?). The storage implementation provides a basis service required by the Core to run. Once the Core is up and running it acts as container for the various DM plugins.

See ticket 391.

comment:51 Changed 8 years ago by Jörg Richter

Core: move tests to new dm4-test module (#391).

The circular (test scope) dependency between Core and Storage is resolved.

The Neo4j Storage module is removed from the hot-deploy list.
The Neo4j Storage module is included in the reactor build.

A reactor build with a pristine Maven repo works.
Thanks dgf for advise!

See ticket 391.

comment:52 Changed 8 years ago by Jörg Richter

Core fix: indexing of HTML fields (#391).

For HTML topics: the HTML tags are stripped before indexing.

BREAKING CHANGES
(for Storage implementers only)

DeepaMehtaStorage? API:

  • storeTopicValue() has additional "indexValue" parameter (optional)
  • storeAssociationValue() has additional "indexValue" parameter (optional)

So, there are separate 2 values provided: the value to be *stored* and the value to be *indexed*.
The latter one is optional. If not specified (null) the former one is used for both operations.

See ticket 391.

comment:53 Changed 8 years ago by jri

  • Status changed from accepted to closed
  • Resolution set to fixed
Note: See TracTickets for help on using tickets.