Ticket #1089 (closed Feature Request: fixed)

Opened 4 years ago

Last modified 3 years ago

Identity-Value data model

Reported by: jri Owned by: jri
Priority: Major Milestone: Release 5.0
Component: DeepaMehta Standard Distribution Version: 4.8.8
Keywords: Cc: Malte, JuergeN, dgf, robert.schuster
Complexity: 13 Area: Data Model
Module: deepamehta-core

Description

A type is either an identy type or a value type.

The instances of value types are immutable.
The instances of identity types are updated in-place.

Value types can be simple or composite.
Identity types can be simple or composite.

Value types can have both, value child types, and identity child types.
Identity types can have both, value child types, and identity child types.

Examples:

"Address" is a value type. An address never changes. If a person's address changes the person is assigned to another (possibly newly created) address object.

"Person" is a identity type. If the address of a person changes, the person remains the same. It keeps its identity. No other person object is created.

Objectives:

  • Object sharing

True semantic objects: things like Names, Adresses, Dates, Times, Geo Coordinates, ... exist only once and are shared among many composites. They are never changed in-place. Updating arbitrary complex composite structures causes no semantic side effects.

  • Simpler frontend develoment

The data-update intelligence moves from client to server. The process how new data is integrated with existing data is governed by the server. The client fetches a composite object from the server, updates it (through client-side data-binding), and sends it back to the server. The client is not required to send IDs or refs. It just sends values.

Change History

comment:1 Changed 4 years ago by Jörg Richter <jri@…>

In d7acf9a57df597ed47de5643417610bdf1d5c266/deepamehta:

Meta model: add "Identity Type" (#1089)

See #1089

comment:2 Changed 4 years ago by Jörg Richter <jri@…>

In 00414ed23b34d86d98c3284e352c621762d6a2bf/deepamehta:

Revise DB bootstrap (#1089)

Topic type "Role Type" is created via regular migration.

See #1089

comment:3 Changed 4 years ago by Jörg Richter <jri@…>

comment:4 Changed 4 years ago by Jörg Richter <jri@…>

In 26585969cb07208afbaa15261c1489c56080dec6/deepamehta:

Unification of simple topics (#1089)

See #1089

comment:5 Changed 4 years ago by Malte

Hey, great to see this realized. One remark (which may be) of minor importance. When you state that:

True semantic objects: things like Names, Adresses, Dates, Times, Geo Coordinates, ... exist only once and are shared among many composites. They are never changed in-place. Updating arbitrary complex composite structures causes no semantic side effects.

I think it would be more correct to say that "... exist only once per workspace" as the same values will inevitably exist more than once, namely in different workspaces / collaborative scenarios / a.k.a SharingModes?.

The reference pool of values already known to the server are different for every users issueing a request to the server. Which, for frontend development would mean, that (the standard case is that a user is working in and member of many workspaces) the same value may exist in more than one workspace. Subsequently the client would still need to send an ID, e.g. a workspace_id so the server can determine which value to use/reference. Which is in contrast to your statement "The client is not required to send IDs or refs. It just sends values." The workspace_id is in any way required right now, so probably nothing will change but I just wanted to have this new way clearly described for frontend developers in advance here. Thanks for your time!

comment:6 Changed 4 years ago by jri

The client is not required to send IDs or refs.

IDs here refers to topic IDs in composite structures.

Thanks for you hint though.

comment:7 Changed 4 years ago by jri

Yes, the same topic can exist more than once, e.g. in private workspaces.
However this does not affect the new data model and update approach described here.
The Core update logic still runs under the request's authority, that is when checking if a certain topic already exists, the Core will not "see" any topic the user has no read permission for.
So, the Core would not share things the user have no permission for. This would make no sense.

Also the workspace approach is not affected by the new data model.
The workspace cookie is still required.

comment:8 Changed 4 years ago by jri

I think this is the way it is supposed to work:
When a user creates e.g. a Person in a private workspace and enter "Berlin" in its address, and a "Berlin" topic exists already in another workspace, where the user has READ permission, that "Berlin" topic will be shared in this case. The Person topic is private, the association to "Berlin" is private, but the "Berlin" topic is shared.

The workspace cookie affects only things being created (as it is now in DM4).

comment:9 Changed 4 years ago by jri

  • Cc robert.schuster added

comment:10 Changed 4 years ago by jri

Regarding frontend developer experience:
Lets say a Person with Berlin assigned exists already.
Currently in DM4 if a frontend shows a Person form, and Berlin is not changed by the user when submitting the form, the frontend is supposed to send the ID (ref_id:...) in the update request in order to refer to the existing Berlin topic. Finding out all the IDs puts a big burden on the frontend developer, and also brings the danger of unwanted side effects. When "Berlin" would be just send back as is (as a value instead of a ref ID), the existing "Berlin" topic would be overwritten with e.g. "Munich"! That is the city Berlin is renamed to "Munich"!

Finding out all the ref IDs at client-side is a hard task, in particular when it comes to referring to possibly existing composites like e.g. addresses. (This is why the longstanding problem of "aggregated composites", #337, was never solved in DM4.) Of course the frontend could send several requests just in order to find out if a particular street-postalcode-city-country combination exists already in the DB. But it is much more convenient for the frontend developer if the update logic is located at the server side. Thus all possible frontends -- current and future ones -- will benefit.

With the DM5 approach the frontend just sends the data (just values, no IDs) back to the server as received. This is the best possible frontend developer experience I can think of. And the problem of "aggregated composites" is eventually solved once and for all.

comment:11 Changed 4 years ago by Jörg Richter <jri@…>

In 583a6fe3615d9e8109f39cff584aee89178feaba/deepamehta:

Type editor: add "Identity Type" field (#1089)

See #1089

comment:12 Changed 4 years ago by jri

I have a question: are you thinking more Identity types do exist, or more Value types? I would like make this the default then.

comment:13 Changed 4 years ago by Jörg Richter <jri@…>

comment:14 Changed 4 years ago by Jörg Richter <jri@…>

In 07684c6a871bac30cab13efd63e5708641a41924/deepamehta:

Refactor migration manager exception handling (#1089)

See #1089

comment:15 Changed 4 years ago by Jörg Richter <jri@…>

In 1c2d5ceac2a4575439751cf8fc484b65ed7a8d35/deepamehta:

Topic type "Value Type" replaces "Identity Type" (#1089)

See #1089

comment:16 Changed 4 years ago by Jörg Richter <jri@…>

In f5b9275aaa23941854417343230cf8ecb3f2cde0/deepamehta:

Unification of simple identity topics (#1089)

See #1089

comment:17 Changed 4 years ago by Jörg Richter <jri@…>

In d3488286d1d1fedbf5152acdb8fac43df6f0831f/deepamehta:

Refactor persistence layer traversal (#1089)

See #1089

comment:18 Changed 4 years ago by Jörg Richter <jri@…>

In 0856e25158ad9891b37abc2b4ec9d2745790290f/deepamehta:

Update composite identity topics (#1089)

See #1089

comment:19 Changed 4 years ago by Jörg Richter <jri@…>

In b917831d89c6deb52f55860755733e9ad8a2638e/deepamehta:

Contacts data model: define value types (#1089)

These types are defined as value types:

"Person Name", "First Name", "Last Name"
"Institution Name",
"Phone", "Email Address",
"Address", "Street", "Postal Code", "City","Country"

Note that also composite types are value types ("Person Name", "Address").

IMPORTANT:
in order for the unification to work *simple* value types MUST have a "KEY" index.

See #1089

comment:20 Changed 4 years ago by Jörg Richter <jri@…>

In 0b1c3627adab24e1643092abdc85322762d300d3/deepamehta:

Date/Time? data model: define value types (#1089)

These types are now defined as value types:

"Date", "Month", "Day", "Year"
"Time", "Hour", "Minute"
"Date/Time?"

Note that also composite types (of any nesting depth) can be value types ("Date", "Time", "Date/Time?").

IMPORTANT

in order for the unification to work *simple* value types MUST have a "KEY" index.
Example (JSON migration):

{
  "value":          "Year",
  "uri":            "dm4.datetime.year",
  "dataTypeUri":    "dm4.core.number",
  "isValueType":    true,
  "indexModeUris": ["dm4.core.key"]
}

See #1089

comment:21 Changed 4 years ago by Jörg Richter <jri@…>

In c872029b04daa1ac1948f15798136c1ae8c79b99/deepamehta:

Update composite value types (#1089)

See #1089

comment:22 Changed 4 years ago by Jörg Richter <jri@…>

In 7c943d73225166dc0ac680bf79fbe00a37a41262/deepamehta:

Identity types: fix in-place child assignments (#1089)

See #1089

comment:23 Changed 4 years ago by Jörg Richter <jri@…>

In b152cc935e599ca95c7c0007bbc31f994c18a500/deepamehta:

Identity types: fix memory update (#1089)

See #1089

comment:24 Changed 4 years ago by Jörg Richter <jri@…>

comment:25 Changed 4 years ago by Jörg Richter <jri@…>

In abc527c476faef008b6f4d49660706d8cb701101/deepamehta:

Don't create empty simple value topics (#1089)

See #1089

comment:26 Changed 4 years ago by Jörg Richter <jri@…>

In ad9b595b1bd04480bd7916cc319f9ced900f21e7/deepamehta:

Don't create empty simple identity topics (#1089)

See #1089

comment:27 Changed 4 years ago by Jörg Richter <jri@…>

In 0104ec294d9a8b72207c7ccabfe94d01b4f34938/deepamehta:

Type editor: edit Value Type flag (#1089)

See #1089

comment:28 Changed 4 years ago by Jörg Richter <jri@…>

comment:29 Changed 4 years ago by Jörg Richter <jri@…>

In fa478bdc1753aa70f81d3dbd4ad6ea0ecda06499/deepamehta:

Identify parent by identity attribute(s) (#1089)

See #1089

comment:30 Changed 4 years ago by Jörg Richter <jri@…>

In 3ef04772da7a241872e34ae5cdc7ee594d8fd655/deepamehta:

Drop concept "simple identity type" (#1089)

Simple types are ALWAYS value types.

See #1089

comment:31 Changed 4 years ago by Jörg Richter <jri@…>

In df601465f1a7d892c7faea2717c34067f98d6b94/deepamehta:

Fix memory update on unifying composites (#1089)

See #1089

comment:32 Changed 4 years ago by Jörg Richter <jri@…>

In 05014ae09ebf3c147e1a4f012912296029085cc8/deepamehta:

Plugin data model: add index to "Migration Nr." (#1089)

See #1089

comment:33 Changed 4 years ago by Jörg Richter <jri@…>

In 21ae88cc6808c42dd2aea47386649dd4ffb66b4e/deepamehta:

Notes data model: add KEY indexes (#1089)

See #1089

comment:34 Changed 4 years ago by Jörg Richter <jri@…>

comment:35 Changed 4 years ago by Jörg Richter <jri@…>

comment:36 Changed 4 years ago by Jörg Richter <jri@…>

In edcfd21efa106e414cbc46eeafda12d042643e2f/deepamehta:

Fix type creation (#1089)

Interactively created types get a KEY index (besides FULLTEXT).

See #1089

comment:37 Changed 4 years ago by Jörg Richter <jri@…>

comment:38 Changed 4 years ago by Jörg Richter <jri@…>

In e50f3a94a06b018001ee4554c319900c58ab11d6/deepamehta:

Fix child topic unification (#1089)

See #1089

comment:39 Changed 4 years ago by Jörg Richter <jri@…>

In dfd09a59552c7874bd79274a34cd8368441b093d/deepamehta:

Meta model: add "Identity Attribute" (#1089)

See #1089

comment:40 Changed 4 years ago by Jörg Richter <jri@…>

comment:41 Changed 3 years ago by jri

  • Status changed from new to closed
  • Resolution set to fixed

Identity-Value data model basically done, in master (DM5)

Note: See TracTickets for help on using tickets.