Plugin hot deployment

[jri]

Plugin hot deployment is currently only supported in the development environment (when build from source) and in the Karaf Distribution, but not in the Standard Distribution.

The Standard Distribution should come with a "deploy" directory, which is empty be default. Bundles moved/removed to/from it should be deployed/undeployed automatically.

[Jörg Richter <jri@…>]

In 929f14401ff69d76f4517520788e42a86ffe9e26/deepamehta:

Add hot deployment for standard distro (#846).

The Standard Distribution comes with a "bundle-deploy" directory.
This is the hot deploy directory, and is empty by default.

Drop bundles here to deploy them.
Remove to undeploy.

So, to install e.g. a DeepaMehta plugin (.jar file) just drop it here.
Restarting DeepaMehta is not required.

See #846.

[Malte]

Though i think of this generally as an imnprovement i feel a bit lost now. How can i continue to hot deploy via "mvn clean package", when a) having build dm4 from source and b) when not?

For a), is it the "modules-external" folder all my jars must be copied into?
For b), is it the "bundles-deploy" folder all my jars must be copied into?

I definitely will not relocate (and possibly copy) all my plugins source code into the (various) "bundle-deploy" folders of various dm4-distributions. So, what is to be adapted at the third party plugin developers maven side so ".jars" do not end up in an unwatched /target/ folder but in the resp. bundle folders (mentioned above)?

I guess we must either configure or additionally provide some kind of path to the folders (mentioned above), at best when doing something like "mvn clean package".

Thanks for your help & Cheers!

[jri]

Replying to Malte:

How can i continue to hot deploy via "mvn clean package", when a) having build dm4 from source and b) when not?

For a), is it the "modules-external" folder all my jars must be copied into?

See https://trac.deepamehta.de/ticket/847#comment:8

So, yes, the standard case is to put your plugin source in "modules-external". In your plugin directory, when you perform "mvn clean package", the target jar is hot deployed automatically then (by being copied to "bundle-deploy", see https://trac.deepamehta.de/ticket/847#comment:4). No configuration is required.

At the moment in 4.8-SNAPSHOT there is only support for the standard case. When you want to keep your plugin source at another location additional configuration is required on your side. I'll add such a configuration option soon.

For b), is it the "bundles-deploy" folder all my jars must be copied into?

See https://trac.deepamehta.de/ticket/847#comment:3

So, the standard case is to put your plugin source in "bundle-dev". In your plugin directory, when you perform "mvn clean package", the target jar is hot deployed automatically then (by being copied to the distro's "bundle-deploy", see https://trac.deepamehta.de/ticket/846#comment:2). No configuration is required.

At the moment in 4.8-SNAPSHOT there is only support for the standard case. When you want to keep your plugin source at another location additional configuration is required on your side. I'll add such a configuration option soon.

Thanks for your question!

[jri]

A configuration option needs to be added in case the developer want keep hers plugin sources at a non-standard location (other than "bundle-dev" resp. "modules-external").

[Malte]

Thank you, a configuration option for that would be really great (for my workflow)!

[jri]

Replying to jri:

A configuration option needs to be added in case the developer want keep hers plugin sources at a non-standard location (other than "bundle-dev" resp. "modules-external").

The configuration option to come will work as follows: in your plugin's pom.xml you will configure the directory where your plugin gets hot deployed (once build with "mvn clean package").

[Malte]

I would be glad if you could find a way to leave a platform run configuration (when building dm from source) in one of the platforms source config files.

If there is no way around i will surely get used to it but it doesn't sound good to me to spread the deploy paths into several configuration files of the plugins (e.g. eight in the kiezatlas setup).

[Malte]

Please consider the following:
Normally, programmers (and thus plugin developers) will make errors, in this case, they may configure their plugin wrong (e.g. importModels-setting) or deploying a wrong version of a plugin.

How will the core/resp. the startup process/activation phase be visible in the log with this new approach? Both these "failure cases" are wonderfully covered by the current solution, e.g. the gogo shell allows to spot and possible restart and investigate a mis-configured bundle easily.

I just want to make sure here that there is no regression or unnecessary complication for the plugin developers workflow. I would be glad if the core developers will give this a try for themselves with one of their own 3rd party plugins to be able to assess the improvement properly.

Subsequently: What is the workflow around bundles incompatible with the current platform being started? would you expect that plugin devs clean this folder manually during each change in work projects?

Will compiled version of other 3rd party plugins (, plugins which are not part of the current project and thus not current to the dm4 platform version covering the modules-external|bundle-dev but there at the time of startup) stay in these new folders?

Thanks for your efforts on improving the plugin development workflow.

[Jörg Richter <jri@…>]

In 96ef7c9e923c02a947e9478cec5e443e296e465b/deepamehta:

Build: introduce dm4.deploy.dir property (#847).

The developer can keep hers plugin sources at a non-standard location (other than "bundle-dev" resp. "modules-external"). In order to hot (re)deploy the plugin every time it is build the developer must add the property dm4.deploy.dir to the plugin's pom.xml. The value is a path to the directory where DM copies the plugin jar to once build.

The path can be absolute:

<properties>
    <dm4.deploy.dir>/home/terry/deepamehta/bundle-deploy</dm4.deploy.dir>
</properties>

or relative (to the plugin pom.xml):

<properties>
    <dm4.deploy.dir>${project.basedir}/../../deepamehta/bundle-deploy</dm4.deploy.dir>
</properties>

The default value for dm4.deploy.dir is ../../bundle-deploy, which is fine in case the plugin sources are kept at the standard location ("bundle-dev" resp. "modules-external").

See #847.
See #846.

[jri]

Replying to Jörg Richter <jri@…>:

or relative (to the plugin pom.xml):

<properties>
    <dm4.deploy.dir>${project.basedir}/../../deepamehta/bundle-deploy</dm4.deploy.dir>
</properties>

Sorry, this was misleading as actually an absolute path is specified.
You don't need ${project.basedir}/
Just write:

<properties>
    <dm4.deploy.dir>../../deepamehta/bundle-deploy</dm4.deploy.dir>
</properties>

The path is relative to the plugin pom.xml

[jri]

Replying to Malte:

I would be glad if you could find a way to leave a platform run configuration (when building dm from source) in one of the platforms source config files.

The config property felix.fileinstall.dir is still there (in global pom.xml). You still can add your own (target) directories there. DM will still scan all specified directories for deployable items. So, from my point of view your complex dev scenario is still supported.

And with DM 4.8-SNAPSHOT it even works if you don't build from source as the felix.fileinstall.dir config property exist in the binary distro's config.properties file as well.

2 things are to be considered tough:

• If you still want deploy your plugins via setting the felix.fileinstall.dir config property manually you possibly want disable the plugin from being copied automatically to the "bundle-deploy" directory once build (as DM 4.8-SNAPSHOT does). I have to provide such a disabling mechanism.

• Currently in DM 4.8-SNAPSHOT the config property felix.fileinstall.filter is dropped. It was used to watch only for jar files for deployment. Otherwise Felix File Install would try also to deploy the target's classes directory, which often fails. I have to check this out and possibly must re-add the felix.fileinstall.filter setting.

Thank you for presenting your dev scenario!

[Jörg Richter <jri@…>]

In dcd3ab1220382f854862e9b1e64ad9f43e338ad2/deepamehta:

Build: add dm4.deploy.disable property (#847).

To disable the plugin from being copied automatically to the "bundle-deploy" directory once build run Maven with the dm4.deploy.disable property:

mvn clean package -Ddm4.deploy.disable

See #847.
See #846.

[Jörg Richter <jri@…>]

In 4093c9c2e1d0d9cd547b56b8f1a59ff4636607c9/deepamehta:

Config files: explain hot deploy directory (#846).

The property felix.fileinstall.dir is more prominent in the config files and has an explanation.

In pom.xml of a from-source installation the property <felix.fileinstall.dir> is located in the global <properties> element (instead of being buried in a <profile> element).
It has its own <!-- Hot Deployment --> section.

Explanation:
DeepaMehta plugins (jar files) from this directory are hot deployed (like any other OSGi bundle).
You can add further directories here, separated by comma.
IMPORTANT: the trimmed value of the <felix.fileinstall.dir> element must not contain any spaces.
Otherwise the first and the last directory is ignored. Line breaks and tab characters are allowed however.

In config.properties of the binary distribution the property felix.fileinstall.dir has its own ## Hot Deployment ## section.

Explanation:
DeepaMehta plugins (jar files) from this directory are hot deployed (like any other OSGi bundle).
You can add further directories here, separated by comma.
Also you can remove the standard directory ("bundle-deploy") if you like.
Line breaks, tab characters and spaces are allowed.
IMPORTANT: if the value of felix.fileinstall.dir spreads several lines the line break characters must be escaped with backslash (\).

See #846.
See #847.

[jri]

Replying to jri:

2 things are to be considered tough:

• If you still want deploy your plugins via setting the felix.fileinstall.dir config property manually you possibly want disable the plugin from being copied automatically to the "bundle-deploy" directory once build (as DM 4.8-SNAPSHOT does). I have to provide such a disabling mechanism.

This is done now.
See comment:14

• Currently in DM 4.8-SNAPSHOT the config property felix.fileinstall.filter is dropped. It was used to watch only for jar files for deployment. Otherwise Felix File Install would try also to deploy the target's classes directory, which often fails. I have to check this out and possibly must re-add the felix.fileinstall.filter setting.

I've checked that. We don't need the felix.fileinstall.filter property anymore.

The topic "revising the hot deployment mechanism" is considered complete now.
This state is supposed to be released as DM 4.7.1 (if no objection occur ;-)