Ticket #815 (closed Enhancement: fixed)

Opened 5 years ago

Last modified 4 years ago

Per-workspace file repositories

Reported by: jri Owned by: jri
Priority: Major Milestone: Release 4.7
Component: DeepaMehta Standard Distribution Version: 4.6.1
Keywords: Cc: dgf, Malte, JuergeN
Complexity: 5 Area:
Module: deepamehta-files

Description

DM's file repo needs to be under access control too (#592). Access to a file could be governed by the regular access control applied to the representing File topic.

Change History

comment:1 Changed 5 years ago by jri

  • Status changed from new to accepted

comment:2 Changed 5 years ago by Jörg Richter <jri@…>

In a86855efd089c90e3f2943ed15160d19819bedeb/deepamehta:

Files plugin: refactor security (#815).

BREAKING CHANGES

1 interface dropped:

de.deepamehta.core.service.SecurityHandler

Use this listener instead:

de.deepamehta.core.service.event.ResourceRequestFilterListener

See #815.

comment:3 Changed 5 years ago by Jörg Richter <jri@…>

In 8b790f7fd42c2898110b9c4eb234b9c8cf7a2716/deepamehta:

Per-workspace file repositories (#815).

There is a new config property:

dm4.filerepo.per_workspace=false|true

Set to "true" to put the file repository under access control. DM then creates a separate file repository for each workspace beneath the file repository base path. This allow for e.g. private and collaborative file repositories. If set to "false" (the default) there is one big public file repository shared by all users.

See #815.

comment:4 Changed 5 years ago by Jörg Richter <jri@…>

In c7dac274f9965c35e9bb8b9282e28ada7e4c1aa6/deepamehta:

Per-workspace file repos: upload works (#815).

See #815.

comment:5 Changed 5 years ago by Jörg Richter <jri@…>

In 31ed4138b417d5b8f5accf960093e8998856d0cc/deepamehta:

File content rendering works (#815).

1 new interface:

de.deepamehta.core.service.DirectoryResourceMapper {

    // Maps a resource name to an URL
    URL getResource(String name);
}

BREAKING CHANGE

de.deepamehta.core.osgi.PluginActivator?'s publishDirectory() method takes a DirectoryResourceMapper.
Pass null for default mapping.

See #815.

comment:6 follow-up: ↓ 8 Changed 5 years ago by Malte

Is this (=handling of file-repositories via plugins, e.g. using createFolder()) going to be opaque for the plugin developer?

For example, consider the images and mail plugin, both creating specifically named folders to store the files they manage.

The folders are (commonly, as of 4.5) created in imperative migrations consuming the FilesService?.createFolder().

Such a folder would then need to be created in every file-repo by the file-repo manager. (if the per_workspace- is set to true).

I am pretty sure you already have this in mind, just wanted to comment it since i just started upgrading the images and mail plugin to 4.6.x respectively.

Cheers!

comment:7 Changed 5 years ago by Jörg Richter <jri@…>

In 2a4bf8349f9dbad416072ceeb92722906aa3bc36/deepamehta:

Cross-workspace file revealing/browsing (#815).

A user can reveal File/Folder? topics created in other workspaces provided she has access to. Folder topics belonging to other workspaces can also be used for filesystem browsing.

See #815.

comment:8 in reply to: ↑ 6 Changed 5 years ago by jri

Replying to Malte:

Is this (=handling of file-repositories via plugins, e.g. using createFolder()) going to be opaque for the plugin developer?

Good question :-)
Unfortunately the per-workspace file repos feature can't be fully transparent to the plugin developer.
In case dm4.filerepo.per_workspace is set to true DM needs to know to which workspace your Files Service call applies.

When you call createFolder(folderName, path) with "/" as the path DM evaluates the workspace Cookie and creates the folder in the workspace-specific directory then. The workspace-specific directory is created automatically by DM.

If no cookie is present an exception is thrown. This means you can't call createFolder(..., "/") from a migration.

When calling createFolder() with another path than "/" DM expects the path to contain the workspace-specific base path, e.g.:

/workspace-1234/attachments

In order to realize per-workspace file repos DM manages workspace-specific directories beneath the file repo root directory. The workspace-specific directories have a fixed name: workspace-xxx where xxx is the workspace ID.

In this case no Cookie is required as the workspace is coded already in the path.

For example, consider the images and mail plugin, both creating specifically named folders to store the files they manage.

The folders are (commonly, as of 4.5) created in imperative migrations consuming the FilesService?.createFolder().

Such a folder would then need to be created in every file-repo by the file-repo manager. (if the per_workspace- is set to true).

My first suggestion is to create the folders lazily, that is when they are needed the 1st time, e.g. when uploading a file. At this time the Webclient knows which workspace the user is in.

Creating the folders in a migration might not be suitable as the user can create new workspaces interactively.

This is just my first suggestion. The per-workspace file repos feature is still in flux.

I am pretty sure you already have this in mind, just wanted to comment it since i just started upgrading the images and mail plugin to 4.6.x respectively.

That's great you're upgrading he mail and images plugins to 4.6!

BTW: Once sending mail works in 4.6 I think extending the personal workspaces with receiving mails would be a great feature we could offer as our first myDM premium feature.

comment:9 Changed 5 years ago by Jörg Richter <jri@…>

In 5ec4e085ab3ec0ff273bb577047d00bd45533462/deepamehta:

Secure file repo (#815).

The service which serves the static file repo files is now secured.
If a GET /filerepo/{repo_path} request is not authorized the response is 401 Unauthorized.
Authorization is performed by checking access to the File topic that corresponds to the repo path.
If that File topic is missing the response is 500 Server Error.

Example of a file repo request (multi-user setup):

GET /filerepo/workspace-1234/images/img1.png

Here 1234 is the ID of the workspace the file belongs to.

All this applies to the situation when the DM file repo is put under access control, that is when the dm4.filerepo.per_workspace config property is set to true.

In contrast in a single-user situation, that is when dm4.filerepo.per_workspace is false (the default), no file repo access control is performed. In this case the existence of a corresponding File topic for each repo file is not a requirement. Consider the DM default configuration, which is "single-user" and regards the entire hard disc as the file repo.

Example of a file repo request (single-user setup):

GET /filerepo/home/terry/images/img1.png

No workspace-specific directory appears here.

CHANGES

2 new methods in de.deepamehta.core.service.accesscontrol.AccessControl:

String getUsername(HttpServletRequest request);

String username(HttpSession session);

See #815.

comment:10 Changed 5 years ago by jri

Unfortunately it occurred to me that file repo security is not working due to a bug in Pax Web.
As a consequence only public file repos are working. The files of private and collaborative repos are not accessible for authorized users.

I filed a bug report and the Pax Web developers confirmed it:
https://ops4j1.jira.com/browse/PAXWEB-885

Unfortunately this is a show stopper for the myDM launch and we have to wait for the Pax Web fix.

comment:11 Changed 5 years ago by Jörg Richter <jri@…>

comment:12 Changed 5 years ago by Jörg Richter <jri@…>

In 703eadca66aabc826e0222ffc55f853d4e6eb521/deepamehta:

Add per-user disk quota config definition (#815).

Not yet functional.

See #815.

comment:13 Changed 5 years ago by jri

Unfortunately the Pax Web issue (see comment:10) remain unsolved. After discussing the topic with the Pax Web developer he claimed the Pax Web implementation is compliant with the OSGi HTTP Service specification and that DM's mode of operation is unusual. He closed the ticket as "works as designed".
https://ops4j1.jira.com/browse/PAXWEB-885

Technical background: effectively Pax Web binds a separate session manager to each HttpContext (as passed to the HTTP Service's register[Resources|Servlet] method). As a consequence a session created in one HttpContext? is not available in another HttpContext?. There is some reason in implementing it this way as a HttpContext? kind of represents a Servlet API ServletContext which in turn represents the web application as a whole, and a web container is supposed to protect web applications from each other. But it means the session concept (a user using an application) can only be maintained if the application uses the same HttpContext? instance for all its register calls. DM however utilizes 3 different HttpContext? classes while one of it is instantiated many times (per-plugin). DM does so for its different HTTP needs: 1) serving plugin resources (e.g. icons), 2) serving the file system (DM file repository), 3) serving the REST services (via the Jersey servlet). So, a DM session (which is created by the Jersey servlet) is not available in the HttpContext? which handles the security for the file repository. That's the reason while private and collaborative file repositories are not working at the moment.

What remains open at the moment is weather the OSGi HTTP Service specification strictly dictates this relationship of session manager and HttpContext as implemented by Pax Web.

An interesting fact is that the problem does not occur in Apache Felix HTTP Jetty, another implementation of the OSGi HTTP Service specification.

Dgf, possibly you're deeper into the OSGi specs, and can provide some info about this subject, possibly by commenting on the Pax Web ticket mentioned above.

After all, my current plan is to return back to Apache Felix HTTP Jetty. DM file repository security will work then.

Another disadvantage of Pax Web are the incomplete exports of the Java Mail API which causes severe trouble to the DM4 Mail plugin (see ticket:716#comment:24). With Apache Felix HTTP Jetty this problem does not occur as it bundles no Java Mail packages at all.

Furthermore the original cause for preferring Pax Web over Felix HTTP -- interoperability in conjunction with Karaf environments -- has faded as DM is meanwhile compatible with both Pax Web and Felix HTTP, and can easily deployed in existing Karaf environments via "feature install" in conjunction with Maven central.

Meanwhile there are newer versions of Apache Felix HTTP which bundle a more up-to-date Jetty server.

If there are no objections I will replace Pax Web with Felix HTTP in the DM Standard Distribution soon.

comment:14 Changed 5 years ago by jri

Just a status update here.

Pending file repo related work includes:

  1. Collaborative file browsing and Access Control.

File and Folder topics need special workspace assignments. The standard assignments do not work. Consider this situation: within hers private workspace a user reveals an existing directory topic that is part of a public file repository. Then she continues browsing from that directory, that is revealing contained files/folders. File and Folder topics created in the course are assigned to the user's private workspace then -- despite the contained files/folders are public. (Keep in mind DM file repo browsing causes on-the-fly creation of File and Folder topics.) Then when another user tries to browse the same files/folders an exception occurs as he has no permission to access the File/Folder? topics created by the first user. (Keep in mind that DM represents each file repo file by an unique topic, identified by path.)

The solution are special workspace assignment rules for File and Folder topics. Instead of involving the selected workspace the assignments must reflect the actual repo ownership. Note: in a DM installation where per-worksapce repos are not configured, the entire single repo must be regarded as public (it still can be browsed by different users).

  1. File browsing and anonymous users (see #573).

At the moment anonymous can't browse public file repos although she is supposed to do so. That is because DM file repo browsing causes on-the-fly creation of File and Folder topics, which are POST operations, and anonymous is not allowed to send POST requests.

The solution is to design the file browsing operations as GET operations.

  1. File upload error handling.

An error occurring while uploading a file -- e.g. a disk quota exceedance -- must be reported to the Webclient user.

  1. Switch back to Felix HTTP.

At the moment accessing files in private and collaborative file repos doesn't work due to an oddity in Pax Web (see comment:13). The solution is to switch back to Felix HTTP.

These issues will require some more work days.

Last edited 5 years ago by jri (previous) (diff)

comment:15 Changed 5 years ago by Jörg Richter <jri@…>

In d71435d5df374cdbaef0943c4b906551811d0b0d/deepamehta:

Fix cross-workspace file repo browsing (#815).

File and Folder topics are always assigned to the workspace the file repo actually belongs to. When a File or Folder topic is created (while browsing) the currently selected workspace is not relevant.

Thus collaborative cross-workspace file repo browsing is now supported.

Consider the situation when a user browses a public file repo from within hers private workspace (by revealing existing Folder topics), and then another user browses the same files.

When per-workspace file repos are switched off (the default) the entire file repo belongs to the public "DeepaMehta" workspace.

CHANGES

de.deepamehta.core.service.accesscontrol.AccessControl API:

2 new methods:

Topic getWorkspace(String uri)

long getDeepaMehtaWorkspaceId()

See #815.

comment:16 Changed 5 years ago by Jörg Richter <jri@…>

In e505bd8650b78d9a1bf2d6df4b78ecc65acb2cff/deepamehta:

Failed uploads are reported to the user (#815).

If an error occurs while a file upload the error is reported to the user by the means of a dialog box.

CHANGES

Client-side dm4c API:

1 new method:

dm4c.open_error_dialog(server_response)

Opens an error dialog and renders a server response.

The server response will be rendered as
1) either arbitrary error text or as a chain of exceptions, and
2) the server status.

A chain of exceptions is rendered if the server response meets these conditions: "content_type" is
"application/json" and "content" is the JSON representation of a Java exception (as occurred at server-side):
an object with "exception", "message", and "cause" properties. The "cause" value is again an exception object.
The final exception has no "cause" property.
{{
@param server_response The server response to render. An object with 4 properties:

content_type -- media type of the error content (string)
content -- the error content (string)
status_code -- the server status code (number)
status_text -- the server status text (string)

}}}

See #815.

See #815.
}}}

comment:17 follow-up: ↓ 19 Changed 5 years ago by Jörg Richter <jri@…>

In eccde4ffc3a91f6695cc2b8311c620376368167a/deepamehta:

File repo security Pax Web SUCCESS!!! (#815).

File repo security finally works with Pax Web!
Access to private and collaborative file repos is restricted to authorized users.

I worked around the Pax Web HttpContext? issue (see comment:13): to enable proper session management DM now uses the same HttpContext? instance for all its HTTP service registrations. This HttpContext? is called MasterHttpContext. It dispatches to the various subordinate HttpContexts? based on the request URI.

Phew!
That was fun ;-)

We don't need to switch back to Felix HTTP after all.
Thanks, dgf, for the phone call! That was motivating.

See #815.

comment:18 Changed 5 years ago by Jörg Richter <jri@…>

In af4b10f5f211b7d6c0a065732ab16304fa42e737/deepamehta:

Fix: removing contexts from master context (#815).

See #815.

comment:19 in reply to: ↑ 17 Changed 5 years ago by jri

Replying to comment:17:

File repo security Pax Web SUCCESS!!! (#815).

Oooh no, we have a new problem with that solution.
Hot redeploying does not work properly anymore. After redeploying a bundle Pax Web doesn't serve some resources of some other bundles (which are completely independent from the one redeployed) anymore as it does not call our MasterHttpContext?'s getResource() method anymore, thus producing some 404s. At the moment this is a complete mystery to me. I've no clue what's going on in Pax Web.

comment:20 Changed 5 years ago by Jörg Richter <jri@…>

In 21520a4bffaf38ed60af2d97651230e96ad9d902/deepamehta:

Switch from Pax Web to Felix HTTP (#815).

File repo security and bundle hot redeployment works immediately.

Not yet merged into master.

See #815.

comment:21 Changed 5 years ago by jri

Just a note: the MasterHttpContext? solution works flawless in Felix HTTP (although it doesn't require it at all). Hot redeployment works without any problem.

comment:22 Changed 5 years ago by jri

In 7e836265e5fb0ea6fb87f9cc6c05dc79d7048fd5:

Fix Felix HTTP shutdown (#815).

See #815.

comment:23 Changed 5 years ago by jri

  • Status changed from accepted to closed
  • Resolution set to fixed

comment:24 Changed 5 years ago by Malte

It would be great if you could provide an additional API call to the FilesService?, like

filesService.getAbsoluteFileRepoPath()

so third party plugins can write files directly into the correct file-repo (relative to the request).

See #871 for details, in preparation to re-vitalize the SVG/JSON export option per Topicmap.

comment:25 Changed 5 years ago by jri

This works already. Use the FileService?'s getFile() method to get the absolute repo path. If called with "/" you get the proper repo path for both cases:

  • if per_workspace=false the repo's root directory is returned
  • if per_workspace=true the root directory of the current workspace is returned. This is done by inspecting the workspace cookie.

So, to create a file in the repo:

File repoDir = filesService.getFile("/");
filesService.createFile(in, repoDir + "/exported/topicmap1.svg")

in is the InputStream the file content is read from.
/exported is just an example. You can put any path here, or drop it (to create the file in the root directory).

Last edited 5 years ago by jri (previous) (diff)

comment:26 Changed 5 years ago by Malte

So, getting the absoluteFileRepoPath() via using getFile("/") works very well (if a developer knows about this), thanks to your hint, SVG Exporting works OK (though there is no file topic created yet). But that is another issue.

Now when i do what you recommend, and per-workspace filerepos are ON:

String jsonFileName = "topicmap-" + topicmapId + ".txt";
File repoDir = filesService.getFile("/");
return filesService.createFile(in, repoDir + jsonFileName);

I get the following error:

Nov 11, 2015 12:41:21 PM net.abriraqui.dm4.importexport.ImportExportPlugin exportTopicmapToJSON
INFORMATION: Exporting Topicmap ######### 2247
Nov 11, 2015 12:41:21 PM de.deepamehta.plugins.topicmaps.TopicmapsPlugin getTopicmap
INFORMATION: Loading topicmap 2247 (includeChilds=true)
Nov 11, 2015 12:41:21 PM de.deepamehta.plugins.files.FilesPlugin getFile
INFORMATION: Accessing the file at "/"
Nov 11, 2015 12:41:21 PM de.deepamehta.plugins.files.FilesPlugin checkPath
INFORMATION: Checking path "/home/malted/Schreibtisch/filerepo-test/workspace-2237"
  dm4.filerepo.path="/home/malted/Schreibtisch/filerepo-test" => PATH OK
Nov 11, 2015 12:41:21 PM de.deepamehta.plugins.files.FilesPlugin checkExistence
INFORMATION: Checking existence of "/home/malted/Schreibtisch/filerepo-test/workspace-2237" => EXISTS
Nov 11, 2015 12:41:21 PM de.deepamehta.plugins.files.FilesPlugin createFile
INFORMATION: Creating file (from input stream) at repository path "/home/malted/Schreibtisch/filerepo-test/workspace-2237topicmap-2247.txt"
Nov 11, 2015 12:41:21 PM de.deepamehta.plugins.files.FilesPlugin checkPath
INFORMATION: Checking path "/home/malted/Schreibtisch/filerepo-test/home/malted/Schreibtisch/filerepo-test/workspace-2237topicmap-2247.txt"
  dm4.filerepo.path="/home/malted/Schreibtisch/filerepo-test" => PATH OK
Nov 11, 2015 12:41:21 PM de.deepamehta.core.util.UniversalExceptionMapper logException
SCHWERWIEGEND: Request "POST /import-export/export/json" failed. Responding with 500 (Internal Server Error). The original exception/error is:
java.lang.RuntimeException: Export failed
	at net.abriraqui.dm4.importexport.ImportExportPlugin.exportTopicmapToJSON(ImportExportPlugin.java:54)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
	at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
	at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
	at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.ResourceObjectRule.accept(ResourceObjectRule.java:100)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1480)
	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1411)
	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1360)
	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1350)
	at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
	at org.apache.felix.http.base.internal.handler.ServletHandler.doHandle(ServletHandler.java:339)
	at org.apache.felix.http.base.internal.handler.ServletHandler.handle(ServletHandler.java:300)
	at org.apache.felix.http.base.internal.dispatch.ServletPipeline.handle(ServletPipeline.java:93)
	at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:50)
	at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31)
	at org.apache.felix.http.base.internal.dispatch.FilterPipeline.dispatch(FilterPipeline.java:76)
	at org.apache.felix.http.base.internal.dispatch.Dispatcher.dispatch(Dispatcher.java:49)
	at org.apache.felix.http.base.internal.DispatcherServlet.service(DispatcherServlet.java:67)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
	at org.eclipse.jetty.server.Server.handle(Server.java:370)
	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
	at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:971)
	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1033)
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644)
	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Creating file (from input stream) at repository path "/home/malted/Schreibtisch/filerepo-test/workspace-2237topicmap-2247.txt" failed
	at de.deepamehta.plugins.files.FilesPlugin.createFile(FilesPlugin.java:229)
	at net.abriraqui.dm4.importexport.ImportExportPlugin.exportTopicmapToJSON(ImportExportPlugin.java:52)
	... 51 more
Caused by: java.io.FileNotFoundException: /home/malted/Schreibtisch/filerepo-test/home/malted/Schreibtisch/filerepo-test/workspace-2237topicmap-2247.txt (No such file or directory)
	at java.io.FileOutputStream.open(Native Method)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
	at de.deepamehta.plugins.files.FilesPlugin.createFile(FilesPlugin.java:219)
	... 52 more

comment:27 Changed 5 years ago by Malte

See #871 for the issue around a) "file topic could not be created" and b) "file topic was not created to due the implementation of a custom XML document writer".

comment:28 Changed 5 years ago by Malte

Clarified question on how to "getAbsoluteFileRepoPath()" and for the issue on how to store a file using the FilesService? when per_workspace=true, we may better use #879. Sorry for crossing issues here.

comment:29 Changed 5 years ago by jri

You've missed a slash. It must be:

createFile(in, repoDir + "/" + jsonFileName)

Look at my example.

Also the FileNotFoundException? gives the hint:

/workspace-2237topicmap-2247.txt

comment:30 Changed 5 years ago by Malte

I tried it again but having a slash in between or not does not make any difference for the outcome. If i just look at the beginning o fthe path (it contains two times my filerep-path) i do not wonder about a FileNotFoundException?.

Nov 11, 2015 3:08:34 PM net.abriraqui.dm4.importexport.ImportExportPlugin exportTopicmapToJSON
INFORMATION: Exporting Topicmap ######### 2247
Nov 11, 2015 3:08:34 PM de.deepamehta.plugins.topicmaps.TopicmapsPlugin getTopicmap
INFORMATION: Loading topicmap 2247 (includeChilds=true)
Nov 11, 2015 3:08:34 PM de.deepamehta.plugins.files.FilesPlugin getFile
INFORMATION: Accessing the file at "/"
Nov 11, 2015 3:08:34 PM de.deepamehta.plugins.files.FilesPlugin checkPath
INFORMATION: Checking path "/home/malted/Schreibtisch/filerepo-test/workspace-2237"
  dm4.filerepo.path="/home/malted/Schreibtisch/filerepo-test" => PATH OK
Nov 11, 2015 3:08:34 PM de.deepamehta.plugins.files.FilesPlugin checkExistence
INFORMATION: Checking existence of "/home/malted/Schreibtisch/filerepo-test/workspace-2237" => EXISTS
Nov 11, 2015 3:08:34 PM de.deepamehta.plugins.files.FilesPlugin createFile
INFORMATION: Creating file (from input stream) at repository path "/home/malted/Schreibtisch/filerepo-test/workspace-2237/topicmap-2247.txt"
Nov 11, 2015 3:08:34 PM de.deepamehta.plugins.files.FilesPlugin checkPath
INFORMATION: Checking path "/home/malted/Schreibtisch/filerepo-test/home/malted/Schreibtisch/filerepo-test/workspace-2237/topicmap-2247.txt"
  dm4.filerepo.path="/home/malted/Schreibtisch/filerepo-test" => PATH OK
Nov 11, 2015 3:08:34 PM de.deepamehta.core.util.UniversalExceptionMapper logException
SCHWERWIEGEND: Request "POST /import-export/export/json" failed. Responding with 500 (Internal Server Error). The original exception/error is:
java.lang.RuntimeException: Export failed
	at net.abriraqui.dm4.importexport.ImportExportPlugin.exportTopicmapToJSON(ImportExportPlugin.java:55)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
	at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
	at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
	at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.ResourceObjectRule.accept(ResourceObjectRule.java:100)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1480)
	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1411)
	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1360)
	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1350)
	at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
	at org.apache.felix.http.base.internal.handler.ServletHandler.doHandle(ServletHandler.java:339)
	at org.apache.felix.http.base.internal.handler.ServletHandler.handle(ServletHandler.java:300)
	at org.apache.felix.http.base.internal.dispatch.ServletPipeline.handle(ServletPipeline.java:93)
	at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:50)
	at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31)
	at org.apache.felix.http.base.internal.dispatch.FilterPipeline.dispatch(FilterPipeline.java:76)
	at org.apache.felix.http.base.internal.dispatch.Dispatcher.dispatch(Dispatcher.java:49)
	at org.apache.felix.http.base.internal.DispatcherServlet.service(DispatcherServlet.java:67)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
	at org.eclipse.jetty.server.Server.handle(Server.java:370)
	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
	at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:971)
	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1033)
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644)
	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Creating file (from input stream) at repository path "/home/malted/Schreibtisch/filerepo-test/workspace-2237/topicmap-2247.txt" failed
	at de.deepamehta.plugins.files.FilesPlugin.createFile(FilesPlugin.java:229)
	at net.abriraqui.dm4.importexport.ImportExportPlugin.exportTopicmapToJSON(ImportExportPlugin.java:52)
	... 51 more
Caused by: java.io.FileNotFoundException: /home/malted/Schreibtisch/filerepo-test/home/malted/Schreibtisch/filerepo-test/workspace-2237/topicmap-2247.txt (No such file or directory)
	at java.io.FileOutputStream.open(Native Method)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
	at de.deepamehta.plugins.files.FilesPlugin.createFile(FilesPlugin.java:219)
	... 52 more

comment:31 Changed 5 years ago by Malte

I experienced a similar issue when using createFolder:

When wanting to create a folder via the FilesService? when per_workspace filrepos are ON:
When doing this, the same error appears (=the absolute path to the filerepo is handled as a "repository path" and thus appears twice in the logfile, exactly as above):

File repoDir = fileService.getFile("/")
fileService.createFolder(FILEREPO_IMAGES_SUBFOLDER, repoDir);

Just when i do it this way, i am able to successfully create a folder (and it does so in the correct filerepo, works with global and per-workspace filerepos)

fileService.createFolder(FILEREPO_IMAGES_SUBFOLDER, "/");

comment:32 Changed 5 years ago by jri

If i just look at the beginning o fthe path (it contains two times my filerep-path)

Sorry, my example was wrong:

File repoDir = filesService.getFile("/");
filesService.createFile(in, repoDir + "/exported/topicmap1.svg")

... because createFile() expects a repository path, not an absolute path.
Please give me some time to think about it.

comment:33 follow-up: ↓ 34 Changed 4 years ago by Malte

Note: It is now up to the plugin developer to construct a "Workspace" folder in the file repository (in the case the per-workspace filerepo setting is set to "true"). For example if one wants to create a subfolder in e.g "/workspace-1234/images" the "workspace-1234" might not exist and must be created manually.

I do not exactly know how it was handled before (it must have been opaque to me) so this might be considered a BREAKING CHANGE. See #952

comment:34 in reply to: ↑ 33 Changed 4 years ago by jri

Replying to Malte:

Note: It is now up to the plugin developer to construct a "Workspace" folder in the file repository (in the case the per-workspace filerepo setting is set to "true"). For example if one wants to create a subfolder in e.g "/workspace-1234/images" the "workspace-1234" might not exist and must be created manually.

No, there is no breaking change. The concept now and then is that the developer must not care about creating the workspace folders. This is supposed to happen automatically. But there was a bug which is fixed meanwhile. See ticket:965#comment:6 and following.

Note: See TracTickets for help on using tickets.