dm4-files: when re-configuring the filerepo path, subsequent uploads fail due to the quota check

[Malte]

I just re-configured my file-repo path for testing purposes and i did not move my existing files. As of 4.7 this can result in the state that there are file topics in the DB which are not in my filerepo anymore. This is the scope of this ticket.

The current result is:
Storing any new file in the newly configured file-repo fails due to an error invovling the quota check. Here is the full exception log:

INFORMATION: ### Parsing multipart/form-data request (1 parts)
Nov 10, 2015 11:04:12 AM de.deepamehta.plugins.files.provider.UploadedFileProvider parseMultiPart
INFORMATION: ### field "upload" => file "tumblr_my4lrkdoFe1s87u37o1_1280.jpg" (image/jpeg, 205700 bytes)
Nov 10, 2015 11:04:12 AM de.deepamehta.plugins.images.ImagePlugin upload
INFORMATION: upload image tumblr_my4lrkdoFe1s87u37o1_1280.jpg
Nov 10, 2015 11:04:12 AM de.deepamehta.plugins.files.FilesPlugin getResourceInfo
INFORMATION: Getting resource info for repository path "images"
Nov 10, 2015 11:04:12 AM de.deepamehta.plugins.files.FilesPlugin checkPath
INFORMATION: Checking path "/home/malted/Schreibtisch/filerepo-test/images"
  dm4.filerepo.path="/home/malted/Schreibtisch/filerepo-test" => PATH OK
Nov 10, 2015 11:04:12 AM de.deepamehta.plugins.files.FilesPlugin checkExistence
INFORMATION: Checking existence of "/home/malted/Schreibtisch/filerepo-test/images" => EXISTS
Nov 10, 2015 11:04:12 AM de.deepamehta.plugins.images.ImagePlugin createImagesDirectoryInFileRepo
INFORMATION: ImagesPlugin regards everything as FINE with the current FileRepo!
Nov 10, 2015 11:04:12 AM de.deepamehta.plugins.files.FilesPlugin storeFile
INFORMATION: Storing file "tumblr_my4lrkdoFe1s87u37o1_1280.jpg" (image/jpeg, 205700 bytes) at repository path "images"
Nov 10, 2015 11:04:12 AM de.deepamehta.plugins.files.FilesPlugin checkPath
INFORMATION: Checking path "/home/malted/Schreibtisch/filerepo-test/images"
  dm4.filerepo.path="/home/malted/Schreibtisch/filerepo-test" => PATH OK
Nov 10, 2015 11:04:12 AM de.deepamehta.plugins.files.FilesPlugin checkExistence
INFORMATION: Checking existence of "/home/malted/Schreibtisch/filerepo-test/images" => EXISTS
Nov 10, 2015 11:04:12 AM de.deepamehta.plugins.files.FilesPlugin getFile
INFORMATION: Accessing the file of File topic 5112
Nov 10, 2015 11:04:12 AM de.deepamehta.plugins.files.FilesPlugin checkPath
INFORMATION: Checking path "/home/malted/Schreibtisch/filerepo-test/\dissertation-grosser-fotos\2014-06-20 16.00.17.jpg"
  dm4.filerepo.path="/home/malted/Schreibtisch/filerepo-test" => PATH OK
Nov 10, 2015 11:04:12 AM de.deepamehta.plugins.files.FilesPlugin checkExistence
INFORMATION: Checking existence of "/home/malted/Schreibtisch/filerepo-test/\dissertation-grosser-fotos\2014-06-20 16.00.17.jpg" => NOT FOUND
Nov 10, 2015 11:04:12 AM de.deepamehta.core.util.UniversalExceptionMapper logException
SCHWERWIEGEND: Request "POST /images/upload?CKEditor=editor1&CKEditorFuncNum=0&langCode=en" failed. Responding with 500 (Internal Server Error). The original exception/error is:
java.lang.RuntimeException: java.lang.RuntimeException: Storing file "tumblr_my4lrkdoFe1s87u37o1_1280.jpg" (image/jpeg, 205700 bytes) at repository path "images" failed
	at de.deepamehta.plugins.images.ImagePlugin.upload(ImagePlugin.java:64)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
	at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
	at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
	at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.ResourceObjectRule.accept(ResourceObjectRule.java:100)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1480)
	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1411)
	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1360)
	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1350)
	at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
	at org.apache.felix.http.base.internal.handler.ServletHandler.doHandle(ServletHandler.java:339)
	at org.apache.felix.http.base.internal.handler.ServletHandler.handle(ServletHandler.java:300)
	at org.apache.felix.http.base.internal.dispatch.ServletPipeline.handle(ServletPipeline.java:93)
	at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:50)
	at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31)
	at org.apache.felix.http.base.internal.dispatch.FilterPipeline.dispatch(FilterPipeline.java:76)
	at org.apache.felix.http.base.internal.dispatch.Dispatcher.dispatch(Dispatcher.java:49)
	at org.apache.felix.http.base.internal.DispatcherServlet.service(DispatcherServlet.java:67)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
	at org.eclipse.jetty.server.Server.handle(Server.java:370)
	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
	at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Storing file "tumblr_my4lrkdoFe1s87u37o1_1280.jpg" (image/jpeg, 205700 bytes) at repository path "images" failed
	at de.deepamehta.plugins.files.FilesPlugin.storeFile(FilesPlugin.java:205)
	at de.deepamehta.plugins.images.ImagePlugin.upload(ImagePlugin.java:59)
	... 51 more
Caused by: java.lang.RuntimeException: Writing uploaded file to disk failed (file "tumblr_my4lrkdoFe1s87u37o1_1280.jpg" (image/jpeg, 205700 bytes))
	at de.deepamehta.plugins.files.UploadedFile.write(UploadedFile.java:111)
	at de.deepamehta.plugins.files.FilesPlugin.storeFile(FilesPlugin.java:197)
	... 52 more
Caused by: java.lang.RuntimeException: An error occurred in the CheckDiskQuotaListener of plugin "DeepaMehta 4 Access Control"
	at de.deepamehta.core.impl.EventManager.deliverEvent(EventManager.java:110)
	at de.deepamehta.core.impl.EventManager.fireEvent(EventManager.java:63)
	at de.deepamehta.core.impl.EmbeddedService.fireEvent(EmbeddedService.java:531)
	at de.deepamehta.plugins.files.provider.UploadedFileProvider.check(UploadedFileProvider.java:75)
	at de.deepamehta.plugins.files.UploadedFile.write(UploadedFile.java:108)
	... 53 more
Caused by: java.lang.RuntimeException: Accessing the file of File topic 5112 failed
	at de.deepamehta.plugins.files.FilesPlugin.getFile(FilesPlugin.java:357)
	at de.deepamehta.plugins.accesscontrol.AccessControlPlugin.getOccupiedSpace(AccessControlPlugin.java:575)
	at de.deepamehta.plugins.accesscontrol.AccessControlPlugin.checkDiskQuota(AccessControlPlugin.java:526)
	at de.deepamehta.plugins.files.FilesPlugin$1.deliver(FilesPlugin.java:75)
	at de.deepamehta.core.impl.EventManager.deliverEvent(EventManager.java:97)
	... 57 more
Caused by: de.deepamehta.plugins.files.FileRepositoryException: File or directory "/home/malted/Schreibtisch/filerepo-test/\dissertation-grosser-fotos\2014-06-20 16.00.17.jpg" does not exist
	at de.deepamehta.plugins.files.FilesPlugin.checkExistence(FilesPlugin.java:720)
	at de.deepamehta.plugins.files.FilesPlugin.getFile(FilesPlugin.java:354)
	... 61 more

Nov 10, 2015 11:04:12 AM de.deepamehta.core.impl.TransactionFactory$TransactionResourceFilter$2 filter
WARNUNG: ### Rollback transaction of AbstractSubResourceMethod(ImagePlugin#upload)

[jri]

Replying to Malte:

I just re-configured my file-repo path for testing purposes and i did not move my existing files. As of 4.7 this can result in the state that there are file topics in the DB which are not in my filerepo anymore.

When you change the file-repo path you MUST move the existing repo physically. Otherwise you create an inconsistent state.

[Malte]

Yes, exactly as i wrote.

What i can also confirm is that the quota check fails alos (and thus the repo becomes unusable, esp. no uploads possible anymore) if one removes a file from the hard-disk manually.

[jri]

The DM file repo is under control of DM. You MUST not manipulate it manually.

Analogously you would not delete some files from the "deepamehta-db" folder and than be surprised, DM is not working as expected, right?