Access to local filesystem through DM proxy

[JuergeN]

It would be more than cool if one could access the local filesystem through the DeepaMehta proxy server. The proxy server should be configureable in two terms:

1. who has access => configurable IP ranges, e.g. 127.0.0.1/32 per default, but a list of IP ranges should be configurable like in Apache: 192.168.0.0/24 172.16.0.1/24, (and so on).
2. what directory => '/' per default, but could also be /opt/deepamehta/deepamehta-files

Access to the local filesystem is pretty much like access to web resources. The only difference is that we access a file:/whatever. Directories appear as topics in the canvas. Once I click on a directory, the files are listed in the content panel. When I click on a file in the content panel it appears on the canvas and if the mime type is supported by the browser, its contents show up in the contents panel. That's t! :) A great enhancement!

[Jörg Richter]

Files module: begin of directory browsing (#90).

Add command "New File Browser" to the "Create" menu.
Directory listing not yet functional.

You must reset your DB.

See ticket 90.

• Changeset: a532cb35d80ec2dbbb8e1ad6f965f8ebd22b0cab
• URL: ​https://github.com/jri/deepamehta/commit/a532cb35d80ec2dbbb8e1ad6f965f8ebd22b0cab

[Jörg Richter]

New module: deepamehta-proxy (#90).

Move proxy functionality from "Files" module to its own module.

See ticket 90.

• Changeset: 6c96bb8f7b3a77f63c93b05f4dfab089a339604b
• URL: ​https://github.com/jri/deepamehta/commit/6c96bb8f7b3a77f63c93b05f4dfab089a339604b

[Jörg Richter]

Files module: file browser works! (#90)

Choose "New File Browser" from the create menu.

See ticket 90.

• Changeset: d2c09c29833ab1e2a8b7c5806515b26f5f521930
• URL: ​https://github.com/jri/deepamehta/commit/d2c09c29833ab1e2a8b7c5806515b26f5f521930

[jri]

Proxy module: configurable IP ranges (#90).

In ./pom.xml there is a new property:

<dm4.proxy.net.filter>127.0.0.1/32</dm4.proxy.net.filter>

The default is as shown.
Only *one* range is supported for the moment.

• Changeset: [d94b2b952e1fef5b035b5adfbee8dd0e241d6b10]
• URL: ​https://github.com/jri/deepamehta/commit/d94b2b952e1fef5b035b5adfbee8dd0e241d6b10

[JuergeN]

Great to see the progress here! I wanted to test this today and changed the path for the proxy dir to a specific directory, let's say:

/home/me/deepamehta/deepamehta-filedir

But when I open the "New file browser" in the GUI it, still tries to open "/" instead of the path that is given in the config file. So I think this needs some changes.

My suggestion would be though, that in the GUI you stick with "/" to represent the starting directory instead of displaying the full path "/home/me/deepamehta/deepamehta-filedir". It is very common, that the relative path is what you see in the GUI insted of the full path. (Think of e.g. apache webserver, where you also define the directory root for your server with an absolute path, but just stick with the relative path for all the documents, pictures and pages. For other servers like Samba or likewiese it is all the same.)

So in other words:

1. I create a directory "/home/me/deepamehta/deepamehta-filedir" and put some files in it.
2. I change the config file ./pom.xml to

   <dm4.proxy.files.path>/home/me/deepamehta/deepamehta-filedir</dm4.proxy.files.path>
   

3. In the GUI I open click "New file browser"
4. On the canvas I see a directory-Icon
5. In the content panel I see the list of files from "/home/me/deepamehta/deepamehta-filedir".
6. The path shown in the GUI for this directory is not the full path on the system, but just the relative path from the starting directory, in this case still "/".

I hope you understand what I mean. Otherwise, let's just talk on Monday. But anyways, right now, the configuration does not fully work yet, I think.

[jri]

Thanks for testing and feedback!

Although the dm4.proxy.files.path property exists already in pom.xml this feature is *not* yet implemented.

For the moment just the dm4.proxy.net.filter feature is implemented (both features are independant).

Regarding the dm4.proxy.files.path feature: I understand what you mean and implemented it exactly in the fashion you described. I pay attention the file repo's root directory is *never* exposed. Neither in the GUI nor via REST API.

I'll commit the dm4.proxy.files.path feature very soon.

[Jörg Richter]

Proxy: configurable file repo directory (#90).

In ./pom.xml there is a new property:

<dm4.proxy.files.path><dm4.proxy.files.path>

This is the file repository's root directory: An absolute path with *no* slash at the end.

To enable the entire file system specify an *empty string* (instead of "/"), as shown above. This is the default.

Example:

<dm4.proxy.files.path>/home/me/deepamehta/deepamehta-files<dm4.proxy.files.path>

See ticket 90.

• Changeset: 7c1977034094d9877b502aa8b95993eda8b13861
• URL: ​https://github.com/jri/deepamehta/commit/7c1977034094d9877b502aa8b95993eda8b13861

[Jörg Richter]

Files/Proxy? modules: handle and report NOT FOUND.

404 NOT FOUND happens e.g. if the server is restarted with a different dm4.proxy.files.path setting.
File and folder topics retrieved before are not valid anymore. A 404 (Not Found) message is displayed for them.

See ticket 90.

• Changeset: b472940a5368d4f0a563caf2ce388419d4da629d
• URL: ​https://github.com/jri/deepamehta/commit/b472940a5368d4f0a563caf2ce388419d4da629d

[Jörg Richter]

Proxy: fix REST API call getResourceInfo() (#90).

The dm4.proxy.files.path setting is respected.

See ticket 90.

• Changeset: ed0f2b9a1e779fb72cfeff29287dc0b18e6f7f0a
• URL: ​https://github.com/jri/deepamehta/commit/ed0f2b9a1e779fb72cfeff29287dc0b18e6f7f0a

[jri]

Hint: for the binary distribution both settings are found in ./conf/config.properties

dm4.proxy.files.path
dm4.proxy.net.filter

[Malte]

deepamehta console log of requested file not found

[Malte]

i downloaded and used the "4.0.4-20110912"-Snapshot (thus running on in here described default configuration?)

• browsing directories on my hard disk beginning from "/" works fine, pretty cool feature!!!
• when selecting a file, e.g. an audio-file, the correct plugin according to mime/type configuration gets loaded firefox but it cannot play the file.. and the deepamehta-console says: "Requested file/directory NOT FOUND.." see also attached log-file

ubuntu 11.04 running on Gnome 3, running the webclient in firefox 5

[jri]

Thanks, Malte, for reporting!

For the moment I don't understand what's happening on your machine. Strange is there are two consecutive "getResource" requests to the same resource, while the first one is properly decoded and the second one is not. The second one causes the NOT FOUND because the file's path is invalid.

INFO: Requesting resource "file:/home/malted/Downloads/AMC Mixtape/AMC Mixtape/01 the world is listening.mp3" (mediaType="audio/mpeg", size=9091958)
INFO: Requesting resource "file:%2Fhome%2Fmalted%2FDownloads%2FAMC%20Mixtape%2FAMC%20Mixtape%2F01%20the%20world%20is%20listening.mp3?size=9091958&type=audio%2Fmpeg" (mediaType="null", size=0)

These requests are issued by your audio player plugin.

Can you try other player plugins? (And other webbrowsers as well?)

Have you also problems with displaying/playing other files types like images, text, pdf, video?

Once we've found the problem's origin we can fix/workaround it.

[JuergeN]

I experience the same problem, both in firefox and in chromium.

[JuergeN]

This seems to be a general problem with playing embedded mp3 files in firefox and likewise browsers in Ubuntu. There is a whole bunuch of bug reports about this on the web. After longer searching I found the following solution:

aptitude install gecko-mediaplayer

That did the job on my computer. Now I can play a file of type mp3 within DeepaMehta. The only caveat for me is that the player is only displayed half in the content panel. The upper half is cut off. That was the same with the other layer before. So I guess it nothing particular eith this recent plugin, but a generla issue. See attachment Cut_off_Player.png

[Malte]

numberformat excetion for input string 0:0:0:0:0:0:1...

[Malte]

thanks juergen for investigating this issue further! and i can confirm that a "cut-off"-embed was also shown to me with that setup, i will be able to retest things with that setup by friday.

to jri: i remember that for example image-files from the file system could be displayed in the content panel with that particular setup..

today on my pc i ran into another problem with the default setup provided with the "4.0.4-20110912"-Snapshot: After creating and selected a "new file browser" the following message was rendered in the folder-content"-field of the page-renderer with the result that i could not surf into any folder..

"FolderContentRendererError: AJAX GET request failed, server response: 500 (For input string: "0:0:0:0:0:0:0:1"), exception: undefined"

and in the deepamehta-console a "java.lang.NumberFormatException?: For input string: "0:0:0:0:0:0:0:1" appeared, which seems to be unrelated to the former issue.
see also the correspeonding log-file "dm-numberFormatException.txt" attached here..

[jri]

java.lang.NumberFormatException??: For input string: "0:0:0:0:0:0:0:1

This is an IPv6 issue.

It is fixed in master.

See #92.

[Jörg Richter]

Files module fix: audio player is not cropped (#90).

See ticket 90.

• Changeset: 3129a2dbb0ec1a6c3e82461dd0b3da5a3a768bc2
• URL: ​https://github.com/jri/deepamehta/commit/3129a2dbb0ec1a6c3e82461dd0b3da5a3a768bc2

[jri]

The only caveat for me is that the player is only displayed half in the content panel.
The upper half is cut off.

To my knowledge there is no general solution to the "plugin size" problem. The <embed> element requires width and height attributes but has no knowledge about size requirements of the plugin in use. So what DM does here is a best guess.

The height for the audio plugin is now increased from 16 to 64 pixel. The Gecko Mediaplayer should be rendered fine now.

[Malte]

syntax error in filename

[Malte]

cool, thanks for fixing that ipv6 one! and also for giving the field-render which <embeds> a bit more space..!

i could additionally just test the new snapshot with a windows xp/firefox setup and i could successfully create a "new file browser" topic which correctly starts to list all files and folders in the "Folder Content"-field but could neither reveal a new file-topic nor a new folder-topic..
.. see also the attached windows console output; my guess is the first slash should be removed instead of replaced for windows machines...

Caused by: java.lang.RuntimeException?: Checking file repository access failed (file="\C:\bootfont.bin")
Caused by: java.io.Exception: Die Syntax für den Dateinamen, Verzeichnisnamen oder die Datenträgerbezeichnung ist falsch" at ProxyPlugin?.checkFileAccess(ProcyPlugin?.java:122)

[Malte]

for this windows specific issue see also #91, both tickets talk about the same error output.

[jri]

See ticket #91