File Folder Annotation

[swhitley]

I think it's important to offer users a way to organize file uploads. I'd like to suggest the following annotation to help with that:

{
    "type": "net.app.x.filefolders",
    "value": {
        "path": "folder1/folder2/folder3/folderN",
    }
}

path = A slash-delimited list of virtual folders.

Clients would read the annotation, parse the folder names, and represent the folders using standard hierarchical folder conventions (as appropriate for the client). The file would not be included in the path. The file is assumed to be contained within the right-most folder.

Folders are dependent on files. You cannot create a set of folders without a file. Likewise, when the last file is removed, the containing folder(s) will no longer exist.

[berg]

Good idea. I think this is a good candidate for a core annotation.

There are some issues that I see, mainly around escaping/path separation. We could sidestep them altogether by supplying a list of path components instead of expecting them to be slash-separated. What do you think, @swhitley?

[swhitley]

@berg Thanks for the thoughts. Something simple like below? I'd just want to make sure that everyone understood the relationship between array index and folder level (i.e. index 0 folders would all be displayed at the same level).

{
    "type": "net.app.x.filefolders",
    "value": {
        "path": [
            "folder1",
            "folder2",
            "folder3",
            "folderN"
        ]
    }
}

[berg]

Yeah, that seems sensible. Let me think about this a little bit more. I'm happy to make it core.

[ludolphus]

I have one concern, how are we going to build the 'entire' directory structure ? Don't think it is an option to first get all files from the api and then build the directory structure. I'm thinking hundreds/thousands of files here...

[berg]

We're not planning to build an internal tree index of files here, so you'd have to pull them all down.

[ludolphus]

yeah kinda thought that otherwise you would probably already have build something in the File api. Also from your latest podcast it's clear that the File api is not intended to be something like dropbox

[ludolphus]

could do some caching, but files can get deleted or moved to another folder even by updating the annotation. If you all do it within the same app then there's no 'problem', with multiple apps there will be

[swhitley]

@ludolphus Yes, it's the same with S3 and tools like S3Fox. There could be some inconsistencies if the cache is out of date, but I think it's still better than not having a method for organizing files.

If this is not intended to build a complex directory structure what about just having a bunch of tags on the file ?
My 2 cents

[swhitley]

@sklouvi The intent is to be able to represent a simple directory structure with the path attribute. Tags would be useful too and I wouldn't be opposed to turning this into a file attribute annotation.

@swhitley The separeted path proposition is good.
@ludolphus We can rely on File id to figure out what happen on a move/rename operation or did I get that wrong ?

[ludolphus]

The file id never changes. But you will have to do something (query the api) to handle deleted files. Building a cache of files is an option and then use pagination parameters to get new files. Only thing you won't know is if files already in the cache have been deleted. On accessing those files you will know or by doing a full fetch of all files at set interval e.g. once a day, to not hit the api too much and keep things fast in a client.

Maybe we need a include_deleted for files, just as there is for posts and messages. Querying a file that was deleted now gives you an error with code 403 and error_message 'Forbidden'. I would like to suggest return either a code that says the specified file was deleted or have the include_deleted option or even an endpoint to query deleted files.

[neuroscr]

I don't think pathing is a modern way to address this issue. We should look at tagging like sklouvi said, so a file can belong to more than one folder/tag. (Like Google docs) Tags can be hierarchical too.

[mlv]

Idea: have a type of file be a folder. It would contain the names and file ids that are intended to be part of that folder. Some of those ids could of course be other folders.

[swhitley]

@mlv That might actually be brilliant. Because you can filter by file_types, you could select all of the "...folder..." types to get the folder structure without having to enumerate every file.

I'll think about this a bit more.