[Feature Request] Add frontend to host VFS Layer

[webgeek1234]

This is a feature request to add a virtual file system layer between Retroarch and the host. This would allow remote files systems such as WebDAV, sshfs, cifs, and NFS to be mounted and ROMs, saves, bios, and and thumbnails to be accessed through them.

My use case is sharing the ROMs and saves between a couple Linux machines and several Android devices. I already use Kodi with sshfs and mysql to share sources and last viewed positions among them. I would like to do the same with Retroarch and ROMs / saves. Play for a while on my Shield Android TV, then pick up my Shield Portable and continue the same game with the same save point with no manual syncing or such.

---

There is a $70 open bounty on this issue. Add to the bounty at Bountysource.

[webgeek1234]

This ticket has been given a bounty on bountysource:
https://www.bountysource.com/issues/43321648-add-frontend-to-host-vfs-layer

I would consider my portion of the bounty complete with a baseline vfs layer and a sshfs implementation since that's my use case. Twinaphex or team members may want other implementations for the additional funds.

[Hedda]

Smartest would probably be to copy the new VFS ABI code (and VFS addons) from Kodi / XBMC:

xbmc/xbmc#8966

If it was copied from Kodi / XBMC then could always keep that as upstream.

See @notspiff repositories list for some example of VFS addons:

https://github.com/notspiff?utf8=%E2%9C%93&tab=repositories&q=vfs

Kodi / XBMC already have VFS addons for SMB, NFS, AFP, SFTP, and RAR, and more planned.

[notspiff]

i very much doubt you want to do that! you'd have to replicate a lot of the kodi add-on environment including callbacks etc.

[inactive123]

Preface to any code submission - it has to be written as C code. Kodi is C++, so you could look at it for inspiration since the license they use is pretty much the same, but in terms of implementation, you would have to rewrite it so that its C code.

The RetroArch codebase has to remain maintainable long after this bounty is but a shallow memory in people's mind. And we have chosen on C for maximum portability and performance.

Also, in no way do I ever want to be beholden to some 'upstream' that is not RetroArch itself for RetroArch. So no, you can look at it for inspiration, but that is pretty much it. I am not going to be dragging in unaltered C++ code from a separate project.

[ofry]

Maybe this can help? https://www.icculus.org/physfs/

Code is mostly on C.

Is this considered done since VFS has been merged into master now?

[Ferk]

VFS has been merged long ago, but other than on the Windows Store / Xbox One port I don't think it has been given much use in Retroarch's frontend. Or has it?

I love how Kodi's libretro frontend can load ROMs from zip files without having to extract the files to disk thanks to the VFS support. This zip VFS feature alone would help in all these issues: #4331 (with another bunty), #4094 and could be the basis for #6534

But it would be best if those who contributed to this bounty for this ticket clarified exactly if just having the VFS support is enough to claim the bounty or if some form of "remote file system" support was actually intended, like it seems to be hinted in the issue description.
There's also a very related issue with another bounty, that could be solved by having remote FS support, but applied to the save directory in particular: #6875

[webgeek1234]

My intent for the bounty is remote filesystem support. I wouldn't necessarily mind splitting that into a couple different bounties, though. Like get a generic modular vfs layer working, then implement different protocols. My end goal is to be able to access stuff over sshfs, like Kodi supports. I haven't kept up with development here as much lately, so I don't know what shape the current vfs implementation is in.

[Ghabry]

As long as there are no directory functions in the VFS (at least "read dir") I wouldn't even consider the VFS fully implemented because currently any core which works on directory instead of file level (= not an emulator which just loads a single rom file) can't use it.

[krzys-h]

The current VFS is used mostly to route file access calls through the frontend to platform-specific functions (with builtin fallbacks in the cores), but it does look feasible to actually implement proper VFS functions into it. I had to improve the VFS support for the StorageFile implementation for UWP - it now has directory support, as well as I fixed the calls to path_is_valid, path_is_directory etc. for files on the VFS. All that has to be done now is to add something on the frontend to route the calls to different things (sshfs, zip, etc.), I will probably try to do it after I'm done with StorageFile.

[erfg12]

@krzys-h was sshfs, zip, etc. done?