Creating Virtual File System v1 & v2 in .NET
The IT Hit User File System is provided with file system examples that you can use as a starting point for your Virtual File System. While samples have a lot of functionality that allows you to fine-tune your file system implementation, in this article we will concentrate on the minimum affords required to create a file system.
In most cases, you will modify classes that provide high-level file and folder abstractions, such as UserFile, UserFolder, UserFileSystem, and Mapping as well as will replace classes responsible for synchronization from your remote storage to the user file system, such as RemoteStorageMonitor and ServerToClientSync classes.
This article is about the legacy version of the User File System. For the latest version please refer to the articles in this section.
Folders and Files
The sample virtual file system implementation provides UserFile and UserFolder classes that represent a high-level abstraction of files and folders in your file system. The UserFile and the UserFolder classes are derived from UserFileSystemItem class that contains methods common for both files and folders.
Listing Folder Content
The folder listing is done inside the UserFolder.EnumerateChildrenAsync() method:
public async Task<IEnumerable<FileSystemItemBasicInfo>> EnumerateChildrenAsync(string pattern)
{
// This method has a 60 sec timeout.
// To process longer requests modify the IFolder.GetChildrenAsync() implementation.
IEnumerable<MyRsFileSystemInfo> remoteStorageChildren = new MyRsDirectoryInfo(RemoteStoragePath).EnumerateFileSystemInfos(pattern);
List<FileSystemItemBasicInfo> userFileSystemChildren = new List<FileSystemItemBasicInfo>();
foreach (MyRsFileSystemInfo remoteStorageItem in remoteStorageChildren)
{
FileSystemItemBasicInfo itemInfo = Mapping.GetUserFileSysteItemBasicInfo(remoteStorageItem);
userFileSystemChildren.Add(itemInfo);
}
return userFileSystemChildren;
}
You will update this method to list the folder content of your remote storage. The Engine will convert the array of the FileSystemItemBasicInfo items returned from this method into file and folder placeholders and return them to the platform.
Note that applications, as well as the platform itself (operating system), can generate a large amount of folder content listings calls and expect a very fast response from the file system. Mapping all such calls into your remote storage requests will typically make the file system unresponsive and unusable. To avoid this situation, the Engine does NOT map all file system calls into your code calls. Instead, only a first call is directly mapped into EnumerateChildrenAsync() call, which will create file and folder placeholders. After that, you will update the list of files and folders using a pull or push approach described below in this article.
In the provided sample code, the EnumerateChildrenAsync() method is both called during first folder enumeration and during periodic remote storage to the user file system synchronization in ServerToClientSync class.
Reading File Content
When an application is reading file content the UserFile.ReadAsync() method is called:
public async Task<byte[]> ReadAsync(long offset, long length)
{
// This method has a 60 sec timeout.
// To process longer requests modify the IFolder.TransferDataAsync() implementation.
await using (Stream stream = MyRsFile.OpenRead(RemoteStoragePath))
{
stream.Seek(offset, SeekOrigin.Begin);
byte[] buffer = new byte[length];
int bytesRead = await stream.ReadAsync(buffer, 0, (int)length);
return buffer;
}
}
In this method, you will read file content from your remote storage and return it as a return parameter.
Sending Changes to Remote Storage
When a file or folder is modified in the user file system, the UserFile.UpdateAsync() and UserFolder.UpdateAsync() methods are called. Below is a UserFile.UpdateAsync() method:
public async Task<string> UpdateAsync(IFileBasicInfo fileInfo, Stream content = null)
{
return await CreateOrUpdateFileAsync(RemoteStoragePath, fileInfo, FileMode.Open, content);
}
protected static async Task<string> CreateOrUpdateFileAsync(string remoteStoragePath, IFileBasicInfo newInfo, FileMode mode, Stream newContentStream = null)
{
MyRsFileInfo remoteStorageItem = new MyRsFileInfo(remoteStoragePath);
await using (Stream remoteStorageStream = remoteStorageItem.Open(mode, FileAccess.Write, FileShare.None))
{
...
// Update remote storage file content.
if (newContentStream != null)
{
await newContentStream.CopyToAsync(remoteStorageStream);
remoteStorageStream.SetLength(newContentStream.Length);
}
// Update remote storage file basic info.
remoteStorageItem.Attributes = newInfo.Attributes;
remoteStorageItem.CreationTime = newInfo.CreationTime;
remoteStorageItem.LastWriteTime = newInfo.LastWriteTime;
remoteStorageItem.LastAccessTime = newInfo.LastAccessTime;
remoteStorageItem.LastWriteTime = newInfo.LastWriteTime;
return /* return ETag received from your remote storage here*/;
}
}
The UpdateSync() method provides the updated file information in fileInfo parameter, which contains creation date, modified date, attributes, etc. as well as new file content in content parameter. The content parameter is null if the content of the was not modified and should not be sent to the remote storage.
Creating Files and Folders
When a new file or folder is created in the user file system the UserFolder.CreateFileAsync() and UserFolder.CreateFolderAsync() methods are called:
public async Task<string> CreateFileAsync(IFileBasicInfo fileInfo, Stream content)
{
string itemPath = Path.Combine(RemoteStoragePath, fileInfo.Name);
return await CreateOrUpdateFileAsync(itemPath, fileInfo, FileMode.CreateNew, content);
}
public async Task<string> CreateFolderAsync(IFolderBasicInfo folderInfo)
{
string itemPath = Path.Combine(RemoteStoragePath, folderInfo.Name);
return await CreateOrUpdateFolderAsync(itemPath, folderInfo, FileMode.CreateNew);
}
Remote Storage to User File System Synchronization
To reflect changes made on the server in your remote storage, you will need to either periodically pull your server for changes or implement a push from your remote storage to the client machines. In many cases, you will implement both approaches. The sample code provides two classes: the RemoteStorageMonitor class implements a push approach and the ServerToClientSync implements a pull approach. In both cases, you will use the UserFileSystemItem class to apply changes received from your remote storage into the user file system. This class is provided with the sample code and wraps PlaceholderFolder and PlaceholderFile classes with additional checks.
The sample RemoteStorageMonitor class contains FileSystemWatcher that monitors the folder that simulates remote storage for changes and applies changes in the user file system. In your real-life application, you will rewrite this class replacing it with web sockets client or any other technology that will listen to your server changes.
The sample ServerToClientSync class implements simple full remote storage to user file system synchronization recursively comparing all files and folders using ETag. In the sample code, the ETag is simulated by file/folder modification date. In your real-life application, you may want to optimize the comparison by implementing a more advanced comparison, such as based on SyncID algorithm.
To create files and folders in the user file system you can call the UserFileSystemItem.CreateAsync() static method provided with the sample:
string userFileSystemPath = Mapping.ReverseMapPath(remoteStoragePath);
string userFileSystemParentPath = Path.GetDirectoryName(userFileSystemPath);
// Because of the on-demand population, the file or folder placeholder may not exist in the user file system
// or the folder may be offline.
if (Directory.Exists(userFileSystemParentPath)
&& !new DirectoryInfo(userFileSystemParentPath).Attributes.HasFlag(System.IO.FileAttributes.Offline))
{
FileSystemItemBasicInfo newItemInfo = Mapping.GetUserFileSysteItemBasicInfo(remoteStorageItem);
await UserFileSystemItem.CreateAsync(userFileSystemParentPath, new[] { newItemInfo });
}
To update a file or folder call the UserFileSystemItem.UpdateAsync() method:
FileSystemItemBasicInfo itemInfo = Mapping.GetUserFileSysteItemBasicInfo(remoteStorageItem);
if (FsPath.Esxists(userFileSystemPath)
&& !await ETag.ETagEqualsAsync(userFileSystemPath, itemInfo))
{
await new UserFileSystemItem(userFileSystemPath).UpdateAsync(itemInfo);
}
To delete a file or folder call UserFileSystemItem.DeleteAsync() method.
if (FsPath.Exists(userFileSystemPath))
{
await new UserFileSystemItem(userFileSystemPath).DeleteAsync();
}
To move or rename a file or folder call UserFileSystemItem.MoveToAsync() method:
if (FsPath.Exists(userFileSystemOldPath))
{
await new UserFileSystemItem(userFileSystemOldPath).MoveToAsync(userFileSystemNewPath);
}