Creating Virtual Drive in .NET

In this article, we describe advanced features such as move and delete operations on the Microsoft Windows platform, temporary files filtering (typically created by Microsoft Office and AutoCAD transactional save operation), storing custom data associated with files and folders, displaying data custom columns in Windows Explorer, thumbnails implementation, packaged installation and items locking.

Move and Delete Operations on Microsoft Windows Platform

Unlike with other operations, the platform provides the means for the move/rename and delete operations cancellation. In the case, the operation fails, because of the lost connection, validation errors, or any errors in your remote storage, you can cancel it, keeping your files in sync.

Both move and delete provide two methods, that are being called before and after the operation. The Engine calls IFileSystemItem.MoveToAsync() and IFileSystemItem.DeleteAsync() before the operation and IFileSystemItem.MoveToCompletionAsync() and IFileSystemItem.DeleteCompletionAsync() - after. You can confirm the operation in the "before"-method, by calling the IConfirmationResultContext. ReturnConfirmationResult() method or cancel the opertion by calling IConfirmationResultContext.ReturnErrorResult().

If you do not call IConfirmationResultContext.ReturnConfirmationResult() or IConfirmationResultContext.ReturnErrorResult() inside your move or delete implementation, and the method completes without exceptions, the Engine calls ReturnConfirmationResult(), automatically confirming the operation. If your code fails with an exception, the Engine will automatically call ReturnErrorResult() and the item will NOT be moved or deleted.

If the IFileSystemItem.MoveToCompletion() method completes without exceptions the item is marked as successfully moved and information about the old location of the item is erased (note that this information may still be stored by the platform, for example, to perform the undo operation).

If the IFileSystemItem.DeleteCompletion() completes without exceptions the item is marked as deleted and any information about it is erased (note that the item can still remain in the recycle bin or may still be stored by the platform), otherwise, the information about the deleted item is kept stored by the Engine, so the Engine can retry deleting the item in your remote storage at a later time.

Implementing Move on Windows

Move Operation Sequence

Below we will describe the sequence of the move operation for the move without overwriting.

In the case of a file, the platform opens the file handle and blocks it for reading and writing. The Engine calls IFileWindows.OpenCompletionAsync() method.
The platform starts the move operation. The Engine calls IFileSystemItem.MoveToAsync() method.
The implementer calls IConfirmationResultContext.ReturnConfirmationResult() or IConfirmationResultContext.ReturnErrorResult() with error code.
- If the ReturnErrorResult() method is called, the platform displays an error message with Try Again and Cancel buttons. If the user cancels the operation the platform does NOT rename/move the item. In the case of a file, the platform closes the file handle and the Engine calls IFileWindows.CloseCompletionAsync(). The Engine does NOT call IFileSystemItem. MoveToCompletionAsync() in this case and the move operation completes.
- If the ReturnConfirmationResult() method is called, the platform moves the item in the file system.
If the target is the offline folder, the platform lists the content of the target folder. The Engine calls IFolder. GetChildrenAsync() on the target folder.
The Engine calls IFileSystemItem.MoveToCompletionAsync() and the IFileWindows.CloseCompletionAsync() method. These methods may be called in any sequence.

If the MoveToCompletion() completes without exceptions the item is marked as in-sync and any information about the old location of the item is erased (note that this information may still be stored by the platform, for example, to perform the undo operation). To repeat the failed move operation you can call the EngineWindows.ProcessAsync() method.

Below is a move implementation with a file system being used as a remote storage simulation:

public async Task MoveToAsync(
    string targetUserFileSystemPath, 
    byte[] targetFolderRemoteStorageItemId, 
    IOperationContext operationContext = null, 
    IConfirmationResultContext resultContext = null,
    CancellationToken cancellationToken = default)
{
}

/// <inheritdoc/>
public async Task MoveToCompletionAsync(
    string targetUserFileSystemPath, 
    byte[] targetFolderRemoteStorageItemId, 
    IMoveCompletionContext operationContext = null, 
    IInSyncStatusResultContext resultContext = null,
    CancellationToken cancellationToken = default)
{
    string remoteStorageOldPath = Mapping.GetRemoteStoragePathById(RemoteStorageItemId);
    FileSystemInfo remoteStorageOldItem = FsPath.GetFileSystemItem(remoteStorageOldPath);

    if (remoteStorageOldItem != null)
    {
        string remoteStorageNewParentPath = WindowsFileSystemItem.GetPathByItemId(targetFolderRemoteStorageItemId);
        string remoteStorageNewPath = Path.Combine(remoteStorageNewParentPath, Path.GetFileName(targetUserFileSystemPath));

        if (remoteStorageOldItem is FileInfo)
        {
            (remoteStorageOldItem as FileInfo).MoveTo(remoteStorageNewPath, true);
        }
        else
        {
            (remoteStorageOldItem as DirectoryInfo).MoveTo(remoteStorageNewPath);
        }
    }
}

Move with Overwrite on Folders

In the case the target folder exists, the platform requests confirmation from the user for the folder to overwrite. If the user confirms, the following sequence is executed:

The platform starts the source folder deletion. The Engine calls IFileSystemItem.DeleteAsync() for the source folder.
The platform moves each file from the source folder to the target folder. The Engine calls the sequence described in 'Move without Overwrite' section above - OpenCompletionAsync() -> MoveToAsync()-> ReturnConfirmationResult() -> MoveToCompletionAsync() -> CloseCompletionAsync() method for each file and folder.
The platform completes the source folder delete operation. The Engine calls IFileSystemItem.DeleteCompletionAsync().

Implementing Delete on Windows

Delete Operation Sequence

Below we will describe the sequence of the delete operation.

In the case of a file, the platform opens the file handle and blocks it for reading and writing. The Engine calls IFileWindows.OpenCompletionAsync() method.
The platform starts the delete operation. The Engine calls IFileSystemItem.DeleteAsync() method.
The implementer calls IConfirmationResultContext.ReturnConfirmationResult() or IConfirmationResultContext.ReturnErrorResult() with error code.
- If the ReturnErrorResult() method is called, the platform displays an error message with Try Again and Cancel buttons. If the user cancels the operation the platform does NOT delete the item. In the case of a file, the platform closes the file handle and the Engine calls IFileWindows.CloseCompletionAsync(). The Engine does NOT call IFileSystemItem.DeleteCompletionAsync() in this case and the delete operation completes.
- If the ReturnConfirmationResult() method is called, the platform deletes the item in the file system.
The Engine calls IFileSystemItem.DeleteCompletionAsync() and the IFileWindows.CloseCompletionAsync() method. These methods may be called in any sequence. Note that the DeleteCompletionAsync() is called for the item that has already been deleted and does not exist in the user file system.

If the DeleteCompletionAsync() completes without exceptions the item is marked as deleted from your back-end storage and any information about it is erased (note that the item can still remain in the recycle bin or information about the item may still be stored by the platform), otherwise, the information about the deleted item is kept stored by the Engine, so the Engine can retry deleting the item at a later time. To repeat the failed delete operation you can call the EngineWindows.ProcessAsync() method.

Below is a delete operation implementation in WebDAV Drive sample:

public async Task DeleteAsync(...)
{
}

public async Task DeleteCompletionAsync(...)
{
    string remoteStoragePath = Mapping.GetRemoteStoragePathById(RemoteStorageItemId);

    FileSystemInfo remoteStorageItem = FsPath.GetFileSystemItem(remoteStoragePath);
    if (remoteStorageItem != null)
    {
        if (remoteStorageItem is FileInfo)
        {
            remoteStorageItem.Delete();
        }
        else
        {
            (remoteStorageItem as DirectoryInfo).Delete(true);
        }
    }
}

The delete sequence is different for every application and is also different for populated and unpopulated folders. When deleting unpopulated folder via Windows Explorer, it will delete this folder only. The platform has no information about content of the unpopulated folder and will not perform deletion for the children items. The Engine will call DeleteAsync() and DeleteCompletionAsync() methods for this folder only.

If the folder is populated, Windows Explorer will recursively delete all its children first. The Engine will call DeleteAsync() and DeleteCompletionAsync() methods for every item. After that Windows Explorer will delete the folder itself.

In addition to this Windows Explorer may also repeat deletion several times. You will see several DeleteAsync() and DeleteCompletionAsync() calls for every item in a folder.

Filtering Temporary Files

Transactional File Save Operation

Some applications, for example, Microsoft Office and Autodesk AutoCAD, are saving documents in a certain sequence that minimizes chances of data loss in case the file save is failed. In addition to the transactional save, Microsoft Office and AutoCAD apps create a lock-file(s) intended to indicate that the file is opened for editing. This sequence is different for every application, however, the approximate sequence looks like the following:

The lock-file(s) are created: $file.ext.
The original file is renamed (backup file is created): document.ext -> document.bak. In some cases, such a file is also marked with a hidden or temporary flag.
The new file is created: document.tmp. The new content is saved into it.
The new file is renamed to the original file name: document.tmp -> document.ext.
The backup file (document.bak) is deleted.
The lock-file ($file.ext) is deleted.

In our case, we want to avoid any temporary, hidden and backup files being uploaded to our remote storage. Here are the typical tasks you would like to do while handling the transactional save operation:

Avoid temporary files, backup files as well as lock-files being uploaded to your remote storage.
Avoid renaming and deleting the original file in your remote storage.
Avoid the file content upload to your remote storage until the file is closed (optional).
Automatically lock the file on open and unlock on close events. This is achieved by the locking mechanism provided by the User File System.

We will describe how to achieve the first three tasks in this article below. Custom data preservation requires storing such data outside of the provided storage inside a placeholder (as the placeholder will be deleted as part of the transactional save). The automatic locking is provided by the Engine in combination with IEngine.FilterAsync() implementation and is described in the Locking section in this article below.

Note that starting with Engine v4 Beta 2, the Engine automatically stores and manages remote storage ID as well as you can associate and store the eTag and any other data with the item. You also do not need to monitor changes in your user file system starting Engine v4 Beta 2.

Avoiding temporary files upload

To avoid temporary files, backup files as well as lock-files being uploaded to your remote storage you will implement the IEngine.FilterAsync() method:

public override async Task<bool> FilterAsync(
    OperationType operationType, 
    string userFileSystemPath, 
    string userFileSystemNewPath = null, 
    IOperationContext operationContext = null)
{
    switch(operationType)
    {
        // To send file content to the remote storage only when the MS Office or
        // AutoCAD document is closed, uncommnt the Create and Update cases below:
        //case OperationType.Create:
        //case OperationType.Update:

        case OperationType.Unlock:
            return FilterHelper.AvoidSync(userFileSystemPath) 
                || FilterHelper.IsAppLocked(userFileSystemPath);

        case OperationType.Move:
        case OperationType.MoveCompletion:
            // When a hydrated file is deleted, it is moved to a Recycle Bin.
            return FilterHelper.IsRecycleBin(userFileSystemNewPath) 
                || FilterHelper.AvoidSync(userFileSystemNewPath);

        default:
            return FilterHelper.AvoidSync(userFileSystemPath);
    }
}

The FilterAsync() method provides source and target item names, as well as the operation name (create, update, delete, move, open, close, lock, unlock). This method also provides information about the environment, from which you can extract the process name if needed. However, the environment info is not provided for the create operation and operations called via the EngineWindows.ProcessAsync() method.

As soon as you have filtered some files from being processed by the Engine and underlying platform, such files are NOT converted into placeholders by the Engine and remain in the user file system as regular files. The events are not fired for such items and they can not be tracked inside your IFile and IFolder interface s implementations.

Storing Custom Data Associated with Items

With the IT Hit User File System you can associate custom data of any size with each item in your file system. The PlaceholderItem class provides Properties dictionary that you can use to assoiate data with each file or folder. The dictionary contains IDataItem items that provide methods for reading and writing data.

To add new items you can use the IDictionary.Add() method or the ICustomData.AddOrUpdateAsync() method:

string eTag = ...
PlaceholderItem placeholder = engine.Placeholders.GetItem(UserFileSystemPath);
await placeholder.Properties.AddOrUpdateAsync("ETag", eTag);

To read the data you can use the indexer and IDataItem.GetValueAsync() method:

string eTag = await placeholder.Properties["ETag"].GetValueAsync<string>();

or you can use the IDataItem.TryGetValue() method:

string eTag = null;
if (placeholder.Properties.TryGetValue("ETag", out IDataItem propETag))
{
    propETag.TryGetValue<string>(out eTag);
}

The cusom data is being moved with the item and is deleted when the item is deleted. Note that the custom data is NOT being copied when you copy an item in your user file system.

To make sure there is no any remains of your cusom data left after your application uninstall we recommend using the packed installer.

Starting Engive v4 Beta 2 you do not need to store any data outside of the file system.

Locking

Locking and Unlocking Items

To lock items, the IT Hit User File System provides the ILock interface. You will implement it on files and folders that support locking. This interface provides LockAsync(), UnlockAsync() and GetLockModeAsync() methods.

Inside your ILock.LockAsync() method implementation you will lock the file in your remote storage as well as you will save the LockMode, passed as a parameter, somewhere in your local storage, on the client machine:

public async Task LockAsync(
    LockMode lockMode, 
    IOperationContext operationContext = null,
    CancellationToken cancellationToken = default)
{
    LockInfo lockInfo = await Program.DavClient.LockAsync(
        new Uri(RemoteStoragePath), 
        LockScope.Exclusive, 
        false, 
        null, 
        TimeSpan.MaxValue);

    ServerLockInfo serverLockInfo = new ServerLockInfo
    {
        LockToken = lockInfo.LockToken.LockToken,
        Exclusive = lockInfo.LockScope == LockScope.Exclusive,
        Owner = lockInfo.Owner,
        LockExpirationDateUtc = DateTimeOffset.Now.Add(lockInfo.TimeOut)
    };

    // Save lock-token and lock-mode.
    PlaceholderItem placeholder = Engine.Placeholders.GetItem(UserFileSystemPath);
    await placeholder.Properties.AddOrUpdateAsync("LockInfo", serverLockInfo);
    await placeholder.Properties.AddOrUpdateAsync("LockMode", lockMode);
}

In your ILock.UnlockAsync() method you will unlock the item in your remote storage:

public async Task UnlockAsync(
    IOperationContext operationContext = null,
    CancellationToken cancellationToken = default)
{
    // Read the lock-token.
    PlaceholderItem placeholder = Engine.Placeholders.GetItem(UserFileSystemPath);
    var lockInfo = placeholder.Properties["LockInfo"];
    string lockToken = (await lockInfo.GetValueAsync<ServerLockInfo>())?.LockToken;

    LockUriTokenPair[] lockTokens = new LockUriTokenPair[] 
    { 
        new LockUriTokenPair(new Uri(RemoteStoragePath), lockToken)
    };

    // Unlock the item in the remote storage.
    try
    {
        await Program.DavClient.UnlockAsync(new Uri(RemoteStoragePath), lockTokens);
    }
    catch (ITHit.WebDAV.Client.Exceptions.ConflictException)
    {
        // The item is already unlocked.
    }

    // Delete lock-mode and lock-token info.
    placeholder.Properties.Remove("LockInfo");
    placeholder.Properties.Remove("LockMode");
}

Inside your ILock.GetLockModeAsync() you will return your stored LockMode to the Engine:

public async Task<LockMode> GetLockModeAsync(
    IOperationContext operationContext = null,
    CancellationToken cancellationToken = default)
{
    PlaceholderItem placeholder = Engine.Placeholders.GetItem(UserFileSystemPath);

    IDataItem property;
    if (placeholder.Properties.TryGetValue("LockMode", out property))
    {
        return await property.GetValueAsync<LockMode>();
    }
    else
    {
        return LockMode.None;
    }
}

Storing the LockMode

During the ILock.LockAsync() call you must store the LockMode passed to this method on your local machine. When the file handle is closed the Engine will request the LockMode by calling your ILock.GetLockModeAsync() method implementation. You will typically return the LockMode passed into LockAsync() from this method. If your GetLockModeAsync() method returns LockMode.Auto, the Engine will unlock the file by calling the ILock. UnlockAsync() method.

Do NOT store the LockMode in your remote storage, as the ILock. GetLockModeAsync() method may be called from the Windows Explorer context menu and it expects a fast response in this case.

Manual Locking

To lock or unlock the document on the client machine, for example from your custom Windows Explorer lock context menu command, you will call methods of the IClientNotifications interface which is returned by the IEngine.ClientNotifications() call:

IClientNotifications cn = engine.ClientNotifications(@"C:\Users\User1\VD\myfile.ext");
cn.LockAsync(LockMode.Manual);

The IClientNotifications.LockAsync() method has a parameter that indicates if the document should be automatically unlocked when the file handle is closed. This parameter will be passed to your ILock.LockAsync() method implementation. Typically you will pass the LockMode.Manual parameter to IClientNotifications.LockAsync() call, which will indicate that you will unlock the document manually, by calling the IClientNotifications.UnlockAsync() method. If you pass the LockMode.Auto, the file will be unlocked by calling ILock.UnlockAsync() when the file handle is closed.

Automatic Locking

On the Windows platform, the IT Hit User File System supports automatic document locking and unlocking. Typically automatic document locking is useful to lock the document when it is opened by Microsoft Office and unlock on close.

To enable the automatic documents locking set the EngineWindows. AutoLock property to true when creating the Engine instance.

If the automatic documents locking is enabled, when a file handle is opened with the FILE_SHARE_READ or FILE_SHARE_DELETE or FILE_SHARE_NONE sharing mode, the Engine will call the ILock. LockAsync() method passing LockMode.Auto. When the file handle is closed it will call ILock.UnlockAsync() method.

Locking Events Design

Note that the locking event occurs after the file is being opened for writing. The failed locking event can NOT prevent the file from being opened for editing. This is because the Engine and underlying platform API do not provide any events that occur during the file open. The reason behind such a design is that file opening operation is a frequent event and must be instant, while any requests to the remote storage will slow down the file system performance. In addition to that, your file system may be in offline mode, with a server being unavailable and unable to lock the file.

In case your file failed to lock, either because it is being locked by another user, or because of any other reasons, you can mark the file as read-only, so it can NOT be saved. While the read-only attribute does NOT protect the file from modifications, still most applications, including Microsoft Office, will respect the read-only attribute and will NOT update the file.

Another option is to notify clients via web sockets (or another channel) and set the read-only flag on the file as well as render the lock icon in Windows Explorer on the client machines when the file is locked on the server.

Open and Close Events

The Engine provides the IFileWindows.OpenCompletionAsync() and the IFileWindows.CloseCompletionAsync() methods, that are called when the file handle is being opened and closed. Note that both events occur after the file is opened/closed and you can not prevent a file handle from being opened or closed.

For performance reasons, neither the Engine nor the underlying platform provides events that occur before the file handle open or close events or any events that can prevent the file from opening/losing.

Packaged Installation on Microsoft Windows Platform

The packaged installation is a new type of installer on Windows that provides the following major features:

Automatic COM components installation and uninstallation. All COM components are automatically unregistered leaving a clean environment after uninstallation.
Simplifies COM components debugging. You can debug your thumbnails handler, custom columns (CustomStateHandler) and context menu by simply starting your packaging project.
Automatic uninstall cleanup. Your sync root registration will be automatically unregistered and any data created by your application or by the Engine will be deleted.
Installation without admin privileges.
Deployment to the Windows Store.

The package can be also used for direct deployment to users.

The Virtual Drive sample and WebDAV Drive sample projects include the Windows Application Packaging projects that support all the above features.

Implementing Thumbnails Support for Windows

Programming

Creating Virtual Drive in .NET

In this article

Creating Virtual Drive in .NET

Move and Delete Operations on Microsoft Windows Platform

Implementing Move on Windows

Move Operation Sequence

Move with Overwrite on Folders

Implementing Delete on Windows

Delete Operation Sequence

Filtering Temporary Files

Transactional File Save Operation

Avoiding temporary files upload

Storing Custom Data Associated with Items

Locking

Locking and Unlocking Items

Storing the LockMode

Manual Locking

Automatic Locking

Locking Events Design

Open and Close Events

Packaged Installation on Microsoft Windows Platform

Next Article: