Deserialize: Merging

[hippyau]

Hi There!

I have implemented a function to deserialize a filesystem created with Serialize...

// deserialize show file
bool LoadFileSystem(std::string FileName = "")
{
    if (FileName.size() == 0)
        FileName = "last.fs";

    ifstream in(FileName, ios::in | ios::binary);

    if (in.is_open())
    {
        in.seekg(0, std::ios::end);
        std::streampos length = in.tellg();
        in.seekg(0, std::ios::beg);

        std::vector<char> buffer(length);
        in.read(&buffer[0], length);
        in.close();

        try
        {
            g_VFS.Deserialize(buffer);
        }
        catch (VFS::CVFSException e)
        {
          //  spdlog::error("Failed to deserialize: '{}' -> {}",FileName, e.what());     
            return false;
        }

        return true;
    }

    // spdlog::error("Could not open file: {}", FileName);
    return false;
}

This is working, but say create some default directories, eg "/sys", I save the file system, then load a file system, I have now in PrintDirs two "/sys" folders.

Is there a simple way to merge or overlay these?

Also is there a better way to load, so I don't load the entire thing into memory here when loading? I couldn't really think of one.

Cheers,

[tostc]

Hello,

there is currently no deserialize and merge method. Anyway you could create a temporarly filesystem which deserializes your file. Now you can loop over all nodes of the temp one and transfer it to your global one.

[hippyau]

My app would have one root virtual filesystem that always exists, at startup I would...

        vfs.CreateDir("/bin");
        vfs.CreateDir("/boot");
        vfs.CreateDir("/dev");
        vfs.CreateDir("/etc");
        // create /etc/defaults.conf....        

because they always should exist, even if a vfs is loaded later.

...we do some work...

I then save the state, I serailize the filesystem to vfs.vfs

Next startup, if exists we load vfs.vfs and the contents should merge with the pre-created content, and it does...

but there are duplicate /etc & /dev etc...

This way can 'patch the application' with just a vfs file to use the newer overlayes files?

[tostc]

As I mentioned earlier, you can create a temp filesystem which deserialize the file and then you must iterate over each dir and merge them with your main filesystem.

Maybe following pseudo code helps you. Inside your LoadFileSystem you can do following:

CVFS tempFileSystem;

...
//Inside your try block replace g_VFS by tempFileSystem
tempFileSystem.Deserialize(buffer); // Contains now your saved filesystem from disk
...
// Go over each directory beginning from / and add all files which are missing or different to your `g_VFS`. An example how to get files from a directory you can look into the `PrintDirs` function of the `main.cpp`

[hippyau]

Ah, okay,

because I get... (where "show" is the VFS instance)

  // bring a file from the real world into the VFS
    uint ImportFile(std::string localPathFileName, std::string importPathFileName)
    {       
        uint success = 0; // assume we failed...

        //Reads the local path/file.ext into the virtual filesystem at importPathFileName
          // open local file
            ifstream in(localPathFileName, ios::in);            
            if (in.is_open())
            {
                try
                {
                    auto fs = VFS::show.Open(importPathFileName, VFS::FileMode::RW);
                    while (in.good())
                    {
                        char c;
                        in.read(&c, sizeof(c));
                        if (in.good()){               

                        fs->Write(&c, sizeof(c));   // EXCEPTION...:   

                            success++;
                        }
                    }


            inline size_t Write(const char *Data, size_t Size)
            {
                if((m_Mode & FileMode::WRITE) == FileMode::WRITE)
                {

                return m_File->Write(Data, Size); // EXCEPTION: Segfault

                }

                return 0;
            }

                }
                catch (VFS::CVFSException &e)
                {
                    spdlog::error("importfile: Exception: {} --> {} : {}", localPathFileName, importPathFileName, e.what());
                    success = 0;
                }

                in.close();

                return success;

            } else {
                spdlog::error("importfile: local file not available: '{}'", importPathFileName);
            }
       
        return success;
    }

[tostc]

...
// I'm not sure what happens if importPathFileName parent directories don't exist. Maybe this causes your exception
auto fs = VFS::show.Open(importPathFileName, VFS::FileMode::RW);
...

For example if you have a file system with the path /tmp/folder1 and you write your file to /tmp/folder1/subfolder/MYFILE.txt, where subfolder doesn't exists.