Check transferred data

After transferring files (either uploading or downloading) between the repository and your computer, you should always check that all the files have been transferred completely. When transferring a large number of files and/or large files, something may go wrong (e.g. a short interruption in the network connection or an unexpected server downtime). As such, we strongly recommend checking that your data have been transferred correctly. If you have uploaded files via the web portal, this check has already been done by the system and you have been alerted if any of the files was not transferred correctly.

If you encounter problems or errors when checking your data transfer, you can contact the ICT helpdesk (email: icthelpdesk@ru.nl, telephone: 0031 (0)24 3622222).

Step 1: Obtain the manifest file

First, you will need a manifest file to compare your transferred data with. Do not use the MANIFEST.txt file currently in your collection, but create one by following these steps:

  1. In the RDR webpage, select your collection

  2. If your collection is in the Internal review or External review state, first change it back to Editable by clicking on the Switch to Editable button on the top left

  3. Click on the Manifest tab

  4. Click on Generate manifest file. Large collections with more than 500 (sub)folders or more than approximately 20.000 files might result in a failed download. In those cases, please contact RDM support in order to receive your manifest file

  5. Move the downloaded manifest file to the folder of your computer that contains the local copy of the files you want to check

Note: Once a collection is published or archived, it is no longer possible to generate a manifest file via the Manifest tab. Instead, the manifest is available in the collection as part of the data (under the Files tab), called MANIFEST.txt.

Step 2: Check the transferred files

Now you can check your transfer by computing the checksum. Be aware that this can take some time, especially if there are a lot of files and/or particularly large files. How you can compute the checksum, depends on your operating system. Click on your preferred OS to open further instructions:

Windows

  1. Change the manifest file's extension from .txt to .sha256.

    If the file extensions are not visible, go to the Windows explorer folder with the manifest file and click on the View tab (Beeld) in the top of your folder. Then tick the box File name extensions (bestandsnaamextensies).

  2. If you are a Radboud University employee and have access to the Software Centre, you can download the hashcheck tool (named 'HashCheck') via the Software Centre. Otherwise, you can go to https://github.com/gurnec/HashCheck/releases/tag/v2.4.0 and download the file HashCheckSetup-v2.4.0.exe.

    ../../../_images/check_data_hashcheck_tool.png

    Install the tool by running the .exe file. Restart your computer after installation.

  3. Double-click on the manifest file with .sha256 extension you have downloaded. This should open the tool you have just downloaded. It will open a window that lists the files that are matched and non-matched.

The check will probably result in three 'Unreadable' results for MANIFEST.txt, LICENSE.txt and ABOUT.txt. This occurs because the tool is unable to compare these three automatically generated files in your RDR collection to similar files in your local folder, because they are not present in your local folder. You can ignore the warning for these three files.

MacOS

  1. Open the command terminal and change the directory to the folder that contains the local copy of the files from the repository:

    $ cd ~/PathToLocalCollection

  2. Run the shasum command with the name of the manifest file you downloaded (in our example this is manifest.txt):

    $ shasum -a 256 -c manifest.txt

    To see only the names of the non-matched files, run the following command:

    $ shasum -a 256 -c manifest.txt | grep -v 'OK'

    If shasum is not installed on your system, you can use Homebrew to install it. See http://brew.sh for details.

The terminal now shows the matched and non-matched files in the collection directory. The check will probably result in three 'FAILED' results for MANIFEST.txt, LICENSE.txt and ABOUT.txt. This occurs because the tool is unable to compare these three automatically generated files in your RDR collection to similar files in your local folder, because they are not present in your local folder. You can ignore the warning for these three files.

Linux

  1. Open the command terminal and change the directory to the folder that contains the local copy of the files from the repository:

    $ cd ~/PathToLocalCollection

  2. Run the sha256sum command with the name of the manifest file (in our example this is manifest.txt):

    $ sha256sum -c manifest.txt

    If sha256sum is not installed on your computer, you can install it by running the following code:

    $ apt-get install coreutils

The terminal now shows the matched and non-matched files in the collection directory. The check will probably result in three 'FAILED' results for MANIFEST.txt, LICENSE.txt and ABOUT.txt. This occurs because the tool is unable to compare these three automatically generated files in your RDR collection to similar files in your local folder, because they are not present in your local folder. You can ignore the warning for these three files.