Question:
I have to export many, many files from the last ten years from a single project. When I try to export, Spitfire times out. Is there a different way to bulk export files that would give me better results?
Answer:
Yes! If you have the PAGE | Can Export Files from the Catalog permission.
Note: this article applies to the Power UX Catalog. For the Classic UI, see Classic UI: How to Export Files
Exporting a Bunch of Files
If you need export under 300 files with a total native size under 6GB, you can use the “Files Only” view of the Catalog, set your backend filter and then select your files. The grid supports selecting multiple rows: click on the first, shift or control click on the second. Then right click and choose “Download” from the context menu. Respond YES to the confirmation.
If you have selected too many files, you will see a “Too many files” message.
Using the Bulk Export Tool
The bulk export tool is designed to export ALL files associated with a project using a series of zip files.
Again, start with the Files Only view of the catalog. Specify the desired project id as a filter. No other filters matter, but for speed you can specify a date range. When you have results, click on the “Select All” button, then the Export button.
The Bulk Export From Catalog tool opens and finds the project id you specified and attempts to discern the range of years. You can broaden or narrow the range of years. You cannot specify exact dates here.
The “Include Outbound Email PDFs” increases the number of files by including the small PDFs that were stored to document each and every route via email. This helps document that you notified the route recipient. If you do not need these files, uncheck the box.
You can specify a number of files per zip — but there is a size limit that you cannot control. We find that around 400 is typically safe.
When you are ready, click SCAN. The grid will fill with a list of zip files that will be generated and downloaded. The process will begin when you click START. It will take a long
time.
When there are more files than will fit in a single zip file, the tool creates a series of zips. The files from that year will be placed into the zip files in date order. File 1 of 4 will have January through (perhaps) March. File 2 of 4 will begin in March and continue.
The bottom of the grid indicates how much space. PDF files do not compress much when zipped since they are already compressed. Be sure you have enough local disk space.
When you click START, the process of building the ZIP files will begin. You should choose a time when there are very few users.
The grid will update to indicate progress. It is possible that some files will fail.
Make good notes so you can go back for the missed files!
Getting Missed Zips
In the example above, there were 3 pieces missing from 2019.
- Repeat the process to open the tool
- Specify the year you need as the starting and ending year
- Important: You *must* use exactly the same value for “# Per Zip” and the “include outbound email PDFs” options. This is essential so the tool breaks up the ZIP files in exactly the same way. If you change these options, re-download the entire year!
- Hold the CTRL key and click SCAN and specify “Skip” when prompted
- Click on the Status column in the row(s) for the pieces you still need and change the status from Skip to Pending
- Click Start