As we continue to improve upon our JSON data feeds, we wanted to provide a means to download them as efficiently as possible. First we made them available via HTTP, accessible via our portal itself. Now we are additionally providing the ability to download the files in larger batches of tar.gz files rather than the individual files. They can be extracted via this index URL: http://publisherfeeds.connexity.com/feeds/batch/index.txt.gz.
This index provides links to tar.gz batch files with the following naming convention: batch-[hashId].tar.gz. These tar.gz files bundle the same content into a fewer number of larger files (<100 files) for more efficient downloading. Once extracted the batch files provide the same feed files that could otherwise be downloaded from the individual direct links. For example:
tar –zxvf batch-67y.tar.gz #>> produces: 123_45600.json, 987_45600.json, 987_45601.json, …
More details can be found in the feed documentation. Please contact us if you need a copy of the documentation. Make sure to remember to append your publisher credentials in order to access the content!
We are continually working to improve the content we provide, and how to access it, and we welcome your feedback and any recommendations for improvement.