I had this issue where I was trying to migrate a client to the cloud and I kept getting problems with the zip file not extracting properly. I kept getting various errors. I started to trace the files history, it gets created on the client deployment server, then ftp’d to our ftp server and the downloaded from the ftp server.
This technique is really handy if you are going to move huge bits of data, and want to know that the file is sane before you start to use it in anger.
download windows fciv https://www.microsoft.com/en-us/download/details.aspx?id=11533
I check my client copy and see:
C:\Users\shannonm\Downloads>c:\temp\fciv.exe DV910include.zip
//
// File Checksum Integrity Verifier version 2.05.
//
062172c62ed68fc54422629fb593c8a6 dv910include.zip
okay, not check the ftp server copy – this is a linux box, not a different utility that gives the same information – phew!
sudo apt-get install md5sum
myriad@mywebftp:/var/www/html/upload/files$ md5sum DV910include.zip
062172c62ed68fc54422629fb593c8a6 DV910include.zip
That shows me that I’m downloading (destination) exactly what is on the FTP server and these are the same.
Then I log into the source server
H:\data>fciv DV910include.zip
//
// File Checksum Integrity Verifier version 2.05.
//
0964db2477f1724ec8e5cd5c275cebc1 dv910include.zip
TOTALLY different (well, from an md5 point of view) – what does this mean? https://en.wikipedia.org/wiki/Checksum
I’ve moved from using the FTP server to using s3 buckets directly in AWS. I was not going to do this because of problems with getting to https://aws.amazon.com on the client site, but then noticed that I could install the AWS command line tools and run them without any problems. Using AWS S3, I have no problems with the zip files or the extraction.
Note that I also get VERY good speed on the up and down, because there are some fat pipes going into AWS. About 24 GB download in 15 minutes
No comments:
Post a Comment