Something bad happened / rsync

DevKinsta Bug Reports
1

Q: Date/Time this occurred (Provide your time zone also)
A: 2022-09-07 06:01 AM AST

Q: DevKinsta Version
A: 2.7.0.4846

Q: OS Version
A: Mac OS 12.5 (M1)

Q: Docker Desktop Version
A: 4.11.1 (84025)

Q: Were any error codes or messages observed? If so, what were they?
A: Something bad happened

Q: Detailed Description of the Problem
**A: I am trying to sync a remote staging site with my local copy. It gets to about 1.6GB of 1.7 and then generates the error: Something bad happened. I’ve tried restarting several times. The download step of the sync process takes several minutes (>5 minutes). I checked the log, I saw that during that step, rsync is running. When I manually run rsync, it completes in a minute. I’m not sure what DevKinsta is doing for rsync that forces it to download every single file. See the log below.

[2022-09-07 06:01:25.473] [warn]  [terminalExec] Error: 
rsync: connection unexpectedly closed (8573955 bytes received so far) [receiver]

[2022-09-07 06:01:25.480] [warn]  [terminalExec] Error: rsync error: error in rsync protocol data stream (code 12) at io.c(228) [receiver=3.2.3]

[2022-09-07 06:01:25.483] [warn]  [terminalExec] Error: rsync: connection unexpectedly closed (1127478 bytes received so far) [generator]
rsync error: timeout in data send/receive (code 30) at io.c(228) [generator=3.2.3]

[2022-09-07 06:01:25.538] [info]  [ProgressIndicator] { isFailed: false, isOpen: true }
[2022-09-07 06:01:25.685] [info]  [terminalExec] Child process exited with code 30
[2022-09-07 06:01:25.688] [error] [downloadSite] Error while downloading files Error: Encountered an error in child process 50625: 30. rsync: connection unexpectedly closed (1127478 bytes received so far) [generator]
rsync error: timeout in data send/receive (code 30) at io.c(228) [generator=3.2.3]

    at ChildProcess.<anonymous> (/Applications/DevKinsta.app/Contents/Resources/app.asar/main.prod.js:2:821249)
    at ChildProcess.emit (node:events:394:28)
    at ChildProcess.emit (node:domain:475:12)
    at maybeClose (node:internal/child_process:1064:16)
    at Socket.<anonymous> (node:internal/child_process:450:11)
    at Socket.emit (node:events:394:28)
    at Socket.emit (node:domain:475:12)
    at Pipe.<anonymous> (node:net:672:12)
[2022-09-07 06:01:25.696] [error] [ipcMainStep] Error in operation SITE_PULL, step rsync: Error: Encountered an error in child process 50625: 30. rsync: connection unexpectedly closed (1127478 bytes received so far) [generator]
rsync error: timeout in data send/receive (code 30) at io.c(228) [generator=3.2.3]

    at ChildProcess.<anonymous> (/Applications/DevKinsta.app/Contents/Resources/app.asar/main.prod.js:2:821249)
    at ChildProcess.emit (node:events:394:28)
    at ChildProcess.emit (node:domain:475:12)
    at maybeClose (node:internal/child_process:1064:16)
    at Socket.<anonymous> (node:internal/child_process:450:11)
    at Socket.emit (node:events:394:28)
    at Socket.emit (node:domain:475:12)
    at Pipe.<anonymous> (node:net:672:12)
[2022-09-07 06:01:25.757] [info]  [editGlobalPhpIni] Getting content from  /www/kinsta/public/hidden/.user.ini
[2022-09-07 06:01:25.758] [info]  [dockerUtil/getContainer] Get 'devkinsta_fpm' Docker container
[2022-09-07 06:01:25.774] [info]  [ProgressIndicator] { isFailed: true, isOpen: true }
[2022-09-07 06:01:25.850] [info]  [containerExec] Command 'test -f /www/kinsta/public/hidden/.user.ini && echo $?' on devkinsta_fpm finished with exit code 1
[2022-09-07 06:01:25.851] [info]  [PhpIniEditor] Loading content into editor:
2

Hi @shomari, thanks for reaching out. Every time I see this sort of issue with rsync, it’s due to large files/directories on the origin site. It’s usually old backup/zip files or node_modules/.git directories. Moving those out of ~/public before the pull usually helps it complete.

Unfortunately there’s currently no way to exclude directories/files from the Pull, however we are working on that feature.

Please try deleting or moving any large and unnecessary files before trying again. I’m assuming the bulk of your site size is coming from the files and not the database.

3

Ok thanks. After a few attempts it worked but now I’m back at it again since I decided to start using GIT. It looks like the DevKinsta dashboard doesn’t include .git when calculating the files but it actually does attempt to transfer them. For now, I’ll move the .git folder before syncing.

1 Like
4

We’ve been having similar problems over here. We have a site with some very large files. The backup is nearly 2Gb in size. The automatic sync always fails during the download step. You can retry multiple times and it will get to different places but generally always fail. This has been tried now on 4 different systems 3 Linux Ubuntu 22.04 and a Mac running 10.5.1.

The solution has been to generate a downloadable backup of the desired environment in Kinsta, and install it manually with the recipe found here: Manually Import a Backup - Kinsta®.

I’m not familiar with where the log files are on the Mac platform (which is what I’m using now), but if you want additional data, point me to where those are and I can provide.

My take is that the download process in DevKinsta isn’t robust enough to recover from certain network glitches that cause it to fail. A retry starts over, which is not at all useful given the size of the site. Interestingly enough, the file download process built into the browser has no problems whatsoever with what I presume is the same backup file DevKinsta doesn’t seem to be able to handle.

My gut feeling is that if the backup download were to be improved, even if it just resumed where it left off, this problem would for all purposes go away.